Software Engineer - Observe

Snowflake Snowflake · Data AI · CA-Menlo Park, United States · Engineering

Software Engineer for the Data Intake team at Snowflake, focusing on building the agent, ingestion infrastructure, and in-product onboarding experience for an AI-powered observability platform. The role involves contributing to a custom OpenTelemetry collector distribution, designing control-plane APIs, and debugging complex ingestion issues to ensure data flows correctly to the Observe platform.

What you'd actually do

  1. Contribute to the Observe Agent, a custom OpenTelemetry collector distribution, building receivers, processors, and exporters that collect and forward customer telemetry at scale
  2. Build and maintain configuration-heavy systems including OTel collector configs, Helm charts, Kubernetes manifests, and AWS infrastructure templates
  3. Design and implement control-plane APIs and data models that represent ingestion state, pipeline errors, and operational events for both internal tooling and user-facing experiences
  4. Collaborate with product and platform teams to deliver in-product onboarding flows that help customers connect their systems and validate data is flowing correctly
  5. Debug complex ingestion issues spanning agents, pipelines, and backend systems, often across codebases and environments you did not originally build

Skills

Required

  • 2+ years of experience in software engineering with a focus on backend systems, distributed systems, or infrastructure
  • Proficiency in Go or another statically typed language
  • Experience with distributed systems, data pipelines, or agent-based software architectures
  • Familiarity with Kubernetes, container orchestration, and cloud environments (AWS, GCP, or similar)
  • Comfort reading, navigating, and reasoning about large open-source codebases and systems you did not originally write
  • A systematic approach to debugging distributed systems and pipelines across production environments
  • Ability to move fluidly between configuration, infrastructure, and application code

Nice to have

  • Experience with OpenTelemetry, observability agents, or telemetry pipeline architectures
  • Background building or operating control planes for distributed systems or agent infrastructure
  • Prior work on developer onboarding flows, configuration management, or user-facing diagnostics for infrastructure products
  • Exposure to platform, infrastructure, or SRE-adjacent team environments

What the JD emphasized

  • custom OpenTelemetry collector distribution
  • agent
  • ingestion infrastructure
  • control-plane APIs
  • data models
  • debug complex ingestion issues
  • systems that are easy to reason about