Staff Software Engineer - Logs Observability Pipelines [new York]

Datadog Datadog · Enterprise · New York, NY · Dev Eng

Staff Software Engineer focused on building and scaling high-throughput systems for log processing, routing, and transformation within customer-managed observability pipelines. The role involves driving architecture, leading cross-team initiatives, and ensuring system performance and scalability in distributed environments.

What you'd actually do

  1. Drive the architecture and technical direction for unifying observability pipelines and on-prem observability storage into a cohesive BYOC Logs experience
  2. Build and scale high-throughput systems for log processing, routing, and transformation across distributed environments
  3. Lead cross-team initiatives, aligning engineers, product managers, and stakeholders to deliver complex, multi-team projects
  4. Design and implement software that runs reliably within customer-managed infrastructure, including on-prem and cloud environments
  5. Improve system performance, scalability, and cost efficiency through thoughtful trade-off analysis and capacity planning

Skills

Required

  • experience building and operating software deployed in customer-managed or on-prem environments
  • strong expertise in distributed systems, including scalability, performance optimization, and high-throughput data processing
  • proficient in systems-level programming (e.g., Go, Rust, or C++)
  • understand how software runs across diverse environments (e.g., Linux systems)
  • experience with cloud platforms (e.g., AWS, Azure, or GCP)
  • comfortable troubleshooting infrastructure and networking issues
  • experienced with containerization and orchestration technologies such as Kubernetes in production environments
  • track record of leading large, cross-functional engineering efforts and influencing technical direction across teams

Nice to have

  • Go
  • Rust
  • C++

What the JD emphasized

  • customer-managed infrastructure
  • on-prem observability storage
  • high-throughput systems
  • distributed environments
  • customer-managed or on-prem environments
  • distributed systems
  • high-throughput data processing
  • systems-level programming
  • cloud platforms
  • Kubernetes in production environments
  • large, cross-functional engineering efforts