Senior Data Engineer

Together AI Together AI · Data AI · San Francisco, CA · Engineering

Senior Data Engineer to build and operate data infrastructure for billing, analytics, and BI tools. Requires expertise in stream processing, real-time analytics, and IaC. Role involves designing, building, and scaling data platforms in a fast-paced environment.

What you'd actually do

  1. Identify, design, and develop foundational data infrastructure components capable of handling millions or billions of events daily
  2. Analyze and improve the robustness and scalability of existing data processing infrastructure
  3. Partner with product teams to understand functional requirements and deliver solutions that meet business needs
  4. Write clear, well-tested, and maintainable infra-as-code and software for both new and existing systems
  5. Conduct design and code reviews, create developer documentation, and develop testing strategies for robustness and fault tolerance

Skills

Required

  • 5+ years of demonstrated experience in building large scale, fault tolerant, distributed data platforms, stream processing pipelines, ETLs, etc
  • Expert-level skills in designing, building, and operating stream processing pipelines with services like AWS Kinesis, Apache Kafka, or Redpanda
  • Expert-level knowledge of building real-time customer facing analytics systems using services like AWS TimeStream or Clickhouse
  • Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform, AWS CDK, or Pulumi
  • Proficiency in version control practices and integrating IaC with CI/CD pipelines.
  • Proficiency in implementing and managing GitOps workflows with tools such as ArgoCD, Github Actions, TeamCity, or similar
  • Proficiency in one or more of Golang, Rust, Python, Java, or TypeScript
  • Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical experience

Nice to have

  • Experience with Kubernetes, or containers a plus