Senior Software Engineer - Data Infrastructure Services

Weights & Biases Weights & Biases · Data AI · Bellevue, WA +1 · Technology

Senior Software Engineer role focused on building and scaling data infrastructure services, including managed databases, data ingestion, and stream processing, for an AI-focused cloud platform. The role involves designing, implementing, and operating these systems, ensuring performance, reliability, and security, and establishing data governance guidelines.

What you'd actually do

  1. Design and implement the platform to deliver data to teams with a focus on providing managed solutions through APIs
  2. Participate in operations and scaling of relational data platforms.
  3. Develop a stream processing architecture and solve for scalability and reliability.
  4. Improve the performance, security, reliability, and scalability of our data platforms, and related services and participate in the teams on-call rotation.
  5. Establish guidelines, guard rails for data access and storage for stakeholder teams.

Skills

Required

  • 5+ years of experience in a software or infrastructure engineering industry
  • experience operating services in production and at scale
  • reliability engineering concepts
  • CAP theorem and concurrency models
  • define data models and establish guidelines around data management
  • operating these systems at scale
  • Kubernetes
  • Linux distro, shell scripting, and the Linux storage and networking stacks
  • Go/Python/Java

Nice to have

  • experience with designing and operating these systems at scale
  • interest or comfortable with using it for event-driven and/or stateful orchestration
  • contribute to a Kubernetes operator in order to manage data systems
  • transform problems in elastic solutions, decompose them into achievable tasks, and socialize both to your teammates
  • interested in contributing to open source
  • new research in data structures
  • distributed NewSQL datastores such as CockroachDB, TiDB, YDB, Yugabyte
  • stream processing tools such as NATS or Kafka

What the JD emphasized

  • Ensure compliance with standards for data protection regulation.