Senior Software Engineer, Data Platforms

Lyft Lyft · Consumer · Toronto, ON · Infrastructure

Senior Software Engineer for Lyft's Streaming Compute Team, focusing on designing, developing, and maintaining low latency, high throughput streaming systems. These systems are critical for various teams and almost all ML models at Lyft, handling use cases like persisting service events, detecting issues, computing prime-time, and creating/storing/serving ML features for training and inference. The role involves leading projects, improving architecture, and ensuring operational excellence for real-time streaming platforms using technologies like Apache Kafka, Flink, and Beam, often in a cloud environment.

What you'd actually do

  1. Design, develop, deploy, monitor, operate and maintain scalable streaming solutions that constitute our streaming platform
  2. Work with technologies such as Apache Kafka, Flink, Beam etc. to enhance our data platform's streaming offerings
  3. Collaborate with cross-functional teams to integrate streaming solutions into the broader data architecture
  4. Analyze our internal systems and processes and locate areas for improvement/automation
  5. Collaborate with product org stakeholders to address and prioritize custom edge cases

Skills

Required

  • 5+ years of experience with building, deploying and maintaining low latency, high throughput services
  • 5+ years of experience building and developing large-scale infrastructure, distributed systems or networks, and/or experience with compute technologies
  • 3+ years of software engineering industry experience and with distributed streaming solutions like Apache Flink, Beam, Spark streaming, Streaming SQL etc.
  • Experience working with Kubernetes and container technologies (e.g. Docker, cri-o, etc)
  • Familiar with a cloud-based environments such as AWS/GCP/Azure

What the JD emphasized

  • low latency and high throughput systems
  • ML features for training and inference purposes
  • near-real-time streaming systems
  • delivering high-quality software solutions that solve critical business needs
  • dive deep into problems with lots of ambiguity
  • leading architectural design and building robust, well-tested technical solutions
  • take complete ownership of a project or a feature
  • operational excellence