Director, Data Platform Engineering

Lila Sciences Lila Sciences · AI Frontier · San Francisco, CA · Software

Director of Data Platform Engineering to lead a team responsible for Lila's product data platform, owning the end-to-end architecture, delivery, reliability, and developer/data scientist experience. The platform supports analytical and ML workloads, including AI inference workflows. The role involves team leadership, technical strategy, stakeholder management, and driving innovative solutions for data interfaces, exploration, query, analytics, and ML/inference at scale.

What you'd actually do

  1. Build, mentor, and manage a high-performing team of 8-12 data engineering experts.
  2. Define and execute the technical roadmap for our data platform, aligning with Lila’s overall data strategy;
  3. Partner with data scientists, data engineers, lab scientists, product managers, and other stakeholders to understand their data processing needs and requirements;
  4. Drive innovative, agentic, and low-code solutions to deliver data interfaces - exploration, query, analytics, and ML/inference solutions at scale.
  5. Represent Lila’s data platform work at external conferences;

Skills

Required

  • 12+ years of software development experience, with a focus on data processing at scale.
  • 5+ years of experience leading senior engineers.
  • Experience with building on AWS/GCP primitives like S3 + Athena/BigQuery, and query engines.
  • Operated data platforms at petabyte scale with sub-second query latency requirements.
  • Experience managing data infrastructure supporting 100+ concurrent ML training and inference workloads.
  • Familiarity with LLM/AI-native data patterns — vector stores, embedding pipelines, pre/mid/post training.
  • Track record of building data platforms in high-growth or early-stage environments where speed-to-value mattered as much as long-term architecture.
  • Hands-on coding in Python and modern backend frameworks.
  • Experience with infrastructure-as-code and containerized deployments (Kubernetes).
  • BS, MS, or Ph.D. in Computer Science or a related field of study.

Nice to have

  • Thought leadership in the community via presentations in conferences or blog posts.
  • Experience building and growing teams focusing on open source technologies.
  • Scientific data management and quality experience
  • Built self-service data products/platforms where developer experience was a first-class product concern.

What the JD emphasized

  • Operated data platforms at petabyte scale with sub-second query latency requirements.
  • Experience managing data infrastructure supporting 100+ concurrent ML training and inference workloads.
  • Track record of building data platforms in high-growth or early-stage environments where speed-to-value mattered as much as long-term architecture.

Other signals

  • Own the data platform and infrastructure end-to-end — architecture, delivery, reliability, and developer/data scientist experience.
  • Platform supports analytical and machine learning workloads across Lila, serving autonomous DBTL cycles, instrument data pipelines, and AI inference workflows.
  • Drive innovative, agentic, and low-code solutions to deliver data interfaces - exploration, query, analytics, and ML/inference solutions at scale.