Data Engineer, Notifications

Whatnot · Consumer · San Francisco, CA · Engineering

Data Engineer on the Notifications Platform responsible for owning and evolving high-volume data systems that support product development, experimentation, and machine learning.

What you'd actually do

  1. Own data architecture end-to-end. Define how we capture, model, and serve critical business data—then implement it in production. You’ll make architectural decisions around storage formats, compute patterns, and SLAs that balance cost, scalability, and consistency.
  2. Build mission-critical pipelines. Develop and operate batch data workflows that process high-volume events related to notifications with tight guarantees for latency, completeness, and accuracy.
  3. Design and implement canonical models. Create domain-oriented data models that serve as the source of truth for analytics, ML, and production applications. Establish and enforce modeling standards, ownership boundaries, and data contracts across teams.
  4. Enforce data quality at scale. Build tests, lineage, monitoring, and reconciliation systems that make every dataset observable and every anomaly actionable.
  5. Automate operational workflows. Partner with business systems and platform teams to eliminate manual data handoffs and reconcile data across services, warehouses, and external systems.

Skills

Required

  • SQL
  • Python
  • Kafka
  • Debezium
  • dbt
  • Spark
  • Flink
  • Dagster
  • Airflow
  • Snowflake
  • BigQuery
  • Redshift
  • CI/CD
  • infrastructure-as-code

Nice to have

  • ML Engineers
  • Analytics Platform team
  • Monte Carlo
  • Great Expectations

What the JD emphasized

  • 5+ years of experience in the data or software engineering domain
  • high-volume events
  • hundreds of millions of events per day
  • hundreds of millions of rows per day