Senior Data Engineer

Polymarket Polymarket · Fintech · New York, NY · Engineering

Senior Data Engineer to scale Polymarket's data platform, powering user-facing products with low-latency analytics and serving data layers. Responsibilities include owning the OLAP analytics layer, partnering on the OLTP serving layer, shaping streaming and data lake infrastructure, and designing data models at scale. Requires deep knowledge of OLTP/OLAP architectures, columnar warehouses, data lakes, streaming pipelines, data modeling, PostgreSQL, and SQL.

What you'd actually do

  1. Own the OLAP analytics layer. Drive our columnar warehouse environment end-to-end: raw event ingestion → cleaned facts and dimensions → business aggregates → API-serving views. You'll own materialized view design, refresh cadences, dictionary catalog, query planning, and cost optimization.
  2. Partner on the OLTP serving layer. Work closely with the team on high-write serving tables that back our product APIs – partition strategy, indexing, trigger pipelines, autovacuum tuning, and bloat monitoring – with sub-100ms read-path discipline.
  3. Shape streaming and data lake infrastructure. Define Kafka topic schema contracts, evolve the S3 lake layout with modern table formats, and contribute to parity-validation tooling that guards data correctness under migration pressure.
  4. Design data models at scale. Work with event-sourced, append-mostly data with chain-reorg semantics. Design the derivative analytics – PnL, realized/unrealized position tracking, cohort metrics – and formalize ownership boundaries between upstream ingestion and downstream analytics.
  5. Coordinate across teams. Negotiate schema contracts with the warehouse-owning team and downstream consumers including frontend, notifications, and third-party integrators.

Skills

Required

  • 5+ years of data engineering on production systems serving real users at scale
  • Deep knowledge of OLTP/OLAP split architectures
  • Columnar warehouse expertise: ClickHouse strongly preferred; Snowflake, BigQuery, Redshift, or Apache Pinot accepted if fundamentals are solid
  • Data lake experience: Parquet, Iceberg (or Delta/Hudi), compaction strategies, S3 layout discipline
  • Streaming pipeline experience: Kafka, exactly-once vs. at-least-once reasoning, backpressure, consumer-group patterns, schema evolution
  • Strong data modeling fundamentals: star/snowflake, SCD patterns, CDC, idempotent event sourcing, dimensional vs. event-log tradeoffs
  • PostgreSQL at scale: partitioning, index design, autovacuum/bloat remediation, query planning, CDC triggers vs. logical replication
  • SQL fluency at warehouse scale: window functions, CTEs, dictionary-based enrichment, dialect specifics
  • Distributed systems reasoning: consistency models, event ordering, replay semantics, write-once vs. mutable state, reorg handling

Nice to have

  • EVM indexing experience: rindexer, subgraphs, or comparable
  • Rust: you'll touch indexer and validation tooling codebases; comfortable reading and contributing
  • Domain knowledge in DeFi, prediction markets, or order-book systems
  • Observability and SLO thinking: Prometheus metrics design, dashboard discipline, alert-fatigue avoidance
  • Python for SQL tooling, ad-hoc analysis, and one-off migrations
  • Track record shipping a platform migration or greenfield data stack under a hard deadline

What the JD emphasized

  • sub-100ms latency targets
  • senior operator
  • drive entire workloads end-to-end
  • make Polymarket's analytics and serving data layer something the rest of the company can build on without thinking
  • Schema drift, parity gaps under migration, latency regressions, and cost blowouts all land on this role
  • you're the one who can say yes confidently, or redesign the slice that makes it possible
  • sub-100ms read-path discipline
  • parity-validation tooling that guards data correctness under migration pressure
  • chain-reorg semantics
  • ownership boundaries
  • hard deadline