Software Engineer - Database Integrations

ClickHouse ClickHouse · Data AI · EMEA · Engineering

Software Engineer for the ClickPipes - Database Integrations team, responsible for building and maintaining a platform for real-time data replication from various databases into ClickHouse at petabyte scale. This involves solving complex distributed systems problems related to database internals, schema evolution, data type compatibility, latency, and data consistency. The role also includes ensuring end-to-end reliability through debugging, improving infrastructure, and participating in on-call rotations, as well as driving product innovation by collaborating with customers and internal teams.

What you'd actually do

  1. Design and develop high-throughput integrations with databases (Postgres, MySQL, MongoDB), data lakes (Iceberg, Delta Lake), and data warehouses (BigQuery, Snowflake).
  2. Handle edge cases in real-world production scenarios: unconventional database setups, internals of data types, database upgrades/failovers, large transactions, etc.
  3. Design integration solutions to enable users to fully harness ClickHouse's performance and throughput.
  4. Debug complex issues in production leveraging runtime diagnostics (e.g. pprof, parca) and observability tools (e.g. metrics, logging, tracing).
  5. Build and improve infrastructure and tools to increase system reliability, reduce incident response time, and simplify/automate operations.

Skills

Required

  • Go
  • Systems programming
  • Cloud-native experience
  • Kubernetes
  • Production debugging

Nice to have

  • Database replication technologies (CDC, logical replication)
  • Durable execution frameworks (Temporal)
  • Data formats and protocols (Avro, Parquet, Protobuf)
  • Modern data processing frameworks (Kafka, Spark, Flink)
  • Maintaining/contributing to open-source software

What the JD emphasized

  • 5+ years of industry experience building data-intensive software solutions.
  • Proficient in Go, or experienced in systems programming with willingness to ramp up quickly in Go.
  • Strong problem solver and solid production debugging skills.