Software Engineer - Database Integrations

ClickHouse ClickHouse · Data AI · EMEA · Engineering

Software Engineer role focused on building and maintaining real-time data replication platform (ClickPipes) for ClickHouse, enabling data ingestion from various databases and data lakes at petabyte scale. Responsibilities include designing high-throughput integrations, ensuring end-to-end reliability, debugging complex issues, and driving product innovation through customer collaboration.

What you'd actually do

  1. Design and develop high-throughput integrations with databases (Postgres, MySQL, MongoDB), data lakes (Iceberg, Delta Lake), and data warehouses (BigQuery, Snowflake).
  2. Handle edge cases in real-world production scenarios: unconventional database setups, internals of data types, database upgrades/failovers, large transactions, etc.
  3. Design integration solutions to enable users to fully harness ClickHouse's performance and throughput.
  4. Debug complex issues in production leveraging runtime diagnostics (e.g. pprof, parca) and observability tools (e.g. metrics, logging, tracing).
  5. Build and improve infrastructure and tools to increase system reliability, reduce incident response time, and simplify/automate operations.

Skills

Required

  • Go
  • Systems programming
  • Cloud-native experience (AWS/GCP/Azure)
  • Kubernetes
  • Production debugging
  • Data-intensive software solutions

Nice to have

  • Database replication technologies (CDC, logical replication)
  • Durable execution frameworks (Temporal)
  • Data formats and protocols (Avro, Parquet, Protobuf)
  • Modern data processing frameworks (Kafka, Spark, Flink)
  • Maintaining/contributing to open-source software

What the JD emphasized

  • 5+ years of industry experience building data-intensive software solutions
  • Proficient in Go, or experienced in systems programming with willingness to ramp up quickly in Go
  • Cloud-native experience deploying and operating services on at least one major cloud platform (AWS/GCP/Azure)
  • Practical experience with Kubernetes
  • Strong problem solver and solid production debugging skills