Senior Backend Engineer - Data Ingestion (clickpipes)

ClickHouse ClickHouse · Data AI · Engineering

Senior Backend Engineer focused on data ingestion pipelines (ClickPipes) for ClickHouse, a real-time analytics and data warehousing platform. The role involves developing and enhancing integrations with diverse data sources, improving system scalability and resilience, and working with distributed systems and cloud services at petabyte scale.

What you'd actually do

  1. Develop and enhance integrations with various data sources including streaming platforms, databases, data lakes, and object stores.
  2. Continuously improve our systems based on operational metrics, customer feedback, and evolving business requirements.
  3. Drive technical discussions and contribute to architectural decisions that impact our platform's scalability and resilience.
  4. Participate in on-call rotations to ensure system reliability and respond to production incidents.

Skills

Required

  • 5+ years of relevant software development industry experience building data-intensive software solutions.
  • Strong knowledge of Golang and experience with its ecosystem.
  • Experience with distributed systems and microservices architecture
  • The ability to design and build robust ETL data pipelines that can handle large volumes of data reliably and efficiently.
  • Understanding data replication methodologies like CDC.
  • Good knowledge of cloud-native architecture and practical experience with at least one major CSP.
  • Excellent communication skills and the ability to work well within a team and across engineering teams.
  • Strong problem solver and have solid production debugging skills.

Nice to have

  • Experience with Java is a plus.

What the JD emphasized

  • real-time data ingestion
  • change data capture (CDC)
  • distributed systems
  • cloud services
  • Golang
  • Java
  • ETL data pipelines
  • data replication methodologies like CDC
  • cloud-native architecture
  • production debugging skills