Senior Cloud Data Infrastructure Engineer

ClickHouse ClickHouse · Data AI · Engineering

The Cloud AutoScaling Engineer will build and maintain auto-scaling capabilities for the ClickHouse cloud environment, aiming to transform it into a server-less database solution. This involves improving the metrics pipeline, developing algorithms for autoscaling statistics and recommendations, and working with Kubernetes operators for vertical and horizontal scaling. The role also requires collaboration with core database and data plane teams to support autoscaling use cases and other infrastructure improvements, architecting robust, scalable, and highly available distributed systems.

What you'd actually do

  1. Build a cutting-edge Cloud Native platform on top of the public cloud.
  2. Improve the metrics pipeline and build algorithms to generate better autoscaling statistics and recommendations.
  3. Work on the autoscale and Kubernetes operator to support seamless Vertical and Horizontal Auto-scaling.
  4. Work closely with our ClickHouse core development team and other data plane teams, partnering with them to support auto-scaling use cases as well as other internal infrastructure improvements.
  5. Architecting and building a robust, scalable, and highly available distributed infrastructure

Skills

Required

  • Go
  • C++
  • Kubernetes
  • AWS
  • GCP
  • Azure
  • distributed systems
  • fault-tolerant systems
  • scalable systems
  • controller runtime

Nice to have

  • Python
  • uv
  • rye
  • fastAPI
  • Pandas
  • NumPy
  • Spark
  • Kafka

What the JD emphasized

  • 5+ years of relevant software development industry experience building and operating scalable, fault-tolerant, distributed systems.
  • Experience building operators with Kubernetes, controller runtime
  • Production experience with programming languages like Go, C++
  • Expertise with a public cloud provider (AWS, GCP, Azure) and their infrastructure as a service offering (e.g., EC2).