Sr. Engineer - Data & ML Platform (hybrid, Ind)

CrowdStrike CrowdStrike · Enterprise · Bangalore, India

This role focuses on building and scaling an ML Experimentation Platform, encompassing data preparation, feature engineering, model training, and model serving. It involves developing scalable ML pipelines, modularizing ML code, establishing patterns for model development and deployment, and leveraging workflow orchestration tools. The role also involves integrating with cloud services and CI/CD frameworks, with future plans for generative AI use cases.

What you'd actually do

  1. Help design, build, and facilitate adoption of a modern Data+ML platform
  2. Modularize complex ML code into standardized and repeatable components
  3. Establish and facilitate adoption of repeatable patterns for model development, deployment, and monitoring
  4. Build a platform that scales to thousands of users and offers self-service capability to build ML experimentation pipelines
  5. Leverage workflow orchestration tools to deploy efficient and scalable execution of complex data and ML pipelines

Skills

Required

  • Python
  • distributed computing
  • orchestration technologies (Kubernetes, Airflow)
  • infrastructure-as-code tools (Terraform, FluxCD)
  • CI/CD frameworks (GitHub Actions)
  • containerization frameworks
  • ML Platform tools (Jupyter Notebooks, MLFlow, Ray, Vertex AI)
  • data platform tools (Apache Spark, Flink)
  • Machine Learning concepts
  • Distributed Systems Knowledge
  • Data Platform Experience

Nice to have

  • Java
  • Scala
  • Go
  • Iceberg
  • Pinot
  • Jenkins
  • Parquet
  • Protocol Buffers/GRPC

What the JD emphasized

  • B.S. in Computer Science, Data Science, Statistics, Applied Mathematics, or a related field and 10+ years related experience; or M.S. with 8+ years of experience; or Ph.D with 6+ years of experience
  • 3+ years experience developing and deploying machine learning solutions to production
  • 3+ years experience with ML Platform tools like Jupyter Notebooks, NVidia Workbench, MLFlow, Ray, Vertex AI etc.
  • Experience building data platform product(s) or features with (one of) Apache Spark, Flink or comparable tools in GCP
  • Expert level experience with Python
  • Expert level experience with CI/CD frameworks such as GitHub Actions
  • Expert level experience with containerization frameworks
  • Distributed Systems Knowledge
  • Data Platform Experience
  • Machine Learning concepts

Other signals

  • ML Experimentation Platform from the ground up
  • Data Preparation, Cataloging, Feature Engineering, Model Training, and Model Serving
  • production-focused culture that bridges the gap between model development and operational success
  • generative AI investments