Principal Data Engineer, Llm/ai Platforms (remote)

CrowdStrike CrowdStrike · Enterprise · United States · Remote

Principal Data Engineer focused on building and optimizing data infrastructure for LLMs, RAG, and agentic systems at Exabyte scale. This role involves designing, implementing, and deploying scalable solutions, mentoring engineers, and establishing MLOps/DataOps best practices for AI services within a cybersecurity context.

What you'd actually do

  1. Architect, implement, and optimize data platforms and pipelines specifically designed to support LLMs, Retrieval-Augmented Generation (RAG), and sophisticated AI agentic systems at Exabyte scale.
  2. Drive the adoption and deployment of agentic workflows and agent harnessing techniques to create autonomous, data-driven security features.
  3. Design and implement highly scalable, fault-tolerant, and cost-effective data solutions, emphasizing rapid iteration and high-quality deployment.
  4. Write elegant, production-ready code with a focus on performance, maintainability, and testing rigor, ensuring the ability to ship fast without compromising quality.
  5. Provide technical leadership and deep expertise in data modeling, normalization, and semantic cataloging for AI/ML workloads.

Skills

Required

  • LLM integration
  • agentic workflows
  • agent harnessing
  • data platforms and pipelines
  • RAG
  • distributed data processing frameworks (Spark, Dask, Flink)
  • cloud platforms (AWS, GCP, or OCI)
  • containerization and orchestration (Docker, Kubernetes)
  • message queuing and streaming technologies (Kafka, Pulsar)
  • Data Warehousing (Snowflake, BigQuery)
  • Data Orchestration (Airflow, Kubeflow)
  • Python
  • designing and delivering large-scale distributed systems
  • engineering practices (code reviews, architecture design, testing)

Nice to have

  • MLOps Tools (MLflow, Sagemaker, Vertex AI)
  • agentic workflow frameworks (LangChain, LlamaIndex)
  • JVM technologies
  • cybersecurity, intelligence, or high-compliance industries
  • contributions to open-source projects related to data or AI/ML

What the JD emphasized

  • Exabyte scale
  • agentic workflows
  • agent harnessing
  • LLM integration
  • shipping fast
  • high-quality code
  • massive scale

Other signals

  • Designing, building, and deploying cutting-edge data infrastructure that powers our next generation of AI-driven security products.
  • Significant hands-on experience in LLM integration, agentic workflows, and agent harnessing to deliver high-impact, scalable solutions.
  • Architect, implement, and optimize data platforms and pipelines specifically designed to support LLMs, Retrieval-Augmented Generation (RAG), and sophisticated AI agentic systems at Exabyte scale.