Data Engineer, Infrastructure Finops

Anduril Anduril · Defense · Costa Mesa, CA · Corporate Technology : Information Security

Data Engineer to build and scale a next-generation data platform, focusing on designing and maintaining scalable data pipelines, applying software engineering principles to data infrastructure, architecting and modeling data, and ensuring data quality and reliability.

What you'd actually do

  1. Design and Build Scalable Data Pipelines: Architect, develop, and maintain robust and efficient ETL/ELT pipelines to process petabyte-scale data from a wide variety of sources using platforms like Google BigQuery , Databricks, and Palantir Foundry .
  2. Apply Software Engineering Principles: Treat data infrastructure as a software product. Develop reusable frameworks, libraries, and tools to accelerate data pipeline development and ensure reliability , using best practices like version control, code reviews, and CI/CD.
  3. Architect and Model Data: Design and implement well-structured, performant data models and schemas in our data warehouse and data lakehouse environments, optimized for both analytical querying and application use.
  4. Ensure Data Quality and Reliability: Implement comprehensive data quality checks, monitoring, and alerting to proactively identify and resolve data issues. Troubleshoot and optimize performance bottlenecks in data processing and storage layers.
  5. Be a Force Multiplier: Collaborate closely with data scientists, analysts, software engineers, and business stakeholders

Skills

Required

  • Python
  • Google BigQuery
  • Databricks
  • Palantir Foundry
  • SQL
  • data modeling
  • ETL/ELT
  • data warehousing
  • data lakehouse
  • GCP
  • AWS
  • Azure
  • Docker
  • Kubernetes

Nice to have

  • Spark
  • PySpark
  • dbt
  • Terraform
  • CloudFormation
  • Kafka
  • Flink
  • Pub/Sub
  • CI/CD
  • GitHub Actions
  • Jenkins
  • data services
  • data products
  • API design
  • Tableau

What the JD emphasized

  • Must be eligible to obtain and maintain a US Top Secret security clearance.