Senior Data Platform Engineer

Upstart · Fintech · Remote · Data Engineering

Senior Data Platform Engineer responsible for designing and building scalable data platform infrastructure, including distributed systems for data discovery, metadata management, and data access. The role involves developing self-serve tooling, improving data reliability and observability, and partnering with ML engineers and analysts. A key aspect is contributing to platform evolution in areas like metadata services, natural language data access, and AI-enabled data interactions.

What you'd actually do

  1. Design and build scalable data platform infrastructure, including distributed systems that power data discovery, metadata management, and data access
  2. Develop and maintain tooling that enables teams to self-serve data transformations, testing, and deployment workflows within the Lakehouse
  3. Improve data reliability and observability by building systems for monitoring, alerting, and data quality validation
  4. Partner with machine learning engineers, analysts, and product teams to understand data needs and drive adoption of platform capabilities
  5. Contribute to platform evolution in areas such as metadata services, natural language data access, and AI-enabled data interactions

Skills

Required

  • Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field (or equivalent practical experience) plus 4+ years of experience
  • Experience building and operating distributed data systems or data platform infrastructure in a cloud environment (e.g., AWS)
  • Proficiency in at least one programming language such as Python, Java, Scala, Javascript or Kotlin
  • Experience working with data processing frameworks or infrastructure (e.g., Spark, Kafka, Airflow, or similar technologies)
  • Experience designing, deploying, and maintaining production-grade data systems, including monitoring and reliability practices

Nice to have

  • Experience building or supporting data platforms, developer tooling, or internal infrastructure products
  • Knowledge of metadata management, data cataloging, or data governance systems
  • Experience working with Lakehouse architectures, Databricks, or similar modern data platforms
  • Ability to collaborate effectively with cross-functional stakeholders and translate data concepts into practical solutions
  • Interest in applying AI/ML capabilities to improve data accessibility, discovery, or user experience

What the JD emphasized

  • AI-enabled data interactions