Senior Software Engineer - Data Infrastructure

Marqeta Marqeta · Fintech · Job Requisition - Premium+/Premium/National · ML, AI, Data Platform

Senior Software Engineer on the Data Infrastructure team, responsible for building and maintaining the data lakehouse, streaming, orchestration, and catalog platforms that support analytics and AI at Marqeta. This role involves owning and delivering projects, leading design, and ensuring reliability and performance of data infrastructure components.

What you'd actually do

  1. Define and deliver well-scoped milestones for projects, and may be a technical lead for projects on your team.
  2. Lead design for small-to-medium components and participate in design for larger components, moving from task-level impact to small-to-medium project-level impact.
  3. Have significant ownership of a component of the data infrastructure, including its reliability, performance, and developer experience.
  4. Identify gaps and propose plans towards addressing those gaps.
  5. Independently write code that is clear, well-tested, and appropriately flexible, reusable, and adaptable to changing requirements.

Skills

Required

  • Python
  • SQL
  • Distributed systems
  • Apache Kafka
  • Apache Spark
  • Apache Airflow
  • Parquet
  • Hudi
  • Iceberg
  • Delta Lake
  • AWS
  • Terraform
  • CloudFormation
  • CDK
  • Monitoring
  • Alerting

Nice to have

  • Payments
  • FinTech
  • Apache Iceberg
  • CDC tooling
  • Debezium
  • Kafka Connect
  • DataHub
  • Java
  • Kubernetes
  • Container-based workloads
  • Scalable software platforms

What the JD emphasized

  • required part of the software engineering role
  • 5+ years of experience as a software engineer
  • Strong proficiency in Python in a production context
  • Proficiency with SQL
  • Experience working with distributed systems
  • Production experience with multiple core data technologies, including Apache Kafka, Apache Spark, and Apache Airflow (or close equivalents)
  • Production experience with modern table formats (Parquet, Hudi, Iceberg, or Delta Lake)
  • Production experience with AWS or another major cloud
  • Experience with infrastructure-as-code (Terraform, CloudFormation, CDK, or equivalent)
  • Experience monitoring and alerting on production systems
  • Ability to decompose business problems into solutions that may require designs with multiple interacting components
  • Ability to build correct, maintainable software components
  • Ability to work with cross-functional stakeholders to identify technical blindspots and clarify ambiguity
  • Ability to present clear and concise messages tailored to various audiences; able to communicate clearly and respectfully both verbally and in writing
  • A focus on quality and meeting deadlines