Senior Software Engineer - Data Platform (spark & Trino)

Workday Workday · Enterprise · Vancouver, BC

This role focuses on building and enhancing a large-scale distributed data processing platform using Spark and Trino in a cloud environment. The responsibilities include designing fault-tolerant architectures, implementing self-healing mechanisms, and developing comprehensive telemetry for observability and scalability. The role is primarily engineering-focused on data infrastructure.

What you'd actually do

  1. Developing data processing algorithms and techniques that work on large datasets, facilitating an interactive querying experience over large volumes of data.
  2. Resiliency: Designing and implementing strategies for high availability, fault isolation, and graceful degradation of Spark and Trino clusters.
  3. Observability: Building end-to-end tracing, deep metrics, and advanced alerting frameworks to proactively identify performance bottlenecks and system anomalies.
  4. Scalability: Optimizing resource allocation, cluster auto-scaling, and multi-tenant isolation to support highly elastic and unpredictable workloads.

Skills

Required

  • Java
  • Scala
  • Linux/Unix
  • AWS or GCP
  • database internals
  • query processing
  • distributed systems
  • SQL
  • Apache Spark
  • Trino
  • PrestoSQL
  • Presto
  • high availability
  • fault tolerance
  • observability
  • telemetry
  • Prometheus
  • Grafana
  • OpenTelemetry
  • JMX metrics

Nice to have

  • BS in Computer Science or a related field
  • MS/PhD in Computer Science
  • AI coding tools like Cursor

What the JD emphasized

  • 8+ years of software development experience in Java and/or Scala in Linux/Unix environment.
  • Experience operating large-scale distributed systems natively within public cloud environments (AWS or GCP)
  • 3+ years in database internals, query processing, or distributed systems
  • Proven experience architecting and tuning distributed systems for high availability, fault tolerance, and massive horizontal scale.