Senior Software Engineer

The Trade Desk The Trade Desk · Media · Irvine, CA · Software Engineering

Senior Software Engineer role focused on architecting and optimizing large-scale distributed systems and data/ML pipelines. Responsibilities include leading the lifecycle of data products, enhancing algorithmic systems with data-driven insights, applying graph algorithms, and mentoring junior engineers. Requires strong experience in Python, Scala, SQL, distributed computing (Spark), CI/CD, cloud platforms, and data analysis.

What you'd actually do

  1. Architect and optimize large-scale distributed systems that handle petabyte-scale data across cloud and on-premise data centers, ensuring high availability and performance.
  2. Design and develop robust data and ML pipelines with strong emphasis on automation, scalability, data governance, quality assurance, and privacy compliance.
  3. Lead the end-to-end lifecycle of data products, including design, development, validation, and production deployment to meet client and business needs.
  4. Leverage data-driven insights to enhance large-scale algorithmic systems, improving operational efficiency and accuracy through analytical experimentation.
  5. Develop and apply advanced graph algorithms to solve complex real-world challenges.

Skills

Required

  • Python
  • Scala
  • SQL
  • Agile software development
  • Data product lifecycle management
  • Version control systems
  • REST API design and development
  • CI/CD implementation
  • Spark
  • Scalable algorithm design
  • Graph-based algorithms
  • Columnar database systems
  • Cloud computing platforms
  • Data analysis
  • Data visualization
  • Workflow orchestration frameworks
  • Data and ML pipeline development

What the JD emphasized

  • petabyte-scale data
  • data governance
  • quality assurance
  • privacy compliance
  • large-scale algorithmic systems
  • graph algorithms
  • Python, Scala or other high-level programming languages
  • SQL and relational database systems
  • working with Agile software development environments
  • participating in the full lifecycle of data products, including design, testing, validation, and production deployment
  • version control systems
  • designing, developing, and maintaining REST APIs for data and software systems
  • implementing continuous integration and continuous deployment (CI/CD) practices for data and software systems
  • distributed computing frameworks including Spark
  • designing and implementing scalable algorithms, including graph-based approaches
  • columnar database systems for data analytics
  • cloud computing platforms and tools
  • data analysis and visualization for large-scale (terabytes or larger) datasets
  • workflow orchestration frameworks
  • developing and maintaining data and machine learning pipelines