Senior Data Engineer

Adobe Adobe · Enterprise · San Francisco, CA

Senior Data Engineer to build and optimize data architecture and ETL pipelines for Adobe's Pro Design products, enabling AI/ML features, analytics, and AI-driven insights. Role involves data governance, pipeline optimization, and experience with AI/ML data pipelines and agentic systems.

What you'd actually do

  1. Build, scale and optimize the data architecture and ETL/pipeline on Databricks across Pro Design products, spanning shared platform data and in-app usage telemetry.
  2. Partner with Product, Engineering, and Data Science to define and implement data logging and instrumentation that enables product experimentation, AI/ML features, and analytics.
  3. Set and evolve governance standards regarding data quality, privacy, security, lineage, and SLAs in Unity Catalog.
  4. Build automated reporting and the data foundations that power conversational, AI-driven insights across our reporting stack.
  5. Keep production pipelines and queries healthy and performant at scale and drive optimization for cost and latency.

Skills

Required

  • SQL
  • Python
  • PySpark
  • Airflow
  • Databricks workflows
  • Data engineering (5+ years)
  • Production pipelines at scale
  • High-volume event/telemetry data
  • Integrating AI tools
  • Building multi-agent AI systems or agentic workflows
  • AI/ML data pipelines (feature stores, embeddings, or retrieval)
  • Modern BI/reporting tools
  • Partnering with Product, Engineering, and Data Science

Nice to have

  • Databricks
  • Unity Catalog
  • Azure and Azure Cloud platforms
  • Retrieval Augmented Generation (RAG)
  • data privacy and compliance for user-behavior data (e.g., GDPR, CCPA)

What the JD emphasized

  • Experience integrating AI tools to optimize/redesign workflows towards higher efficiency, quality, or latency.
  • Experience building multi-agent AI systems or agentic workflows.
  • Working knowledge of AI/ML data pipelines (feature stores, embeddings, or retrieval)
  • A track record of partnering with Product, Engineering, and Data Science to turn ambiguous questions into reliable data models.

Other signals

  • Data Engineering
  • ETL/Pipelines
  • AI/ML Features
  • Data Governance
  • AI-driven Insights
  • Agentic Workflows