Senior Software Engineer

Adobe Adobe · Enterprise · San Jose, CA

Senior Software Engineer role focused on building and maintaining high-performance data ingestion pipelines and storage layers for Adobe Experience Platform. The role involves handling petabytes of real-time customer profile data, ensuring scalability, resiliency, and data correctness. Requires strong programming skills in Java or Scala and experience with distributed processing systems and cloud data lake storages.

What you'd actually do

  1. Collaborate with a team of engineers & product managers in building high-performance data ingestion pipelines and data store to serve the use cases of Segmentation and Activation.
  2. Own responsibility for design and implementation of key components of ingesting and maintaining petabyte of Profile data
  3. Develop systems to support high volume data ingestion pipelines handling both streaming and batch processing.
  4. Leverage popular file and table formats to design storage models to support the required ingestion volumes and data access patterns.
  5. Explore tradeoffs across different formats and schema layouts driven by workload and application characteristics

Skills

Required

  • Java
  • Scala
  • Distributed processing systems like Apache Spark, Hadoop Stack, or Apache Kafka
  • Data Lake cloud storages like Azure Data Lake Storage or AWS (Amazon Web Services) S3
  • building resilient data pipelines at scale

Nice to have

  • Apache Parquet
  • Databricks Delta
  • Apache Iceberg
  • Apache Hudi
  • NoSQL databases like Apache HBase, Cassandra, Mongo, or Azure Cosmos DB

What the JD emphasized

  • 15+ years of industry background