Sr. Software Development Engineer

Adobe Adobe · Enterprise · San Jose, CA

This role focuses on building and maintaining high-performance data ingestion pipelines and storage layers for Adobe Experience Platform, handling petabytes of data for real-time customer profiles. It involves developing resilient, scalable, and efficient systems for both streaming and batch processing, leveraging distributed processing systems and cloud storage.

What you'd actually do

  1. Collaborate with a team of engineers & product managers in building high-performance data ingestion pipelines and data store to serve the use cases of Segmentation and Activation.
  2. Own responsibility for design and implementation of key components of ingesting and maintaining petabyte of Profile data
  3. Develop systems to support high volume data ingestion pipelines handling both streaming and batch processing.
  4. Leverage popular file and table formats to design storage models to support the required ingestion volumes and data access patterns.
  5. Deploy production services and iteratively improve them based on customer feedback

Skills

Required

  • Master’s degree in Information Systems Management, Computer Science, Computer Engineering, Information Technology, Software Engineering or related field and 5+ years of experience
  • B.S. or M.S. in Computer Science or a related field or equivalent experiences
  • Experience with distributed processing systems like Apache Spark, Hadoop Stack, or Apache Kafka
  • Strong programming skills with extensive experience in Java or Scala
  • Leadership skills to collaborate and drive cross-team efforts
  • Excellent communication skills

Nice to have

  • Understanding of file formats like Apache Parquet and table formats such as Databricks Delta, Apache Iceberg or Apache Hudi
  • Understanding of NoSQL databases like Apache HBase, Cassandra, Mongo, or Azure Cosmos DB
  • Practical experience in building resilient data pipelines at scale

What the JD emphasized

  • Experience with distributed processing systems like Apache Spark, Hadoop Stack, or Apache Kafka
  • Experience with Data Lake cloud storages like Azure Data Lake Storage or AWS (Amazon Web Services) S3
  • Practical experience in building resilient data pipelines at scale is preferred