Comcast brings together the best in media and technology. We drive innovation to create the world's best entertainment and online experiences. As a Fortune 50 leader, we set the pace in a variety of innovative and fascinating businesses and create career opportunities across a wide range of locations and disciplines. We are at the forefront of change and move at an amazing pace, thanks to our remarkable people, who bring cutting-edge products and services to life for millions of customers every day. If you share in our passion for teamwork, our vision to revolutionize industries and our goal to lead the future in media and technology, we want you to fast-forward your career at Comcast.
Job Summary
We are seeking an experienced Senior Spark Engineer to design, develop, and optimize large-scale streaming systems using Apache Spark. This role involves building high-performance data pipelines, and collaborating with cross-functional teams to enable advanced analytics and machine learning.
Job Description
Key Responsibilities
Data Pipeline Development:
Design and implement scalable streaming jobs in Cloud and on-prem
Optimize Spark jobs for performance and cost efficiency.
Dev Ops:
Develop and maintain automated tooling for operational montoring and maintenance of complex pipelines
Implement best practices for distributed data processing and storage.
Collaboration:
Work closely with data scientists, analysts, and business stakeholders to deliver reliable datasets.
Data Quality & Governance:
Implement data validation, monitoring, and lineage tracking.
Ensure compliance with security and privacy standards.
Continuous Improvement:
Research and adopt new technologies to improve Spark-based data engineering processes.
Required Skills & Qualifications
Technical Expertise:
Strong proficiency in Spark/Scala
Development Experience, familiarity with code repos, CI/CD
Programming & Tools:
Proficiency in Scala or Java.
Cloud & Infrastructure:
Hands-on experience with cloud platforms (AWS EMR, Azure Databricks, GCP Dataproc).
Familiarity with CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation).
Experience:
5+ years in engineering roles, with at least 2 years focused on Spark/Scala.
Education:
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
Nice to have
Experience with real-time streaming (Kafka, Spark Streaming).
Dockers / Kubernetes Experience
Databricks Experience
Knowledge of performance tuning and troubleshooting Spark jobs.
Certifications in cloud platforms or big data technologies.
Strong problem-solving and communication skills.
We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools that are personalized to meet the needs of your reality—to help support you physically, financially and emotionally through the big milestones and in your everyday life.
Please visit the benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Certifications (if applicable)
Relevant Work Experience
7-10 Years
Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.