Job Requisition ID #
26WD97610
Position overview
We are building a large-scale, API-first streaming platform on Apache Flink, Apache Kafka, and AWS. The platform powers near-real-time ingestion, processing, and delivery of data for many internal and product-facing use cases. Everything is exposed and governed behind well-designed APIs, and teams consume capabilities through a self-serve portal as well as programmatic interfaces. We are hiring a Principal Engineer to own architecture and technical design for this platform: from stream topology and operational model to control-plane services, developer experience, and production excellence. This is a high-impact role with real architectural ownership - you will partner across product, platform, and data teams, mentor engineers, and raise the bar on reliability, security, and velocity.
Responsibilities
- Architect and design the streaming platform
- Define end-to-end architecture for stream processing: ingest, stateful processing, delivery, replay, and integration with downstream systems on Flink and Kafka
- Design API-first boundaries (REST and event-driven patterns as appropriate) so streams, jobs, schemas, and operational workflows are manageable and safe at scale
- Shape multi-tenant or multi-team patterns: quotas, isolation, naming, observability, and upgrade strategies for long-running Flink workloads
- Evolve AWS-native deployment and operations (networking, IAM, scaling, resilience patterns) aligned with organizational security and compliance requirements
- Build and evolve platform surfaces
- Lead design of Spring Boot-based services that implement the platform control plane, metadata, orchestration hooks, and integration with Kafka/Flink operations
- Contribute to the self-serve portal and related UX: work with product and design and apply solid frontend engineering practices so internal users can onboard, configure, and operate without excessive manual support
- Drive developer experience: documentation, SDKs or clients where applicable, golden paths, and reducing time-to-first-successful pipeline
- Engineer for scale, reliability, and cost
- Define and refine SLIs/SLOs, capacity planning, backpressure handling, and failure modes for streaming workloads
- Institutionalize testing (unit, integration, contract, load) and progressive delivery for streaming jobs and services
- Champion observability: metrics, tracing, structured logging, and actionable alerting for Kafka, Flink, and surrounding services
- CI/CD, infrastructure, and operations
- Own and improve CI/CD pipelines for application code, infrastructure, and safe promotion of Flink jobs and configuration
- Partner on Infrastructure as Code, environment strategy, and operational runbooks; participate in on-call or operational escalation as appropriate for a principal (you build it, you run it mindset)
- Lead technically and collaborate
- Translate ambiguous requirements into clear technical strategies, RFCs, and phased execution plans
- Mentor engineers, lead architecture and code reviews, and align teams on standards for Java, APIs, streaming, and AWS usage
- Work with security, networking, and platform teams on secure-by-design streaming and API patterns
What success looks like
- A clear, documented architecture for the streaming platform that teams can adopt without heroics
- Reliable, observable Kafka and Flink estates with predictable scaling and incident response
- APIs and self-serve flows that reduce tickets, manual operations, and time-to-production for stream owners
- Stronger engineering practices - CI/CD, testing, reviews - and a more capable team around you
Minimum qualifications
- 10+ years of professional software engineering experience, including deep hands-on ownership of distributed systems in production
- Expert-level experience with Apache Kafka (topics, partitioning, consumer groups, reliability, operations) and Apache Flink (state, checkpoints, rescaling, fault tolerance, performance tuning)
- sustained production ownership, not occasional use
- Strong proficiency in Java as the primary language for large, maintainable systems
- Demonstrated experience with Spring Boot (or equivalent JVM microservices frameworks) building production APIs and services
- Hands-on AWS experience designing and operating services (compute, networking, IAM, observability patterns; familiarity with Kubernetes or managed container/orchestration patterns where the platform runs)
- Proven CI/CD experience (pipeline design, artifact promotion, testing gates) and infrastructure management (IaC concepts, environments, drift, operational change)
- Experience shipping user-facing or internal portals or significant full-stack contributions - enough to partner credibly with frontend specialists and own end-to-end delivery for self-serve workflows
- Excellent communication skills; ability to drive consensus across engineering, product, and operations
Preferred qualifications
- Experience building or operating a streaming/data platform used by many teams (not a single standalone pipeline)
- Familiarity with schema governance, serialization formats, and compatibility across producers and consumers
- Experience with AWS managed streaming or related services (e.g., MSK, Kinesis) where relevant to hybrid architectures
- Background in observability stacks tailored to Kafka and Flink
- Contributions to technical standards, guilds, or cross-org architecture forums
**The Ideal Candidate **
- You are a builder and systems thinker: you care about clean abstractions, thoughtful APIs, and operable designs
- You are comfortable with ambiguity and turn it into architecture, milestones, and measurable outcomes
- You balance velocity with sustainability - performance, security, and long-term maintainability matter as much as shipping
#LI-PW1
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.
When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.
Diversity & Belonging We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
**Are you an existing contractor or consultant with Autodesk? **
Please search for open jobs and apply internally (not on this external site).