Principal Data Engineer -general Application

Autodesk Autodesk · Enterprise · Kraków, Poland +1

Principal Data Engineer role focused on improving data and data pipeline architecture, optimizing data flow and collection, and supporting data projects for software developers, data analysts, and data scientists. The role involves building data pipeline infrastructure using SQL and AWS 'big data' technologies, creating analytics tools, and working with data scientists and analysts. While the preferred qualifications mention designing prompts and staying updated on NLP/AI advancements, the core responsibilities are data engineering.

What you'd actually do

  1. Create data pipeline architecture
  2. Assemble complex data sets that meet functional / non-functional requirements
  3. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
  4. Build the infrastructure required for extraction, transformation, and loading of data from different data sources using SQL and AWS 'big data' technologies
  5. Build analytics tools that use the data pipeline to provide applicable insights into employee experience, operational efficiency and other main performance metrics

Skills

Required

  • 5 + years of experience in Data Engineer role
  • Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
  • Strong command of English
  • AWS (Batch, CloudFormation, IAM, Lambda, ECS, StepFunctions)
  • Terraform
  • Python
  • SQL
  • Relational databases (e.g., PostgreSQL)
  • API (RESTful)
  • BI Tools (QuickSight)

Nice to have

  • Familiarity with Agile and SCRUM methodologies
  • Experience working with PowerBI to develop dashboards
  • Analytical skills related to working with unstructured datasets
  • A successful history of processing value from large, disconnected datasets
  • Experience working with agile, globally distributed teams
  • Design and develop prompts for various applications, including text generation, translation, question answering, and creative writing
  • Collaborate with product teams, data scientists, and engineers to understand user needs and translate them into effective prompts
  • Analyse and iterate on performance metrics and user feedback to ensure high-quality outputs
  • Conduct experiments and research to test and optimize existing workflows
  • Stay up to date on the latest advancements in natural language processing (NLP) and AI, and apply those insights to your work
  • Document and communicate your work clearly and concisely to technical and non-technical audiences