Data Engineer

Caterpillar Caterpillar · Industrial · Chicago, IL

Data Engineer role focused on designing, building, and maintaining scalable data pipelines, microservices, and cloud-based data systems using Python and AWS services. The role supports the development of AI models and autonomous systems within the industrial domain.

What you'd actually do

  1. Design, build, and optimize data pipelines and microservices using Python for real-time and batch processing
  2. Develop cloud-based data ingestion solutions using AWS services (Kinesis, S3, DynamoDB, EventBridge, etc.)
  3. Build and maintain data integration workflows for source data pipelines aligned to CI Autonomy
  4. Translate business requirements into reliable data workflows, mappings, and system designs
  5. Implement automated testing and validation to ensure data integrity across distributed systems

Skills

Required

  • Python
  • AWS services
  • data pipelines
  • microservices
  • real-time processing
  • batch processing
  • data ingestion
  • data integration
  • automated testing
  • data integrity
  • distributed systems
  • agile development

Nice to have

  • Java
  • CI/CD tools
  • API development
  • SQL
  • relational databases
  • noSQL databases
  • streaming platforms
  • monitoring
  • automation