Principal Digital Architect

Caterpillar Caterpillar · Industrial · Peoria, IL

Design and implement large-scale data lakes and enterprise data warehousing (EDW) projects using on-premise and cloud-native solutions. Drive development and execution of automated data processing ETL pipelines using various tools and technologies. Lead data architecture development for enterprise-scale systems in public cloud environments (AWS, Azure, hybrid). Implement data lakes, big data transformations, cloud data processing, SQL, and NoSQL data modeling. Manage data modeling requirements for structured and non-structured data using various database management technologies.

What you'd actually do

  1. Design largescale data lakes and enterprise data warehousing (EDW) projects using on-premise and cloud native data warehousing solutions such as Redshift & Snowflake.
  2. Drive development and execution of designing, implementing, and supporting end-to-end automated data processing ETL pipelines using a variety of tools and technologies such as Hadoop Technologies, AWS EMR, and AWS Glue.
  3. Lead data architecture development for enterprise scale systems in public cloud in AWS, Azure and hybrid eco-systems.
  4. Implement data lakes, big data transformations, cloud data processing, SQL, and NoSQL data modeling.
  5. Manage data modeling requirements for structured and non-structured data using various database management technologies such as AWS RDS (mySQL, PostagreSQL, Aurora), AWS DynamoDB, and NoSQL database.

Skills

Required

  • Bachelor’s degree or foreign equivalent in Computer Science, Electrical Engineering, Information Technology or a related field
  • 10 years of experience in software development or related occupation
  • Designing, developing, deploying and maintaining software at scale
  • Developing software applications using relational and NoSQL databases
  • Application architectural patterns, such as MVC, Microservices, Event-driven, etc.
  • Deploying software using CI/CD tools such as Jenkins, GoCD, Azure Devops etc.
  • Deploying and maintaining software using public clouds such as AWS or Azure
  • Working within an Agile framework such as Scrum
  • Batch or stream processing systems such as Apache Spark, Flink, Akka, Storm
  • Message brokers such as Kafka, Rabbitmq, AWS SQS, AWS SNS, Apache Active MQ, Kinesis
  • Writing API proxies on platforms such as Apigee Edge, AWS API Gateway or Azure API Gateway
  • Datastores such as MongoDB, Cassandra, Redis, Elasticsearch, MySQL, Oracle
  • Debugging and maintaining software in Linux or Unix platforms

What the JD emphasized

  • Designing, developing, deploying and maintaining software at scale
  • Developing software applications using relational and NoSQL databases
  • Application architectural patterns, such as MVC, Microservices, Event-driven, etc.
  • Deploying software using CI/CD tools such as Jenkins, GoCD, Azure Devops etc.
  • Deploying and maintaining software using public clouds such as AWS or Azure
  • Working within an Agile framework such as Scrum
  • Batch or stream processing systems such as Apache Spark, Flink, Akka, Storm
  • Message brokers such as Kafka, Rabbitmq, AWS SQS, AWS SNS, Apache Active MQ, Kinesis
  • Writing API proxies on platforms such as Apigee Edge, AWS API Gateway or Azure API Gateway
  • Datastores such as MongoDB, Cassandra, Redis, Elasticsearch, MySQL, Oracle
  • Debugging and maintaining software in Linux or Unix platforms