Senior Data Engineer

Socure Socure · Vertical AI · United States · Remote · Tech

Senior Data Engineer to design and build scalable data platforms and pipelines for identity verification products and analytics, focusing on automated data ingestion, ML feature engineering, and analytics. The role involves end-to-end delivery of data initiatives, evolving the data platform, automating data operations, and optimizing performance. It requires strong programming, distributed data processing (Spark), and AWS experience, with a focus on building and maintaining production-grade data systems.

What you'd actually do

  1. Design and build batch and streaming data pipelines to support automated data ingestion, ML feature engineering and analytics across multiple product domains.
  2. Own end-to-end delivery of complex, ambiguous data initiatives, including architecture, implementation, testing, deployment, monitoring, and documentation.
  3. Develop and evolve the data platform to support large-scale data processing using modern cloud-native technologies.
  4. Automate data operations (validation, quality checks, alerting, backfills, and recovery workflows) to reduce manual effort and improve consistency.
  5. Optimize cost, performance, and reliability of data workloads.

Skills

Required

  • 5+ years of hands-on data engineering experience
  • building and maintaining production-grade data platforms and pipelines
  • Strong programming skills in general-purpose language (such as Python or Scala)
  • SQL for data analytics
  • Deep experience with distributed data processing frameworks, such as Apache Spark
  • performance tuning and optimization
  • Proven experience building data solutions using services on AWS (EMR, Lambda, s3, etc)
  • Strong understanding of data modeling and data warehousing concepts
  • partitioning, schema design for large-scale datasets
  • Experience operating and supporting production pipelines
  • monitoring, alerting, incident response
  • improving reliability over time
  • Solid foundation in software engineering practices
  • version control, CI/CD, testing strategies, and code review
  • Strong communication and collaboration skills

Nice to have

  • Experience with streaming or near-real-time data processing (Kafka, Kinesis, etc)
  • Hands-on experience with data orchestration tools (Airflow, Step Functions, etc)
  • Familiarity with modern data platform patterns such as Data Lakehouse, Data Mesh, and large-scale data sharing across teams
  • Experience with prompt engineering using modern GenAI, Large Language Models (LLM)
  • Experience mentoring other engineers and contributing to engineering-wide standards, best practices

What the JD emphasized

  • ML feature engineering
  • large-scale data processing
  • production-grade data platforms and pipelines
  • operating and supporting production pipelines

Other signals

  • ML feature engineering
  • data platforms and pipelines
  • large-scale data processing
  • GenAI-assisted ingestion