Senior Data Platform Engineer

Samsara Samsara · Enterprise · Remote - WA · Business Systems

Senior Data Platform Engineer responsible for administering and monitoring Databricks workspaces and underlying AWS infrastructure. The role involves developing platform admin tools, assisting teams with Databricks features, and collaborating with data engineers, analysts, and data scientists to operationalize and optimize workflows and data pipelines. This includes overseeing the data-engineering dev ecosystem, implementing IaaC and CI/CD automation, ensuring security and compliance, and incorporating DevOps best practices. The role also involves cross-functional collaboration to understand data needs, design scalable solutions, and provide mentorship.

What you'd actually do

  1. Administer and monitor Databricks workspaces and underlying AWS Infra. Develop platform admin tools and capabilities to effectively administer, monitor, track and troubleshoot Databricks resources: clusters, catalogs, users, groups, databases, storage and security.
  2. Act as the primary Databricks SME, assisting and guiding teams on platform features and best practices.
  3. Collaborate with data engineers, analysts, and data scientists to operationalize and optimize workflows and data pipelines.
  4. Oversee data-engineering dev ecosystem, including dev-tooling, CI/CD pipelines, and monitoring frameworks.
  5. Own implementation and management of IaaC (Terraform, CloudFormation) and CI/CD automation for Databricks and AWS resources.

Skills

Required

  • Databricks administration
  • AWS services
  • Cloud Infrastructure provisioning
  • SQL
  • Python
  • Data pipelines
  • ETL
  • Change-data capture
  • Incremental loading
  • Batch loading
  • Problem-solving
  • Communication
  • Stakeholder management

Nice to have

  • Databricks and AWS Infra for 100s of users
  • Technical lead experience
  • DevOps tools and practices (GitHub/GitLab, CI/CD, Terraform, Datadog, CloudWatch)
  • Data governance

What the JD emphasized

  • 10+ years experience in a Software Engineering, Platform Engineering, Data Engineering, DevOps/DataOps or similar technical role, including at least 2+ years experience in data infrastructure or platform focused role.
  • Solid knowledge of Databricks features and administration, including Unity Catalog, cluster management, security, conducting troubleshooting and root-cause analysis, and performance optimization.
  • Experience and familiarity with AWS services and Cloud Infrastructure provisioning, management, monitoring, and security (e.g., S3, IAM, RDS, Lambda, API-Gateway, VPC, EC2, ECS/EKS).