Senior Software Engineer, AI Productivity

Instacart Instacart · Consumer · Canada · Remote · Software Engineering

Instacart's AI Productivity team builds AI-powered platforms and tools to enhance engineer and operator productivity. This role involves designing, building, and operating production systems that integrate LLMs into workflows, focusing on code assistants, knowledge search, and chat. The engineer will own projects end-to-end, ensuring reliability, security, and compliance, and will partner with various teams to ship scalable AI solutions.

What you'd actually do

  1. Design, build, and operate AI-powered services and internal applications—such as code and test assistants, knowledge search, and intelligent chat workflows—that improve developer and operator productivity at scale.
  2. Implement robust back-end services in languages such as Python, Go, or Java; integrate foundation models via APIs and open-source frameworks; and apply techniques like retrieval-augmented generation and vector search where appropriate.
  3. Own projects end-to-end, including requirements, technical design, implementation, testing, deployment, observability, on-call participation, and iterative improvements focused on reliability, latency, and cost efficiency.
  4. Partner with Developer Platform, ML Platform, Security, and Legal to ensure safe and compliant AI usage, including data governance, guardrails, content/prompt safety, and privacy-by-design.
  5. Define and track success with clear metrics (e.g., adoption, task completion time, DORA/SPACE signals, satisfaction), and run experiments and A/B tests to validate impact and guide iteration.

Skills

Required

  • 5+ years of professional software engineering experience building and operating production services.
  • 3+ years of hands-on experience in at least one of: Python, Go, or Java.
  • 2+ years of experience integrating ML or LLM capabilities into applications (e.g., via OpenAI, Anthropic, or open-source frameworks such as LangChain or LlamaIndex) and shipping them to production.
  • Practical experience with cloud platforms (AWS or GCP), containers (Docker), and orchestration (Kubernetes).
  • Proficiency with SQL and data modeling; experience building or maintaining data pipelines (e.g., Airflow, Dagster, or similar).
  • Experience owning services end-to-end, including CI/CD, automated testing, observability (logging, metrics, tracing), and participation in an on-call rotation.
  • Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Nice to have

  • Experience adapting or fine-tuning foundation models and implementing retrieval-augmented generation with vector databases (e.g., Pinecone, Weaviate, FAISS, or Elasticsearch with vector search).
  • Track record of building internal developer tools or platforms adopted by 100+ engineers.
  • Experience measuring developer productivity (e.g., DORA/SPACE), instrumenting product analytics, and running A/B or multivariate experiments.
  • Knowledge of AI safety, privacy, and compliance practices (e.g., PII redaction, guardrails, model/content moderation, data residency considerations).
  • Experience with event-driven systems and microservices (e.g., Kafka, Pub/Sub) and designing resilient distributed systems.
  • Ability to build lightweight front-ends for internal tools (e.g., React, TypeScript) to accelerate adoption and feedback loops.
  • Demonstrated mentorship and technical leadership within cross-functional teams.

What the JD emphasized

  • building and operating production services
  • integrating ML or LLM capabilities into applications
  • shipping them to production
  • owning services end-to-end
  • building internal developer tools or platforms adopted by 100+ engineers

Other signals

  • AI-powered platforms and tools
  • large language models and intelligent automation
  • internal developer tools or platforms