Software Development Engineer, Agi - Web Information Retrieval

Amazon Amazon · Big Tech · Boston, MA · Software Development

Software Development Engineer to develop a multi-modal, multi-lingual web-scale information retrieval system that combines LLMs with retrieval systems for question-answering, knowledge synthesis, and information grounding. The role involves building data pipelines, optimizing core algorithms and ranking models, designing data structures and indexing techniques, optimizing performance and scalability, integrating ML models, and developing control plane systems for the IR system infrastructure. Experience with Rust, large-scale distributed systems, and ML integration is required.

What you'd actually do

  1. Build high-throughput, cost-effective data pipelines to support feature extraction and indexing for our web-scale Information Retrieval (IR) system
  2. Develop and optimize the core algorithms in Rust and ranking models that power the search engine's ability to retrieve and rank relevant results for user queries.
  3. Design and implement efficient data structures and indexing techniques to store and retrieve massive amounts of web data and content using Rust programming language.
  4. Optimize the performance, scalability, and reliability of the search engine's core components, including query parsing, retrieval, ranking, and result rendering.
  5. Collaborate with machine learning teams to integrate and deploy advanced machine learning models for query understanding, ranking, and personalization.

Skills

Required

  • 3+ years of professional software development experience
  • 2+ years of design or architecture experience
  • 1+ years of experience building large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl
  • 1+ years of Object Oriented Design experience
  • Experience programming with at least one software programming language
  • Rust

Nice to have

  • 3+ years of full software development life cycle experience
  • Bachelor's degree in computer science or equivalent

What the JD emphasized

  • web-scale information retrieval system
  • combining the power of large language models with retrieval systems
  • multi-modal, multi-lingual
  • Rust programming language
  • large-scale distributed systems

Other signals

  • web-scale information retrieval system
  • combining the power of large language models with retrieval systems
  • question-answering, knowledge synthesis, and information grounding
  • multi-modal, multi-lingual
  • feature extraction and indexing
  • ranking models
  • integrate and deploy advanced machine learning models