Senior Data/ai Engineering

AT&T AT&T · Telecom · Plano, TX

Develops high-performance distributed computing systems for GenAI using Big Data technologies, Spark, Databricks, and Azure Cloud. Designs and delivers GenAI technologies at scale with real-time results using OpenAI and LLMs for chat experiences. Builds RAG models, truth set generators, and integrates LangChain loaders and Vector DB for optimized data retrieval and configuration. Develops Python Fast API rest services and WebSocket ends for GenAI integration and voice streaming.

What you'd actually do

  1. Responsible for the development of high performance, distributed computing systems using Big Data technologies for GenAI development.
  2. Build scalable multi-threaded Spark clusters using Databricks interfacing with NoSQL, including data mining using various distributed technologies on the Azure Cloud platform.
  3. Design and deliver GenAI technologies at scale with real time results using OpenAI and large language models to service chat experience for business partners across the company.
  4. Build auto LLM to automate discovery of the best configuration to be identified in leaderboard results using RAG models.
  5. Build truth set auto generator to optimize leaderboard ranking based on standardized questions and answers.

Skills

Required

  • Spark
  • Databricks
  • NoSQL
  • Azure Cloud
  • OpenAI
  • large language models
  • RAG models
  • Lang Chain
  • Vector DB
  • Python Fast API
  • WebSocket
  • Hadoop
  • text mining

Nice to have

  • GenAI
  • chat experience
  • data mining
  • leaderboard results
  • truth set auto generator
  • model packages
  • data retrieval
  • Snowflake
  • ServiceNow
  • voice streaming

What the JD emphasized

  • GenAI development
  • large language models
  • RAG models
  • Vector DB
  • Python Fast API rest services
  • voice streaming GenAI experience

Other signals

  • GenAI development
  • large language models
  • RAG models
  • Vector DB