The AI Senior Specialist is responsible for overseeing the training, validation, and continuous improvement of Agentic Capabilities—AI-powered agents designed to autonomously process a variety of ticket types within workflow management systems.
This role owns AI agent quality: ensuring reliable, high-quality outcomes; rapidly reviewing and resolving exception (“fallout”) tickets; applying corrections and re-ingesting updates; improving training data and fine-tuning artifacts; updating the agent knowledge base; and rerunning tickets to validate fixes—continuously strengthening agentic capabilities.
Key Responsibilities
Agentic Capability Management
- Monitor the performance of Agentic Capabilities as they autonomously process various ticket types.
- Ensure seamless integration of AI agents into new or existing workflows, optimizing for efficiency and accuracy.
Fallout Review & Correction
- Review fallout tickets (cases where AI agents cannot resolve issues) within a workflow management tool.
- Diagnose root causes, make necessary corrections, and re-ingest updated information to the AI system.
- Ensure all fallout tickets are actioned within a 48-hour window; unresolved tickets revert to the human-worked queue.
Training & Continuous Improvement
- Analyze fallout patterns to identify knowledge gaps, process inefficiencies, or opportunities for AI improvement.
- Develop and implement training protocols to enhance Agentic Capabilities, leveraging prompt engineering, model validation, and knowledge base updates.
- Collaborate with cross-functional teams (product, engineering, support) to align AI behaviors with business needs and compliance requirements.
Knowledge Base & Documentation
- Maintain and update the agent knowledge base, ensuring accurate, current, and comprehensive content for AI agents.
- Document training methodologies, ticket resolutions, and process improvements for knowledge sharing and auditing.
Quality Assurance & Compliance
- Validate AI performance through systematic review, testing, and user/stakeholder feedback.
- Ensure all processes comply with regulatory standards, ethical guidelines, and company policies.
Reporting & Communication
- Track and report on key metrics: ticket resolution rates, fallout frequency, review turnaround times, and AI improvement outcomes.
- Continuous Improvement practices
- Communicate insights, best practices, and recommendations to stakeholders and leadership.
Skills & Qualifications
Required Skills & Qualifications
- Understanding of the business function and M&Ps; align AI behavior with policy and process; plus knowledge of supported workflows and tools with experience operating the systems involved (e.g., ticketing/workflow platforms).
- Strong analytical and problem-solving skills with meticulous attention to detail; proven root cause analysis (RCA) capability.
- General AI literacy and understanding of agentic systems; basic prompt engineering (iteration, testing, versioning).
- Ability to manage fallout within SLAs, triage tickets, and drive rapid resolution; strong prioritization in fast-paced environments.
- Advanced scripting/automation and experience writing/maintaining Markdown-based runbooks and KB articles.
- Prior experience with AI in production settings and A/B testing platforms.
- Knowledge base authoring and maintenance; clear documentation of training methods, resolutions, and changes for auditability.
- Compliance/data privacy/ethical guidelines awareness; maintain auditable processes and change logs.
- Effective communication: synthesize findings, report metrics, and present recommendations to stakeholders.
** **
Preferred Skills
- Advanced observability (distributed tracing, SLO/SLA design) and incident response practices.
- Experiment tracking and ML operations tooling, feature flags, canary/rollback strategies.
- Familiarity with fine-tuning pipelines, retrieval/RAG, vector databases, and content ingestion pipelines.
- SQL/BI tools for advanced analytics and dashboarding; ability to build executive-ready reports.
- Version control (Git) for prompts, KB content, and evaluation artifacts; change management discipline.
- Workflow orchestration for scheduled re-ingestion, evaluations, and reporting.
- Experience in training, quality assurance, documentation, or knowledge management, including taxonomy/ontology design.
Weekly Hours:
40
Time Type:
Regular
Location:
Bengaluru, India
AT&T and its subsidiaries are committed to equal employment opportunity. All hiring, promotion, and other employment decisions remain merit-based and free from discrimination on the basis of race, color, religion, religious creed, national origin, ancestry, age, sex, sexual orientation, gender, gender identity, gender expression, physical disability, mental disability, pregnancy, medical condition, genetic information, marital status, citizenship status, military status, veteran status, or any other characteristic protected by federal, state, or local laws. In addition, AT&T will provide reasonable accommodations to qualified individuals with disabilities. AT&T is a fair chance employer and does not initiate a background check until an offer is made. Click here to learn more or request an application accommodation here.