AI Quality Assurance Intern

Cresta Cresta · Vertical AI · Toronto, ON

The AI Quality Assurance Intern will evaluate the behavior, accuracy, and reliability of Cresta’s AI Agents and Agent Assist systems, focusing on AI behavior analysis, intent recognition evaluation, conversation quality assessment, model output auditing, and prompt testing. Responsibilities include executing test plans, validating AI outputs, identifying inconsistencies, and providing feedback for improvement.

What you'd actually do

  1. Execute structured test plans for AI Agent implementations
  2. Validate that intents, entities, and workflows trigger correctly
  3. Identify behavioral inconsistencies, misclassifications, or hallucinations
  4. Review and validate AI-generated suggestions, summaries, and classifications
  5. Provide actionable feedback to improve intent models and redaction/PII handling

Skills

Required

  • AI/ML, NLP, or chatbots interest
  • Product quality and user experience interest
  • Linguistics, conversation analysis, or human–AI interaction interest
  • Exploring a career in AI evaluation, AI ops, or product QA
  • Strong analytical thinking
  • Curiosity about how intelligent systems behave

Nice to have

  • No advanced ML experience required

What the JD emphasized

  • AI Agents
  • Agent Assist systems
  • AI behavior analysis
  • Intent recognition evaluation
  • Model output auditing
  • Light prompt testing and refinement
  • behavioral inconsistencies
  • misclassifications
  • hallucinations
  • AI-generated suggestions
  • redaction/PII handling
  • bias
  • redaction
  • transcription accuracy checks
  • emerging issues

Other signals

  • AI Quality Assurance
  • AI Agents
  • Agent Assist systems
  • AI behavior analysis
  • Intent recognition evaluation
  • Model output auditing