Principal Data Scientist

Autodesk Autodesk · Enterprise · San Francisco, CA +1

Principal Data Scientist to build frameworks for evaluating AI systems within an agentic product platform. The role involves designing predictive models, establishing telemetry and observability standards, developing frameworks for analyzing non-deterministic user experiences, and integrating intelligence into agent orchestration. Requires experience with LLMs, RAG, vector databases, and guiding experimentation strategies.

What you'd actually do

  1. Design and implement predictive models to analyze and anticipate user behavior, intent, and outcomes across multi-agent workflows
  2. Define and establish data instrumentation, telemetry, and observability standards for agent-based systems and user interactions
  3. Develop frameworks and prototypes for analyzing and optimizing non-deterministic user experiences (including user interfaces) driven by AI agents
  4. Collaborate with product and engineering teams to integrate predictive intelligence into agent orchestration and decision-making systems
  5. Create analytical models and reporting frameworks to generate business intelligence, forecasting, and performance insights

Skills

Required

  • 8+ years in data science or applied ML
  • product analytics or user behavior modeling
  • predictive modeling (classification, survival analysis, sequence models, LTV/propensity)
  • designing instrumentation and event schemas for complex, stateful systems
  • define metrics and measurement frameworks for new product spaces
  • working on or adjacent to AI/ML-powered products
  • LLMs and related technologies (frameworks, embedding models, vector databases, RAG) in production
  • data modeling
  • system architectures
  • processing techniques
  • 2D/3D geometric data representations
  • translate theoretical concepts into practical solutions
  • work autonomously while effectively collaborating
  • technical writing and communication

What the JD emphasized

  • agentic product platform
  • evaluation techniques
  • predictive modeling
  • behavioral analytics
  • AI system design
  • non-deterministic user experiences
  • agent orchestration
  • LLMs
  • vector databases
  • Retrieval-Augmented Generation (RAG)

Other signals

  • agentic product platform
  • evaluation techniques
  • predictive modeling
  • behavioral analytics
  • AI system design