Human-robot Interaction Applied Scientist , Fauna

Amazon Amazon · Big Tech · NY +1 · Applied Science

This role focuses on developing cutting-edge human-robot interaction (HRI) systems, making robots feel alive, personal, and fun. The scientist will work on verbal and non-verbal conversational systems, social dynamics, memory, and long-term relationship formation between robots and people. Key responsibilities include developing interactive systems using LLMs, multimodal inputs/outputs, and RLHF, designing conversational systems that handle context and social cues, integrating perceptual sensor streams, and developing memory/personalization systems for robots. The role also involves leading technical projects, mentoring, and bridging research with engineering.

What you'd actually do

  1. Develop interactive systems that leverage large language models, multimodal inputs and outputs, reinforcement learning from human feedback, or other advanced techniques to achieve fluid, engaging, and socially appropriate robot behavior
  2. Design and implement intelligent conversational systems that handle turn-taking, grounding, interruption, and incorporates context drawn from a robot's physical environment and shared history with a user
  3. Integrate perceptual sensor streams including gaze, facial expression, gesture, posture, and more to understand social context and produce coherent, lifelike interactions.
  4. Develop memory and personalization systems that allow robots to form lasting relationships with individual users, learn their environments, and adapt their behavior over weeks and months
  5. Stay updated on advancements in HRI, NLP, multimodal AI, and cognitive and social science to apply cutting-edge techniques to robot interaction challenges

Skills

Required

  • 3+ years of building machine learning models for business application experience
  • PhD, or Master's degree and 6+ years of applied research experience
  • Experience programming in Java, C++, Python or related language
  • Experience with neural deep learning methods and machine learning
  • Strong publication record at major HRI/ML/AI/HCI conferences
  • Experience designing, running, and analyzing user studies with real human participants
  • Experience programming in Python or related language
  • Demonstrated track record of leading technical projects
  • Experience mentoring junior scientists/engineers
  • Strong understanding of one or more of: dialogue systems, multimodal interaction, social signal processing, affective computing, or cognitive modeling
  • Experience applying machine learning (LLMs, multimodal models, reinforcement learning, imitation learning, or related techniques) to interactive or socially situated systems

Nice to have

  • Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc.
  • Experience with large scale distributed systems such as Hadoop, Spark etc.
  • History of impactful first-author publications at major conferences
  • Experience deploying interaction systems on physical robots or other embodied platforms
  • Experience with LLM-based agents, tool use, planning, or grounding language in perception and action
  • Experience with long-term memory, personalization, or user modeling for conversational agents
  • History of technical leadership and cross-functional collaboration
  • Experience bridging research with practical engineering implementation in robotics or consumer AI products
  • Extensive programming skills in Python and PyTorch/JAX
  • Experience integrating perception systems (e.g., vision, audio, and touch sensors) into interaction pipelines
  • Familiarity with safety, privacy, and ethical considerations for socially interactive AI systems

What the JD emphasized

  • Strong publication record at major HRI/ML/AI/HCI conferences
  • Demonstrated track record of leading technical projects
  • Experience applying machine learning (LLMs, multimodal models, reinforcement learning, imitation learning, or related techniques) to interactive or socially situated systems
  • Experience deploying interaction systems on physical robots or other embodied platforms
  • Experience with LLM-based agents, tool use, planning, or grounding language in perception and action
  • Experience with long-term memory, personalization, or user modeling for conversational agents

Other signals

  • human-robot interaction
  • conversational systems
  • multimodal AI
  • reinforcement learning from human feedback
  • large language models