ML Research Scientist I/ii, Multimodal Data Extraction

Lila Sciences Lila Sciences · AI Frontier · One Charles Park, Cambridge, MA · Physical Sciences AI

Research Scientist role focused on developing foundation models for multimodal data extraction in scientific domains, involving fine-tuning LLMs and vision-language models, and building data pipelines.

What you'd actually do

  1. Research and develop AI systems that extract and structure knowledge from diverse scientific sources.
  2. Design and fine-tune large language, multi-modal and specialized models for factual, interpretable data extraction.
  3. Build scalable pipelines for unstructured and heterogeneous scientific data, integrating text, tables, and visuals.
  4. Collaborate with domain experts to align extracted data with real-world discovery workflows.
  5. Publish research that advances the state of the art in multimodal understanding and AI-driven knowledge extraction.

Skills

Required

  • PhD (or equivalent research experience) in Computer Science, Chemistry, Materials Science, or related field.
  • Expertise in machine learning, NLP, and vision–language modeling using PyTorch and Hugging Face Transformers.
  • Proven ability to train, fine-tune, and evaluate LLMs and multimodal models for scientific data extraction.
  • Strong understanding of data structures and representations used in the physical sciences.
  • Demonstrated research impact through publications, preprints, or open-source work (e.g., NeurIPS, ICLR, ICML, ACL, EMNLP, Scientific Journals).

Nice to have

  • Experience with multimodal fusion architectures and document-level understanding.
  • Knowledge of scientific document parsing (OCR, table extraction, figure-caption linking).
  • Familiarity with knowledge graph construction or reasoning systems for science.
  • Experience with noisy or heterogeneous real-world scientific data.

What the JD emphasized

  • PhD (or equivalent research experience)
  • machine learning
  • NLP
  • vision–language modeling
  • PyTorch
  • Hugging Face Transformers
  • LLMs
  • multimodal models
  • scientific data extraction
  • publications
  • preprints
  • open-source work

Other signals

  • foundation models
  • multimodal
  • knowledge extraction
  • scientific data