Research, Evals

Exa
San Francisco
Workplace: OnsiteFull timeUSD 150,000 - 300,000 annuallyFunction: Research & Scientific (R&D)Skills: ["Problem-solving","Communication","Collaboration","Analytical thinking"]

Design and build an evaluation stack for Exa’s ML models in a search context, creating comprehensive eval suites, golden datasets, and scalable pipelines to measure performance, drift, and quality across billions of documents. Collaborate with ML researchers, data engineers, and infra teams to shape feedback loops that improve search models in an AI/LLM world.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Exa
Exa
10 months ago

Research, Evals

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Design and build an evaluation stack for Exa’s ML models in a search context, creating comprehensive eval suites, golden datasets, and scalable pipelines to measure performance, drift, and quality across billions of documents. Collaborate with ML researchers, data engineers, and infra teams to shape feedback loops that improve search models in an AI/LLM world.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Research & Scientific (R&D)

Key Responsibilities

  • •Design and build the eval stack for ML models in a search context
  • •Create and manage eval datasets, benchmarks, and test suites
  • •Develop scalable, reliable eval pipelines to monitor regressions, drift, and quality signals
  • •Collaborate with ML researchers, data engineers, infra engineers, and product to shape feedback loops
  • •Define what constitutes “good search” through measured evaluations and experiments

Pay and Benefits

Salary: USD 150,000 - 300,000 annually
Equity and Bonus:Equity
Perks:Equity

Key Requirements

  • •Hands-on ML experience (training, finetuning, or evaluating models) (bonus if embeddings/LLMs)
  • •Strong engineering fundamentals with ability to build reliable systems (Python, Rust, distributed pipelines, GPU/cluster jobs)
  • •Enjoy building eval sets, inspecting edge cases, and designing measurement strategies
  • •Ability to design and implement evaluation frameworks for search
  • •Experience designing scalable eval pipelines tracking regressions, drift, and quality signals
Experience:AIMLSearch
Skills:Problem-solvingCommunicationCollaborationAnalytical thinking
Languages:English
Tech Stack:PythonRustDistributed pipelinesGPUCluster

Company Brief

Exa
Builds an embeddings-based neural search engine and web search API for AI applications, offering crawling, retrieval, and deep research tools to serve developers and AI agents with up-to-date web knowledge.
Industry: API Platforms
Company Size: Small (11 to 50 employees)
Growth: Scaleup
Valuation: USD 500M to 1B
Funding: Series B
Headquarters: San Francisco, United States
Founded: 2021
WebsiteLinkedIn