Research Engineer - Evals

Firecrawl
San Francisco, Toronto
Workplace: OnsiteFull timeUSD 250,000 - 290,000 annuallyFunction: Research & Scientific (R&D)Experience: 3+ yearsSkills: ["Ownership","Problem-solving","Fast iteration","Quality mindset","Communication"]

Design and build evaluation systems that rigorously measure Firecrawl’s ability to convert messy web URLs into clean, structured, LLM-ready data. Own metrics and quality definitions, create the pipelines and harnesses that scale measurement across vast edge cases, and generate trustworthy datasets. Close the loop by feeding output quality back into model and product decisions, turning “did that work?” into actionable answers for the team.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Firecrawl
Firecrawl
1 month ago

Research Engineer - Evals

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 6 hours agoStatus: Live

Job Summary

Design and build evaluation systems that rigorously measure Firecrawl’s ability to convert messy web URLs into clean, structured, LLM-ready data. Own metrics and quality definitions, create the pipelines and harnesses that scale measurement across vast edge cases, and generate trustworthy datasets. Close the loop by feeding output quality back into model and product decisions, turning “did that work?” into actionable answers for the team.
Location: San Francisco, Toronto
Workplace: Onsite
Employment Type: Full time
Job Function: Research & Scientific (R&D)
Seniority: Mid level

Key Responsibilities

  • •Design metrics that define “good output” across millions of sites, formats, and edge cases.
  • •Build pipelines and harnesses to measure quality rigorously and at scale.
  • •Generate and curate datasets to make evaluation trustworthy.
  • •Own the feedback loop from output quality back to model and product decisions.
  • •Translate “did that work?” into answers the whole team can act on.

Pay and Benefits

Salary: USD 250,000 - 290,000 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVision401kPaid LeaveParental LeaveWellness StipendLearning BudgetLife Insurance

Key Requirements

  • •4+ years of experience in ML, research engineering, or data-heavy backend with real evaluation work.
  • •Have the engineering depth to build evaluation systems, not just run existing benchmarks.
  • •Care deeply about defining what “good” output means and measuring it rigorously.
  • •Be comfortable owning ambiguous problems where the metric itself must be invented.
  • •Move fast and iterate by shipping, measuring, and improving the evaluation loop.
Experience:3+ yearsMLResearch engineering
Skills:OwnershipProblem-solvingFast iterationQuality mindsetCommunication

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Firecrawl
example.com is an IANA-reserved domain maintained for use in documentation, examples, and testing. It is not an operating commercial company and exists solely to provide a stable, non-resolvable hostname for illustrative purposes.
Industry: Other
Website