Software Engineer, Evaluation Platform / Infra

Thinking Machines Lab
San Francisco
Workplace: OnsiteFull timeUSD 300,000 - 475,000 annuallyFunction: Software EngineeringExperience: 2+ yearsEducation: bachelorsSkills: ["Problem-solving","Reliability","Maintainability","Cross-functional collaboration","Ownership"]

Design and build an end-to-end evaluation platform for frontier AI research: author evaluations, run and reproduce them at scale, and turn results into actionable insights for experiments and model releases. Work across Python/Rust backend systems, data pipelines, APIs, and user-facing applications, partnering with pre-training, post-training, and applied teams to improve evaluation quality, reliability, and reproducibility.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Thinking Machines Lab
Thinking Machines Lab
2 days ago

Software Engineer, Evaluation Platform / Infra

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Design and build an end-to-end evaluation platform for frontier AI research: author evaluations, run and reproduce them at scale, and turn results into actionable insights for experiments and model releases. Work across Python/Rust backend systems, data pipelines, APIs, and user-facing applications, partnering with pre-training, post-training, and applied teams to improve evaluation quality, reliability, and reproducibility.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Design, build, and maintain the evaluation platform for authoring, running, tracking, and analyzing model evaluations used in day-to-day work and model releases.
  • •Deliver end-to-end capabilities across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications.
  • •Create flexible abstractions for evaluation tasks, environments, graders, datasets, and model outputs without limiting fast-moving research.
  • •Ensure evaluation results are reproducible and trustworthy via versioning, provenance, observability, failure recovery, and robust quality controls.
  • •Partner with researchers to identify bottlenecks and convert bespoke workflows into self-serve systems across teams.

Pay and Benefits

Salary: USD 300,000 - 475,000 annually
Perks:Health InsuranceDentalVisionPaid LeavePaid ParentalRelocation

Key Requirements

  • •A bachelor’s degree, or equivalent practical experience, in computer science, engineering, machine learning, or a related field.
  • •Two years of post-grad work experience as a software engineer or ML engineer, exclusive of internships.
  • •Hands-on experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
  • •Strong software engineering fundamentals and experience building reliable, maintainable systems.
  • •Proficiency in at least one backend language (primarily Python and Rust) and experience across data-intensive infrastructure (databases, data pipelines, distributed systems).
Experience:2+ yearsAIMachine learningLarge language modelsMultimodal modelsDeveloper toolsDistributed systemsData pipelines
Education:Bachelor's
Skills:Problem-solvingReliabilityMaintainabilityCross-functional collaborationOwnership
Tech Stack:PythonRustReactTypeScriptAPIsDatabasesData pipelinesDistributed systemsObservabilityVersioningProvenanceFailure recovery

Eligibility

Work Authorization:Sponsorship available.

Company Brief

Thinking Machines Lab
Develops enterprise AI solutions, custom large language models, and ML platforms to help organizations deploy intelligent applications. Services include data engineering, model development, and AI consulting for scale and production readiness.
Industry: AI & Machine Learning
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Headquarters: Mumbai, India
Website