Software Engineer - Evals

X AI
Palo Alto
Workplace: OnsiteFull timeUSD 175,000 - 275,000Function: Software EngineeringExperience: 3+ yearsSkills: ["Communication","Attention to detail","Urgency","Product taste","First-principles reasoning"]

Own and elevate a mission-critical research platform by building the evaluation tooling that measures model capabilities and behaviors. Collaborate with the evals team to create instruments for cleaning datasets, providing feedback for experiments, tracking progress during training, and diagnosing agent failures. Design, build, and ship robust systems that engineers rely on daily, with strong attention to detail and first-principles reasoning.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
X AI
X AI
1 month ago

Software Engineer - Evals

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 13 hours agoStatus: Live

Job Summary

Own and elevate a mission-critical research platform by building the evaluation tooling that measures model capabilities and behaviors. Collaborate with the evals team to create instruments for cleaning datasets, providing feedback for experiments, tracking progress during training, and diagnosing agent failures. Design, build, and ship robust systems that engineers rely on daily, with strong attention to detail and first-principles reasoning.
Location: Palo Alto
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Own and elevate the mission-critical research platform that accelerates xAI’s frontier work.
  • •Build evaluation instruments to measure model capabilities and behaviors.
  • •Use evals to clean datasets, provide feedback for experiments, and track training growth.
  • •Diagnose why agents fail and identify improvements for short- and long-term outcomes.
  • •Design and manage evaluation execution infrastructure, datasets, and grading schemes.

Pay and Benefits

Salary: USD 175,000 - 275,000
Equity and Bonus:Equity
Perks:Health InsuranceVisionDental401k

Key Requirements

  • •Software engineering proficiency with 3+ years of experience actively writing code.
  • •Ship high-quality projects with a strong sense of urgency.
  • •Exceptional product taste: translate quality/helpfulness/truthfulness/safety into measurable criteria.
  • •Strong attention to detail.
  • •Familiarity with AI/ML workflows, annotation tools, or evals is a bonus.
Experience:3+ yearsAI/ML workflowsEvalsEarly-stage startups
Skills:CommunicationAttention to detailUrgencyProduct tasteFirst-principles reasoning
Languages:English

Company Brief

X AI
Develops advanced artificial intelligence models and research aimed at building safe, general AI and understanding the fundamental nature of the universe. Focuses on large-scale AI systems, research publications, and building foundational AI capabilities.
Industry: AI & Machine Learning
Company Size: Medium (51 to 250 employees)
Growth: Early Stage Startup
Headquarters: San Francisco, United States
Founded: 2023
Website