Software Engineer, Agent Evaluation and Quality

Cursor
San Francisco
Workplace: OnsiteFull timeFunction: Software EngineeringSkills: ["Collaboration","Communication","Problem-solving","Teamwork","Data-driven"]

Software Engineer on Cursor's Agent Quality team develops measurement, evaluation, and feedback-loop infrastructure to improve the core agent. You’ll instrument critical metrics, build pipelines to analyze agent behavior at scale, collaborate with research, product, and infrastructure, and turn insights into improvements affecting multiple Cursor products and decisions around model quality and cost.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Cursor
Cursor
5 months ago

Software Engineer, Agent Evaluation and Quality

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Software Engineer on Cursor's Agent Quality team develops measurement, evaluation, and feedback-loop infrastructure to improve the core agent. You’ll instrument critical metrics, build pipelines to analyze agent behavior at scale, collaborate with research, product, and infrastructure, and turn insights into improvements affecting multiple Cursor products and decisions around model quality and cost.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Design and build an AI evaluation system including curated datasets, offline replay, scorers/judges, regression alerts, and dashboards.
  • •Design feedback loops from real usage by collecting, cleaning, and interpreting user signals to inform model and harness changes.
  • •Develop analysis tooling and workflows for debugging agent behavior, including failure-mode deep dives and thematic clustering.
  • •Improve reliability and guardrails by making quality measurable and operational: define good/bad/degraded sessions, alerts, and triage primitives.
  • •Collaborate with research, product, and infrastructure teams to turn insights into improvements across Cursor products.

Key Requirements

  • •Built and operated evaluation or measurement systems (e.g., AI evals, experimentation, ranking/relevance, or search quality) and translated ambiguous quality questions into concrete metrics and pipelines.
  • •Strong data acumen and ability to collaborate with data scientists and researchers.
  • •Taste and opinions on model and agent behaviors; able to stay up-to-date on emerging research and industry trends.
  • •Strong software engineering fundamentals and experience shipping production systems.
  • •Experience turning insights into concrete product or system improvements.
Experience:AIMachine learningSoftware engineering
Skills:CollaborationCommunicationProblem-solvingTeamworkData-driven
Languages:English

Company Brief

Cursor
Builds Cursor, an AI-powered development platform and code editor that helps developers write, review, and maintain code using large language models and developer-focused automation tools for enterprise engineering teams.
Industry: Developer Tools
Company Size: Large (251 to 1,000 employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: San Francisco, United States
Founded: 2022
Glassdoor
Glassdoor: 3.0
WebsiteLinkedInGlassdoor