Staff Software Engineer, Agent Quality

Databricks
New York, San Francisco
Workplace: OnsiteFull timeUSD 200,000 - 265,000 annuallyFunction: Software EngineeringExperience: 6+ yearsSkills: ["Technical ownership","Technical quality","Systems design","Collaboration","Influence"]

Build and scale foundational evaluation infrastructure for Databricks AI agents, enabling rigorous benchmarking, regression detection, and quality measurement across research and product teams. Create a feedback flywheel that connects production signals and evaluation results back into agent improvement across the full lifecycle. Drive the long-term technical direction for agent quality infrastructure, partnering with researchers and engineers to turn ideas into robust internal platforms and tooling.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
15 hours ago

Staff Software Engineer, Agent Quality

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 13 hours agoStatus: Live

Job Summary

Build and scale foundational evaluation infrastructure for Databricks AI agents, enabling rigorous benchmarking, regression detection, and quality measurement across research and product teams. Create a feedback flywheel that connects production signals and evaluation results back into agent improvement across the full lifecycle. Drive the long-term technical direction for agent quality infrastructure, partnering with researchers and engineers to turn ideas into robust internal platforms and tooling.
Location: New York, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Stand up foundational evaluation infrastructure for Genie Agents, including benchmarking, regression detection, and quality measurement.
  • •Build the flywheel connecting evaluation results to agent improvement from production signals through training and iterative development.
  • •Define the technical direction for agent quality infrastructure and how Databricks measures and improves first-party agents.
  • •Scale infrastructure, tooling, and developer workflows for rigorous agent evaluation across teams.
  • •Partner with researchers and engineers to turn evaluation ideas into robust internal platforms.

Pay and Benefits

Salary: USD 200,000 - 265,000 annually
Equity and Bonus:Equity
Perks:Annual BonusEquity

Key Requirements

  • •6+ years of industry experience building software systems.
  • •Strong Python skills building production or research infrastructure.
  • •Experience building or operating distributed systems, data pipelines, or large-scale infrastructure with reliability and operational maturity.
  • •Ability to design pragmatic, rigorous systems that generate trustworthy, reproducible signals for complex applications.
  • •Ability to partner across ambiguous research and product boundaries to build robust internal platforms.
Experience:6+ yearsLLM and agent evaluationDistributed systemsData pipelines
Skills:Technical ownershipTechnical qualitySystems designCollaborationInfluence
Languages:English
Tech Stack:Python

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn