Senior Software Development Engineer

Zillow
United States
Workplace: RemoteFull timeFunction: Solutions Engineering & Sales EngineeringExperience: 4+ yearsSkills: ["Ownership","Growth mindset","First-principles thinking","Problem-solving","Collaboration"]

Build and scale evaluation frameworks for agentic AI experiences, including tracing, observability, and quality measurement systems for production AI agents. Create platform infrastructure so domain teams can build on the Evals platform without bottlenecks, integrate new evaluation capabilities into production, and help evolve how Zillow measures quality, reliability, trustworthiness, and product impact. Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Zillow
Zillow
3 days ago

Senior Software Development Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 2 hours agoStatus: Live
Reposted: similar role first listed 3 months ago

Job Summary

Build and scale evaluation frameworks for agentic AI experiences, including tracing, observability, and quality measurement systems for production AI agents. Create platform infrastructure so domain teams can build on the Evals platform without bottlenecks, integrate new evaluation capabilities into production, and help evolve how Zillow measures quality, reliability, trustworthiness, and product impact. Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.
Location: United States
Workplace: Remote
Employment Type: Full time
Job Function: Solutions Engineering & Sales Engineering
Seniority: Mid level

Key Responsibilities

  • •Design, build, and scale evaluation frameworks for agentic AI experiences.
  • •Build tracing, observability, and quality measurement systems for production AI agents.
  • •Create platform infrastructure enabling domain teams to build on the Evals platform without bottlenecks.
  • •Integrate new AI evaluation capabilities into production systems in partnership with applied scientists and ML engineers.
  • •Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.

Pay and Benefits

Equity and Bonus:Equity

Key Requirements

  • •4+ years of backend engineering experience designing, shipping, and operating scalable production ML services.
  • •Experience building platform infrastructure or developer-facing services used by multiple teams, with strong API and reliability thinking.
  • •Hands-on experience with ML evals and observability frameworks (Databricks MLflow preferred; evolving LLM eval tools such as LangSmith, Braintrust, Promptfoo, or similar).
  • •Experience evaluating third-party AI platforms and making principled build-vs-buy decisions for platform infrastructure.
  • •Comfort with LLMs, agentic systems, and evaluation frameworks, including orchestration tools (LangChain, LangGraph, MCP, or similar).
Experience:4+ yearsMachine learningLLMAgentic AI
Skills:OwnershipGrowth mindsetFirst-principles thinkingProblem-solvingCollaboration
Tech Stack:Databricks MLflowLangSmithBraintrustPromptfooLangChainLangGraphMCPLLM tooling

Company Brief

Zillow
Operates an online real estate marketplace providing listings, valuation estimates, rentals, and agent tools. Connects buyers, sellers, renters, and real estate professionals with data-driven search, home Zestimate valuations, and advertising services.
Industry: Online Marketplaces
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Seattle, United States
Founded: 2006
WebsiteLinkedIn