Software Engineer, Agent Platform

Retool
San Francisco
Workplace: HybridFull timeUSD 163,800 - 306,000 annuallyFunction: Software EngineeringExperience: 6+ yearsSkills: ["Ownership","Independent problem-solving","Quality-focused","Iterative improvement","Tradeoff analysis"]

Own model-driven, probabilistic behavior in production systems for AI-powered agent features. Partner across product, infrastructure, and evaluation to design prompting/retrieval/tool-use strategies, build evaluation and regression-detection frameworks, and diagnose issues like hallucinations or instruction drift. Define guardrails, fallbacks, and degradation paths so systems remain trustworthy under variance—shaping runtime behavior across the stack.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Retool
Retool
2 days ago

Software Engineer, Agent Platform

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Own model-driven, probabilistic behavior in production systems for AI-powered agent features. Partner across product, infrastructure, and evaluation to design prompting/retrieval/tool-use strategies, build evaluation and regression-detection frameworks, and diagnose issues like hallucinations or instruction drift. Define guardrails, fallbacks, and degradation paths so systems remain trustworthy under variance—shaping runtime behavior across the stack.
Location: San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Own model-driven behavior for AI-powered features across multiple product surfaces, including quality, safety, variance, and failure modes.
  • •Design and evolve prompting, retrieval, routing, and tool-use strategies that manage non-determinism.
  • •Build and maintain evaluation systems that measure model performance using statistical signals, distributions, and trends.
  • •Detect, diagnose, and resolve non-deterministic failures such as hallucinations, partial correctness, instruction drift, and sensitivity to context changes.
  • •Define guardrails, fallbacks, and degradation paths so systems remain useful when models behave unexpectedly.

Pay and Benefits

Salary: USD 163,800 - 306,000 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVision401k

Key Requirements

  • •6+ years of professional engineering experience with ownership of complex production systems.
  • •Demonstrated experience owning AI/LLM behavior beyond basic integration, including mitigation of variance and failure modes.
  • •Ability to reason about probabilistic systems and tradeoffs such as quality vs. cost and recall vs. precision.
  • •Experience designing or maintaining evaluation frameworks, golden datasets, regression detection, or human-in-the-loop feedback loops.
  • •Comfort operating independently in ambiguous problem spaces and setting quality standards for others.
Experience:6+ years
Skills:OwnershipIndependent problem-solvingQuality-focusedIterative improvementTradeoff analysis
Tech Stack:TypeScriptNode.jsReactLLMsLLM callsRAGAgentic systemsTool-using modelsVector databasesEmbeddingsRetrieval pipelinesFine-tuningModel routingPost-training techniques

Company Brief

Retool
Provides a low-code platform for building internal tools and business applications, enabling developers and teams to assemble UIs on top of databases and APIs to accelerate operational workflows.
Industry: Developer Tools
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: San Francisco, United States
Founded: 2017
WebsiteLinkedIn