Data Scientist, Agent

Lovable
Stockholm
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningSkills: ["Extreme ownership","High velocity","Low-ego collaboration","Entrepreneurial mindset","Working in ambiguity"]

Own how Lovable’s AI agent is measured and improved by defining quality metrics and building eval systems that detect whether changes raise success rates or introduce errors. Develop experimentation and A/B-tested rollouts, translate agent traces and telemetry into actionable fixes with the agent engineering team, and build continuous evaluation tooling as the agent evolves.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Lovable
Lovable
12 hours ago

Data Scientist, Agent

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Own how Lovable’s AI agent is measured and improved by defining quality metrics and building eval systems that detect whether changes raise success rates or introduce errors. Develop experimentation and A/B-tested rollouts, translate agent traces and telemetry into actionable fixes with the agent engineering team, and build continuous evaluation tooling as the agent evolves.
Location: Stockholm
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning

Key Responsibilities

  • •Define and own metrics for agent quality, including success, completion, error rates, and the behaviors driving them.
  • •Build eval systems and an experiment framework to decide whether agent changes ship, including A/B-tested rollouts.
  • •Turn agent traces and telemetry into concrete fixes in partnership with the agent engineering team.
  • •Build tooling and agents that produce these evaluations continuously as the agent evolves.
  • •Set the bar for judging agent behavior in the absence of an answer key.

Key Requirements

  • •Define and own agent quality metrics, driving improvements in success, completion, and error rates.
  • •Experience (or strong interest) in LLM evaluation and observability, including building evals and tracing agent behavior.
  • •Strong SQL and Python skills plus applied statistics and experimentation for noisy outcomes and A/B test design.
  • •Ability to build continuous insight systems (not one-off analyses) that evaluate how agent changes affect user behavior.
  • •Good intuition for what “good” agent behavior looks like when there is no clean answer key, and how to measure it.
Skills:Extreme ownershipHigh velocityLow-ego collaborationEntrepreneurial mindsetWorking in ambiguity
Languages:English
Tech Stack:SQLPythonBraintrustOpenTelemetryBigQueryPubSubHexA/B testingGCP

Company Brief

Lovable
Lovable builds a conversational AI platform that lets users create apps and websites by chatting with AI, automating full‑stack development and integrations to rapidly produce production-ready software.
Industry: Developer Tools
Company Size: Medium (51 to 250 employees)
Revenue: USD 100M to 250M
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series B
Headquarters: Stockholm, Sweden
Founded: 2023
WebsiteLinkedInGlassdoor