Quality Manager, Applied AI

LILT AI
Boston, District of Columbia, Indianapolis, New York, London, San Francisco
Workplace: HybridFull timeUSD 125,000 - 160,000 annuallyFunction: Quality & Regulatory (Non-Software)Experience: 6+ yearsEducation: mastersSkills: ["Stakeholder management","Written and verbal communication","Translating research goals into measurable targets","Error analysis","Partnership"]

Own end-to-end quality for applied AI deliverables, ensuring every release meets measured targets for rater agreement, rubric reproducibility, and audit-ready reporting. Build and run quality frameworks, tooling, and quality infrastructure across multiple concurrent ML/LLM programs. Detect systemic issues early from workflow data, manage go/no-go delivery sign-off, and resolve customer quality disputes using quantitative analysis and SLA-driven remediation.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
LILT AI
LILT AI
2 days ago

Quality Manager, Applied AI

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 42 minutes agoStatus: Live

Job Summary

Own end-to-end quality for applied AI deliverables, ensuring every release meets measured targets for rater agreement, rubric reproducibility, and audit-ready reporting. Build and run quality frameworks, tooling, and quality infrastructure across multiple concurrent ML/LLM programs. Detect systemic issues early from workflow data, manage go/no-go delivery sign-off, and resolve customer quality disputes using quantitative analysis and SLA-driven remediation.
Location: Boston, District of Columbia, Indianapolis, New York, London, San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: Quality & Regulatory (Non-Software)
Seniority: Manager level

Key Responsibilities

  • •Ensure applied AI deliverables ship with measured quality targets across rater agreement, rubric reproducibility, and customer-auditable reporting.
  • •Own technical tooling and quality infrastructure so quality metrics are available without manual compilation and systemic issues are detected from workflow data.
  • •Maintain operational quality at scale by meeting program targets for acceptance rates, rework, throughput, and SLA adherence.
  • •Run delivery sign-off with documented go/no-go decisions using program QA and readiness memos; ensure escapes remain below threshold via process changes.
  • •Resolve customer quality disputes with quantitative analysis and SLA-driven re-adjudication, reducing recurring defect classes over time.

Pay and Benefits

Salary: USD 125,000 - 160,000 annually

Key Requirements

  • •B.S./M.S. in a quantitative field (CS, Statistics, Data Science, Engineering, Linguistics, HCI) or equivalent practical experience.
  • •6+ years building or operating quality programs for ML/LLM systems (data labeling, benchmark creation, model evaluation, or human-in-the-loop pipelines).
  • •Ability to design and run end-to-end quality frameworks (rubrics, sampling plans, QC gates, acceptance criteria, escalation paths) across multiple concurrent programs.
  • •Strong understanding of reliability and agreement methods (e.g., Krippendorff’s α, Cohen’s κ) with power/sample-size intuition and executive reporting.
  • •Experience with rater/annotator operations including calibration, adjudication, retraining, drift monitoring, and integrity/fraud detection in high-throughput evaluation programs.
Experience:6+ yearsML/LLMHuman-in-the-loop
Education:Master's in CS, Statistics, Data Science, Engineering, Linguistics, HCI
Skills:Stakeholder managementWritten and verbal communicationTranslating research goals into measurable targetsError analysisPartnership
Tech Stack:Label StudioLarge language models (LLMs)Machine translationHuman-in-the-loop

Company Brief

LILT AI
Provides an AI-powered enterprise translation and localization platform combining machine translation, adaptive learning, automation workflows, and human verification to help organizations create and manage multilingual content at scale.
Industry: SaaS
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Funding: Series C
Headquarters: San Francisco, United States
Founded: 2015
Glassdoor
Glassdoor: 3.9
WebsiteLinkedInGlassdoor