Senior Data Labeler

White Circle
Paris, London
Workplace: HybridFull timeUSD 60,000 - 100,000 annuallyFunction: Laboratory & Clinical OperationsSkills: ["People management","Operational excellence","Communication","Cross-functional collaboration","Continuous improvement"]

Lead the Data Labeling team from scratch—hiring, coaching, and managing performance. Own annotation guidelines, quality standards, QA processes, calibration, and auditing systems. Partner with AI researchers and engineers to convert research objectives into labeling workflows, prioritizing projects by business and research needs. Run complex AI evaluation initiatives across safety and preference ranking (including RLHF and benchmark creation), monitor quality/throughput/cost, and build dashboards to communicate operational insights.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
White Circle
White Circle
2 months ago

Senior Data Labeler

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Lead the Data Labeling team from scratch—hiring, coaching, and managing performance. Own annotation guidelines, quality standards, QA processes, calibration, and auditing systems. Partner with AI researchers and engineers to convert research objectives into labeling workflows, prioritizing projects by business and research needs. Run complex AI evaluation initiatives across safety and preference ranking (including RLHF and benchmark creation), monitor quality/throughput/cost, and build dashboards to communicate operational insights.
Location: Paris, London
Workplace: Hybrid
Employment Type: Full time
Job Function: Laboratory & Clinical Operations
Seniority: Sr. Director level

Key Responsibilities

  • •Build from scratch and lead the Data Labeling team, including hiring, coaching, and performance management.
  • •Define annotation guidelines, quality standards, and evaluation frameworks.
  • •Develop quality assurance processes, calibration sessions, and auditing systems to maintain consistency.
  • •Partner with AI researchers and engineers to translate research objectives into labeling workflows and prioritize projects.
  • •Lead AI evaluation initiatives (safety, preference ranking, RLHF, policy evaluation, benchmark creation) and analyze edge cases to improve guidelines and model performance.

Pay and Benefits

Salary: USD 60,000 - 100,000 annually
Equity and Bonus:Equity
Perks:Paid LeaveRelocation

Key Requirements

  • •Experience leading data annotation or AI evaluation teams.
  • •Strong people management and operational skills for scaling processes without losing quality.
  • •Deep understanding of AI model evaluation, LLM behavior, and modern annotation workflows.
  • •Experience managing annotation programs for LLMs, generative AI, or machine learning.
  • •Experience with RLHF, preference data collection, safety evaluations, or benchmark creation.
Experience:AI safetyLLM evaluationData annotationTrust & SafetyContent moderation
Skills:People managementOperational excellenceCommunicationCross-functional collaborationContinuous improvement
Languages:English
Tech Stack:LLMsRLHFPrompt engineeringSQLPythonData analysisBenchmarksDashboardsAnnotation toolingAutomation

Company Brief

White Circle
Builds AI-driven products and services to help businesses automate workflows, extract insights from data, and improve decision-making using machine learning and natural language processing technologies.
Industry: AI & Machine Learning
Website