Human Data Quality Engineer (Founding Team)

Prolific
United Kingdom, United States
Workplace: RemoteFull timeFunction: QA, Test & Release EngineeringSkills: ["Analytical thinking","Statistical thinking","Stakeholder management","Communication","Builder mindset"]

Build the quality systems for Prolific’s AI Data Services team, defining what “good” means for human data used in frontier ML and evaluation. Partner with clients and AI labs to translate model needs into robust human data and evaluation strategies across the full lifecycle—from study design and participant strategy to launch readiness and delivery. Engineer scalable measurement frameworks, automated checks, and dashboards using Python and SQL, and help raise quality capability as the team grows.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Prolific
Prolific
1 month ago

Human Data Quality Engineer (Founding Team)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 15 hours agoStatus: Live

Job Summary

Build the quality systems for Prolific’s AI Data Services team, defining what “good” means for human data used in frontier ML and evaluation. Partner with clients and AI labs to translate model needs into robust human data and evaluation strategies across the full lifecycle—from study design and participant strategy to launch readiness and delivery. Engineer scalable measurement frameworks, automated checks, and dashboards using Python and SQL, and help raise quality capability as the team grows.
Location: United Kingdom, United States
Workplace: Remote
Employment Type: Full time
Job Function: QA, Test & Release Engineering
Seniority: Mid level

Key Responsibilities

  • •Design quality frameworks for complex human data programmes, from evaluation rubrics through launch readiness.
  • •Work with clients as a strategic thought partner, challenging annotation schemas and data requirements to deliver the needed model signal.
  • •Build quality upstream across recruitment, screening, training, guideline writing, and calibration sessions.
  • •Build scalable quality systems, measurement frameworks, and automated checks using Python and SQL.
  • •Investigate data quality and integrity issues, identify root causes, and turn insights into scalable improvements; architect dashboards and monitoring for operational performance.

Pay and Benefits

Equity and Bonus:Equity
Perks:Remote WorkEquity

Key Requirements

  • •5+ years building quality, evaluation, or annotation systems within AI, machine learning, LLMs, or human data environments.
  • •Strong Python and SQL skills, with a passion for solving complex quality problems with data.
  • •Solid understanding of machine learning pipelines and how human data impacts model performance.
  • •Strong analytical and statistical thinking, with experience designing scalable quality frameworks.
  • •Confidence and credibility to interact with stakeholders at frontier labs and act as a partner.
Experience:AIMachine learningLLMsHuman data
Skills:Analytical thinkingStatistical thinkingStakeholder managementCommunicationBuilder mindset
Languages:English
Tech Stack:PythonSQL

Company Brief

Prolific
Operates an online participant recruitment platform connecting researchers and companies with vetted participants for academic and market research studies, offering survey hosting, participant management, and data quality tools.
Industry: SaaS
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Funding: Series A
Headquarters: Oxford, United Kingdom
Founded: 2014
WebsiteLinkedIn