Data Quality Engineer, AI Business

Prolific
North America, United Kingdom
Workplace: RemoteFull timeFunction: QA, Test & Release EngineeringExperience: 5+ yearsSkills: ["Problem-solving","Communication","Collaboration"]

Data Quality Engineer for Prolific AI Data Services overseeing end-to-end quality design, measurement systems, and automation for managed service studies. You’ll work with Product, Engineering, Operations, and Client teams to embed quality and authenticity into study design and execution, building scalable gating, calibration, and monitoring to ensure trustworthy data for AI training and research.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Prolific
Prolific
7 months ago

Data Quality Engineer, AI Business

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 10 hours agoStatus: Live

Job Summary

Data Quality Engineer for Prolific AI Data Services overseeing end-to-end quality design, measurement systems, and automation for managed service studies. You’ll work with Product, Engineering, Operations, and Client teams to embed quality and authenticity into study design and execution, building scalable gating, calibration, and monitoring to ensure trustworthy data for AI training and research.
Location: North America, United Kingdom
Workplace: Remote
Employment Type: Full time
Job Function: QA, Test & Release Engineering

Key Responsibilities

  • •Own end-to-end quality design for Prolific managed service studies, including rubrics, acceptance criteria, defect taxonomies, severity models, and clear definitions of done.
  • •Define, implement, and maintain quality measurement systems, including sampling plans, golden sets, calibration protocols, agreement targets, adjudication workflows, and drift detection.
  • •Build and deploy automated quality checks and launch gates using Python and SQL, such as schema and format validation, completeness checks, anomaly detection, consistency testing, and label distribution monitoring.
  • •Design and run launch readiness processes, including pre-launch checks, pilot calibration, ramp criteria, full-launch thresholds, and pause/rollback mechanisms.
  • •Partner with Product and Engineering to embed in-study quality controls and authenticity checks into workflows, tooling, and escalation paths.

Pay and Benefits

Perks:Remote Work

Key Requirements

  • •5+ years of experience in quality engineering, data or annotation quality, analytics engineering, trust and integrity, or ML/LLM evaluation operations.
  • •Strong proficiency in Python and SQL, with comfort applying statistical concepts such as sampling strategies, confidence levels, and agreement metrics.
  • •Proven track record turning ambiguous or messy quality problems into clear metrics, automated checks, and durable process improvements.
  • •Strong quality systems thinking with ability to translate edge cases into clear rules, tests, rubrics, and governance mechanisms.
  • •Hands-on experience instrumenting workflows and implementing pragmatic automation that catches quality and integrity issues early.
Experience:5+ yearsAIData qualityML/LLM evaluation
Skills:Problem-solvingCommunicationCollaboration
Languages:English
Tech Stack:PythonSQL

Company Brief

Prolific
Operates an online participant recruitment platform connecting researchers and companies with vetted participants for academic and market research studies, offering survey hosting, participant management, and data quality tools.
Industry: SaaS
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Funding: Series A
Headquarters: Oxford, United Kingdom
Founded: 2014
WebsiteLinkedIn