Model Data Operations Specialist, Evaluation & Judgement Center, Platform Responsibility

TikTok
Singapore
Workplace: OnsiteFull timeFunction: Product ManagementEducation: bachelorsSkills: ["Problem-solving","Root-cause analysis","Prioritization","Stakeholder collaboration","Analytical thinking"]

Lead model data operations for an evaluation and judgment center that uses LLMs to strengthen TikTok content safety. Drive AI special projects including data annotation, model iteration evaluation, and prompt optimization to improve risky-content recall. Own the Golden Set definitions and translate policy into annotation SOPs for Data Ops. Establish adjudication and rule-routing logic for consistent risk judgments, and build an AI content quality management system through continuous diagnosis and optimization.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
1 month ago

Model Data Operations Specialist, Evaluation & Judgement Center, Platform Responsibility

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 22 hours agoStatus: Live

Job Summary

Lead model data operations for an evaluation and judgment center that uses LLMs to strengthen TikTok content safety. Drive AI special projects including data annotation, model iteration evaluation, and prompt optimization to improve risky-content recall. Own the Golden Set definitions and translate policy into annotation SOPs for Data Ops. Establish adjudication and rule-routing logic for consistent risk judgments, and build an AI content quality management system through continuous diagnosis and optimization.
Location: Singapore
Workplace: Onsite
Employment Type: Full time
Job Function: Product Management
Seniority: Mid level

Key Responsibilities

  • •Undertake AI special projects for product R&D, including annotation of model training data samples and evaluation of model iteration effectiveness.
  • •Standardize Golden Set definitions across risk domains and convert qualitative policy documents into quantitative annotation SOPs for the Data Ops team.
  • •Own ground-truth delivery activities such as demand analysis, acceptance, solution customization and training, and quality assurance.
  • •Establish an adjudication mechanism for internal cases to align consistent judgments across different risk levels.
  • •Design and continuously optimize LLM rule routing logic and build an AI content quality management system through diagnosis, data analysis, and model optimization.

Key Requirements

  • •Bachelor’s degree or above with 3+ years of experience in model operations, AI/ML operations, or related domains.
  • •Ability to lead community policy development and collaborate across Product, Engineering, and Operations teams.
  • •Hands-on familiarity with mainstream LLM products (e.g., ChatGPT, Gemini) and experience in model training, evaluation, or prompt engineering.
  • •Strong analytical capabilities to translate data insights into actionable strategies and drive resolution of operational/model issues.
  • •Understanding of community safety frameworks and risk governance, with ability to identify content/model/process risks and design scalable mitigations.
Experience:AI/MLLLMsContent safetyInternet products
Education:Bachelor's
Skills:Problem-solvingRoot-cause analysisPrioritizationStakeholder collaborationAnalytical thinking
Tech Stack:LLMLarge language modelsPrompt engineeringChatGPTGeminiAIData pipelinesAnnotation

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn