LLM Evaluation Product Operations Specialist – TikTok

TikTok
Singapore
Workplace: OnsiteFull timeFunction: Communications, PR & CommunityEducation: bachelorsSkills: ["Written communication","Abstract thinking","Detail-oriented","Logical thinking","Collaboration"]

Own the end-to-end design and implementation of an evaluation framework for a TikTok in-app AI chatbot experience. Use user research and data analysis to define experience quality standards and guide improvements in model capability and user satisfaction. Partner with strategy and algorithm teams to build scenario-based evaluation metrics, and with international delivery teams to execute evaluation programs using automated evaluation methods that meet time and quality targets.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
1 month ago

LLM Evaluation Product Operations Specialist – TikTok

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 10 hours agoStatus: Live

Job Summary

Own the end-to-end design and implementation of an evaluation framework for a TikTok in-app AI chatbot experience. Use user research and data analysis to define experience quality standards and guide improvements in model capability and user satisfaction. Partner with strategy and algorithm teams to build scenario-based evaluation metrics, and with international delivery teams to execute evaluation programs using automated evaluation methods that meet time and quality targets.
Location: Singapore
Workplace: Onsite
Employment Type: Full time
Job Function: Communications, PR & Community

Key Responsibilities

  • •Own the end-to-end design and implementation of the AI chatbot experience evaluation framework for Tako, supporting ongoing iteration and optimization.
  • •Leverage user research and data analysis to identify user needs and define experience quality standards to improve model capability and user satisfaction.
  • •Collaborate with strategy and algorithm teams to build scenario-based evaluation metrics and provide analytical insights and recommendations.
  • •Partner with international delivery teams to execute evaluation programs and use automated evaluation methods to meet time and quality standards.

Key Requirements

  • •Bachelor's degree or above.
  • •Strong written communication skills and the ability to think abstractly to define user scenarios and experience quality standards.
  • •Highly detail-oriented and well-organized with strong logical thinking and a solid data mindset.
  • •Excellent communicator and collaborative team player.
  • •Prior experience in LLM evaluation or user-facing product roles (preferred).
Experience:LLM evaluationProduct operationsUser researchData analysis
Education:Bachelor's
Skills:Written communicationAbstract thinkingDetail-orientedLogical thinkingCollaboration
Tech Stack:LLM evaluationData analysisAutomated evaluation methods

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn