Software Test Engineer (AI), TikTok Search

TikTok
Singapore
Workplace: OnsiteFull timeFunction: QA, Test & Release EngineeringEducation: bachelorsSkills: ["Debugging","Problem-solving","Cross-functional collaboration","Analytical thinking"]

Own end-to-end quality for TikTok Search, building AI-powered testing and evaluation for query understanding, retrieval, ranking, and generative search (LLM Answers and RAG). Develop LLM-based evaluation pipelines, multilingual query sets, and golden datasets. Drive impact analysis with offline replay, side-by-side evaluations, and interleaving experiments to strengthen release quality gates. Partner across search engineering, algorithms, product, and operations to improve reliability and engineering productivity with automated test tooling and self-healing regression suites.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
15 hours ago

Software Test Engineer (AI), TikTok Search

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 15 hours agoStatus: Live

Job Summary

Own end-to-end quality for TikTok Search, building AI-powered testing and evaluation for query understanding, retrieval, ranking, and generative search (LLM Answers and RAG). Develop LLM-based evaluation pipelines, multilingual query sets, and golden datasets. Drive impact analysis with offline replay, side-by-side evaluations, and interleaving experiments to strengthen release quality gates. Partner across search engineering, algorithms, product, and operations to improve reliability and engineering productivity with automated test tooling and self-healing regression suites.
Location: Singapore
Workplace: Onsite
Employment Type: Full time
Job Function: QA, Test & Release Engineering

Key Responsibilities

  • •Design and deliver AI-powered testing and evaluation solutions for TikTok Search across query understanding, retrieval, ranking, and generative search (LLM Answers and RAG).
  • •Build LLM-based evaluation pipelines for relevance, answer quality (faithfulness/groundedness), freshness, coverage, diversity, and safety, and maintain multilingual multi-market query sets and golden datasets.
  • •Perform AI-assisted change impact analysis across the search stack (index, retrieval, ranking models, prompts, and models) using offline replay, side-by-side evaluation, and interleaving experiments to strengthen release quality gates.
  • •Partner with search engineering, algorithm, product, and content operations teams to identify quality and productivity pain points and drive practical AI-powered solutions into production.
  • •Improve engineering productivity with AI agents and tooling for automated test case generation, log-driven bug analysis, root-cause attribution, and self-healing regression suites.

Key Requirements

  • •Bachelor’s degree or above in Computer Science, Software Engineering, Information Retrieval, Data Science, or a related technical discipline.
  • •Experience in software quality assurance or software testing, preferably for search, recommendation, ranking, or other large-scale data/algorithm-driven systems.
  • •Solid understanding of testing methodologies, debugging, and QA best practices across the SDLC, including offline evaluation, online A/B testing, and production monitoring.
  • •Familiarity with distributed systems and search/recommendation infrastructure (e.g., inverted indexes, vector indexes, retrieval, ranking, embeddings) plus basic iOS and/or Android knowledge.
  • •Hands-on experience in testing domains such as automation testing, performance/stress testing, algorithm evaluation, or test platform/tool development.
Experience:SearchRecommendationRankingGenerative AILLMsRAGMultilingualA/B testingDistributed systems
Education:Bachelor's in Computer Science, Software Engineering, Information Retrieval, Data Science, or related technical discipline
Skills:DebuggingProblem-solvingCross-functional collaborationAnalytical thinking
Tech Stack:PythonGoJavaC++LLMsRAGLLM AnswersAI agentsEmbeddingsInverted indexesVector indexesRetrievalRankingIOSAndroidA/B testingNDCGMRRRecallSide-by-side (SBS) evaluation

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn