AI Content Red Team Analyst - Trust and Safety

TikTok
San Francisco
Workplace: OnsiteFull timeFunction: Content & Editorial (Writing/Editing)Experience: 3+ yearsSkills: ["Judgment","Creativity","Analytical rigor","Evidence-based decision making","Cross-functional collaboration"]

Conduct structured adversarial testing of generative AI models and features to uncover emerging trust and safety risks. Probe behavior across contexts and user journeys, investigate jailbreaks, evasions, and prompt-based attacks, and document evidence with reproduction steps and severity assessments. Partner with policy, product, business, and engineering stakeholders to validate mitigations and drive root-cause closure, while maintaining testing playbooks and staying current on evolving abuse trends.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
1 month ago

AI Content Red Team Analyst - Trust and Safety

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 28 minutes agoStatus: Live

Job Summary

Conduct structured adversarial testing of generative AI models and features to uncover emerging trust and safety risks. Probe behavior across contexts and user journeys, investigate jailbreaks, evasions, and prompt-based attacks, and document evidence with reproduction steps and severity assessments. Partner with policy, product, business, and engineering stakeholders to validate mitigations and drive root-cause closure, while maintaining testing playbooks and staying current on evolving abuse trends.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Content & Editorial (Writing/Editing)
Seniority: Mid level

Key Responsibilities

  • •Conduct structured adversarial testing on AI models, features, and policies to identify vulnerabilities and emerging risks.
  • •Explore product behavior across contexts and user journeys to find model failure modes not captured in standard evaluations.
  • •Investigate jailbreaks, evasions, prompt-based attacks, and other adversarial techniques relevant to content safety.
  • •Document findings clearly, including risk descriptions, reproduction steps, severity assessments, and mitigation recommendations.
  • •Partner with cross-functional stakeholders to validate mitigations and drive root cause closure; support testing playbooks and stay updated on emerging adversarial trends.

Key Requirements

  • •3+ years of experience in Trust & Safety, cybersecurity, risk/adversarial testing, or related fields.
  • •Experience with prompt testing, jailbreak analysis, LLM evaluation, or adversarial QA.
  • •Familiarity with AI safety risks including jailbreaks, hallucinations, bias, and misuse patterns.
  • •Ability to independently investigate ambiguous problems, identify non-obvious failure modes and abuse patterns, and produce evidence-based conclusions.
  • •Ability to manage multiple priorities and collaborate effectively with cross-functional teams.
Experience:3+ yearsAI safetyTrust and safetyCybersecurityAdversarial testingLLM evaluation
Skills:JudgmentCreativityAnalytical rigorEvidence-based decision makingCross-functional collaboration
Tech Stack:LLMDeepfakesMultimodal manipulation

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn