Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

TikTok
Los Angeles
Full timeFunction: Data Science & Machine LearningExperience: 5+ yearsEducation: bachelorsSkills: ["Communication","Collaboration","Stakeholder management","Problem-solving"]

Build state-of-the-art analytics products for Internal Audit by designing, deploying, and monitoring large-scale ML and LLMs across the full model lifecycle. Conduct model evaluations and audits focused on robustness, fairness, interpretability, alignment, compliance, and risk controls. Define performance and hallucination metrics, identify vulnerabilities, automate continuous auditing via ETL pipelines and dashboards, and collaborate with auditors and data partners to deliver actionable risk insights at scale.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
1 month ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

âś“ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 18 hours agoStatus: Live

Job Summary

Build state-of-the-art analytics products for Internal Audit by designing, deploying, and monitoring large-scale ML and LLMs across the full model lifecycle. Conduct model evaluations and audits focused on robustness, fairness, interpretability, alignment, compliance, and risk controls. Define performance and hallucination metrics, identify vulnerabilities, automate continuous auditing via ETL pipelines and dashboards, and collaborate with auditors and data partners to deliver actionable risk insights at scale.
Location: Los Angeles
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Audit model performance and controls across the model lifecycle from training through deployment and monitoring.
  • •Identify and mitigate model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation.
  • •Define and assess metrics for model performance, fairness, calibration, robustness, confidence scoring, and stability, including hallucination-rate measurement for LLMs.
  • •Partner with auditors to provide data analytics services—conduct interviews/observations, develop queries and testing strategies, and deploy data quality checks.
  • •Build and maintain data warehouses, ETL pipelines, and dashboards to support continuous auditing and automation use cases.

Key Requirements

  • •5+ years of applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models.
  • •Hands-on experience designing, deploying, and monitoring large-scale ML models with lifecycle risk/control awareness, plus strong SQL and Python proficiency.
  • •Expertise defining and assessing model performance metrics (accuracy, precision/recall, F1, calibration, robustness, fairness) and evaluating LLM hallucination rates, bias/fairness, confidence scoring, and stability.
  • •Extensive knowledge of transformer-based LLM architectures (e.g., GPT, BERT, T5, PaLM) and classical ML algorithms.
  • •Bachelor’s degree in a quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).
Experience:5+ yearsSaaSMedia techE-commerceSocial media platformsFintech
Education:Bachelor's in quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics)
Skills:CommunicationCollaborationStakeholder managementProblem-solving
Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorGPTBERTT5PaLMETLDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduce

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn