Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

TikTok
New York
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningExperience: 5+ yearsEducation: bachelorsSkills: ["Communication","Collaboration","Stakeholder management","Risk assessment","Problem-solving"]

Build continuous auditing analytics by designing and auditing large-scale ML and LLM models across the full lifecycle—from training and deployment through monitoring. Define model performance and risk metrics (fairness, robustness, calibration, hallucination rates), identify vulnerabilities, and recommend remediation. Partner with internal auditors to deliver data pipelines, data quality checks, data warehousing, dashboards, and automated insights from unstructured and structured data, communicating findings to technical and non-technical stakeholders.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
TikTok
TikTok
1 month ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 18 hours agoStatus: Live

Job Summary

Build continuous auditing analytics by designing and auditing large-scale ML and LLM models across the full lifecycle—from training and deployment through monitoring. Define model performance and risk metrics (fairness, robustness, calibration, hallucination rates), identify vulnerabilities, and recommend remediation. Partner with internal auditors to deliver data pipelines, data quality checks, data warehousing, dashboards, and automated insights from unstructured and structured data, communicating findings to technical and non-technical stakeholders.
Location: New York
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Audit model lifecycle risks from training through deployment and monitoring, ensuring compliance with quality, performance, fairness, and risk-management standards.
  • •Identify model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation strategies.
  • •Define and assess performance and statistical validation metrics for ML/LLM models, including hallucination rates, fairness quantification, confidence scoring, and stability analyses.
  • •Partner with auditors to support audit engagements with data analytics services such as interviews, query/testing strategy design, and deploying data quality checks.
  • •Build automation for continuous auditing by creating ETL data pipelines and dashboards, and developing AI/ML models to surface insights from unstructured and structured data at scale.

Key Requirements

  • •Bachelor’s degree in a quantitative discipline (e.g., Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).
  • •5+ years professional experience in applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models, plus 5+ years in data science/analytics from the technology sector.
  • •Hands-on experience designing, deploying, and monitoring large-scale ML models, with strong proficiency in SQL and Python (including Hugging Face Transformers, TensorFlow, PyTorch, scikit-learn) and ML pipeline orchestration platforms.
  • •Expertise defining and assessing model performance and risk metrics, including accuracy/precision-recall/F1/calibration/robustness/fairness, hallucination rates, bias/fairness quantification, confidence scoring, and stability analyses.
  • •Extensive knowledge of transformer-based LLM architectures and classical ML algorithms, with a track record supporting or performing AI/ML model audits or evaluations in a corporate, regulatory, or advisory context.
Experience:5+ yearsLLMsMachine learningAI researchTechnology sectorSaaSFintechE-commerceSocial media
Education:Bachelor's
Skills:CommunicationCollaborationStakeholder managementRisk assessmentProblem-solving
Languages:Mandarin
Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorLLM auditingETLData warehousingDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduceGPTBERT

Company Brief

TikTok
Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Growth: Scaleup
Headquarters: Singapore, Singapore
Founded: 2016
WebsiteLinkedIn