1 day ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Build state-of-the-art analytics products for Internal Audit by designing, deploying, and monitoring large-scale ML and LLMs across the full model lifecycle. Conduct model evaluations and audits focused on robustness, fairness, interpretability, alignment, compliance, and risk controls. Define performance and hallucination metrics, identify vulnerabilities, automate continuous auditing via ETL pipelines and dashboards, and collaborate with auditors and data partners to deliver actionable risk insights at scale.

Location: Los Angeles

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Audit model performance and controls across the model lifecycle from training through deployment and monitoring.
•Identify and mitigate model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation.
•Define and assess metrics for model performance, fairness, calibration, robustness, confidence scoring, and stability, including hallucination-rate measurement for LLMs.
•Partner with auditors to provide data analytics services—conduct interviews/observations, develop queries and testing strategies, and deploy data quality checks.
•Build and maintain data warehouses, ETL pipelines, and dashboards to support continuous auditing and automation use cases.

Key Requirements

•5+ years of applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models.
•Hands-on experience designing, deploying, and monitoring large-scale ML models with lifecycle risk/control awareness, plus strong SQL and Python proficiency.
•Expertise defining and assessing model performance metrics (accuracy, precision/recall, F1, calibration, robustness, fairness) and evaluating LLM hallucination rates, bias/fairness, confidence scoring, and stability.
•Extensive knowledge of transformer-based LLM architectures (e.g., GPT, BERT, T5, PaLM) and classical ML algorithms.
•Bachelor’s degree in a quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).

Experience:5+ yearsSaaSMedia techE-commerceSocial media platformsFintech

Education:Bachelor's in quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics)

Skills:CommunicationCollaborationStakeholder managementProblem-solving

Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorGPTBERTT5PaLMETLDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduce

Company Brief

TikTok

Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

TikTok

1 day ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Location: Los Angeles

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Audit model performance and controls across the model lifecycle from training through deployment and monitoring.
•Identify and mitigate model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation.
•Define and assess metrics for model performance, fairness, calibration, robustness, confidence scoring, and stability, including hallucination-rate measurement for LLMs.
•Partner with auditors to provide data analytics services—conduct interviews/observations, develop queries and testing strategies, and deploy data quality checks.
•Build and maintain data warehouses, ETL pipelines, and dashboards to support continuous auditing and automation use cases.

Key Requirements

•5+ years of applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models.
•Hands-on experience designing, deploying, and monitoring large-scale ML models with lifecycle risk/control awareness, plus strong SQL and Python proficiency.
•Expertise defining and assessing model performance metrics (accuracy, precision/recall, F1, calibration, robustness, fairness) and evaluating LLM hallucination rates, bias/fairness, confidence scoring, and stability.
•Extensive knowledge of transformer-based LLM architectures (e.g., GPT, BERT, T5, PaLM) and classical ML algorithms.
•Bachelor’s degree in a quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).

Experience:5+ yearsSaaSMedia techE-commerceSocial media platformsFintech

Education:Bachelor's in quantitative discipline (Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics)

Skills:CommunicationCollaborationStakeholder managementProblem-solving

Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorGPTBERTT5PaLMETLDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduce

Company Brief

TikTok

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn