1 day ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Build continuous auditing analytics by designing and auditing large-scale ML and LLM models across the full lifecycle—from training and deployment through monitoring. Define model performance and risk metrics (fairness, robustness, calibration, hallucination rates), identify vulnerabilities, and recommend remediation. Partner with internal auditors to deliver data pipelines, data quality checks, data warehousing, dashboards, and automated insights from unstructured and structured data, communicating findings to technical and non-technical stakeholders.

Location: New York

Workplace: Onsite

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Audit model lifecycle risks from training through deployment and monitoring, ensuring compliance with quality, performance, fairness, and risk-management standards.
•Identify model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation strategies.
•Define and assess performance and statistical validation metrics for ML/LLM models, including hallucination rates, fairness quantification, confidence scoring, and stability analyses.
•Partner with auditors to support audit engagements with data analytics services such as interviews, query/testing strategy design, and deploying data quality checks.
•Build automation for continuous auditing by creating ETL data pipelines and dashboards, and developing AI/ML models to surface insights from unstructured and structured data at scale.

Key Requirements

•Bachelor’s degree in a quantitative discipline (e.g., Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).
•5+ years professional experience in applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models, plus 5+ years in data science/analytics from the technology sector.
•Hands-on experience designing, deploying, and monitoring large-scale ML models, with strong proficiency in SQL and Python (including Hugging Face Transformers, TensorFlow, PyTorch, scikit-learn) and ML pipeline orchestration platforms.
•Expertise defining and assessing model performance and risk metrics, including accuracy/precision-recall/F1/calibration/robustness/fairness, hallucination rates, bias/fairness quantification, confidence scoring, and stability analyses.
•Extensive knowledge of transformer-based LLM architectures and classical ML algorithms, with a track record supporting or performing AI/ML model audits or evaluations in a corporate, regulatory, or advisory context.

Experience:5+ yearsLLMsMachine learningAI researchTechnology sectorSaaSFintechE-commerceSocial media

Education:Bachelor's

Skills:CommunicationCollaborationStakeholder managementRisk assessmentProblem-solving

Languages:Mandarin

Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorLLM auditingETLData warehousingDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduceGPTBERT

Company Brief

TikTok

Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

TikTok

1 day ago

Senior Data Scientist, Model Risk & Data Analytics, Internal Audit - AMS

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Location: New York

Workplace: Onsite

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Audit model lifecycle risks from training through deployment and monitoring, ensuring compliance with quality, performance, fairness, and risk-management standards.
•Identify model vulnerabilities such as bias, fairness violations, harmful hallucinations, and security risks, and recommend remediation strategies.
•Define and assess performance and statistical validation metrics for ML/LLM models, including hallucination rates, fairness quantification, confidence scoring, and stability analyses.
•Partner with auditors to support audit engagements with data analytics services such as interviews, query/testing strategy design, and deploying data quality checks.
•Build automation for continuous auditing by creating ETL data pipelines and dashboards, and developing AI/ML models to surface insights from unstructured and structured data at scale.

Key Requirements

•Bachelor’s degree in a quantitative discipline (e.g., Mathematics, Statistics, Computer Science, Financial Engineering, Operations Research, or Economics).
•5+ years professional experience in applied data science, machine learning engineering, or AI research with hands-on work on LLMs and traditional ML models, plus 5+ years in data science/analytics from the technology sector.
•Hands-on experience designing, deploying, and monitoring large-scale ML models, with strong proficiency in SQL and Python (including Hugging Face Transformers, TensorFlow, PyTorch, scikit-learn) and ML pipeline orchestration platforms.
•Expertise defining and assessing model performance and risk metrics, including accuracy/precision-recall/F1/calibration/robustness/fairness, hallucination rates, bias/fairness quantification, confidence scoring, and stability analyses.
•Extensive knowledge of transformer-based LLM architectures and classical ML algorithms, with a track record supporting or performing AI/ML model audits or evaluations in a corporate, regulatory, or advisory context.

Experience:5+ yearsLLMsMachine learningAI researchTechnology sectorSaaSFintechE-commerceSocial media

Education:Bachelor's

Skills:CommunicationCollaborationStakeholder managementRisk assessmentProblem-solving

Languages:Mandarin

Tech Stack:SQLPythonHugging Face TransformersTensorFlowPyTorchScikit-learnLLMAuditorLLM auditingETLData warehousingDashboardsAWSGCPAzureSnowflakeHadoopFlinkMapReduceGPTBERT

Company Brief

TikTok

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn