3 months ago

Research Program Manager - Model Evals and Safety

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 23 hours agoStatus: Live

Job Summary

Lead the creation of foundational infrastructure for model evaluations and safety at Reflection. Define evaluation frameworks, tooling requirements, and operational processes to assess capabilities, risks, and readiness for release. Stand up model safety operations, embed safety checkpoints across training stages, drive prioritization of eval investments, and establish external safety ecosystem relationships. Build from zero to one with a focus on measurable outcomes and cross-team coordination.

Location: San Francisco

Workplace: Onsite

Employment Type: Full time

Job Function: Program & Project Management (PMO)

Seniority: Sr. Manager level

Key Responsibilities

•Build the foundational infrastructure for model evals and safety at Reflection. Define the evaluation frameworks, tooling requirements, and operational processes that will underpin how we assess model capabilities, risks, and readiness for release.
•Stand up model safety operations as a function, including establishing the workflows, review cadences, and decision frameworks that connect safety evaluation to the model development and release lifecycle.
•Partner with research and engineering leads across pre-training, mid-training, and post-training to embed safety and evaluation checkpoints into the development process in a way that is rigorous without being a bottleneck.
•Drive the scoping and prioritization of eval science and eval infrastructure investments, working with technical leads to determine what to build in-house, what to adopt, and where to invest research effort.
•Establish Reflection's engagement with the external safety ecosystem, including third-party assessments, academic partnerships, and industry safety frameworks. Represent the company's safety posture to external stakeholders with credibility and clarity.

Pay and Benefits

Perks:Health InsuranceDentalVisionLife InsuranceDisability InsuranceParental LeaveRelocationEquity

Key Requirements

•7+ years of experience in technical program management, research operations, or ML engineering, with demonstrated experience standing up new functions, teams, or programs from scratch.
•Familiar with the landscape of model evaluation and AI safety, including evaluation methodologies, red-teaming, alignment research, and the evolving regulatory and industry safety ecosystem.
•Deep enough technically to engage with researchers and engineers on topics like model behavior, evaluation design, data pipelines, and safety-critical system architecture.
•Proven ability to build structures where none exists with clear ownership, measurable outcomes, and durable processes.
•Strong stakeholder management skills spanning deeply technical ICs, research leadership, and external partners.

Experience:7+ yearsAI safetyML engineeringModel evaluation

Skills:Stakeholder managementCommunicationLeadershipProblem-solvingCross-functional collaboration

Languages:English

Company Brief

Reflection AI

Builds frontier autonomous AI systems focused on autonomous coding agents (product: Asimov) to create organizational superintelligence, founded by former DeepMind/Google researchers and hiring across SF, NYC, London.

Industry: AI & Machine Learning

Company Size: Small (11 to 50 employees)

Growth: Growth Stage Startup

Funding: Series A

Headquarters: San Francisco, United States

Founded: 2024

Website LinkedIn Glassdoor

Loading job details...

Preparing the role view and application actions.

Reflection AI

3 months ago

Research Program Manager - Model Evals and Safety

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 23 hours agoStatus: Live

Job Summary

Location: San Francisco

Workplace: Onsite

Employment Type: Full time

Job Function: Program & Project Management (PMO)

Seniority: Sr. Manager level

Key Responsibilities

•Build the foundational infrastructure for model evals and safety at Reflection. Define the evaluation frameworks, tooling requirements, and operational processes that will underpin how we assess model capabilities, risks, and readiness for release.
•Stand up model safety operations as a function, including establishing the workflows, review cadences, and decision frameworks that connect safety evaluation to the model development and release lifecycle.
•Partner with research and engineering leads across pre-training, mid-training, and post-training to embed safety and evaluation checkpoints into the development process in a way that is rigorous without being a bottleneck.
•Drive the scoping and prioritization of eval science and eval infrastructure investments, working with technical leads to determine what to build in-house, what to adopt, and where to invest research effort.
•Establish Reflection's engagement with the external safety ecosystem, including third-party assessments, academic partnerships, and industry safety frameworks. Represent the company's safety posture to external stakeholders with credibility and clarity.

Pay and Benefits

Perks:Health InsuranceDentalVisionLife InsuranceDisability InsuranceParental LeaveRelocationEquity

Key Requirements

•7+ years of experience in technical program management, research operations, or ML engineering, with demonstrated experience standing up new functions, teams, or programs from scratch.
•Familiar with the landscape of model evaluation and AI safety, including evaluation methodologies, red-teaming, alignment research, and the evolving regulatory and industry safety ecosystem.
•Deep enough technically to engage with researchers and engineers on topics like model behavior, evaluation design, data pipelines, and safety-critical system architecture.
•Proven ability to build structures where none exists with clear ownership, measurable outcomes, and durable processes.
•Strong stakeholder management skills spanning deeply technical ICs, research leadership, and external partners.

Experience:7+ yearsAI safetyML engineeringModel evaluation

Skills:Stakeholder managementCommunicationLeadershipProblem-solvingCross-functional collaboration

Languages:English

Company Brief

Reflection AI

Industry: AI & Machine Learning

Company Size: Small (11 to 50 employees)

Growth: Growth Stage Startup

Funding: Series A

Headquarters: San Francisco, United States

Founded: 2024

Website LinkedIn Glassdoor