Director, Research - Evaluation & Training

Snorkel AI
New York, San Francisco
Workplace: HybridFull timeFunction: Education & TrainingExperience: 7+ yearsSkills: ["Communication","Leadership","Problem-solving"]

Lead a multi-quarter roadmap for novel evaluation, error analysis, and data valuation; translate research findings into strategic datasets and data strategies; grow and mentor a research team; bridge findings to Product, GTM, and customers; influence how data drives model performance and benchmarks in AI applications.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Snorkel AI
Snorkel AI
3 months ago

Director, Research - Evaluation & Training

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Lead a multi-quarter roadmap for novel evaluation, error analysis, and data valuation; translate research findings into strategic datasets and data strategies; grow and mentor a research team; bridge findings to Product, GTM, and customers; influence how data drives model performance and benchmarks in AI applications.
Location: New York, San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: Education & Training
Seniority: Director level

Key Responsibilities

  • •Own a multi-quarter roadmap centered on novel evaluation, error analysis, and data valuation techniques
  • •Synthesize and share trends from model-failure analysis and benchmarking into recommendations on the datasets the community should focus on and the ones Snorkel should invest in — making this team a primary input to the company's data strategy.
  • •Focus on data valuation techniques that quantify how Snorkel data meaningfully improves model performance
  • •Lead and grow a team of researchers, setting a high bar for quality, rigor and speed of execution
  • •Act as the primary bridge between the team's findings and Product, GTM, and our customers

Key Requirements

  • •7+ years in applied AI, ML, or research roles, with 4+ years managing technical teams.
  • •A leader who has repeatedly turned research and analysis into business outcomes, and who instinctively connects technical findings to market and customer needs.
  • •Strong business and market judgment in the AI/ML space — you understand the competitive and frontier-lab landscape and can prioritize accordingly.
  • •Technically conversant and credible: enough depth in LLM evaluation, benchmarking, and model behavior analysis to set direction, judge experimental quality, and pressure-test results — without needing to be the deepest technical expert in the room.
  • •A nose for trends: able to look across many evaluation results and failure cases and extract the signal that should drive what gets built next.
Experience:7+ yearsAI/MLData-centricResearch
Skills:CommunicationLeadershipProblem-solving
Languages:English
Tech Stack:LLM evaluationBenchmarkingData valuation

Company Brief

Snorkel AI
Develops a programmatic data labeling platform and tools to accelerate machine learning by enabling teams to generate, manage, and maintain training datasets through weak supervision and automations.
Industry: Data Infrastructure
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Funding: Series C
Headquarters: Palo Alto, United States
Founded: 2019
WebsiteLinkedIn