Principal Research Scientist - Evaluations

Canva
San Francisco
Workplace: HybridFull timeUSD 310,000 - 420,000 annuallyFunction: Data Science & Machine LearningSkills: ["Leadership","Communication","Problem-solving","Mentorship"]

Define and scale evaluation for Canva’s generative models so design quality is measurable and trusted. Own the evaluation strategy across design, image, video, audio, and agentic workflows; validate metric correlations with human preferences and downstream product outcomes; and build shared evaluation layers for cross-region consistency. You’ll troubleshoot evaluation anomalies in production, partner with model and design generation teams to shape training/inference, and mentor research talent.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Canva
Canva
2 days ago

Principal Research Scientist - Evaluations

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Define and scale evaluation for Canva’s generative models so design quality is measurable and trusted. Own the evaluation strategy across design, image, video, audio, and agentic workflows; validate metric correlations with human preferences and downstream product outcomes; and build shared evaluation layers for cross-region consistency. You’ll troubleshoot evaluation anomalies in production, partner with model and design generation teams to shape training/inference, and mentor research talent.
Location: San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Sr. Manager level

Key Responsibilities

  • •Establish evaluation gates that inform launch decisions and own judgement calls when signals are ambiguous.
  • •Diagnose anomalous evaluation results during production training runs and clearly communicate findings.
  • •Partner with Design Generation, Foundation Models, and Agents teams so evaluation shapes training and inference.
  • •Work with designers, creators, and product teams to convert subjective creative judgement into measurable criteria.
  • •Mentor senior research scientists and engineers and raise evaluation craft across the group.

Pay and Benefits

Salary: USD 310,000 - 420,000 annually
Equity and Bonus:Equity
Perks:Health Insurance401kParental LeaveEquity

Key Requirements

  • •Track record defining evaluation frameworks and standards for frontier GenAI evaluation and measurement systems.
  • •Experience linking evaluation metrics to downstream user or business outcomes, including diagnosing metric divergence.
  • •Ability to turn subjective human judgement into reliable, objective evaluation signals via rubric design, human data pipelines, and model training.
  • •Strong grounding in multimodal generative models (diffusion, transformers, VLMs, and MLLMs) with enough depth to know where evaluation breaks.
  • •Experience with reward modelling, preference learning, or alignment methods using human feedback.
Experience:Generative AIMultimodalRLHFAlignment
Skills:LeadershipCommunicationProblem-solvingMentorship
Languages:English (US)
Tech Stack:DiffusionTransformersVLMsMLLMsRLHFRLAIF

Company Brief

Canva
Canva is an online visual communications platform that provides drag-and-drop design tools, templates, and media assets for creating presentations, social media graphics, videos, documents and more, serving individuals and large enterprises globally.
Industry: Enterprise Software
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Decacorn (USD 10B+)
Funding: Series E+
Headquarters: Surry Hills, New South Wales, Australia
Founded: 2012
Glassdoor
Glassdoor: 3.9
WebsiteLinkedInGlassdoor