Lead Assessment Developer

Outsmart
Los Angeles, San Francisco
Workplace: OnsiteFull timeUSD 120,000 - 180,000 annuallyFunction: Software EngineeringExperience: 3+ yearsEducation: mastersSkills: ["Communication","Ownership","Risk identification","Working with ambiguity"]

Own the end-to-end development lifecycle for scalable, high-stakes assessments, from item authoring and field testing to standard setting, scoring, and reporting. Partner with psychometricians, faculty, product managers, and engineers to build rigorous assessment systems while responsibly integrating AI into development and QA workflows. Lead SME panel recruitment and calibration, manage vendor relationships for pilots, and produce core documentation, timelines, and risk/communications for stakeholders.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Outsmart
Outsmart
4 months ago

Lead Assessment Developer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 11 hours agoStatus: Live

Job Summary

Own the end-to-end development lifecycle for scalable, high-stakes assessments, from item authoring and field testing to standard setting, scoring, and reporting. Partner with psychometricians, faculty, product managers, and engineers to build rigorous assessment systems while responsibly integrating AI into development and QA workflows. Lead SME panel recruitment and calibration, manage vendor relationships for pilots, and produce core documentation, timelines, and risk/communications for stakeholders.
Location: Los Angeles, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Manage end-to-end assessment development operations for assigned assessments.
  • •Recruit, train, calibrate, and oversee an SME network and ensure quality.
  • •Lead standard-setting studies, including method selection, panel facilitation, and reporting.
  • •Coordinate vendor and external partner relationships, including SLAs and security protocols.
  • •Run field testing and pilot operations and oversee scoring for constructed-response and performance-based assessments.

Pay and Benefits

Salary: USD 120,000 - 180,000 annually
Perks:Health InsuranceDentalVisionParental LeavePaid LeaveRemote WorkEquityMeal Allowance

Key Requirements

  • •Master’s degree (Ph.D. preferred) in educational measurement, assessment, or a related field.
  • •3+ years managing operational, high-stakes assessment lifecycles end-to-end.
  • •Experience leading standard-setting studies and managing diverse SME panels.
  • •Experience managing external vendors for field tests and pilot studies.
  • •Working knowledge of validity, reliability, and fairness standards (e.g., AERA/APA/NCME).
Experience:3+ years
Education:Master's in educational measurement, assessment
Skills:CommunicationOwnershipRisk identificationWorking with ambiguity
Tech Stack:AERAAPANCMEDOKBloom'sCTTIRTCDMDIFAI-assisted item generationAccreditation reviewsSME panels

Company Brief

Outsmart
Outsmart provides a creative analytics and optimization platform that helps brands and performance marketers improve advertising creative performance using data-driven insights, testing, and automation to increase return on ad spend across digital channels.
Industry: SaaS
Website