Assessment Design Lead

Speak
United States
Workplace: RemoteFull timeUSD 100,000 - 140,000 annuallyFunction: Design (Product/UX/UI/Visual)Experience: 4+ yearsSkills: ["Ownership","Communication","Judgment","Problem-solving","Collaboration"]

Define and sign off on Speak’s language assessment strategy across curriculum mastery, proficiency tests, and onboarding placement. Own assessment constructs, rubrics, and blueprints aligned to CEFR-like proficiency standards, and ensure validity and fairness through quality-bar signoff and bias audits. Run validity evidence plans using criterion studies and partner tightly with Product and ML engineers on scoring, calibration, and feedback generation to improve speaking efficacy at scale.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Speak
Speak
21 hours ago

Assessment Design Lead

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Define and sign off on Speak’s language assessment strategy across curriculum mastery, proficiency tests, and onboarding placement. Own assessment constructs, rubrics, and blueprints aligned to CEFR-like proficiency standards, and ensure validity and fairness through quality-bar signoff and bias audits. Run validity evidence plans using criterion studies and partner tightly with Product and ML engineers on scoring, calibration, and feedback generation to improve speaking efficacy at scale.
Location: United States
Workplace: Remote
Employment Type: Full time
Job Function: Design (Product/UX/UI/Visual)

Key Responsibilities

  • •Own assessment design: define what’s measured, why, and how often across curriculum mastery, proficiency tests, and placement tests, starting with the Proficiency Test.
  • •Define constructs and build rubric/blueprint layers so item writers can generate items and ML engineers can build grading models.
  • •Own validity and quality bar: sign off on content validity, define passing/mastery operationally, and audit items/rubrics for bias and gaming resistance.
  • •Design and run a validity evidence plan using concurrent/criterion studies benchmarked to external proficiency measures.
  • •Partner with Product and ML on automated scoring, calibration, and feedback generation, co-evolving constructs and model outputs.

Pay and Benefits

Salary: USD 100,000 - 140,000 annually

Key Requirements

  • •4+ years designing rubrics, blueprints, and item specs for a real, shipped language assessment product (or equivalent psychometric measurement depth).
  • •Deep familiarity with language proficiency frameworks such as CEFR (or ACTFL, IELTS/TOEFL band descriptors).
  • •Ability to design for fairness across learner populations, including differential item functioning considerations.
  • •Translate qualitative constructs (e.g., pronunciation quality) into concrete, buildable scoring constructs and technical specs.
  • •Comfort with quantitative rigor behind rubrics/rater systems (e.g., inter-rater reliability, classical test theory, basic IRT concepts).
Experience:4+ yearsLanguage learningEdTechAIPsychometrics
Skills:OwnershipCommunicationJudgmentProblem-solvingCollaboration
Tech Stack:AIMLCEFRACTFLIELTSTOEFLIRT

Company Brief

Speak
Provides conversational English practice through live tutors and AI-enhanced lessons, offering personalized language coaching via mobile and web to help professionals and learners improve speaking skills and workplace communication.
Industry: EdTech
Website