Data Annotation Specialist, Generalist

Cohere
Calgary, Winnipeg, Vancouver, Halifax, Edmonton, Toronto, Victoria, Montreal, Canada
Workplace: RemoteContractCAD 30+ hourlyFunction: OtherExperience: 1+ yearsSkills: ["Analytical skills","Judgment","Attention to detail","Reading comprehension","Time management"]

Review, rank, and justify model outputs for data annotation projects that improve large language models for accuracy, helpfulness, tone, and safety. Stress-test and document adversarial failure cases, author prompts and exemplars, and apply evolving rubrics and taxonomies. Annotate and correct multimodal and structured data (text, image, JSON, CSV/TSV, Markdown), participate in calibration, and report model performance trends in a remote, Canada-only independent contractor role.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Cohere
Cohere
1 month ago

Data Annotation Specialist, Generalist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 3 hours agoStatus: Live

Job Summary

Review, rank, and justify model outputs for data annotation projects that improve large language models for accuracy, helpfulness, tone, and safety. Stress-test and document adversarial failure cases, author prompts and exemplars, and apply evolving rubrics and taxonomies. Annotate and correct multimodal and structured data (text, image, JSON, CSV/TSV, Markdown), participate in calibration, and report model performance trends in a remote, Canada-only independent contractor role.
Location: Calgary, Winnipeg, Vancouver, Halifax, Edmonton, Toronto, Victoria, Montreal, Canada
Workplace: Remote
Employment Type: Contract
Seniority: Mid level

Key Responsibilities

  • •Evaluate and rank model outputs using preference and comparison tasks, writing clear justifications aligned to guidelines for accuracy, helpfulness, tone, and safety.
  • •Stress-test models adversarially to surface failure modes, unsafe behavior, and capability gaps, and document reproducible cases for engineering and research teams.
  • •Create datasets by authoring prompts, responses, and exemplars; edit machine- or human-written outputs to spec.
  • •Build and apply rubrics and taxonomies to produce consistent, structured annotations across task types.
  • •Annotate and correct multimodal data (text, image, and structured formats), participate in calibration checks, and report model performance trends.

Pay and Benefits

Salary: CAD 30 hourly

Key Requirements

  • •1+ years of experience in AI data annotation, LLM evaluation, content moderation, research, or a related analytical role, with exposure to quality assurance and/or preference ranking.
  • •Apply detailed guidelines to complex, ambiguous content using strong contextual and sociocultural judgment, sensitivity to nuance, tone, and register.
  • •Comfort flagging unclear edge cases rather than guessing, and working on novel experimental tasks as definitions evolve.
  • •Excellent written English and strong reading comprehension, with the ability to justify evaluations and write clear prompts and exemplars.
  • •Comfort working with annotation platforms and structured formats such as JSON, CSV/TSV, Markdown, XML, and YAML.
Experience:1+ yearsAI data annotationLLM evaluationContent moderationResearch
Skills:Analytical skillsJudgmentAttention to detailReading comprehensionTime management
Languages:English
Tech Stack:JSONCSVTSVMarkdownXMLYAML

Company Brief

Cohere
Builds security-first foundation models and enterprise AI products (LLMs, retrieval, agent platforms) for regulated industries, enabling customizable, private deployments across cloud and on-premises for real-world business applications.
Industry: AI & Machine Learning
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: Toronto, Canada
Founded: 2019
Glassdoor
Glassdoor: 2.9
WebsiteLinkedInGlassdoor