AI Engineer, Product

Mistral
Paris
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningExperience: 3-4 yearsSkills: ["Clear communication","Autonomy","Product mindset"]

Improve AI-powered product features by owning your domain’s AI quality end-to-end. You’ll design and run evaluations, define success and safety metrics, and operate prompt/orchestration iterations. Run A/B tests across prompts and model configurations, set up observability for LLM calls, and manage model releases with canary/shadow traffic and SLO-based rollbacks. Partner with Science on regressions, post-mortems, and measurable improvements to quality, latency, safety, and reliability.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Mistral
Mistral
1 month ago

AI Engineer, Product

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 6 hours agoStatus: Live
Reposted: similar role first listed 1 month ago

Job Summary

Improve AI-powered product features by owning your domain’s AI quality end-to-end. You’ll design and run evaluations, define success and safety metrics, and operate prompt/orchestration iterations. Run A/B tests across prompts and model configurations, set up observability for LLM calls, and manage model releases with canary/shadow traffic and SLO-based rollbacks. Partner with Science on regressions, post-mortems, and measurable improvements to quality, latency, safety, and reliability.
Location: Paris
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Design and run evaluations for product areas using reference tests, heuristics, and model-graded checks.
  • •Define and track key metrics (task success, helpfulness, hallucination proxies, safety flags, latency, cost).
  • •Own prompt and orchestration design by writing, testing, and iterating on prompts and system prompts.
  • •Run A/B tests on prompts, models, and configurations and decide rollouts/rollbacks from data.
  • •Set up observability for LLM calls and operate model releases using canary/shadow traffic and regression detection.

Pay and Benefits

Perks:Health InsuranceParental LeaveRetirementRelocation SupportWellness StipendMeal AllowanceCommuter Benefits

Key Requirements

  • •3-4 years of experience in ML engineering moving toward product or software engineering with real AI/ML production experience.
  • •Strong TypeScript or Python skills.
  • •Production LLM experience including prompts, tool/function calling, and system prompts.
  • •Hands-on experience with evals and A/B testing, including designing metrics.
  • •Comfort implementing directly in product code (not only notebooks), with observability skills like logging and tracing.
Experience:3-4 yearsAI/MLLLMObservabilityExperimentation
Skills:Clear communicationAutonomyProduct mindset
Tech Stack:TypeScriptPythonLLM callsStructured loggingTracingDashboardsAlertsA/B testing

Company Brief

Mistral
Develops state-of-the-art large language models and AI systems, offering models and developer tools for natural language understanding, generation, and enterprise AI integrations. Focuses on open research and production-ready model deployments.
Industry: AI & Machine Learning
Company Size: Medium (51 to 250 employees)
Growth: Early Stage Startup
Funding: Seed
Headquarters: Paris, France
Founded: 2023
WebsiteLinkedIn