Senior LLMOps Engineer

Heidi Health
Melbourne, Sydney
Workplace: HybridFull timeFunction: Solutions Engineering & Sales EngineeringSkills: ["Ownership","End-to-end problem solving"]

Build the operational layer behind Heidi’s production LLMs—deployment health monitoring, tracing from clinician sessions to exact model IDs, and incident response. Create an improvement flywheel that turns Intercom tickets and CSAT feedback into actionable model updates using AI agents, while surfacing execution traces for stakeholders. Own per-model unit economics by linking revenue to inference costs, and raise LLMOps practices (evaluation, observability, feedback) across the model team.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Heidi Health
Heidi Health
20 hours ago

Senior LLMOps Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 10 hours agoStatus: Live

Job Summary

Build the operational layer behind Heidi’s production LLMs—deployment health monitoring, tracing from clinician sessions to exact model IDs, and incident response. Create an improvement flywheel that turns Intercom tickets and CSAT feedback into actionable model updates using AI agents, while surfacing execution traces for stakeholders. Own per-model unit economics by linking revenue to inference costs, and raise LLMOps practices (evaluation, observability, feedback) across the model team.
Location: Melbourne, Sydney
Workplace: Hybrid
Employment Type: Full time
Job Function: Solutions Engineering & Sales Engineering
Seniority: Mid level

Key Responsibilities

  • •Build a deployment health dashboard with live model health metrics, monitoring, and proactive incident alerting.
  • •Create complete session-to-model lineage so degradations can be traced to affected sessions, user profiles, model IDs, impact scope, and root causes.
  • •Stand up an improvement flywheel using Intercom tickets and qualitative CSAT feedback, with AI agents identifying the exact sessions behind feedback.
  • •Retrieve and summarize full execution traces for flagged sessions to support review and action by stakeholders outside engineering.
  • •Own per-model unit economics by measuring revenue vs. inference cost and using results to guide model selection and deployment strategy.

Pay and Benefits

Perks:Learning BudgetWellness StipendHome OfficeParental LeaveFertility SupportRemote WorkEquity

Key Requirements

  • •2–3 years hands-on in an LLMOps role building observability, tracing, evaluation, and feedback systems for production LLMs through real incidents.
  • •Proven ability to ship monitoring and alerting (Datadog or similar), distributed tracing across multi-step LLM pipelines, and session/event data models that scale.
  • •Experience building with LLMs (e.g., using agents to triage feedback and match tickets to sessions).
  • •Comfort combining cost and revenue data into per-model unit economics to support leadership decisions.
  • •Senior-level ownership: design systems from ambiguous mandates and run them in production (no PhD required).
Experience:LLMOpsAI company
Skills:OwnershipEnd-to-end problem solving
Tech Stack:DatadogIntercom

Company Brief

Heidi Health
Builds an AI medical scribe and clinical productivity platform that automates documentation, form-filling, and task management to expand clinician capacity across hospitals, GP clinics and specialist services worldwide.
Industry: HealthTech
Company Size: Medium (51 to 250 employees)
Growth: Scaleup
Valuation: USD 250M to 500M
Funding: Series B
Headquarters: Melbourne, Australia
Founded: 2019
Glassdoor
Glassdoor: 4.7
WebsiteLinkedInGlassdoor