Applied AI Engineer

Monte Carlo
United States
Workplace: RemoteFull timeUSD 180,000 - 240,000 annuallyFunction: Data Science & Machine LearningSkills: ["Problem-solving","Experimentation","Ownership","Cross-functional partnership","Shipping"]

Build end-to-end products for agent observability, taking ambiguous problems from research and prototyping through production deployment. Design and ship agent-powered capabilities like root-cause analysis, incident triage, and monitor generation, and create the eval infrastructure (golden datasets, regression suites, offline/online scoring) that makes these systems safe to iterate. Own retrieval/context pipelines and production instrumentation, partnering with data science and PM to set quality and behavioral standards for trustworthy agents.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Monte Carlo
Monte Carlo
1 day ago

Applied AI Engineer

âś“ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 9 hours agoStatus: Live

Job Summary

Build end-to-end products for agent observability, taking ambiguous problems from research and prototyping through production deployment. Design and ship agent-powered capabilities like root-cause analysis, incident triage, and monitor generation, and create the eval infrastructure (golden datasets, regression suites, offline/online scoring) that makes these systems safe to iterate. Own retrieval/context pipelines and production instrumentation, partnering with data science and PM to set quality and behavioral standards for trustworthy agents.
Location: United States
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Take open-ended problems end-to-end from research and prototyping through production, stopping approaches that don’t work early.
  • •Design and ship agent-powered features (root-cause analysis, incident triage, monitor generation) and integrate them into the platform.
  • •Build eval infrastructure to safely change systems, including golden datasets, regression suites, and offline/online scoring.
  • •Own retrieval and context pipelines plus production instrumentation (traces, failure taxonomies, cost/latency budgets) to close the quality loop.
  • •Partner with data science and PM on detection quality, experiment design, and defining what agents should do versus what they can do.

Pay and Benefits

Salary: USD 180,000 - 240,000 annually
Equity and Bonus:Equity
Perks:Remote WorkEquity

Key Requirements

  • •Built agents in production with real autonomy and internal loops, including keeping them running once real users arrive.
  • •Owned an eval framework for non-deterministic agents (golden datasets, regression suites, offline/online scoring) and monitored performance after launch.
  • •Strong Python daily and backend/model-layer knowledge sufficient to reason about LLM behavior.
  • •Works from ambiguous problems: designs experiments, builds the smallest testable version, and ships what works.
  • •Uses AI tools daily (e.g., Claude) for code and research; backend/model-layer work only (no frontend).
Experience:AI agentsAgent observabilityLLMsData/MLProduction systems
Skills:Problem-solvingExperimentationOwnershipCross-functional partnershipShipping
Tech Stack:PythonClaudeLLMsRAGMCPSnowflakeDatabricksDbtAirflow

Company Brief

Monte Carlo
Provides an end-to-end data and AI observability platform that detects, triages, and resolves data incidents to improve data reliability for enterprise analytics and ML systems.
Industry: Data Infrastructure
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: San Francisco, United States
Founded: 2019
Glassdoor
Glassdoor: 4.5
WebsiteLinkedInGlassdoor