Senior Technical Product Manager - AI Agents, Evals & Reliability

BJAK
United States
Workplace: RemoteFull timeFunction: Product ManagementSkills: ["Judgment","Decision-making","Prioritization","Independent execution","Collaboration"]

Own the end-to-end product and system requirements for an AI-powered proactive assistant built for everyday users. Work with ML, backend, and mobile engineers to translate model capabilities, data constraints, and evaluation results into product decisions. Define evaluation frameworks spanning offline metrics, online experiments, and human feedback, and make trade-offs across quality, latency, cost, reliability, and UX to deliver trustworthy, reliable AI behavior.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
BJAK
BJAK
2 days ago

Senior Technical Product Manager - AI Agents, Evals & Reliability

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 17 hours agoStatus: Live

Job Summary

Own the end-to-end product and system requirements for an AI-powered proactive assistant built for everyday users. Work with ML, backend, and mobile engineers to translate model capabilities, data constraints, and evaluation results into product decisions. Define evaluation frameworks spanning offline metrics, online experiments, and human feedback, and make trade-offs across quality, latency, cost, reliability, and UX to deliver trustworthy, reliable AI behavior.
Location: United States
Workplace: Remote
Employment Type: Full time
Job Function: Product Management
Seniority: Mid level

Key Responsibilities

  • •Research and define end-to-end AI system requirements from capability to behavior and user impact.
  • •Translate model capabilities, data constraints, and evaluation results into clear product and system decisions.
  • •Make hard trade-offs across quality, latency, cost, reliability, and UX.
  • •Define and evolve evaluation frameworks across offline metrics, online experiments, and human feedback.
  • •Ensure systems ship quickly, safely, and reliably with strong feedback loops and end-to-end product quality.

Key Requirements

  • •Strong CS fundamentals, including algorithms, data structures, and system design.
  • •Solid understanding of ML fundamentals and how modern AI systems behave in production.
  • •Hands-on exposure to AI-powered products, including LLM-based systems.
  • •Experience with model evaluation, prompt or pipeline iteration, and feedback loops.
  • •Significant experience owning complex, technical products end-to-end with strong judgment in ambiguous, fast-moving environments.
Experience:AILLMConsumer products
Skills:JudgmentDecision-makingPrioritizationIndependent executionCollaboration
Tech Stack:LLMMLAI agents

Company Brief

BJAK
Bjak is a Malaysia-based insurtech that operates an online insurance comparison and distribution platform across SEA, simplifying purchase and servicing of motor and life insurance using digital tools and AI-enabled services.
Industry: InsurTech
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Headquarters: Selangor, Malaysia
Founded: 2019
Glassdoor
Glassdoor: 2.5
WebsiteLinkedInGlassdoor