Senior Technical Product Manager - AI Agents, Evals & Reliability

BJAK
Seoul
Workplace: HybridFull timeFunction: Product ManagementSkills: ["System design thinking","Strong judgment","Decision-making","Discipline","Collaboration"]

Own end-to-end product quality for an AI assistant used by billions, translating AI capabilities and evaluation results into system and product decisions. Drive requirements from capability to user impact, define evaluation frameworks (offline metrics, online experiments, and human feedback), and make trade-offs across quality, latency, cost, reliability, and UX. Partner closely with ML, backend, and mobile engineers to ship reliably with fast feedback loops and strong judgment in ambiguity.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
BJAK
BJAK
1 day ago

Senior Technical Product Manager - AI Agents, Evals & Reliability

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 17 hours agoStatus: Live

Job Summary

Own end-to-end product quality for an AI assistant used by billions, translating AI capabilities and evaluation results into system and product decisions. Drive requirements from capability to user impact, define evaluation frameworks (offline metrics, online experiments, and human feedback), and make trade-offs across quality, latency, cost, reliability, and UX. Partner closely with ML, backend, and mobile engineers to ship reliably with fast feedback loops and strong judgment in ambiguity.
Location: Seoul
Workplace: Hybrid
Employment Type: Full time
Job Function: Product Management
Seniority: Mid level

Key Responsibilities

  • •Research and define end-to-end AI system requirements from capability to user impact.
  • •Translate model capabilities, data constraints, and evaluation results into product and system decisions.
  • •Make trade-offs across quality, latency, cost, reliability, and UX.
  • •Define and evolve evaluation frameworks across offline metrics, online experiments, and human feedback.
  • •Own product quality end-to-end, driving safe, reliable shipping with strong feedback loops.

Key Requirements

  • •Strong computer science fundamentals, including algorithms, data structures, and system design.
  • •Solid understanding of ML fundamentals and how modern AI systems behave in production.
  • •Hands-on exposure to AI-powered products, including LLM-based systems.
  • •Experience with model evaluation, prompt or pipeline iteration, and feedback loops.
  • •Significant experience owning complex, technical products end-to-end with strong judgment in ambiguous environments.
Experience:AIMachine learningLLMAI agentsConsumer products
Skills:System design thinkingStrong judgmentDecision-makingDisciplineCollaboration
Tech Stack:Machine learningML fundamentalsLLMAI agentsEvaluation frameworksOffline metricsOnline experimentsHuman feedbackPrompt iterationPipeline iteration

Company Brief

BJAK
Bjak is a Malaysia-based insurtech that operates an online insurance comparison and distribution platform across SEA, simplifying purchase and servicing of motor and life insurance using digital tools and AI-enabled services.
Industry: InsurTech
Company Size: Medium (51 to 250 employees)
Growth: Growth Stage Startup
Headquarters: Selangor, Malaysia
Founded: 2019
Glassdoor
Glassdoor: 2.5
WebsiteLinkedInGlassdoor