Box

2 weeks ago

Product Manager II, AI Evaluations

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Own AI evaluations for Box Agents by running rigorous tests on frontier models and continuously improving the evaluation methodology, datasets, and cost-appropriate graders. Build synthetic data pipelines, turn production usage into actionable insights, and lead end-to-end model launches. Partner with Product Marketing and engineering on launch narratives, demos, and external relationships with model providers to determine production readiness, compliance posture, and model deprecations.

Location: Redwood City

Workplace: Hybrid

Employment Type: Full time

Job Function: Product Management

Seniority: Mid level

Key Responsibilities

•Run rigorous evaluations across frontier models (OpenAI, Anthropic, Google/Gemini) and mature the evaluation methodology and process.
•Define evaluation datasets and measurable criteria tied to end-user needs, including robust, cost-appropriate graders.
•Drive synthetic data generation and maintain high-quality, reliable, scalable data-generation pipelines.
•Own online evaluations by translating real production usage into insights on product usage.
•Lead end-to-end model launches, including external narratives, provider relationships, and decisions on production readiness, compliance posture, pricing input, and model deprecations.

Pay and Benefits

Salary: USD 160,500 - 200,500 annually

Perks:Equity

Key Requirements

•Minimum of 2 years of product management experience.
•Deep understanding of the AI landscape and frontier models, with comfort in technical conversations with model providers.
•Statistically literate and hands-on in code (Python, SQL, notebooks) to build evals, run analyses, and prototype data pipelines.
•Ability to drive cross-functional launches across Product Marketing, Communications, Developer Relations, and Engineering.
•Excellent written and verbal communication to present complex technical topics simply to senior stakeholders.

Experience:2+ yearsAI/MLProduct management

Skills:High agencyCommunicationCross-functional collaborationAmbiguity handlingTechnical problem-solving

Languages:English

Tech Stack:PythonSQLNotebooksOpenAIAnthropicGoogle/Gemini

Company Brief

Box

Provides cloud content management, file sharing, and collaboration services for businesses, enabling secure content storage, workflow automation, and integrations with enterprise apps to manage and share files across organizations.

Industry: Enterprise Software

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: Redwood City, United States

Founded: 2005

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

Box