Senior Technical Program Manager, Machine Learning Infrastructure

Cohere
Canada, Boston, Chicago, New York, Miami, New Jersey, Atlanta, Washington, Ottawa, Montreal
Workplace: RemoteFull timeFunction: Program & Project Management (PMO)Experience: 5+ yearsSkills: ["Communication","Leadership","Problem-solving","Cross-functional collaboration","Stakeholder management"]

Technical Program Manager for Machine Learning Infrastructure at a remote-friendly, global AI company. You will own the program portfolio for model inference, serving, efficiency, and endpoints, coordinating across Modeling and customer-facing teams, driving process improvements, prioritizing competing programs, and delivering updates to engineering and leadership. You’ll partner with senior leads, scale infrastructure for internal and external users, and help drive Cohere’s ML initiatives.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Cohere
Cohere
2 months ago

Senior Technical Program Manager, Machine Learning Infrastructure

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 23 hours agoStatus: Live

Job Summary

Technical Program Manager for Machine Learning Infrastructure at a remote-friendly, global AI company. You will own the program portfolio for model inference, serving, efficiency, and endpoints, coordinating across Modeling and customer-facing teams, driving process improvements, prioritizing competing programs, and delivering updates to engineering and leadership. You’ll partner with senior leads, scale infrastructure for internal and external users, and help drive Cohere’s ML initiatives.
Location: Canada, Boston, Chicago, New York, Miami, New Jersey, Atlanta, Washington, Ottawa, Montreal
Workplace: Remote
Employment Type: Full time
Job Function: Program & Project Management (PMO)

Key Responsibilities

  • •Manage the program portfolio covering inference, efficiency, serving, and endpoints to guarantee that Cohere’s infrastructure continues to scale for a rapidly expanding user base of internal and external users
  • •Lead the end-to-end coordination and execution across the program, with a particular focus on cross-functional collaboration with Modeling and customer-facing teams
  • •Identify pain points and establish processes to ensure that the team can focus on development, while meeting needs of internal and external users of the models, and improving engineering best practices
  • •Build a strong culture around continuous improvement, such as liaising with incident management leads, ensuring that problems are accurately root caused, and that required fixes are provided in a timely manner
  • •Manage various overlapping projects and programs, ruthlessly prioritizing asks, to ensure that the company’s top priorities are met

Pay and Benefits

Perks:Health InsuranceMeal AllowanceRrsp MatchingHome OfficePaid LeaveParental Leave

Key Requirements

  • •5+ years of Technical Program Management experience focusing on Machine Learning Infrastructure, specifically covering areas such as model inference, serving, efficiency, and endpoints design & implementation
  • •In-depth technical knowledge around ML infrastructure design and implementation, as well as engineering best practices, supporting a balance of velocity and structure in rapidly growing organizations
  • •Experience working in a chaotic, fast paced, low structure environment. We need an EPM who is pragmatic, can roll up their sleeves when needed, and can function at the tactical and strategic level to do whatever it takes to make Cohere’s models succeed
  • •Optional, but strongly preferred: hands-on technical experience within ML programs, with reputation as a subject matter expert who brings engineering best practices and structure to improve the program
  • •Optional, but preferred: experience working with ML teams and understanding how ML models are built to advocate for infrastructure and modeling needs
Experience:5+ yearsMachine learningInfrastructureMLOps
Skills:CommunicationLeadershipProblem-solvingCross-functional collaborationStakeholder management
Languages:English
Tech Stack:ML infrastructureInferenceServingEndpoints

Company Brief

Cohere
Builds security-first foundation models and enterprise AI products (LLMs, retrieval, agent platforms) for regulated industries, enabling customizable, private deployments across cloud and on-premises for real-world business applications.
Industry: AI & Machine Learning
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series D
Headquarters: Toronto, Canada
Founded: 2019
Glassdoor
Glassdoor: 2.9
WebsiteLinkedInGlassdoor