Senior AI Runtime Engineer

Modular
United States, Canada
Workplace: RemoteFull timeUSD 216,000 - 324,000 annuallyFunction: Data Science & Machine LearningExperience: 5+ yearsSkills: ["Team-oriented","Collaboration","Problem-solving","Creativity","Curiosity"]

Own and evolve a high-performance AI runtime that delivers state-of-the-art CPU/GPU efficiency across diverse model families. Design cross-stack optimizations (including caching and data locality), port the runtime to new hardware platforms, and build an API to streamline integration. Collaborate with compiler, kernels, serving, and models teams, engage with customers on performance needs, and help define automated performance analysis and benchmarking.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Modular
Modular
14 hours ago

Senior AI Runtime Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 3 hours agoStatus: Live

Job Summary

Own and evolve a high-performance AI runtime that delivers state-of-the-art CPU/GPU efficiency across diverse model families. Design cross-stack optimizations (including caching and data locality), port the runtime to new hardware platforms, and build an API to streamline integration. Collaborate with compiler, kernels, serving, and models teams, engage with customers on performance needs, and help define automated performance analysis and benchmarking.
Location: United States, Canada
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Design and develop runtime and cross-stack optimizations to improve CPU and GPU efficiency, addressing CPU overhead, caching, and data locality.
  • •Port the Modular runtime stack to new hardware platforms and develop an API to streamline the process.
  • •Collaborate with compiler, kernels, serving, and models teams to deliver state-of-the-art end-to-end performance on CPU and GPU hardware.
  • •Collaborate with the customer success team to understand performance requirements and use cases.
  • •Collaborate with tooling and infrastructure teams to design systems for automated performance analysis and benchmarking.
Travel: Low travel

Pay and Benefits

Salary: USD 216,000 - 324,000 annually
Equity and Bonus:Equity
Perks:Health InsurancePaid LeaveEquityLearning BudgetWellness Stipend

Key Requirements

  • •5+ years of experience working on high-performance computing systems.
  • •Experience in C++ programming and complex software systems.
  • •Experience with CPU or GPU runtime optimizations and performance analysis on CPUs, GPUs, or AI accelerators.
  • •Proficiency with one or more profiling tools (CPU or GPU).
  • •Creativity and curiosity for solving complex problems, with a team-oriented attitude.
Experience:5+ yearsHigh-performance computing
Skills:Team-orientedCollaborationProblem-solvingCreativityCuriosity
Tech Stack:C++CPUGPUMLIRLLVMMojo

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Modular
Provides infrastructure and developer tools to build, deploy, and scale large AI models and foundation-model applications, including model hosting, orchestration, and SDKs to accelerate AI product development.
Industry: AI & Machine Learning
Website