Manager, Large Language Model Inference

NVIDIA
Santa Clara
Workplace: HybridFull timeUSD 184,000 - 356,500 annuallyFunction: Software EngineeringExperience: 7+ yearsEducation: mastersSkills: ["Leadership","Communication","Mentoring","Problem-solving","Teamwork"]

Engineering Manager leading a team focused on LLM/VLM/VLA inference software, driving kernel development, runtime optimizations, and production-ready libraries for NVIDIA’s next-generation enterprise and edge hardware. You’ll design, integrate cutting-edge technologies, mentor engineers, and coordinate with researchers and GPU architects to ship high-performance, scalable inference runtimes across distributed teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
NVIDIA
NVIDIA
6 months ago

Manager, Large Language Model Inference

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 1 hour agoStatus: Live

Job Summary

Engineering Manager leading a team focused on LLM/VLM/VLA inference software, driving kernel development, runtime optimizations, and production-ready libraries for NVIDIA’s next-generation enterprise and edge hardware. You’ll design, integrate cutting-edge technologies, mentor engineers, and coordinate with researchers and GPU architects to ship high-performance, scalable inference runtimes across distributed teams.
Location: Santa Clara
Workplace: Hybrid
Employment Type: Full time
Job Function: Software Engineering
Seniority: Sr. Manager level

Key Responsibilities

  • •Lead and grow a team responsible for specialized kernel development, runtime optimizations, and frameworks for LLM inference.
  • •Drive the design, development, and delivery of production inference software for NVIDIA’s enterprise and edge hardware platforms.
  • •Integrate cutting-edge technologies developed at NVIDIA and provide an intuitive developer experience for LLM deployment.
  • •Lead software development execution, with responsibility for project planning, milestone delivery, and cross-functional coordination.

Pay and Benefits

Salary: USD 184,000 - 356,500 annually
Equity and Bonus:Equity
Perks:Equity

Key Requirements

  • •MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, or a related technical field.
  • •7+ overall years of software engineering experience, including 3+ years of technical leadership experience.
  • •Proven ability to lead and scale high-performing engineering teams, especially across distributed and cross-functional groups.
  • •Strong background in C++ or Python, with expertise in software design and delivering production-quality software libraries.
  • •Demonstrated expertise in large language models (LLM) and/or vision language models (VLM).
Experience:7+ yearsAILLMInferenceGPU
Education:Master's
Skills:LeadershipCommunicationMentoringProblem-solvingTeamwork
Tech Stack:C++PythonCUDATensorRTLLMVLLMSGLangGPU

Company Brief

NVIDIA
Designs and manufactures GPUs, AI accelerators, and system-on-chip products for gaming, data centers, professional visualization, and automotive markets, enabling advanced graphics, AI, and high-performance computing solutions worldwide.
Industry: Electronics Manufacturing
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Santa Clara, United States
Founded: 1993
Glassdoor
Glassdoor: 4.3
WebsiteLinkedInGlassdoor