NVIDIA

3 months ago

Principal High-Performance LLM Training Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Seeking a deeply technical Principal Engineer to optimize large-scale AI training and post-training workloads across the full NVIDIA hardware/software stack. You will drive end-to-end performance of frontier-scale LLM workloads on thousands of GPUs, influence frameworks like PyTorch/JAX/NeMo, and shape future GPU/system/software roadmaps through rigorous analysis, modeling, and leadership across teams.

Location: Santa Clara

Employment Type: Full time

Job Function: Education & Training

Key Responsibilities

•Lead end-to-end performance analysis and optimization of innovative LLM pre-training and post-training workloads on the latest NVIDIA hardware and software platforms.
•Drive workloads closer to speed-of-light performance by identifying and removing bottlenecks across compute, memory, communication, scheduling, parallelism strategy, kernel efficiency, framework overhead, and system-level scaling.
•Develop production-quality software, tools, models, benchmarks, and analysis infrastructure that improve training performance, efficiency, and developer velocity across NVIDIA’s AI software stack.
•Build and refine performance models, workload characterizations, and simulation methodologies to guide future GPU, networking, system, and software architecture decisions.
•Mentor and provide technical leadership to engineers across the organization, helping establish best practices for large-scale AI performance analysis and optimization.

Pay and Benefits

Salary: USD 272,000 - 431,250 annually

Equity and Bonus:Equity

Key Requirements

•MS or PhD (or equivalent) in Computer Science, Electrical Engineering, Computer Engineering, or related field, with 12+ years of relevant work or research experience.
•Demonstrated principal-level technical impact in large-scale AI training systems, GPU performance optimization, distributed systems, high-performance computing, ML frameworks, compilers/runtimes, or hardware/software co-design.
•Deep hands-on experience analyzing and optimizing performance of large-scale deep learning workloads, especially transformer-based models, LLM pre-training, reinforcement learning, fine-tuning, or other post-training workloads.
•Strong understanding of GPU and AI accelerator architecture from individual accelerators to datacenter-scale systems.
•Experience with distributed training techniques such as data parallelism, tensor parallelism, pipeline parallelism, expert parallelism, sequence parallelism, activation checkpointing, mixed precision training, and communication/computation overlap.

Experience:12+ yearsAIGPULLMDistributed trainingHigh-performance computing

Education:Master's

Skills:PyTorchJAXNeMoNeMo RLCUDAGPUDistributed trainingBenchmarkingProfiling

Tech Stack:PyTorchJAXNeMoNeMo RLCUDAGPUCUDA librariesFrameworks

Company Brief

NVIDIA

Designs and manufactures GPUs, AI accelerators, and system-on-chip products for gaming, data centers, professional visualization, and automotive markets, enabling advanced graphics, AI, and high-performance computing solutions worldwide.

Industry: Electronics Manufacturing

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: Santa Clara, United States

Founded: 1993

Glassdoor: 4.3

Website LinkedIn Glassdoor

Loading job details...

Preparing the role view and application actions.

NVIDIA

3 months ago

Principal High-Performance LLM Training Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Location: Santa Clara

Employment Type: Full time

Job Function: Education & Training

Key Responsibilities

•Lead end-to-end performance analysis and optimization of innovative LLM pre-training and post-training workloads on the latest NVIDIA hardware and software platforms.
•Drive workloads closer to speed-of-light performance by identifying and removing bottlenecks across compute, memory, communication, scheduling, parallelism strategy, kernel efficiency, framework overhead, and system-level scaling.
•Develop production-quality software, tools, models, benchmarks, and analysis infrastructure that improve training performance, efficiency, and developer velocity across NVIDIA’s AI software stack.
•Build and refine performance models, workload characterizations, and simulation methodologies to guide future GPU, networking, system, and software architecture decisions.
•Mentor and provide technical leadership to engineers across the organization, helping establish best practices for large-scale AI performance analysis and optimization.

Pay and Benefits

Salary: USD 272,000 - 431,250 annually

Equity and Bonus:Equity

Key Requirements

•MS or PhD (or equivalent) in Computer Science, Electrical Engineering, Computer Engineering, or related field, with 12+ years of relevant work or research experience.
•Demonstrated principal-level technical impact in large-scale AI training systems, GPU performance optimization, distributed systems, high-performance computing, ML frameworks, compilers/runtimes, or hardware/software co-design.
•Deep hands-on experience analyzing and optimizing performance of large-scale deep learning workloads, especially transformer-based models, LLM pre-training, reinforcement learning, fine-tuning, or other post-training workloads.
•Strong understanding of GPU and AI accelerator architecture from individual accelerators to datacenter-scale systems.
•Experience with distributed training techniques such as data parallelism, tensor parallelism, pipeline parallelism, expert parallelism, sequence parallelism, activation checkpointing, mixed precision training, and communication/computation overlap.

Experience:12+ yearsAIGPULLMDistributed trainingHigh-performance computing

Education:Master's

Skills:PyTorchJAXNeMoNeMo RLCUDAGPUDistributed trainingBenchmarkingProfiling

Tech Stack:PyTorchJAXNeMoNeMo RLCUDAGPUCUDA librariesFrameworks

Company Brief

NVIDIA

Industry: Electronics Manufacturing

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: Santa Clara, United States

Founded: 1993

Glassdoor: 4.3

Website LinkedIn Glassdoor