NVIDIA

1 day ago

Senior System Software Engineer - Local AI

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 25 minutes agoStatus: Live

Job Summary

Build and optimize an on-device AI inference stack for RTX, RTX Pro, and DGX systems, focusing on high-performance local inference, low latency, and efficient memory use. Design inference runtimes and execution stacks across LLMs and multimodal workloads, leveraging frameworks and GPU toolchains. Perform end-to-end model, data pipeline, and runtime optimization (quantization, pruning, sparsity, distillation) plus system-level debugging and performance-accuracy trade-off analysis to ensure production readiness.

Location: Pune

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Partner with NVIDIA software, research, architecture, and product teams to align strategies and technical needs for AI on RTX and DGX PCs.
•Build and optimize a local AI inference stack for RTX, RTX Pro, and DGX GPUs across hardware architectures, focusing on performance, stability, and scalability.
•Architect and develop inference runtimes and execution stacks supporting LLMs and multimodal workloads, using frameworks including Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX.
•Optimize AI models, data pipelines, and inference runtimes across current and next-generation GPU architectures, applying techniques like quantization, pruning, sparsity, and distillation.
•Perform system-level debugging and performance optimization with performance–accuracy trade-off analysis; develop infrastructure for sweeps and establish engineering guidelines for production readiness.

Key Requirements

•5+ years of experience with a Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or a related field (or equivalent experience).
•Excellent C++ programming and debugging skills, with strong understanding of data structures, algorithms, and machine learning.
•Proven experience with AI inferencing pipelines and applications using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT.
•Deep interest in inference backends and runtime internals including scheduling, memory management, KV-cache behavior, graph execution, quantization, and hardware-aware optimization.
•Strong analytical and problem-solving skills, including the ability to multitask effectively in a dynamic environment.

Experience:5+ yearsAIMachine learningInferenceOn-device AI

Education:Bachelor's in Computer Science, Software Engineering, Mathematics (or related field)

Skills:DebuggingProblem-solvingCommunicationAnalytical thinkingCollaboration

Tech Stack:C++Llama.cppVLLMPyTorchWinMLDXCGCTensorRTCUDAKV-cacheQuantizationPruningSparsityDistillationGraph executionSchedulingMemory managementVulkanDirectXRTxDGX

Company Brief

NVIDIA

Designs and manufactures GPUs, AI accelerators, and system-on-chip products for gaming, data centers, professional visualization, and automotive markets, enabling advanced graphics, AI, and high-performance computing solutions worldwide.

Industry: Electronics Manufacturing

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: Santa Clara, United States

Founded: 1993

Glassdoor: 4.3

Website LinkedIn Glassdoor

Loading job details...

Preparing the role view and application actions.

NVIDIA

1 day ago

Senior System Software Engineer - Local AI

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 25 minutes agoStatus: Live

Job Summary

Location: Pune

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Partner with NVIDIA software, research, architecture, and product teams to align strategies and technical needs for AI on RTX and DGX PCs.
•Build and optimize a local AI inference stack for RTX, RTX Pro, and DGX GPUs across hardware architectures, focusing on performance, stability, and scalability.
•Architect and develop inference runtimes and execution stacks supporting LLMs and multimodal workloads, using frameworks including Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX.
•Optimize AI models, data pipelines, and inference runtimes across current and next-generation GPU architectures, applying techniques like quantization, pruning, sparsity, and distillation.
•Perform system-level debugging and performance optimization with performance–accuracy trade-off analysis; develop infrastructure for sweeps and establish engineering guidelines for production readiness.

Key Requirements

•5+ years of experience with a Bachelor’s, Master’s, or PhD in Computer Science, Software Engineering, Mathematics, or a related field (or equivalent experience).
•Excellent C++ programming and debugging skills, with strong understanding of data structures, algorithms, and machine learning.
•Proven experience with AI inferencing pipelines and applications using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT.
•Deep interest in inference backends and runtime internals including scheduling, memory management, KV-cache behavior, graph execution, quantization, and hardware-aware optimization.
•Strong analytical and problem-solving skills, including the ability to multitask effectively in a dynamic environment.

Experience:5+ yearsAIMachine learningInferenceOn-device AI

Education:Bachelor's in Computer Science, Software Engineering, Mathematics (or related field)

Skills:DebuggingProblem-solvingCommunicationAnalytical thinkingCollaboration

Tech Stack:C++Llama.cppVLLMPyTorchWinMLDXCGCTensorRTCUDAKV-cacheQuantizationPruningSparsityDistillationGraph executionSchedulingMemory managementVulkanDirectXRTxDGX

Company Brief

NVIDIA

Industry: Electronics Manufacturing

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: Santa Clara, United States

Founded: 1993

Glassdoor: 4.3

Website LinkedIn Glassdoor