Modular

1 month ago

Inference Optimization Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 20 hours agoStatus: Live

Job Summary

Build and optimize a scalable inference platform for Modular Cloud, profiling real customer workloads end-to-end, applying kernel, engine, and distributed system optimizations to push performance and cost toward the Pareto frontier. Develop tooling to automate optimization loops, collaborate with engineering, product, and GTM, and publish industry-impact blog posts. Remote/work in US or Canada; in-person onboarding in Los Altos, CA.

Location: United States

Workplace: Remote

Employment Type: Full time

Job Function: Software Engineering

Key Responsibilities

•Build the optimization platform that drives inference performance of LLMs served on Modular Cloud to state of the art levels across the latest GPU and ASIC architectures.
•Shape the technical direction of Modular Cloud, delivering LLM performance on the Pareto frontier for agentic use cases and keeping it there as the landscape evolves.
•Partner closely with the GTM team to deliver highly customized LLM inference tuned to specific customer use cases, and collaborate across engineering to drive optimizations spanning the full stack, from GPU kernels to cloud infrastructure.
•Translate insights from customer engagements into technical direction for engineering teams.
•Publish blog posts on innovative approaches to LLM inference optimization that shape industry wide best practices.

Pay and Benefits

Equity and Bonus:Equity

Perks:Health Insurance401kEquityTime Off

Key Requirements

•5+ years of experience in distributed systems or performance engineering.
•A track record of building durable, reusable software tools and libraries that are adopted across teams and functions.
•Sound judgment in evaluating technical tradeoffs and setting priorities, paired with strong communication and technical leadership skills.
•Creativity and curiosity in solving complex problems, a collaborative and team oriented mindset, and alignment with our culture.
•Experience with GPU kernel programming, inference engine internals, or distributed inference architectures is helpful but not required.

Experience:5+ yearsDistributed systemsPerformance engineeringAI inferenceCloud

Skills:CommunicationLeadershipProblem-solving

Tech Stack:GPUKubernetesCloudInferenceKernelsDistributed systems

Company Brief

Modular

Provides infrastructure and developer tools to build, deploy, and scale large AI models and foundation-model applications, including model hosting, orchestration, and SDKs to accelerate AI product development.

Industry: AI & Machine Learning

Website

Loading job details...

Preparing the role view and application actions.

Modular

1 month ago

Inference Optimization Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 20 hours agoStatus: Live

Job Summary

Location: United States

Workplace: Remote

Employment Type: Full time

Job Function: Software Engineering

Key Responsibilities

•Build the optimization platform that drives inference performance of LLMs served on Modular Cloud to state of the art levels across the latest GPU and ASIC architectures.
•Shape the technical direction of Modular Cloud, delivering LLM performance on the Pareto frontier for agentic use cases and keeping it there as the landscape evolves.
•Partner closely with the GTM team to deliver highly customized LLM inference tuned to specific customer use cases, and collaborate across engineering to drive optimizations spanning the full stack, from GPU kernels to cloud infrastructure.
•Translate insights from customer engagements into technical direction for engineering teams.
•Publish blog posts on innovative approaches to LLM inference optimization that shape industry wide best practices.

Pay and Benefits

Equity and Bonus:Equity

Perks:Health Insurance401kEquityTime Off

Key Requirements

•5+ years of experience in distributed systems or performance engineering.
•A track record of building durable, reusable software tools and libraries that are adopted across teams and functions.
•Sound judgment in evaluating technical tradeoffs and setting priorities, paired with strong communication and technical leadership skills.
•Creativity and curiosity in solving complex problems, a collaborative and team oriented mindset, and alignment with our culture.
•Experience with GPU kernel programming, inference engine internals, or distributed inference architectures is helpful but not required.

Experience:5+ yearsDistributed systemsPerformance engineeringAI inferenceCloud

Skills:CommunicationLeadershipProblem-solving

Tech Stack:GPUKubernetesCloudInferenceKernelsDistributed systems

Company Brief

Modular

Industry: AI & Machine Learning

Website