Inference Systems Engineer

Modular
United States
Workplace: HybridFull timeUSD 198,000 - 242,000 annuallyFunction: IT Operations (Systems/Network Admin)Experience: 5+ yearsSkills: ["System design","Cross-team collaboration","Ownership","Performance diagnostics","Defensive engineering"]

Own the next generation of AI inference foundations by unifying today’s inference features into robust, flexible distributed-systems abstractions. Lead cross-team architecture decisions across concurrency, data movement, and system boundaries, and build modular components with clear contracts that improve testability and accelerate future feature delivery. Work with a fast-moving in-house stack spanning from the metal to the cloud, designing extensible interfaces for open-source contributors.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Modular
Modular
1 day ago

Inference Systems Engineer

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 51 minutes agoStatus: Live

Job Summary

Own the next generation of AI inference foundations by unifying today’s inference features into robust, flexible distributed-systems abstractions. Lead cross-team architecture decisions across concurrency, data movement, and system boundaries, and build modular components with clear contracts that improve testability and accelerate future feature delivery. Work with a fast-moving in-house stack spanning from the metal to the cloud, designing extensible interfaces for open-source contributors.
Location: United States
Workplace: Hybrid
Employment Type: Full time
Job Function: IT Operations (Systems/Network Admin)
Seniority: Mid level

Key Responsibilities

  • •Lead cross-team architecture-level decisions on concurrency, data movement, and system boundaries in a fast-moving in-house stack spanning from the metal to the cloud.
  • •Develop flexible, extensible interfaces for open-source model developers and community contributors that promote best practices.
  • •Design highly modular systems with clear contracts for component ownership, testability, and scaffolding for robust agentic development.
  • •Unify underlying structures underneath inference features into distributed-systems abstractions and building blocks that help future features ship faster and more reliably.
  • •Create composable system foundations for long-tail inference reliability as new model architectures and optimization techniques emerge.
Travel: Low travel

Pay and Benefits

Salary: USD 198,000 - 242,000 annually
Equity and Bonus:Equity
Perks:Health Insurance401kPaid Leave

Key Requirements

  • •5+ years of systems programming experience focused on performance, concurrency, and distributed architectures.
  • •Strong instinct for API and abstraction design, with the ability to defend design proposals.
  • •Comfortable working in Python, Rust, Mojo, or similar.
  • •Ability to diagnose and resolve pathological performance bottlenecks in complex systems.
  • •Demonstrated ownership from implementation through production support, including defensive engineering learnings.
Experience:5+ yearsAI infrastructureML inferenceDistributed systemsHigh-performance systemsOpen-source community
Skills:System designCross-team collaborationOwnershipPerformance diagnosticsDefensive engineering
Tech Stack:PythonRustMojoVLLMSGLangTRT-LLMPOSIX APIsNUMAZero-copy transportGraph-based programmingDataflowActor modelGPU/accelerator programming

Company Brief

Modular
Provides infrastructure and developer tools to build, deploy, and scale large AI models and foundation-model applications, including model hosting, orchestration, and SDKs to accelerate AI product development.
Industry: AI & Machine Learning
Website