Software Engineer, LLM Infrastructure

Fireworks AI
San Mateo
Full timeUSD 175,000 - 220,000 annuallyFunction: Software EngineeringExperience: 3+ yearsEducation: bachelorsSkills: ["Collaboration","Code review","Problem-solving"]

Design and build scalable backend infrastructure for a generative AI platform, supporting distributed training, inference, and data pipelines. Own core services like an LLM CI/CD pipeline, control plane, and model serving systems while driving performance, cost efficiency, and reliability improvements across compute, storage, and networking. Collaborate closely with cloud infrastructure, product, and performance teams to translate research and product needs into dependable infrastructure.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Fireworks AI
Fireworks AI
9 months ago

Software Engineer, LLM Infrastructure

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 4 hours agoStatus: Live
Reposted: similar role first listed 1 week ago

Job Summary

Design and build scalable backend infrastructure for a generative AI platform, supporting distributed training, inference, and data pipelines. Own core services like an LLM CI/CD pipeline, control plane, and model serving systems while driving performance, cost efficiency, and reliability improvements across compute, storage, and networking. Collaborate closely with cloud infrastructure, product, and performance teams to translate research and product needs into dependable infrastructure.
Location: San Mateo
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Design and develop scalable backend infrastructure for distributed training, inference, and data pipelines.
  • •Build and maintain core backend services including an LLM CI/CD pipeline, control plane, and model serving systems.
  • •Improve performance, cost efficiency, and reliability across compute, storage, and networking layers.
  • •Develop frameworks and safeguards to ensure strong model quality.
  • •Collaborate with performance, training, and product teams and participate in code reviews and CI/CD.

Pay and Benefits

Salary: USD 175,000 - 220,000 annually
Equity and Bonus:Equity

Key Requirements

  • •Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience).
  • •3 years of software engineering experience focused on infrastructure or machine learning systems.
  • •Strong programming skills in Python, Go, or a similar language.
  • •Proven experience building ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes).
  • •Basic understanding of LLM concepts such as context length, disaggregated prefill, and KV cache memory estimation.
Experience:3+ yearsMachine learningML infrastructureMLOps
Education:Bachelor's
Skills:CollaborationCode reviewProblem-solving
Tech Stack:PythonGoPyTorchMLflowVertex AISageMakerKubernetesVLLMSglangTRT-LLM

Company Brief

Fireworks AI
Develops AI-driven tools to generate and optimize visual marketing content for brands and creators, automating production of short-form videos and multimedia assets for social platforms to improve engagement and scale creative workflows.
Industry: SaaS
Website