Production Engineer, LLM Serving

FuriosaAI
Seoul
Workplace: HybridFull timeFunction: Manufacturing & Production OperationsExperience: 3+ yearsEducation: bachelorsSkills: ["Communication","Collaboration","Quantitative communication","Incident response","Problem diagnosis"]

Own reliability, observability, and service-level performance for LLM serving on FuriosaAI RNGD NPUs. Improve latency, throughput, and resource efficiency using workload-driven measurement, bottleneck analysis, and rigorous benchmarks. Build SLIs/SLOs, metrics, logs, traces, dashboards, and alerts; design load tests and capacity baselines; and optimize routing, caching, and inference execution. Collaborate across infrastructure, inference, runtime, and compiler teams to ship safe deployments, autoscaling, and incident learnings.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
FuriosaAI
FuriosaAI
10 hours ago

Production Engineer, LLM Serving

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Own reliability, observability, and service-level performance for LLM serving on FuriosaAI RNGD NPUs. Improve latency, throughput, and resource efficiency using workload-driven measurement, bottleneck analysis, and rigorous benchmarks. Build SLIs/SLOs, metrics, logs, traces, dashboards, and alerts; design load tests and capacity baselines; and optimize routing, caching, and inference execution. Collaborate across infrastructure, inference, runtime, and compiler teams to ship safe deployments, autoscaling, and incident learnings.
Location: Seoul
Workplace: Hybrid
Employment Type: Full time
Job Function: Manufacturing & Production Operations
Seniority: Mid level

Key Responsibilities

  • •Operate and improve RNGD-based LLM serving services by defining SLIs/SLOs and production readiness criteria.
  • •Build and maintain observability (metrics, logs, traces, dashboards, and alerts) across the serving request path.
  • •Design benchmarks and load tests for realistic input/output lengths, concurrency, and traffic patterns to establish baselines and detect regressions.
  • •Diagnose and improve bottlenecks across routing, queueing, batching, caching, networking, and inference execution to meet latency SLOs.
  • •Automate deployments and support reliability operations including canary rollouts/rollback, on-call incident response, runbooks, and recovery automation.

Key Requirements

  • •3+ years developing and operating production services, with a Bachelor’s degree in Computer Science or a related field (or equivalent experience).
  • •Strong programming skills in Rust, C++, Python, or Go, including the ability to read and instrument production code.
  • •Hands-on experience operating services in Kubernetes, with solid knowledge of Linux, containers, networking, concurrency, and distributed systems.
  • •Understanding of LLM serving (prefill/decode, batching, KV-cache management, and request scheduling) and how it impacts performance.
  • •Experience profiling production services using telemetry/observability to identify bottlenecks across the request path and validate improvements in tail latency, throughput, and efficiency.
Experience:3+ yearsLLM servingProduction servicesKubernetesObservabilityDistributed systems
Education:Bachelor's in Computer Science
Skills:CommunicationCollaborationQuantitative communicationIncident responseProblem diagnosis
Languages:English
Tech Stack:RustC++PythonGoKubernetesLinuxContainersNetworkingIstioMooncakeLlm-dFuriosa-LLMPrometheusGrafanaOpenTelemetryGitOpsVLLMSGLangTensorRT-LLM

Company Brief

FuriosaAI
Designs and develops data‑center AI inference accelerators (RNGD) and a full stack hardware‑software platform to deliver energy‑efficient, high‑performance AI compute for enterprise and cloud customers.
Industry: Hardware Devices
Company Size: Medium (51 to 250 employees)
Growth: Scaleup
Valuation: USD 500M to 1B
Funding: Series C
Headquarters: Seoul, South Korea
Founded: 2017
Glassdoor
Glassdoor: 4.6
WebsiteLinkedInGlassdoor