Machine Learning Engineer Graduate (AML-Engine-Orchestration) - 2027 Start

ByteDance
San Jose
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningEducation: bachelorsSkills: ["Ownership","Collaboration","Problem-solving","Quantitative analysis","Experimentation"]

Build large-scale ML infrastructure for online model serving across ByteDance products, focusing on orchestration, scheduling, and resource management. You’ll design foundational platform capabilities (including Kubernetes operators and job/service lifecycle management), implement multi-tenant quota and preemption systems to improve GPU utilization, and develop serving orchestration with traffic routing, autoscaling, and disaster recovery. Work on systems that directly impact latency, availability, and infrastructure reliability.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
15 hours ago

Machine Learning Engineer Graduate (AML-Engine-Orchestration) - 2027 Start

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 15 hours agoStatus: Live

Job Summary

Build large-scale ML infrastructure for online model serving across ByteDance products, focusing on orchestration, scheduling, and resource management. You’ll design foundational platform capabilities (including Kubernetes operators and job/service lifecycle management), implement multi-tenant quota and preemption systems to improve GPU utilization, and develop serving orchestration with traffic routing, autoscaling, and disaster recovery. Work on systems that directly impact latency, availability, and infrastructure reliability.
Location: San Jose
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Graduate level

Key Responsibilities

  • •Design and build orchestration capabilities for ML platforms, including Kubernetes operators and lifecycle management for jobs, services, and stateful workloads.
  • •Build multi-tenant resource and quota systems supporting priorities, preemption, fair sharing, elasticity, and cross-cluster scheduling to improve GPU utilization and cost efficiency.
  • •Develop lifecycle orchestration for online model serving, including distribution, deployment, upgrades, rollback, autoscaling, multi-cluster operations, and disaster recovery.
  • •Create serving orchestration and traffic management for disaggregated serving clusters, including topology-aware scheduling, KV cache affinity, intelligent request routing, and QoS/SLA management.

Key Requirements

  • •Completing or recently completed a Bachelor's or Master's degree in Computer Science, Software Engineering, Artificial Intelligence, or a related technical field.
  • •Proficiency in at least one of Go, C++, or Python, with strong foundations in data structures, algorithms, and software engineering principles.
  • •Familiarity with Linux and a foundational understanding of operating systems, computer networks, concurrent programming, and distributed systems.
  • •Hands-on and exploratory mindset using source code, metrics, logs, profiling, and experiments to investigate systems.
  • •Systematic quantitative problem solving, including defining measurements, testing hypotheses, and validating improvements.
Experience:Open-sourceInfrastructure
Education:Bachelor's
Skills:OwnershipCollaborationProblem-solvingQuantitative analysisExperimentation
Tech Stack:KubernetesKubernetes OperatorsContainer runtimesLinuxGoC++PythonFinOpsVolcanoKoordinatorOpenKruiseVLLMSGLangTritonKServeRay ServeKV CacheContinuous BatchingPrefill/Decode disaggregationModel parallelism

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn