Software Engineer Graduate (AML-Engine-Orchestration) - 2027 Start

ByteDance
San Jose
Workplace: OnsiteFull timeFunction: Software EngineeringSkills: ["Exploratory","Ownership","Collaboration","Problem solving","Quantitative thinking"]

Build large-scale machine learning orchestration systems that power online model serving across ByteDance products. You’ll design orchestration capabilities for ML platforms, including Kubernetes operators, container lifecycle management, and multi-tenant resource/quota scheduling to improve GPU utilization and cost efficiency. Work on serving orchestration for disaggregated clusters, including traffic management, autoscaling, multi-cluster deployment, and disaster recovery, with measurable impact on latency, availability, and reliability.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
1 day ago

Software Engineer Graduate (AML-Engine-Orchestration) - 2027 Start

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 15 hours agoStatus: Live

Job Summary

Build large-scale machine learning orchestration systems that power online model serving across ByteDance products. You’ll design orchestration capabilities for ML platforms, including Kubernetes operators, container lifecycle management, and multi-tenant resource/quota scheduling to improve GPU utilization and cost efficiency. Work on serving orchestration for disaggregated clusters, including traffic management, autoscaling, multi-cluster deployment, and disaster recovery, with measurable impact on latency, availability, and reliability.
Location: San Jose
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Graduate level

Key Responsibilities

  • •Design and build foundational orchestration capabilities for ML platforms, including Kubernetes Operators, container runtimes, and lifecycle management for jobs, services, and stateful workloads.
  • •Build multi-tenant resource and quota systems supporting priorities, preemption, fair sharing, elasticity, and cross-cluster scheduling to improve GPU utilization and cost efficiency.
  • •Create lifecycle orchestration for online model serving, including model/image distribution, deployment, upgrades, rollback, autoscaling, multi-cluster operation, and disaster recovery.
  • •Develop serving orchestration and traffic management for disaggregated serving clusters, including topology-aware scheduling, KV Cache affinity, intelligent request routing, and QoS/SLA management.

Key Requirements

  • •Completing or recently completed a Bachelor's or Master's degree in Computer Science, Software Engineering, Artificial Intelligence, or a related technical field.
  • •Proficiency in at least one of Go, C++, or Python, with solid foundations in data structures, algorithms, and software engineering principles.
  • •Familiarity with Linux plus foundational understanding of operating systems, computer networks, concurrent programming, and distributed systems.
  • •Strong hands-on exploratory skills using source code, metrics, logs, profiling, and experiments to investigate systems.
  • •A systematic, quantitative problem-solving approach to define measurements, test hypotheses, and validate improvements.
Experience:Machine learningDistributed systemsOpen source
Education:
Skills:ExploratoryOwnershipCollaborationProblem solvingQuantitative thinking
Tech Stack:GoC++PythonLinuxKubernetesContainer runtimesKubernetes OperatorsFinOpsVolcanoKoordinatorOpenKruiseVLLMSGLangTritonKServeRay ServeKV CacheContinuous BatchingPrefill/Decode disaggregationModel parallelism

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn