Production Engineer - Applied Machine Learning Engine (Singapore)

ByteDance
Singapore
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningEducation: bachelorsSkills: ["Ownership","Analytical skills","Problem-solving","Cross-functional collaboration","Incident troubleshooting"]

Build and run high-availability machine learning training, inference, and storage systems for ByteDance’s Data-AML team. Own production operations, including scheduling/orchestration, Kubernetes and GPU cluster operations, distributed training, and Parameter Server/NoSQL storage. Establish SLO/SLA frameworks, observability, alerting, on-call, incident response, and disaster recovery. Improve delivery with CI/CD, canary deployments, automated rollback, health checks, and elastic scaling, while optimizing GPU/CPU/storage/network resources.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
1 month ago

Production Engineer - Applied Machine Learning Engine (Singapore)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 30 days agoStatus: Live
Reposted: similar role first listed 1 month ago

Job Summary

Build and run high-availability machine learning training, inference, and storage systems for ByteDance’s Data-AML team. Own production operations, including scheduling/orchestration, Kubernetes and GPU cluster operations, distributed training, and Parameter Server/NoSQL storage. Establish SLO/SLA frameworks, observability, alerting, on-call, incident response, and disaster recovery. Improve delivery with CI/CD, canary deployments, automated rollback, health checks, and elastic scaling, while optimizing GPU/CPU/storage/network resources.
Location: Singapore
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning

Key Responsibilities

  • •Own production operations management and stability assurance for AML training, inference, and storage systems, including scheduling/orchestration, Kubernetes/GPU clusters, distributed training, online inference serving, and Parameter Server/NoSQL storage.
  • •Build and maintain SLO/SLA frameworks, observability, alerting, on-call processes, incident diagnosis, self-healing, disaster recovery, and post-incident review (postmortem) practices.
  • •Drive engineering capabilities such as CI/CD, canary/gradual deployments, automated rollback, system health inspections, pre-flight checks, capacity forecasting, and elastic scaling.
  • •Lead resource governance and optimization across GPU, CPU, storage, and network infrastructure, including quota management, cost attribution, and performance tuning.
  • •Collaborate across teams to resolve complex technical challenges impacting availability and system performance.

Key Requirements

  • •Bachelor’s degree or above in Computer Science, Software Engineering, Artificial Intelligence, or a related field.
  • •Proficient in Linux and skilled in at least one: Shell, Python, Go, or C++.
  • •Familiar with ML training/inference architectures, Kubernetes, GPU clusters, or distributed storage systems.
  • •Experienced in production issue troubleshooting, performance analysis, and automation platform development.
  • •Strong ownership and analytical/problem-solving skills with ability to drive cross-functional collaboration to resolve complex technical issues.
Experience:Machine learningDistributed systemsKubernetesGPUNoSQLFinOpsOpen source infrastructure
Education:Bachelor's in Computer Science, Software Engineering, Artificial Intelligence, or a related field
Skills:OwnershipAnalytical skillsProblem-solvingCross-functional collaborationIncident troubleshooting
Tech Stack:LinuxShellPythonGoC++KubernetesK8sGPU clustersCI/CDCanary deploymentsDistributed trainingOnline inference servingParameter ServerNoSQLKV CachesMooncakeObservabilityAlertingPostmortemDisaster recovery

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn