Senior Production Engineer - Applied Machine Learning

ByteDance
San Jose
Workplace: OnsiteFull timeFunction: Data Science & Machine LearningEducation: bachelorsSkills: ["Responsibility","Logical thinking","Cross-functional collaboration","Problem-solving"]

Own production management and stability for AML (Applied Machine Learning) training, inference, and storage systems. Build reliability capabilities including SLO/SLA, observability, alerting, on-call operations, incident post-mortems, auto-healing, and disaster recovery. Drive engineering excellence with CI/CD, canary releases, auto-rollback, automated inspections, and capacity forecasting with elastic scaling. Manage GPU/CPU/storage/network governance to improve availability and resource efficiency across core ML pipelines.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
ByteDance
ByteDance
1 month ago

Senior Production Engineer - Applied Machine Learning

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 30 days agoStatus: Live
Reposted: similar role first listed 1 month ago

Job Summary

Own production management and stability for AML (Applied Machine Learning) training, inference, and storage systems. Build reliability capabilities including SLO/SLA, observability, alerting, on-call operations, incident post-mortems, auto-healing, and disaster recovery. Drive engineering excellence with CI/CD, canary releases, auto-rollback, automated inspections, and capacity forecasting with elastic scaling. Manage GPU/CPU/storage/network governance to improve availability and resource efficiency across core ML pipelines.
Location: San Jose
Workplace: Onsite
Employment Type: Full time
Job Function: Data Science & Machine Learning
Seniority: Mid level

Key Responsibilities

  • •Own production management and stability for AML training, inference, and storage systems, covering scheduling/orchestration, K8s/GPU clusters, distributed training, online inference serving, and ParameterServer/NoSQL storage.
  • •Build and maintain reliability mechanisms including SLO/SLA, observability, alerting, on-call processes, fault diagnosis, auto-healing, disaster recovery, and incident post-mortems.
  • •Drive engineering excellence via CI/CD, canary releases, auto-rollback, automated inspections, pre-flight checks, capacity forecasting, and elastic auto-scaling.
  • •Oversee resource governance across GPU/CPU/storage/network, including quota management, cost attribution, and performance tuning to improve availability and utilization.
  • •Collaborate cross-functionally to troubleshoot and resolve complex production issues and improve R&D efficiency.

Key Requirements

  • •Bachelor’s degree or above in Computer Science, Software Engineering, Artificial Intelligence, or related fields.
  • •Familiar with Linux and proficient in at least one of Shell, Python, Go, or C++.
  • •Understanding of ML training/inference architectures, Kubernetes, GPU clusters, or distributed storage systems.
  • •Proven experience in online troubleshooting, performance analysis, and building automation platforms.
  • •Strong sense of responsibility and the ability to drive resolution of complex issues across cross-functional teams.
Experience:Machine learningDistributed systems
Education:Bachelor's
Skills:ResponsibilityLogical thinkingCross-functional collaborationProblem-solving
Tech Stack:LinuxShellPythonGoC++KubernetesGPU clustersDistributed trainingOnline inference servingParameterServerNoSQLCI/CDCanary releasesAuto-rollbackObservabilityAlertingAuto-healingDisaster recoveryCapacity forecastingElastic auto-scaling

Company Brief

ByteDance
Develops consumer internet and content platforms, including TikTok and other apps for short-form video, news, and entertainment. It also builds advertising, commerce, and creator tools that connect audiences, brands, and publishers across global markets.
Industry: Digital Media
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Established Company
Headquarters: Beijing, China
Founded: 2012
WebsiteLinkedIn