Technical Lead Manager - Training Runtime, Data(set) Movement

OpenAI
San Francisco
Workplace: HybridFull timeUSD 295,000 - 445,000 annuallyFunction: Education & TrainingSkills: ["Communication","Problem-solving","Leadership","Collaboration"]

Lead the design and delivery of a unified dataset read platform powering OpenAI's largest model training runs, owning data APIs, storage contracts, versioning, and reliability guarantees across training frameworks. You’ll code hands-on in core data loading paths, align researchers and infrastructure teams, build observability tools, and drive scalable data movement from dataset read to checkpointing, enabling fast, reproducible, and robust training at scale.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
OpenAI
OpenAI
2 months ago

Technical Lead Manager - Training Runtime, Data(set) Movement

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 20 hours agoStatus: Live

Job Summary

Lead the design and delivery of a unified dataset read platform powering OpenAI's largest model training runs, owning data APIs, storage contracts, versioning, and reliability guarantees across training frameworks. You’ll code hands-on in core data loading paths, align researchers and infrastructure teams, build observability tools, and drive scalable data movement from dataset read to checkpointing, enabling fast, reproducible, and robust training at scale.
Location: San Francisco
Workplace: Hybrid
Employment Type: Full time
Job Function: Education & Training
Seniority: Manager level

Key Responsibilities

  • •Design and build a unified dataset read platform for multiple current and future training frameworks.
  • •Define dataset APIs, storage-format expectations, registration/versioning, and migration paths that make data access reproducible and maintainable.
  • •Build reliability into the read path, including stateful iteration, caching, fast restart, recovery, and clear operational contracts.
  • •Build terminal and web-based visualizers that let teams inspect text, multimodal, and reinforcement learning data late in the pipeline.
  • •Write and review production code in core data loading, service, caching, and reliability paths.

Pay and Benefits

Salary: USD 295,000 - 445,000 annually
Equity and Bonus:Equity
Perks:Equity

Key Requirements

  • •Deep hands-on experience designing and building dataset, data loading, storage, or distributed training infrastructure at scale
  • •Experience defining and versioning APIs and storage formats for data pipelines
  • •Proficiency across Python and lower-level systems programming; familiarity with Rust or C++ is a plus
  • •Strong track record writing production code in data loading, service, caching, and reliability paths
  • •Ability to lead technically before a team exists and subsequently manage engineers while maintaining hands-on edge
Experience:AIMachine learningDistributed systemsData movement
Skills:CommunicationProblem-solvingLeadershipCollaboration
Tech Stack:PythonRustC++TorchDistributed systemsData loadingStorageAPIs

Company Brief

OpenAI
Develops and deploys advanced generative AI models (including ChatGPT and DALL·E) and AI infrastructure, providing APIs and consumer products to accelerate safe AGI for broad benefit.
Industry: AI & Machine Learning
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Hectocorn (USD 100B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2015
Glassdoor
Glassdoor: 4.4
WebsiteLinkedInGlassdoor