Staff Software Engineer, Ray Data

Anyscale
San Francisco
Workplace: OnsiteFull timeUSD 240,000 - 270,000 annuallyFunction: Software EngineeringExperience: 6+ yearsSkills: []

Build and optimize the core distributed data processing systems behind Ray Data, focusing on performance, scalability, and reliability. Design execution across pipeline stages in heterogeneous environments, and create data loading/processing for production training and inference. Tackle hard problems in scheduling, resource management, data partitioning, fault tolerance, and performance tuning while making system-level architecture tradeoffs such as batch vs streaming and consistency vs availability.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Anyscale
Anyscale
1 day ago

Staff Software Engineer, Ray Data

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 7 hours agoStatus: Live

Job Summary

Build and optimize the core distributed data processing systems behind Ray Data, focusing on performance, scalability, and reliability. Design execution across pipeline stages in heterogeneous environments, and create data loading/processing for production training and inference. Tackle hard problems in scheduling, resource management, data partitioning, fault tolerance, and performance tuning while making system-level architecture tradeoffs such as batch vs streaming and consistency vs availability.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Design, build, and improve the core systems that power Ray Data with a focus on performance, scalability, and reliability.
  • •Design and optimize distributed execution across stages of data pipelines in heterogeneous environments.
  • •Build data loading and processing solutions for production training and inference workloads.
  • •Solve distributed execution challenges including scheduling, resource management, data partitioning, fault tolerance, and performance optimization.
  • •Make system-level architectural decisions and evaluate tradeoffs in resource allocation, execution models, batch vs streaming, and consistency vs availability.

Pay and Benefits

Salary: USD 240,000 - 270,000 annually
Equity and Bonus:Equity
Perks:Health InsuranceDentalVisionParental Leave

Key Requirements

  • •6+ years building production-grade software, infrastructure, or developer-facing systems, with strong Python engineering experience.
  • •6+ years personally owning core architectural decisions within a distributed data or compute engine (not primarily operating a platform designed by others).
  • •Deep experience with distributed systems internals such as scheduling, fault tolerance, data partitioning, distributed execution, performance optimization, or database/query engine internals.
  • •Demonstrated ability to reason through system-level tradeoffs and defend architectural decisions (batch vs streaming, static vs dynamic resource allocation, consistency vs availability).
  • •Passion for solving large-scale AI infrastructure problems and building systems for the next generation of AI applications.
Experience:6+ yearsDistributed systemsAI infrastructureData processingMachine learningOpen source
Tech Stack:PythonRayAzureCoreWeaveGoogle Cloud

Company Brief

Anyscale
Provides a managed platform and developer tooling built on the Ray open-source project to scale Python and AI applications across clouds, simplifying distributed training, inference and production deployment for engineering teams.
Industry: Developer Tools
Company Size: Large (251 to 1,000 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series C
Headquarters: San Francisco, United States
Founded: 2019
Glassdoor
Glassdoor: 4.3
WebsiteLinkedInGlassdoor