Software Engineer, Data Acquisition

OpenAI
San Francisco
Workplace: OnsiteFull timeUSD 325,000 - 405,000 annuallyFunction: Software EngineeringExperience: 4+ yearsEducation: bachelorsSkills: ["Communication","Teamwork","Problem-solving","Prioritization","Writing"]

A role focused on building and maintaining distributed data acquisition systems for OpenAI, including web crawling, data ingestion, indexing and search, across Kubernetes-based infrastructure. Collaborate with Data Processing, Architecture, and Scaling teams, while ensuring data privacy/compliance and handling petabyte-scale data workloads.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
OpenAI
OpenAI
2 years ago

Software Engineer, Data Acquisition

āœ“ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 16 minutes agoStatus: Live

Job Summary

A role focused on building and maintaining distributed data acquisition systems for OpenAI, including web crawling, data ingestion, indexing and search, across Kubernetes-based infrastructure. Collaborate with Data Processing, Architecture, and Scaling teams, while ensuring data privacy/compliance and handling petabyte-scale data workloads.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • ā€¢Own and lead engineering projects in the area of data acquisition including web crawling, data ingestion, and search.
  • ā€¢Collaborate with other sub-teams, such as Data Processing, Architecture, and Scaling, to ensure smooth data flow and system operability.
  • ā€¢Work closely with the legal team to handle any compliance or data privacy-related matters.
  • ā€¢Develop and deploy highly scalable distributed systems capable of handling petabytes of data.
  • ā€¢Architect and implement algorithms for data indexing and search capabilities.

Pay and Benefits

Salary: USD 325,000 - 405,000 annually
Equity and Bonus:Equity

Key Requirements

  • ā€¢BS/MS/PhD in Computer Science or a related field.
  • ā€¢4+ years of industry experience in software development.
  • ā€¢Experience with large web crawlers a plus
  • ā€¢Strong expertise in large stateful distributed systems and data processing.
  • ā€¢Proficiency in Kubernetes, and Infrastructure-as-Code concepts.
Experience:4+ yearsDistributed systemsData processingWeb crawlers
Education:Bachelor's in Computer Science
Skills:CommunicationTeamworkProblem-solvingPrioritizationWriting
Tech Stack:KubernetesInfrastructure-as-CodeWeb crawlersData ingestionDistributed systemsKey-value databasesSearchGPTBotCloud

Company Brief

OpenAI
Develops and deploys advanced generative AI models (including ChatGPT and DALLĀ·E) and AI infrastructure, providing APIs and consumer products to accelerate safe AGI for broad benefit.
Industry: AI & Machine Learning
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Hectocorn (USD 100B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2015
Glassdoor
Glassdoor: 4.4
WebsiteLinkedInGlassdoor