Software Engineer, Web Crawling

Exa
Singapore
Workplace: OnsiteFull timeUSD 150,000 - 300,000 annuallyFunction: Software EngineeringSkills: ["Problem-solving","Communication","Collaboration","Organization"]

Exa is building a high-scale web crawling platform to power AI applications. The role focuses on designing, building, and scaling distributed web crawlers, optimizing crawling performance, politeness, and content handling, while leveraging Rust/C++ and TypeScript tooling to support a Google-scale crawling system and search indexing.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Exa
Exa
8 months ago

Software Engineer, Web Crawling

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 12 hours agoStatus: Live

Job Summary

Exa is building a high-scale web crawling platform to power AI applications. The role focuses on designing, building, and scaling distributed web crawlers, optimizing crawling performance, politeness, and content handling, while leveraging Rust/C++ and TypeScript tooling to support a Google-scale crawling system and search indexing.
Location: Singapore
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Design, build, and scale a Google-scale web crawler capable of crawling vast portions of the web.
  • •Optimize crawl politeness, rate limiting, and scheduling across thousands of domains.
  • •Work with high-performance languages (C++, Rust) and modern web tooling (TypeScript, Playwright) to index and render content efficiently.
  • •Develop systems to handle dynamic content, anti-bot measures, and robust crawl scheduling.
  • •Collaborate with the team to deliver high-quality knowledge discovery features for AI applications.

Pay and Benefits

Salary: USD 150,000 - 300,000 annually
Equity and Bonus:Equity
Perks:Equity

Key Requirements

  • •Experience building and scaling web crawlers, or eagerness to ramp up quickly.
  • •Experience with a high-performance language (C++, Rust, etc.).
  • •Familiarity with TypeScript, Playwright, modern web design, CDP (Chrome DevTools Protocol).
  • •Comfortable optimizing a system to an exceptional degree.
  • •Care about finding high quality knowledge and its impact on the world.
Experience:Web crawlingSearch infrastructureDistributed systems
Skills:Problem-solvingCommunicationCollaborationOrganization
Tech Stack:RustC++TypeScriptPlaywrightCDPChrome DevTools Protocol

Company Brief

Exa
Builds an embeddings-based neural search engine and web search API for AI applications, offering crawling, retrieval, and deep research tools to serve developers and AI agents with up-to-date web knowledge.
Industry: API Platforms
Company Size: Small (11 to 50 employees)
Growth: Scaleup
Valuation: USD 500M to 1B
Funding: Series B
Headquarters: San Francisco, United States
Founded: 2021
WebsiteLinkedIn