Exa

Recent

Software Engineer, Web Crawling

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 13 hours agoStatus: Live

Job Summary

Join Exa to build Google-scale crawling infrastructure for a new search engine. You will scale web crawlers, work with high-performance languages, and optimize systems to deliver high-quality, rapidly indexed knowledge across massive datasets.

Location: United States

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Key Responsibilities

•Build and maintain crawlers capable of crawling large portions of the web at scale
•Collaborate to design and implement distributed crawling systems and prioritization
•Optimize crawl politeness, rate limits, and handling of dynamic content across domains
•Develop intelligent crawl scheduling and prioritization algorithms for maximum coverage
•Address anti-bot challenges and ensure robust handling of JavaScript-rendered content

Pay and Benefits

Salary: USD 150,000 - 300,000 annually

Equity and Bonus:Equity

Perks:Equity

Key Requirements

•Extensive experience building and scaling web crawlers, or eagerness to ramp up quickly
•Experience with high performance languages (C++, Rust, etc.)
•Familiar with TypeScript, Playwright, modern web design, CDP (Chrome DevTools Protocol)
•Ability to optimize a system to an exceptional degree
•Passion for finding high quality knowledge and its impact on the world

Experience:Web crawlingSearch

Skills:Problem-solvingCollaborationCommunication

Tech Stack:RustC++TypeScriptPlaywrightCDPJavaScript

Eligibility

Visa:OPTSTEM OPTH1BO1E3

Work Authorization:Sponsorship available.

Company Brief

Exa

Builds an embeddings-based neural search engine and web search API for AI applications, offering crawling, retrieval, and deep research tools to serve developers and AI agents with up-to-date web knowledge.

Industry: API Platforms

Company Size: Small (11 to 50 employees)

Growth: Scaleup

Valuation: USD 500M to 1B

Funding: Series B

Headquarters: San Francisco, United States

Founded: 2021

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

Exa