4 months ago

Software Engineer - Web Crawling

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Build and operate distributed data acquisition systems to power real-time AI agents. Design large-scale web pipelines, research browser internals and anti-automation techniques, and develop automated tooling (Playwright, Puppeteer, CDP) to extract high-quality, structured data from live web at scale. Collaborate with AI and infrastructure teams to integrate data into retrieval pipelines, with a focus on performance, resilience, and accurate web interaction. Based in Israel with work authorization required.

Location: Tel Aviv

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Build distributed data acquisition systems that capture and structure the live web.
•Investigate and analyze browser internals, fingerprinting, and anti-automation systems to develop stealthy, adaptive orchestration layers.
•Prototype and deploy intelligent automation frameworks using Playwright, Puppeteer, and low-level browser control protocols (CDP).
•Conduct hands-on research into network flows, JavaScript-based protections, and emerging web standards affecting automation.
•Collaborate with AI and infrastructure teams to integrate real-time web data into retrieval pipelines and LLM-powered agents

Key Requirements

•3–5 years of experience as a backend or systems engineer, ideally working with large-scale, distributed, or web-facing infrastructure
•Strong programming skills in Python or Node.js, with experience in Go or C++ considered a strong plus
•Proven experience building and maintaining browser automation systems (Playwright, Puppeteer, or CDP) in production environments
•Solid understanding of browser internals, network protocols, and web interaction mechanisms
•Experience designing high-performance, resilient systems that handle scale, concurrency, and complex orchestration

Experience:3-5 yearsWeb crawlingDistributed systemsAI agents

Skills:CommunicationProblem-solvingAnalyticalDocumentationTeamwork

Languages:English

Tech Stack:PythonNode.jsGoC++PlaywrightPuppeteerCDPDockerKubernetes

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Tavily

Tavily offers an AI-driven interview preparation and coaching platform that helps candidates practice mock interviews, receive personalized feedback, and track progress with analytics to improve interviewing skills and hiring outcomes.

Industry: EdTech

Website

Loading job details...

Preparing the role view and application actions.

Tavily

4 months ago

Software Engineer - Web Crawling

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Location: Tel Aviv

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Build distributed data acquisition systems that capture and structure the live web.
•Investigate and analyze browser internals, fingerprinting, and anti-automation systems to develop stealthy, adaptive orchestration layers.
•Prototype and deploy intelligent automation frameworks using Playwright, Puppeteer, and low-level browser control protocols (CDP).
•Conduct hands-on research into network flows, JavaScript-based protections, and emerging web standards affecting automation.
•Collaborate with AI and infrastructure teams to integrate real-time web data into retrieval pipelines and LLM-powered agents

Key Requirements

•3–5 years of experience as a backend or systems engineer, ideally working with large-scale, distributed, or web-facing infrastructure
•Strong programming skills in Python or Node.js, with experience in Go or C++ considered a strong plus
•Proven experience building and maintaining browser automation systems (Playwright, Puppeteer, or CDP) in production environments
•Solid understanding of browser internals, network protocols, and web interaction mechanisms
•Experience designing high-performance, resilient systems that handle scale, concurrency, and complex orchestration

Experience:3-5 yearsWeb crawlingDistributed systemsAI agents

Skills:CommunicationProblem-solvingAnalyticalDocumentationTeamwork

Languages:English

Tech Stack:PythonNode.jsGoC++PlaywrightPuppeteerCDPDockerKubernetes

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Tavily

Industry: EdTech

Website