Sr. Software Engineer - Ingestion Core team

Databricks
San Francisco
Workplace: OnsiteFull timeUSD 166,000 - 225,000 annuallyFunction: Software EngineeringExperience: 5+ yearsSkills: ["Collaboration"]

Build distributed platform systems that power seamless ingestion of structured and unstructured, petabyte-scale data into Delta Lake. Work on streaming ingestion, incremental processing, replication, and CDC, reducing end-to-end latency while increasing throughput and lowering costs. Design and optimize distributed workloads for reliability and scale, add monitoring/observability for ingestion workflows, and collaborate on AI use cases like RAG and agentic SDK evaluations.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
3 days ago

Sr. Software Engineer - Ingestion Core team

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 14 hours agoStatus: Live

Job Summary

Build distributed platform systems that power seamless ingestion of structured and unstructured, petabyte-scale data into Delta Lake. Work on streaming ingestion, incremental processing, replication, and CDC, reducing end-to-end latency while increasing throughput and lowering costs. Design and optimize distributed workloads for reliability and scale, add monitoring/observability for ingestion workflows, and collaborate on AI use cases like RAG and agentic SDK evaluations.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Build distributed infrastructure for ingestion from diverse sources, including streaming ingestion, incremental processing, replication, and connector support.
  • •Reduce end-to-end latency while increasing throughput and lowering costs from source systems to Delta Lake availability.
  • •Design and optimize streaming and distributed workloads for throughput, cost, latency, reliability, and scale.
  • •Explore and apply ML techniques to optimize streaming workloads.
  • •Create monitoring and observability capabilities for ingestion workflows and systems.

Pay and Benefits

Salary: USD 166,000 - 225,000 annually

Key Requirements

  • •5+ years of production coding experience in Java, Scala, Go, C++, or Python.
  • •Experience architecting, developing, and deploying large-scale distributed and asynchronous systems.
  • •Hands-on experience with distributed systems, streaming, Spark, databases, data processing, or CDC.
  • •Experience building or operating systems where scale, throughput, latency, reliability, and cost matter.
  • •Comfort using AI tools and defining effective evaluations to speed iteration.
Experience:5+ years
Skills:Collaboration
Languages:English
Tech Stack:JavaScalaGoC++PythonSparkDatabricks SQLDelta LakeSQSADLSGCSOracleSQL ServerMySQLPostgresGoogle DriveSharePointJSONParquetCSV

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn