Senior Staff Software Engineer - Lakeflow Pipelines Datasets

Databricks
Mountain View, San Francisco
Workplace: OnsiteFull timeFunction: Software EngineeringExperience: 10+ yearsEducation: bachelorsSkills: ["Leadership","Communication","Strategic thinking","Collaboration"]

Senior Staff Engineer leading the Lakeflow Pipelines and Datasets platform at Databricks. Own long-term technical direction for declarative dataflow pipelines, materialized views, and streaming components, guiding multi-team architectural initiatives from design to delivery. Drive performance tuning, scale across exabytes of data, and bridge research with production while shaping product strategy and collaborating with engineering leadership to deliver a highly scalable data infrastructure.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Databricks
Databricks
5 months ago

Senior Staff Software Engineer - Lakeflow Pipelines Datasets

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 14 hours agoStatus: Live

Job Summary

Senior Staff Engineer leading the Lakeflow Pipelines and Datasets platform at Databricks. Own long-term technical direction for declarative dataflow pipelines, materialized views, and streaming components, guiding multi-team architectural initiatives from design to delivery. Drive performance tuning, scale across exabytes of data, and bridge research with production while shaping product strategy and collaborating with engineering leadership to deliver a highly scalable data infrastructure.
Location: Mountain View, San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Architect and deliver a highly scalable, fault-tolerant platform for declarative pipelines and materialized views serving thousands of production customers.
  • •Turn ambiguous, large-scale technical challenges into clear execution plans that ship incrementally and hold up at exabyte scale.
  • •Bridge the gap between research and production by bringing ideas from the academic literature or internal R&D into systems that solve real customer problems.
  • •Lead deep systems work: performance diagnosis on large production clusters, low-level debugging, and optimization that moves the needle on cost and latency.
  • •Shape product direction alongside engineering and product leadership, owning technical strategy from design through delivery.

Key Requirements

  • •10+ years of experience building, operating, and evolving large-scale distributed systems in production.
  • •Deep expertise in one or more of: database internals, storage systems, distributed computing, streaming systems, language/API design, or performance engineering.
  • •Track record of executing against a multi-year technical vision through well-sequenced incremental milestones.
  • •Strong algorithmic foundations with practical judgment on when they matter.
  • •BS, MS, or PhD in Computer Science or a related field (or equivalent depth of experience).
Experience:10+ yearsData engineeringDistributed systemsBig dataStreaming
Education:Bachelor's
Skills:LeadershipCommunicationStrategic thinkingCollaboration
Languages:English
Tech Stack:Declarative Dataflow ProcessingStructured StreamingMaterialized ViewsResource managementStorage structuresAutonomic Computing

Company Brief

Databricks
Provides a unified data analytics platform powered by Apache Spark to simplify building, deploying, and scaling data engineering, data science, and machine learning workloads for enterprises.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series E+
Headquarters: San Francisco, United States
Founded: 2013
WebsiteLinkedIn