Technical Lead, Spark

Cloudera
Barcelona, United Kingdom, Ireland
Workplace: RemoteFull timeFunction: Administration & Executive AssistanceExperience: 6+ yearsEducation: bachelorsSkills: ["Java","Scala","Python","Spark","Livy","Parquet","Iceberg","SQL","Distributed systems"]

Lead the design and delivery of distributed data engineering features on Apache Spark and Livy, overseeing production-ready features at scale. Work across Scala/Java/Python, contribute to Spark/Livy, and collaborate with cross-functional teams to solve large-scale data processing challenges on thousands of nodes.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Cloudera
Cloudera
3 months ago

Technical Lead, Spark

âś“ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 21 days agoStatus: Live

Job Summary

Lead the design and delivery of distributed data engineering features on Apache Spark and Livy, overseeing production-ready features at scale. Work across Scala/Java/Python, contribute to Spark/Livy, and collaborate with cross-functional teams to solve large-scale data processing challenges on thousands of nodes.
Location: Barcelona, United Kingdom, Ireland
Workplace: Remote
Employment Type: Full time
Job Function: Administration & Executive Assistance
Seniority: Sr. Manager level

Key Responsibilities

  • •Design new features for Cloudera’s data engineering experience, taking prototypes to production-ready features at scale
  • •Contribute to Apache Spark and Livy projects and integrate them into the platform
  • •Develop new features in Scala/Java/Python on modern platforms to support large-scale data processing
  • •Gain expertise in distributed data processing, SQL planners, data formats (Parquet, Iceberg), and fault tolerance in distributed systems
  • •Collaborate with team members and stakeholders to improve internal infrastructure and deliver customer-focused solutions

Pay and Benefits

Perks:Remote WorkPaid LeaveVolunteer TimeWellness Stipend

Key Requirements

  • •6+ years of professional software development
  • •BSc/MSc in related field or equivalent experience
  • •Experience leading and delivering complex product enhancements
  • •Strong understanding of at least one of Java, Scala, Python; willingness to learn others
  • •Experience with distributed systems and systems design
Experience:6+ yearsDistributed systemsSparkData engineering
Education:Bachelor's
Skills:JavaScalaPythonSparkLivyParquetIcebergSQLDistributed systems
Languages:English
Tech Stack:JavaScalaPythonApache SparkLivyParquetIceberg

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Cloudera
Provides a hybrid data platform for managing, analyzing, and securing enterprise data across cloud and on-premises environments, enabling machine learning, analytics, and data engineering at scale.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Santa Clara, United States
Founded: 2008
Glassdoor
Glassdoor: 3.8
WebsiteLinkedIn