Databricks Data Specialist - R01569707

Brillio
Bengaluru
Workplace: HybridFull timeFunction: Software EngineeringEducation: bachelorsSkills: ["Analytical thinking","Troubleshooting","Problem-solving","Communication","Stakeholder management"]

Design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. Build end-to-end batch and real-time pipelines using PySpark, SQL, Delta Lake, Delta Live Tables, Structured Streaming, and Auto Loader. Implement Lakehouse architectures with Bronze/Silver/Gold layers, manage Delta Lake reliability features, and orchestrate workflows with Databricks Workflows. Establish governance with Unity Catalog and optimize Spark/SQL performance in collaboration with analytics and business stakeholders.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Brillio
Brillio
4 days ago

Databricks Data Specialist - R01569707

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 9 hours agoStatus: Live

Job Summary

Design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. Build end-to-end batch and real-time pipelines using PySpark, SQL, Delta Lake, Delta Live Tables, Structured Streaming, and Auto Loader. Implement Lakehouse architectures with Bronze/Silver/Gold layers, manage Delta Lake reliability features, and orchestrate workflows with Databricks Workflows. Establish governance with Unity Catalog and optimize Spark/SQL performance in collaboration with analytics and business stakeholders.
Location: Bengaluru
Workplace: Hybrid
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
  • •Implement Lakehouse architectures using Bronze/Silver/Gold layers and Delta Lake reliability features.
  • •Create and optimize Delta Live Tables (DLT) pipelines and ingestion processes using Auto Loader.
  • •Develop batch and real-time processing with Structured Streaming and orchestrate workflows using Databricks Workflows.
  • •Establish data governance with Unity Catalog and optimize Spark jobs, SQL queries, and platform performance with cross-functional collaboration.

Key Requirements

  • •Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
  • •Build Lakehouse architectures using Bronze/Silver/Gold layers and manage Delta Lake solutions (ACID, schema enforcement, reliability).
  • •Create, monitor, and optimize Delta Live Tables (DLT) pipelines and data ingestion using Auto Loader.
  • •Develop and manage real-time processing with Structured Streaming and orchestrate workflows using Databricks Workflows.
  • •Establish and enforce data governance, security, and access controls using Unity Catalog; optimize Spark jobs and SQL performance.
Experience:Data engineeringData pipelinesReal-time streamingLakehouseETL
Education:Bachelor's in Computer Science, Data Engineering, Information Technology, or a related discipline
Skills:Analytical thinkingTroubleshootingProblem-solvingCommunicationStakeholder management
Tech Stack:DatabricksPySparkApache SparkDelta LakeDelta Live Tables (DLT)Unity CatalogStructured StreamingAuto LoaderSQLLakehouse ArchitectureDatabricks WorkflowsAzureAzure Data Factory (ADF)Azure Synapse AnalyticsMicrosoft PurviewMicrosoft FabricAWSAWS GlueAWS LambdaAWS Step Functions

Company Brief

Brillio
Provides digital transformation and IT consulting services, including cloud migration, analytics, digital engineering, and managed services to accelerate business outcomes for enterprises across industries.
Industry: Consulting
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Private Equity Backed
Headquarters: Santa Clara, United States
Founded: 2014
WebsiteLinkedIn