Databricks - Sr Technical Lead-Data Engg

Birlasoft
Pune
Workplace: OnsiteFull timeFunction: Administration & Executive AssistanceSkills: ["Collaboration","Mentoring","Stakeholder management","Delivery focus","Problem-solving"]

Design, develop, and optimize Azure-based data platforms using Databricks and PySpark. Own enterprise-grade ingestion frameworks with ADF/Databricks/Spark, implement Bronze-to-Silver transformations, and build scalable ELT/ETL pipelines for structured, semi-structured, and unstructured data. Ensure data quality and governance with Unity Catalog, Delta Lake features, and validation/audit mechanisms. Collaborate with architects and business teams to deliver reliable, analytics-ready models and mentored engineering execution.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
3 days ago

Databricks - Sr Technical Lead-Data Engg

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 4 hours agoStatus: Live

Job Summary

Design, develop, and optimize Azure-based data platforms using Databricks and PySpark. Own enterprise-grade ingestion frameworks with ADF/Databricks/Spark, implement Bronze-to-Silver transformations, and build scalable ELT/ETL pipelines for structured, semi-structured, and unstructured data. Ensure data quality and governance with Unity Catalog, Delta Lake features, and validation/audit mechanisms. Collaborate with architects and business teams to deliver reliable, analytics-ready models and mentored engineering execution.
Location: Pune
Workplace: Onsite
Employment Type: Full time
Job Function: Administration & Executive Assistance

Key Responsibilities

  • •Build and maintain ingestion frameworks using ADF, Databricks, and Spark; implement Bronze-to-Silver transformations aligned to data architecture.
  • •Architect data quality checks including schema validation and contract rules; implement audit and validation mechanisms.
  • •Develop and optimize high-throughput ELT/ETL pipelines integrating on-prem and cloud systems, APIs, and third-party sources.
  • •Implement complex transformations in PySpark with performance optimizations (caching, partitioning, Delta Lake capabilities) and tuning for bottlenecks.
  • •Collaborate with IT/business teams and cross-functional engineering stakeholders to deliver production-ready analytics models, guide junior engineers, and support governance standards in Unity Catalog.

Key Requirements

  • •Strong hands-on experience building with Databricks and PySpark for enterprise data engineering solutions.
  • •Expertise in Azure data platforms, including ingestion frameworks with ADF/Databricks/Spark and robust ELT/ETL pipeline design.
  • •Experience implementing data transformations from Bronze to Silver and building analytics-ready dimensional models (star/snowflake), facts/dimensions, surrogate keys, and SCD handling.
  • •Proficiency with Delta Lake (ACID, schema enforcement/evolution, time travel) and performance tuning (clusters, shuffle, joins, parallelization).
  • •Knowledge of data governance and quality using Unity Catalog, including RBAC/ABAC, naming/metadata policies, and validation frameworks.
Experience:AzureData platformsData pipelinesELTETLDelta LakeDatabricks
Skills:CollaborationMentoringStakeholder managementDelivery focusProblem-solving
Tech Stack:DatabricksPySparkAzureADFSQLSparkDelta LakeUnity CatalogELTETLDimensional modelingStar schemaSnowflake schemaCI/CD

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn