Databricks & PySpark - Sr Technical Lead-Data Engg (INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI, IN)

Birlasoft
Pune
Workplace: OnsiteFull timeFunction: Administration & Executive AssistanceSkills: ["Stakeholder management","Collaboration","Mentoring","Guidance","Alignment"]

Architect, develop, and optimize Azure-based data platforms using Databricks and PySpark. Own end-to-end enterprise-grade ELT/ETL pipelines that ingest from diverse sources, implement Bronze→Silver transformations, and deliver analytics-ready dimensional models and consistent KPIs. Ensure data quality and governance with Unity Catalog, Delta Lake capabilities, and robust validation frameworks. Partner with stakeholders, align delivery to business objectives, and mentor junior engineers as needed.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
3 days ago

Databricks & PySpark - Sr Technical Lead-Data Engg (INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI, IN)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Architect, develop, and optimize Azure-based data platforms using Databricks and PySpark. Own end-to-end enterprise-grade ELT/ETL pipelines that ingest from diverse sources, implement Bronze→Silver transformations, and deliver analytics-ready dimensional models and consistent KPIs. Ensure data quality and governance with Unity Catalog, Delta Lake capabilities, and robust validation frameworks. Partner with stakeholders, align delivery to business objectives, and mentor junior engineers as needed.
Location: Pune
Workplace: Onsite
Employment Type: Full time
Job Function: Administration & Executive Assistance
Seniority: Sr. Manager level

Key Responsibilities

  • •Build and maintain ingestion frameworks using ADF, Databricks, and Spark, including Bronze→Silver transformations and data quality/schema validation.
  • •Implement high-throughput, robust ELT/ETL pipelines integrating on-prem and cloud systems, APIs, and third-party sources using PySpark.
  • •Design dimensional models (star/snowflake), fact/dimension tables, surrogate keys, and SCD handling; translate datasets into analytics-ready models with consistent KPI logic.
  • •Engineer data transformations and performance using Delta Lake features (ACID, schema enforcement/evolution, time travel) and Databricks optimizations.
  • •Set up data governance and quality in Unity Catalog (naming/metadata policies, access controls) and collaborate with stakeholders to validate business rules and metrics.

Key Requirements

  • •Strong hands-on expertise in Databricks and PySpark for building and optimizing data pipelines.
  • •Experience with Azure data engineering, including implementing ingestion and transformation solutions on Azure.
  • •Proficiency with ADF and SQL to orchestrate workflows, pipelines, triggers, and parameterized datasets.
  • •Experience designing enterprise-grade ELT/ETL pipelines, including Bronze→Silver transformations and analytics-ready dimensional models (star/snowflake, SCD).
  • •Knowledge of data quality and governance, including Unity Catalog standards, RBAC/ABAC, and audit/validation frameworks.
Experience:Data engineeringData pipelinesELT/ETL
Skills:Stakeholder managementCollaborationMentoringGuidanceAlignment
Tech Stack:DatabricksPySparkADFSQLAzureSparkDelta LakeUnity CatalogELTETLAPIsCI/CD

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn