Azure Subcontractor

Birlasoft
Hyderabad
Workplace: OnsiteContractFunction: DevOps, Cloud & InfrastructureSkills: ["Problem solving","Team collaboration","Documentation","Technical design","Agile/Scrum methodology"]

Design, develop, and maintain scalable ETL/ELT data pipelines and analytics solutions using Databricks. Build batch and streaming processing with Apache Spark (PySpark), integrate data from databases/APIs/cloud storage, and implement transformation, cleansing, and validation. Create and optimize Delta Lake tables (partitioning, Z-Ordering), develop reusable Databricks notebooks/workflows/jobs, and troubleshoot pipeline performance failures while collaborating with data stakeholders and following CI/CD best practices.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
2 months ago

Azure Subcontractor

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Design, develop, and maintain scalable ETL/ELT data pipelines and analytics solutions using Databricks. Build batch and streaming processing with Apache Spark (PySpark), integrate data from databases/APIs/cloud storage, and implement transformation, cleansing, and validation. Create and optimize Delta Lake tables (partitioning, Z-Ordering), develop reusable Databricks notebooks/workflows/jobs, and troubleshoot pipeline performance failures while collaborating with data stakeholders and following CI/CD best practices.
Location: Hyderabad
Workplace: Onsite
Employment Type: Contract
Job Function: DevOps, Cloud & Infrastructure

Key Responsibilities

  • •Design, develop, and maintain ETL/ELT pipelines using Databricks.
  • •Develop scalable data processing frameworks with Apache Spark (PySpark).
  • •Build and optimize batch and streaming data pipelines.
  • •Develop Delta Lake tables and optimize data storage with partitioning and Z-Ordering.
  • •Monitor and troubleshoot pipeline failures and performance issues while collaborating with stakeholders on data requirements.

Key Requirements

  • •Strong experience with Databricks.
  • •Expertise in Apache Spark/PySpark and proficiency in Python.
  • •Strong SQL skills and ETL/ELT development experience.
  • •Experience with Delta Lake and data warehousing concepts.
  • •Experience with Git/version control and CI/CD best practices.
Experience:Data engineeringCloud data platformsETL/ELTBig dataAnalytics
Skills:Problem solvingTeam collaborationDocumentationTechnical designAgile/Scrum methodology
Tech Stack:DatabricksApache SparkPySparkPythonSQLETL/ELTDelta LakeZ-OrderingGitCI/CDAzure DatabricksADLS Gen2Azure Data FactoryAmazon S3AWS GlueAWS EMRGoogle BigQueryCloud StorageSpark Structured StreamingAirflow

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn