Azure Subcontractor

Birlasoft
Hyderabad
Workplace: OnsiteFull timeFunction: DevOps, Cloud & InfrastructureSkills: ["Data modeling","Problem solving","Team collaboration","Agile/scrum","Documentation"]

Design, develop, and maintain scalable data pipelines and analytics solutions using Databricks. Build ETL/ELT workflows with Apache Spark (PySpark) to support batch and streaming processing, integrate data from multiple sources, and implement transformation, cleansing, and validation. Create and optimize Delta Lake tables with partitioning and Z-Ordering, develop reusable notebooks and jobs, and monitor/troubleshoot performance issues while following Git and CI/CD best practices.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Birlasoft
Birlasoft
2 months ago

Azure Subcontractor

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 5 hours agoStatus: Live

Job Summary

Design, develop, and maintain scalable data pipelines and analytics solutions using Databricks. Build ETL/ELT workflows with Apache Spark (PySpark) to support batch and streaming processing, integrate data from multiple sources, and implement transformation, cleansing, and validation. Create and optimize Delta Lake tables with partitioning and Z-Ordering, develop reusable notebooks and jobs, and monitor/troubleshoot performance issues while following Git and CI/CD best practices.
Location: Hyderabad
Workplace: Onsite
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure

Key Responsibilities

  • •Design, develop, and maintain ETL/ELT pipelines using Databricks.
  • •Develop scalable data processing frameworks using Apache Spark (PySpark).
  • •Build and optimize batch and streaming data pipelines.
  • •Develop Delta Lake tables and optimize storage using partitioning and Z-Ordering.
  • •Monitor and troubleshoot pipeline failures and performance issues while collaborating with stakeholders on data requirements.

Key Requirements

  • •Strong experience with Databricks and expertise in Apache Spark / PySpark.
  • •Proficiency in Python and strong SQL skills.
  • •Experience developing ETL/ELT pipelines and data warehousing concepts.
  • •Experience with Delta Lake and optimizing data storage (partitioning, Z-Ordering).
  • •Experience with Git and version control, following coding standards and CI/CD best practices.
Experience:Data engineeringData pipelinesETL/ELTCloud data engineeringStreaming data
Skills:Data modelingProblem solvingTeam collaborationAgile/scrumDocumentation
Tech Stack:DatabricksApache SparkPySparkPythonSQLETLELTDelta LakeZ-OrderingPartitioningNotebooksWorkflowsGitCI/CDAzure DatabricksADLS Gen2Azure Data FactoryAmazon S3AWS GlueAWS EMR

Company Brief

Birlasoft
Birlasoft is a global IT services and digital solutions provider delivering consulting, enterprise applications, cloud, and infrastructure services to enterprises across industries, helping accelerate digital transformation and business modernization.
Industry: Professional Services
Company Size: Enterprise (1,001+ employees)
Revenue: USD 500M to 1B
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Noida, India
WebsiteLinkedIn