Senior Lead Data Engineer- Data Platform

S&P Global
Hyderabad, Chennai
Workplace: OnsiteFull timeFunction: Administration & Executive AssistanceExperience: 8+ yearsSkills: ["Problem-solving","Troubleshooting","Communication","Collaboration","Mentoring","Documentation","Stakeholder management"]

Lead a hands-on data engineering team supporting offshore stakeholders in India. Own data pipeline onboarding, migration, and ongoing operations within a Databricks-on-AWS lakehouse environment. Help ensure reliable, performant, governed workflows spanning ingestion, transformation, metadata, lineage, and secure access. Partner across platform, governance, and cloud teams to troubleshoot production issues, implement best practices, and drive operational excellence using AWS and Databricks capabilities.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
S&P Global
S&P Global
1 month ago

Senior Lead Data Engineer- Data Platform

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 46 days agoStatus: Live

Job Summary

Lead a hands-on data engineering team supporting offshore stakeholders in India. Own data pipeline onboarding, migration, and ongoing operations within a Databricks-on-AWS lakehouse environment. Help ensure reliable, performant, governed workflows spanning ingestion, transformation, metadata, lineage, and secure access. Partner across platform, governance, and cloud teams to troubleshoot production issues, implement best practices, and drive operational excellence using AWS and Databricks capabilities.
Location: Hyderabad, Chennai
Workplace: Onsite
Employment Type: Full time
Job Function: Administration & Executive Assistance
Seniority: Manager level

Key Responsibilities

  • •Assist in migrating existing data pipelines to the Databricks platform and support ongoing operations, troubleshooting, and optimization post-migration.
  • •Support data onboarding for new assets using standardized processes, including validating and publishing assets into Bronze/Silver/Gold lakehouse layers.
  • •Enable platform integration support, including data quality checks, metadata management, reconciliation, and governed publishing of mastered datasets.
  • •Provide team collaboration and delivery leadership by prioritizing tasks, mentoring junior engineers, reviewing code, and communicating risks and progress.
  • •Drive operational excellence through deployment, monitoring, incident management support, production issue resolution, and pipeline observability using cloud-native tools.

Pay and Benefits

Perks:Health InsurancePaid LeaveLearning Budget

Key Requirements

  • •8+ years of experience in data engineering, pipeline development, or related roles.
  • •Proven experience supporting data onboarding, migration, and pipeline operations.
  • •Hands-on expertise with Spark, SQL, and Python (or similar data engineering tools).
  • •Experience working with Databricks and relevant AWS data engineering services, including Amazon S3 and AWS Glue, plus AWS IAM.
  • •Familiarity with Delta Lake, Databricks Unity Catalog, and exposure to Apache Iceberg or similar lakehouse table formats.
Experience:8+ years
Skills:Problem-solvingTroubleshootingCommunicationCollaborationMentoringDocumentationStakeholder management
Tech Stack:DatabricksAWSAmazon S3AWS GlueAWS IAMAWS Lake FormationDelta LakeUnity CatalogApache IcebergAmazon AthenaAWS CloudWatchCI/CDInfrastructure as codeAWS Step FunctionsAWS LambdaAmazon KinesisSparkSQLPythonPySpark

Company Brief

S&P Global
Provides financial information, analytics, benchmarks, and credit ratings to markets and institutions worldwide, offering data, research, indices, and risk assessment tools across the financial services and commodities sectors.
Industry: Data Infrastructure
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: New York, United States
Founded: 1917
WebsiteLinkedIn