Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director

HSBC
Pune
Workplace: OnsiteFull timeFunction: DevOps, Cloud & InfrastructureEducation: bachelorsSkills: ["Leadership","Coaching","Stakeholder management","Communication","Interpersonal skills"]

Lead end-to-end data platform and pipeline architecture, delivering resilient batch/streaming solutions on GCP. Provide hands-on technical leadership across BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, and orchestration, monitoring, and runbooks to meet SLAs/SLOs. Drive data modelling, metadata, governance, encryption, auditability, and performance optimization for analytical workloads. Establish engineering quality bars, manage delivery dependencies, improve reliability and FinOps, and coach a team of data engineers.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
HSBC
HSBC
5 days ago

Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 2 hours agoStatus: Live

Job Summary

Lead end-to-end data platform and pipeline architecture, delivering resilient batch/streaming solutions on GCP. Provide hands-on technical leadership across BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, and orchestration, monitoring, and runbooks to meet SLAs/SLOs. Drive data modelling, metadata, governance, encryption, auditability, and performance optimization for analytical workloads. Establish engineering quality bars, manage delivery dependencies, improve reliability and FinOps, and coach a team of data engineers.
Location: Pune
Workplace: Onsite
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure
Seniority: Director level

Key Responsibilities

  • •Own end-to-end data platform and pipeline architecture for batch/streaming, defining standards and reference designs aligned to HSBC principles.
  • •Provide hands-on technical leadership for resilient ELT/ETL pipelines including orchestration, scheduling, monitoring, alerting, and runbook-driven operational support to meet SLAs/SLOs.
  • •Drive data modelling, metadata strategy, performance optimisation, and data governance requirements including lineage, access control, encryption, auditability, retention, and privacy-by-design.
  • •Establish engineering quality bars across CI/CD, automated testing, code reviews, data quality checks, release governance, and incident/problem management practices.
  • •Lead and develop a team of data engineers, manage dependencies across squads and third parties, and drive continuous improvement and FinOps for optimising GCP spend and scalability.

Key Requirements

  • •Bachelor’s degree in Computer Science/Engineering (Master’s preferred) and significant experience designing end-to-end data platform and pipeline architectures.
  • •Strong hands-on experience with GCP data/platform services including BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, and Cloud Storage, plus IAM, networking, and encryption/KMS.
  • •Proven expertise in data modelling (e.g., dimensional modelling, data vault/lakehouse patterns) for performant analytical workloads.
  • •Experience building and operating production-grade ELT/ETL pipelines with observability (monitoring/alerting), operational readiness, and runbook-driven support.
  • •Proven leadership managing and developing engineers, along with strong stakeholder management and communication across a complex organisation.
Experience:Data engineeringCloudGCPELT/ETLRegulated environments
Education:Bachelor's in Computer Science, Engineering, or a related field
Skills:LeadershipCoachingStakeholder managementCommunicationInterpersonal skills
Tech Stack:GCPBigQueryCloud StoragePub/SubDataflowDataprocComposerAirflowCloud RunGKEIAMCloud KMSVPCELTETLCI/CDMonitoringAlertingCIAutomated testing

Company Brief

HSBC
Global banking and financial services organisation offering retail, commercial, corporate and investment banking, wealth management, and global markets services across Europe, Asia, the Americas and the Middle East.
Industry: Banking
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: London, United Kingdom
Founded: 1865
Glassdoor
Glassdoor: 3.6
WebsiteLinkedIn