HSBC

1 week ago

Consultant Specialist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 4 hours agoStatus: Live

Job Summary

Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery. Build and maintain batch/stream processing with PySpark, Spark SQL, and Structured Streaming, integrating Kafka CDC events and ensuring idempotent, ordered processing. Migrate MongoDB data, model and land data on Hadoop (HDFS/Hive), orchestrate pipelines with Airflow, and implement data quality/reconciliation controls. Optimize performance, stability, and production support through engineering excellence.

Location: Guangzhou

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Key Responsibilities

•Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery mechanisms.
•Develop and maintain batch/stream processing jobs with PySpark, Spark SQL, and Structured Streaming for cleansing, standardization, mapping, deduplication, and merge.
•Integrate with Kafka CDC events and ensure correct handling of out-of-order/duplicate/late events, replay, and idempotent processing.
•Migrate and incrementally ingest MongoDB data and map it to target models, including document parsing and schema evolution.
•Orchestrate pipelines with Airflow and build data quality/reconciliation controls while optimizing performance and stability.

Key Requirements

•Strong Python fundamentals and hands-on PySpark experience including DataFrames, Spark SQL, and performance troubleshooting.
•Proven experience with Spark Structured Streaming, including watermarking, stateful processing, checkpointing, and delivery semantics.
•Strong Kafka experience covering topic/partition/key design, consumer groups, offset management, replay, ordering, and idempotency.
•Hands-on CDC implementation experience for initial load, incremental continuity, backfill, restorability, and consistency guarantees.
•Experience with Airflow, Hadoop (HDFS/Hive/YARN), and working knowledge of MongoDB and Java (Spring Boot a plus).

Languages:English

Tech Stack:PythonPySparkSpark SQLStructured StreamingKafkaMongoDBJavaSpring BootHadoopHDFSHiveYARNAirflowJSONAvroHLDLLD

Company Brief

HSBC

Global banking and financial services organisation offering retail, commercial, corporate and investment banking, wealth management, and global markets services across Europe, Asia, the Americas and the Middle East.

Industry: Banking

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: London, United Kingdom

Founded: 1865

Glassdoor: 3.6

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

HSBC

1 week ago

Consultant Specialist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 4 hours agoStatus: Live

Job Summary

Location: Guangzhou

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Key Responsibilities

•Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery mechanisms.
•Develop and maintain batch/stream processing jobs with PySpark, Spark SQL, and Structured Streaming for cleansing, standardization, mapping, deduplication, and merge.
•Integrate with Kafka CDC events and ensure correct handling of out-of-order/duplicate/late events, replay, and idempotent processing.
•Migrate and incrementally ingest MongoDB data and map it to target models, including document parsing and schema evolution.
•Orchestrate pipelines with Airflow and build data quality/reconciliation controls while optimizing performance and stability.

Key Requirements

•Strong Python fundamentals and hands-on PySpark experience including DataFrames, Spark SQL, and performance troubleshooting.
•Proven experience with Spark Structured Streaming, including watermarking, stateful processing, checkpointing, and delivery semantics.
•Strong Kafka experience covering topic/partition/key design, consumer groups, offset management, replay, ordering, and idempotency.
•Hands-on CDC implementation experience for initial load, incremental continuity, backfill, restorability, and consistency guarantees.
•Experience with Airflow, Hadoop (HDFS/Hive/YARN), and working knowledge of MongoDB and Java (Spring Boot a plus).

Languages:English

Tech Stack:PythonPySparkSpark SQLStructured StreamingKafkaMongoDBJavaSpring BootHadoopHDFSHiveYARNAirflowJSONAvroHLDLLD

Company Brief

HSBC

Industry: Banking

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: London, United Kingdom

Founded: 1865

Glassdoor: 3.6

Website LinkedIn