HSBC

1 week ago

Consultant Specialist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery. Build and maintain batch/stream data processing pipelines using PySpark, Spark SQL, and Structured Streaming, integrating Kafka CDC events and ensuring idempotent, out-of-order safe handling. Migrate MongoDB data into target models, implement Hadoop (HDFS/Hive/YARN) landing and modeling, orchestrate with Airflow, and enforce data quality and reconciliation controls.

Location: Xi'an

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery mechanisms.
•Develop and maintain batch/stream processing jobs using PySpark, Spark SQL, and Structured Streaming.
•Integrate with Kafka CDC events, including out-of-order/duplicate/late event handling, replay, and idempotent processing.
•Migrate and incrementally ingest MongoDB data using document parsing, schema evolution, and nested/array flattening.
•Orchestrate and govern data pipelines with Airflow and Hadoop, implementing data quality/reconciliation controls and performance/stability tuning.

Key Requirements

•Strong Python fundamentals and hands-on PySpark experience, including DataFrames, Spark SQL, RDD, tuning, and troubleshooting.
•Proven experience with Spark Structured Streaming, including watermarking, stateful processing, checkpointing, and delivery semantics.
•Strong Kafka experience across topic/partition/key strategy, consumer groups, offset management, replay, ordering, and idempotency.
•Hands-on CDC implementation experience covering initial load, incremental continuity, backfill, restorability, and consistency guarantees.
•Experience operating on-prem Hadoop (HDFS/Hive/YARN) and using Airflow DAG patterns for dependency/SLA management, alerting, retries, and backfills.

Skills:Code reviewsDocumentationEngineering practices

Languages:English

Tech Stack:PythonPySparkSpark SQLStructured StreamingKafkaCDCMongoDBJavaSpring BootJSONAvroHadoopHDFSHiveYARNAirflowDLQSQL

Company Brief

HSBC

Global banking and financial services organisation offering retail, commercial, corporate and investment banking, wealth management, and global markets services across Europe, Asia, the Americas and the Middle East.

Industry: Banking

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: London, United Kingdom

Founded: 1865

Glassdoor: 3.6

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

HSBC

1 week ago

Consultant Specialist

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 2 hours agoStatus: Live

Job Summary

Location: Xi'an

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Design and deliver CDMS migration and incremental sync solutions, including CDC cutover, replay/backfill, and recovery mechanisms.
•Develop and maintain batch/stream processing jobs using PySpark, Spark SQL, and Structured Streaming.
•Integrate with Kafka CDC events, including out-of-order/duplicate/late event handling, replay, and idempotent processing.
•Migrate and incrementally ingest MongoDB data using document parsing, schema evolution, and nested/array flattening.
•Orchestrate and govern data pipelines with Airflow and Hadoop, implementing data quality/reconciliation controls and performance/stability tuning.

Key Requirements

•Strong Python fundamentals and hands-on PySpark experience, including DataFrames, Spark SQL, RDD, tuning, and troubleshooting.
•Proven experience with Spark Structured Streaming, including watermarking, stateful processing, checkpointing, and delivery semantics.
•Strong Kafka experience across topic/partition/key strategy, consumer groups, offset management, replay, ordering, and idempotency.
•Hands-on CDC implementation experience covering initial load, incremental continuity, backfill, restorability, and consistency guarantees.
•Experience operating on-prem Hadoop (HDFS/Hive/YARN) and using Airflow DAG patterns for dependency/SLA management, alerting, retries, and backfills.

Skills:Code reviewsDocumentationEngineering practices

Languages:English

Tech Stack:PythonPySparkSpark SQLStructured StreamingKafkaCDCMongoDBJavaSpring BootJSONAvroHadoopHDFSHiveYARNAirflowDLQSQL

Company Brief

HSBC

Industry: Banking

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Public Company

Valuation: Public Company (Market Cap in USD)

Funding: IPO / Publicly Listed

Headquarters: London, United Kingdom

Founded: 1865

Glassdoor: 3.6

Website LinkedIn