NTT

3 months ago

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 5 hours agoStatus: Live

Reposted: first listed 5 hours ago

Job Summary

Own end-to-end datastore migration from on-prem DataLake to an AWS-hosted Lakehouse for a high-visibility program. Refactor extraction logic and pipeline scheduling, execute physical dataset transfers with integrity, and convert legacy SQL/Spark consumption patterns for Snowflake and Iceberg. Reconcile and validate data equivalence using reconciliation frameworks, work with internal data platform workflows, and partner with data owners through handoff and sign-off.

Location: Bengaluru

Workplace: Onsite

Employment Type: Full time

Job Function: Software Engineering

Seniority: Mid level

Key Responsibilities

•Support the datastore-migration Factory team to perform end-to-end migration from on-prem DataLake to an AWS-hosted Lakehouse.
•Refactor and migrate extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment.
•Execute physical migration of underlying datasets while ensuring data integrity and reconciliation.
•Translate and optimize legacy SQL and Spark-based consumption patterns for compatibility with Snowflake and Iceberg.
•Serve as a technical liaison to internal clients for handoff and sign-off, and collaborate on reconciliation frameworks to validate functional equivalence.

Key Requirements

•Bachelor’s or Master’s degree in Computer Science, Applied Mathematics, Engineering, or a related quantitative field.
•Minimum of 3–5 years of hands-on coding experience in a collaborative, team environment.
•Ability to troubleshoot SQL and perform basic scripting.
•Professional proficiency in Python or Java.
•Demonstrated knowledge of data modeling concepts (e.g., SCD Type 2, schema evolution with Iceberg, data partitioning/clustering, and normalization vs. denormalization).

Experience:3-5 yearsData engineering

Skills:Strong integrityCollaborates effectivelyClear communicationGlobal collaborationDelivery-focused ownership

Languages:PythonJava

Tech Stack:PythonJavaKafkaHadoopHDFSHiveSnowflakeApache IcebergIcebergApache SparkANSI SQLSQLSybase IQFTPJSONAvroParquetAWSKubernetesCI/CD

Company Brief

NTT

Dimension Data, operating under NTT Ltd, provides managed IT services, cloud and data center solutions, networking, cybersecurity, and digital transformation services to enterprise customers worldwide.

Industry: Professional Services

Company Size: Enterprise (1,001+ employees)

Revenue: USD 1B+

Growth: Established Company

Valuation: Public Company (Market Cap in USD)

Headquarters: London, United Kingdom

Founded: 1983

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

NTT