4 months ago

Data Engineer - Foundational

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Data Engineer on the Foundational team will build and optimize massive data infrastructure to support multi-modal EO/IR video processing for foundation models. Based in Paris, you’ll create high-throughput ETL/ELT pipelines, synchronize EO/IR data, enable >90% GPU utilization, and implement distributed processing with Spark/Ray/Beam while ensuring data quality, versioning, and cost-efficient storage.

Location: Paris

Workplace: Onsite

Employment Type: Full time

Key Responsibilities

•Multi-Modal Ingestion Pipeline: Build ETL/ELT pipelines to extract, decode, and store raw EO and IR video into optimised formats like WebDataset, TFRecords, or Parquet.
•Sensor Synchronisation & Alignment: Develop algorithms to synchronise EO and IR frames temporally and spatially to provide paired inputs for model training.
•High-Throughput Data Loading: Architect storage-to-GPU pipelines to ensure multi-node training clusters maintain >90% GPU utilisation without I/O bottlenecks.
•Distributed Processing: Write and optimise distributed data processing jobs using Spark, Ray, or Beam to process thousands of hours of tactical video logs.
•Data Quality & Versioning: Implement automated quality checks to filter corrupted or blank frames and maintain reproducible training runs through versioning and lineage tracking.

Key Requirements

•Educational Background: BS or MS in Computer Science, Software Engineering, or Distributed Systems is highly preferred; deep knowledge of operating systems, networking, and parallel computing is essential.
•Technical Experience: 5-6+ years of experience building and maintaining terabyte-scale pipelines for unstructured data (video, images, or point clouds).
•Performance Optimisation: Proven track record of maximising multi-node GPU utlisation and optimising data loaders for PyTorch or JAX.
•Tooling Expertise: Strong command of distributed computing tools (Spark, Ray, Beam) and ML data versioning tools (DVC, Apache Iceberg, or Pachyderm).
•Adaptability & Ownership: A systems-thinker who thrives in a fast-paced startup environment and views messy data as an engineering problem to be solved via automation.

Experience:5-6 yearsDefenseAIMachine learningVideo

Education:Bachelor's

Skills:OwnershipProblem-solvingCommunicationTeamwork

Languages:English

Tech Stack:PythonPyTorchJAXApache SparkRayApache BeamDVCApache IcebergPachydermMinIOS3TFRecordsWebDatasetParquet

Company Brief

Harmattan AI

Develops AI-driven autonomous military systems (drones, electronic warfare, ISR and mission-management software) and scales conflict-ready production for allied armed forces and defense primes.

Industry: Defense Technology

Company Size: Medium (51 to 250 employees)

Growth: Scaleup

Valuation: Unicorn (USD 1B+)

Funding: Series B

Headquarters: Paris, France

Founded: 2024

Website LinkedIn Glassdoor

Loading job details...

Preparing the role view and application actions.

Harmattan AI

4 months ago

Data Engineer - Foundational

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 1 hour agoStatus: Live

Job Summary

Location: Paris

Workplace: Onsite

Employment Type: Full time

Key Responsibilities

•Multi-Modal Ingestion Pipeline: Build ETL/ELT pipelines to extract, decode, and store raw EO and IR video into optimised formats like WebDataset, TFRecords, or Parquet.
•Sensor Synchronisation & Alignment: Develop algorithms to synchronise EO and IR frames temporally and spatially to provide paired inputs for model training.
•High-Throughput Data Loading: Architect storage-to-GPU pipelines to ensure multi-node training clusters maintain >90% GPU utilisation without I/O bottlenecks.
•Distributed Processing: Write and optimise distributed data processing jobs using Spark, Ray, or Beam to process thousands of hours of tactical video logs.
•Data Quality & Versioning: Implement automated quality checks to filter corrupted or blank frames and maintain reproducible training runs through versioning and lineage tracking.

Key Requirements

•Educational Background: BS or MS in Computer Science, Software Engineering, or Distributed Systems is highly preferred; deep knowledge of operating systems, networking, and parallel computing is essential.
•Technical Experience: 5-6+ years of experience building and maintaining terabyte-scale pipelines for unstructured data (video, images, or point clouds).
•Performance Optimisation: Proven track record of maximising multi-node GPU utlisation and optimising data loaders for PyTorch or JAX.
•Tooling Expertise: Strong command of distributed computing tools (Spark, Ray, Beam) and ML data versioning tools (DVC, Apache Iceberg, or Pachyderm).
•Adaptability & Ownership: A systems-thinker who thrives in a fast-paced startup environment and views messy data as an engineering problem to be solved via automation.

Experience:5-6 yearsDefenseAIMachine learningVideo

Education:Bachelor's

Skills:OwnershipProblem-solvingCommunicationTeamwork

Languages:English

Tech Stack:PythonPyTorchJAXApache SparkRayApache BeamDVCApache IcebergPachydermMinIOS3TFRecordsWebDatasetParquet

Company Brief

Harmattan AI

Develops AI-driven autonomous military systems (drones, electronic warfare, ISR and mission-management software) and scales conflict-ready production for allied armed forces and defense primes.

Industry: Defense Technology

Company Size: Medium (51 to 250 employees)

Growth: Scaleup

Valuation: Unicorn (USD 1B+)

Funding: Series B

Headquarters: Paris, France

Founded: 2024

Website LinkedIn Glassdoor