Data/Full Stack Engineer, Data Storage & Ingestion Consultant

Eon Systems
San Francisco
Workplace: OnsiteFull timeUSD 100 - 300 hourlyFunction: Software EngineeringExperience: 5+ yearsSkills: ["Problem-solving","Collaboration","Attention to detail"]

Design and implement a high-throughput data pipeline for multi-petabyte microscopy datasets, from ingestion to long-term storage and downstream processing. Own strategy for on-prem vs cloud storage, BOM, and deployment, while ensuring reliability, cost controls, and integrate with acquisition workflows for segmentation and analysis.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Eon Systems
Eon Systems
10 months ago

Data/Full Stack Engineer, Data Storage & Ingestion Consultant

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 7 hours agoStatus: Live

Job Summary

Design and implement a high-throughput data pipeline for multi-petabyte microscopy datasets, from ingestion to long-term storage and downstream processing. Own strategy for on-prem vs cloud storage, BOM, and deployment, while ensuring reliability, cost controls, and integrate with acquisition workflows for segmentation and analysis.
Location: San Francisco
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering

Key Responsibilities

  • •Architect ingest & storage: Choose and implement an on-prem hardware and data pipeline design or a cloud/S3 alternative with explicit cost and performance tradeoffs at multi-petabyte scale.
  • •Set up a sustained-write ingest path ≥1 GB/s with adequate burst headroom (camera/frame-to-disk), including networking considerations, cooling, and throttling safeguards.
  • •Optimize footprint & cost: Incorporate on-the-fly compression/downsampling options and quantify CPU budget vs. write-speed tradeoffs; document when/where to compress to control $/PB.
  • •Integrate with acquisition workflows ensuring image data and metadata are compatible with downstream stitching/flat-field correction pipelines.
  • •Enable downstream compute: Expose the data to segmentation/analysis stacks (local GPU nodes or cloud).

Pay and Benefits

Salary: USD 100 - 300 hourly

Key Requirements

  • •5+ years designing and deploying high-throughput storage or HPC pipelines (≥1 GB/s sustained ingest) in production.
  • •Deep hands-on with NVMe RAID/striping, ZFS/MDRAID/erasure coding, PCIe topology, NUMA pinning, Linux performance tuning, and NIC offload features.
  • •Proven delivery of multi-GB/s ingest systems and petabyte-scale storage in production (life-sciences, vision, HPC, or media).
  • •Experience building tiered storage systems (NVMe → HDD/object) and validating real-world throughput under sustained load.
  • •Practical S3/object-storage know-how (AWS S3 and/or on-prem S3-compatible systems) with lifecycle, versioning, and cost controls.
Experience:5+ yearsLife-sciencesHPCMicroscopyImagingBig dataOn-prem
Skills:Problem-solvingCollaborationAttention to detail
Tech Stack:NVMeRAIDZFSMDRAIDPCIeNUMALinuxNIC offloadS3Erasure codingRDMARoCEUCSFContainerized

Company Brief

Eon Systems
Developing high-fidelity brain emulation by building large-scale connectomes and using neurobiological data plus AI to create digital emulations (initial targets include Drosophila and mouse models).
Industry: Scientific Research
Company Size: Micro (1 to 10 employees)
Growth: Early Stage Startup
Headquarters: San Francisco, United States
WebsiteLinkedIn