Embedded AI Engineer, On-Device Models

Deepgram
California
Workplace: RemoteFull timeUSD 219,300 - 274,100 annuallyFunction: Data Science & Machine LearningSkills: ["Communication","Builder mindset"]

Build and optimize Deepgram’s speech and conversational models for on-device, real-time inference on resource-constrained hardware. You’ll define on-device inference architecture across diverse processors and accelerators, apply techniques like quantization and pruning, write performance-critical C/C++/Rust runtime code for embedded/RTOS environments, and integrate with edge runtimes and vendor NPU/DSP toolchains. You’ll also own benchmarking, OTA deployment pipelines, and work with silicon partners and research to keep models edge-friendly.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Deepgram
Deepgram
2 months ago

Embedded AI Engineer, On-Device Models

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 21 hours agoStatus: Live

Job Summary

Build and optimize Deepgram’s speech and conversational models for on-device, real-time inference on resource-constrained hardware. You’ll define on-device inference architecture across diverse processors and accelerators, apply techniques like quantization and pruning, write performance-critical C/C++/Rust runtime code for embedded/RTOS environments, and integrate with edge runtimes and vendor NPU/DSP toolchains. You’ll also own benchmarking, OTA deployment pipelines, and work with silicon partners and research to keep models edge-friendly.
Location: California
Workplace: Remote
Employment Type: Full time
Job Function: Data Science & Machine Learning

Key Responsibilities

  • •Run Deepgram’s speech and conversational models on embedded and low-power consumer hardware by defining on-device, real-time inference architecture across processors and accelerators.
  • •Optimize models for constrained targets using techniques such as quantization, pruning, distillation, operator fusion, and architecture-specific compilation.
  • •Write and optimize performance-critical embedded runtime code in C, C++, and/or Rust, including bare-metal and RTOS environments (FreeRTOS, Zephyr).
  • •Integrate with edge inference runtimes and vendor NPU/DSP toolchains, mapping model graphs to on-device accelerators and CPU/GPU/NPU heterogeneity.
  • •Build on-device runtime plumbing (packaging, deployment pipelines, OTA updates, lightweight telemetry) and establish benchmarking/validation to prevent regressions.

Pay and Benefits

Salary: USD 219,300 - 274,100 annually
Equity and Bonus:Equity

Key Requirements

  • •Experience delivering production systems on resource-constrained hardware, including embedded systems, mobile, edge AI, or small low-power devices.
  • •Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
  • •Hands-on experience optimizing models for on-device deployment (e.g., quantization, pruning, knowledge distillation, or architecture-specific compilation).
  • •Familiarity with edge inference runtimes (ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
  • •Experience working close to the metal in bare-metal/RTOS (FreeRTOS, Zephyr), embedded Linux, or microcontroller/edge SoC development.
Experience:Voice AIEmbedded systemsEdge AIOn-device inference
Skills:CommunicationBuilder mindset
Tech Stack:CC++RustFreeRTOSZephyrONNX RuntimeTensorRTTFLiteExecuTorchQuantizationPruningKnowledge distillationOperator fusionModel compilationOver-the-air updates

Company Brief

Deepgram
Deepgram builds a real-time Voice AI platform delivering speech-to-text, text-to-speech, and voice-agent APIs for developers and enterprises, focusing on low-latency, high-accuracy voice models and scalable deployment options.
Industry: API Platforms
Company Size: Medium (51 to 250 employees)
Growth: Scaleup
Valuation: Unicorn (USD 1B+)
Funding: Series C
Headquarters: San Francisco, United States
Founded: 2015
Glassdoor
Glassdoor: 4.5
WebsiteLinkedInGlassdoor