2 days ago

AI Infra Engineer - Large Model Inference Systems (Multimodal/LLM/VLM)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 8 hours agoStatus: Live

Job Summary

Build and evolve inference infrastructure for ultra-large-scale LLM, VLM, and multimodal models. Work on global scheduling across heterogeneous compute, high-concurrency load balancing, efficient batch formation, and low-latency online serving. Optimize distributed inference for 200B+ models using TP/EP/DP strategies, develop high-performance CUDA/Triton kernels for advanced architectures (e.g., MoE), and improve production reliability through deployment pipelines and intelligent operations.

Location: San Jose

Workplace: Onsite

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Build and evolve next-generation inference systems for large-scale online traffic, including global scheduling, high-concurrency load balancing, and efficient batch formation.
•Optimize distributed inference for 200B+ and complex multimodal models using TP/EP/DP strategies to improve throughput and latency in production.
•Develop high-performance kernels for frontier architectures such as MoE, emerging attention mechanisms, and multimodal fusion layers using CUDA and Triton.
•Explore AI-driven infrastructure for inference systems, including AI Agents for kernel optimization, performance tuning, consistency validation, deployment pipelines, and intelligent operations.

Key Requirements

•Bachelor's degree or above in Computer Science, Software Engineering, Artificial Intelligence, Mathematics, or a related field.
•2+ years of experience in high-performance computing, distributed scheduling systems, or large-model inference engine development.
•Familiarity with large-model architectures and strong system design skills for complex, high-concurrency environments.
•Strong understanding of asynchronous scheduling, resource pooling, and load balancing in distributed microservice systems.
•Strong engineering skills in performance optimization and production system development.

Experience:2+ yearsLarge-model inferenceDistributed systemsHigh-performance computingOnline serving

Education:Bachelor's

Skills:System designPerformance optimizationProduction reliabilityLoad balancingResource management

Tech Stack:CUDATritonCutlassVLLMSGLangTPEPDPMoEKV cachePTQQATPD disaggregation

Company Brief

TikTok

Short-form video platform that lets users create, share, and discover entertainment content through algorithmic recommendations. It also offers advertising and creator tools for brands, influencers, and businesses.

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn

Loading job details...

Preparing the role view and application actions.

TikTok

2 days ago

AI Infra Engineer - Large Model Inference Systems (Multimodal/LLM/VLM)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI

Last checked: 8 hours agoStatus: Live

Job Summary

Location: San Jose

Workplace: Onsite

Employment Type: Full time

Job Function: Data Science & Machine Learning

Seniority: Mid level

Key Responsibilities

•Build and evolve next-generation inference systems for large-scale online traffic, including global scheduling, high-concurrency load balancing, and efficient batch formation.
•Optimize distributed inference for 200B+ and complex multimodal models using TP/EP/DP strategies to improve throughput and latency in production.
•Develop high-performance kernels for frontier architectures such as MoE, emerging attention mechanisms, and multimodal fusion layers using CUDA and Triton.
•Explore AI-driven infrastructure for inference systems, including AI Agents for kernel optimization, performance tuning, consistency validation, deployment pipelines, and intelligent operations.

Key Requirements

•Bachelor's degree or above in Computer Science, Software Engineering, Artificial Intelligence, Mathematics, or a related field.
•2+ years of experience in high-performance computing, distributed scheduling systems, or large-model inference engine development.
•Familiarity with large-model architectures and strong system design skills for complex, high-concurrency environments.
•Strong understanding of asynchronous scheduling, resource pooling, and load balancing in distributed microservice systems.
•Strong engineering skills in performance optimization and production system development.

Experience:2+ yearsLarge-model inferenceDistributed systemsHigh-performance computingOnline serving

Education:Bachelor's

Skills:System designPerformance optimizationProduction reliabilityLoad balancingResource management

Tech Stack:CUDATritonCutlassVLLMSGLangTPEPDPMoEKV cachePTQQATPD disaggregation

Company Brief

TikTok

Industry: Digital Media

Company Size: Enterprise (1,001+ employees)

Growth: Scaleup

Headquarters: Singapore, Singapore

Founded: 2016

Website LinkedIn