LLM KV Cache Systems Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 3d ago

About the role

We are looking for an LLM KV Cache Systems Engineer to design and build KV Cache subsystems for large-language-model inference. You will work on memory allocation, lifecycle management, reuse, compression, transfer, tiering, and KV-aware scheduling across single-node and distributed serving systems.

This is a systems role. Direct source-level experience with an LLM KV Cache subsystem is more important than familiarity with serving-framework APIs alone.

What you will do

Design block/page allocators, KV Cache lifecycle management, fragmentation control, eviction, prefix caching, cross-request reuse, and preemption/recovery mechanisms.
Optimize KV layouts, GQA/MQA handling, quantization, compression, or sparsity while balancing accuracy, capacity, memory bandwidth, and compute overhead.
Build KV transfer and disaggregated-serving mechanisms, including prefill/decode separation, RDMA/NVLink/CXL transport, and GPU/CPU/SSD memory tiering or offload.
Develop KV-aware request routing, batching, admission control, scheduling, and multi-tenant isolation under latency and SLA constraints.
Integrate and productionize KV technologies in systems such as vLLM, SGLang, TensorRT-LLM, LMDeploy, Mooncake, or DistServe, with observability and failure handling.

What we are looking for

Master’s degree or Ph.D. in Computer Science, Computer Engineering, Distributed Systems, High-Performance Computing, or a related field.
Deep understanding of Transformer attention, prefill/decode execution, memory behavior, and modern LLM serving architectures.
Direct source-level design, implementation, or research experience in an LLM KV Cache subsystem.
Proficiency in C++/Python, Linux, CUDA or equivalent accelerator development, and distributed-system performance engineering.
Demonstrated results in at least one KV-specific metric: capacity, hit or reuse rate, fragmentation, transfer bandwidth, TTFT, TPOT, throughput, latency, memory cost, or service-level efficiency.

Preferred qualifications

Experience with disaggregated inference, distributed KV transfer, cache-aware routing, or GPU/CPU/SSD tiered KV storage.
Core open-source contributions, publications, patents, or large-scale production deployment in LLM serving or KV Cache systems.

Similar Jobs

Software Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 4d ago

Engineering C++ Programmer Full Time

Responsibilities You will be engaged in design and development of software core technologies. You will use advanced software design and development methodologies such as agile, DevOps, and open-source. You will be expose...

4d ago

Apply

Senior Research Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 2w ago

Engineering Full Time

Responsibilities To conduct cutting-edge research in both lightweight and traditional formal methods (e.g. formal specification and modelling, static/dynamic program analysis, software verification, software testing tech...

2w ago

Apply

Technical Officer (HKCRES0002)

The Hong Kong University of Science and Technology • Full Time • Tai Po District, HK • 2w ago

Engineering Full Time

Company Name: Hong Kong Center for Renewable Energy and Storage (HKCRES) Job Title: Technical Officer Ref. No.: HKCRES0002 About HKCRES The Hong Kong Center for Renewable Energy and Storage (HKCRES) is dedicated to advan...

2w ago

Apply

Executive Director

The Hong Kong University of Science and Technology • Full Time • Tai Po District, HK • 3w ago

Engineering Full Time

Company Name: Power Semiconductors and Applications Center (PowerSAC) Job Title: Executive Director Ref. No.: PowerSAC0006 The Power Semiconductors and Applications Center (PowerSAC), established under the SEAM@InnoHK in...

3w ago

Apply

Data Center Technician - HK - Tai Po - On-site

Engineering • Full Time • Tai Po District, HK • 5m ago

Engineering Full Time

About The Role We are seeking an experienced Data Center Technician for full-time work in Tai Po, HK . This is an on-site position supporting mission-critical infrastructure at premier data center facilities. Engagement...

5m ago

Apply

Recruitment Specialist

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 8h ago

Human Resources / Office Administration Administration Full Time Human Resources

Responsibilities Be responsible for internal recruitment needs of the company and work closely with different departments to understand their hiring needs. Use various recruiting channels and tools to screen and intervie...

8h ago

Apply

Senior Executive Assistant / Executive Assistant (s) (Full-Time / Part-Time) (HKCRES0004)

The Hong Kong University of Science and Technology • Part Time • Tai Po District, HK • 2w ago

Human Resources / Office Administration Part Time Full Time

Company Name: Hong Kong Center for Renewable Energy and Storage (HKCRES) Job Title: Senior Executive Assistant / Executive Assistant (s) (Full-Time / Part-Time) Ref. No.: HKCRES0004 Assist Head of Administration or Line...

2w ago

Apply

Assistant Finance Manager (HKCRES0003)

The Hong Kong University of Science and Technology • Full Time • Tai Po District, HK • 2w ago

Accounting / Audit Full Time

Company Name: Hong Kong Center for Renewable Energy and Storage (HKCRES) Job Title: Assistant Finance Manager Ref. No.: HKCRES0003 About HKCRES The Hong Kong Center for Renewable Energy and Storage (HKCRES) is committed...

2w ago

Apply