LLM KV Cache Systems Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 5d ago

About the role

We are looking for an LLM KV Cache Systems Engineer to design and build KV Cache subsystems for large-language-model inference. You will work on memory allocation, lifecycle management, reuse, compression, transfer, tiering, and KV-aware scheduling across single-node and distributed serving systems.

This is a systems role. Direct source-level experience with an LLM KV Cache subsystem is more important than familiarity with serving-framework APIs alone.

What you will do

Design block/page allocators, KV Cache lifecycle management, fragmentation control, eviction, prefix caching, cross-request reuse, and preemption/recovery mechanisms.
Optimize KV layouts, GQA/MQA handling, quantization, compression, or sparsity while balancing accuracy, capacity, memory bandwidth, and compute overhead.
Build KV transfer and disaggregated-serving mechanisms, including prefill/decode separation, RDMA/NVLink/CXL transport, and GPU/CPU/SSD memory tiering or offload.
Develop KV-aware request routing, batching, admission control, scheduling, and multi-tenant isolation under latency and SLA constraints.
Integrate and productionize KV technologies in systems such as vLLM, SGLang, TensorRT-LLM, LMDeploy, Mooncake, or DistServe, with observability and failure handling.

What we are looking for

Master’s degree or Ph.D. in Computer Science, Computer Engineering, Distributed Systems, High-Performance Computing, or a related field.
Deep understanding of Transformer attention, prefill/decode execution, memory behavior, and modern LLM serving architectures.
Direct source-level design, implementation, or research experience in an LLM KV Cache subsystem.
Proficiency in C++/Python, Linux, CUDA or equivalent accelerator development, and distributed-system performance engineering.
Demonstrated results in at least one KV-specific metric: capacity, hit or reuse rate, fragmentation, transfer bandwidth, TTFT, TPOT, throughput, latency, memory cost, or service-level efficiency.

Preferred qualifications

Experience with disaggregated inference, distributed KV transfer, cache-aware routing, or GPU/CPU/SSD tiered KV storage.
Core open-source contributions, publications, patents, or large-scale production deployment in LLM serving or KV Cache systems.

Similar Jobs

Software Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 5h ago

Engineering C++ Programmer Full Time

Responsibilities You will be engaged in design and development of software core technologies. You will use advanced software design and development methodologies such as agile, DevOps, and open-source. You will be expose...

5h ago

Apply

AI Acceleration Algorithm Engineer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 5d ago

Engineering Full Time

About the role We are looking for an AI Acceleration Algorithm Engineer to research, design, and productionize algorithms that improve end-to-end AI compute, retrieval, storage, and inference efficiency. The role emphasi...

5d ago

Apply

Human Resources Officer

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 3d ago

Human Resources / Office Administration Full Time

Key Responsibilities: HR Data Management & Reporting (50%) Extract HR system data to generate routine workforce reports, prepare and update HR reporting dashboards on a regular schedule. Conduct data review and clean...

3d ago

Apply

Campus Recruiter

Huawei Hong Kong Research Center(HKRC) • Full Time • Tai Po District, HK • 1w ago

Human Resources / Office Administration Full Time

Responsibilities - Support end-to-end campus recruitment, including sourcing, screening, interview coordination, and offer follow-up. - Build relationships with universities, career centers, and student organizations. -...

1w ago

Apply

Accountant

Bytelumio Limited • Full Time • Tai Po District, HK • 3w ago

Accounting / Audit Finance Accounting Full Time

About the Role We are seeking a detail-oriented and experienced Accountant to join our team. The ideal candidate will hold a recognized CPA qualification and possess 3 - 5 years of solid financial accounting experience....

3w ago

Apply