AI and infrastructure news
Filter by type: Paper14
- 18 days ago arxiv.org
Fluxion proposes hybrid sparse attention for long-context inference
Published on May 11, 2026, the article presents a hybrid sparse-attention system for long-context inference using CPU–GPU parallelism.
- 18 days ago
arxiv.orgLoPA uses lookahead to accelerate diffusion model inference
Published on January 11, 2026, the text presents LoPA, a training-free algorithm that uses lookahead to select token-filling orders in diffusion language models.
- 18 days ago arxiv.org
InfLLM-V2 switches between dense and sparse attention
Published on October 13, 2025, the paper presents an attention system that switches between dense and sparse modes for short and long sequences.
- 18 days ago arxiv.org
Router-R1 explores reinforcement learning to route LLMs
Published on October 18, 2025, the text presents Router-R1, which alternates between reasoning and calls to multiple models and reports evaluations on seven question-answering benchmarks.
- 19 days ago arxiv.org
Test-time training layers for sequence modeling
A paper published on September 24, 2026 presents sequence-modeling layers with linear complexity and hidden states updated through self-supervised learning.
- 19 days ago arxiv.org
Dual-Cache: sharing context across LLMs
A paper published on September 26, 2026 presents a latent protocol for transferring information across heterogeneous models by translating KV caches and adding joint mixing between layers.
- 19 days ago arxiv.org
JetSpec speeds up speculative decoding with parallel draft trees
A post published on July 7, 2026 describes JetSpec, which uses a single forward pass to generate coherent draft trees and reports speedups of up to 9.64× on MATH-500 and 4.58× in open-ended conversations.
- 19 days ago aclanthology.org
KV cache: more bits for keys than values
An ACL 2026 paper examines KV cache compression and reports that using 4 bits for keys and 2 for values can work well, while reversing that allocation substantially degrades accuracy.
- 19 days ago arxiv.org
PagedAttention and KV cache management in LLM serving
An article published on September 24, 2026 presents PagedAttention, a virtual-memory-inspired approach to reducing KV cache fragmentation and duplication, which can limit batch size.
- 19 days ago arxiv.org
LLM serving: cache and balancing after a year in production
A paper published on August 21, 2026 analyzes a year of production LLM traffic and examines prefix reuse, KV cache locality, and load balancing.
- 19 days ago arxiv.org
Speculative decoding may reduce inference steps
An article dated September 20, 2026 describes a technique that pairs proposals from an efficient model with parallel verification by a larger model to speed generation without changing the output distribution.
- 19 days ago arxiv.org
StreamDQ moves LLM dequantization into HBM memory
A paper published on July 13, 2026 presents an architecture that dequantizes LLM weights in real time on the HBM base die while preserving conventional GPU load semantics.
- 19 days ago arxiv.org
SVD-LLM proposes model compression with post-truncation updates
Published on September 26, 2026, the paper presents an SVD-based LLM compression method that updates compressed weights after truncating singular values.
- 19 days ago arxiv.org
Switch Attention dynamically combines full and windowed attention
A paper dated September 16, 2026 presents a dynamic, granular approach to combining full and sliding-window attention in hybrid transformers.
Let's assess your project
I reply with scope and a proposal.