AI and infrastructure news
Filter by type: Paper273
- 19 days ago
arxiv.orgIn-context retrieval: study tests scale up to one million tokens
A paper published on July 6, 2026, examines language models as retrievers over one-million-token corpora. Its 0.6-billion-parameter BlockSearch generalizes up to 10 times beyond its training length, according to the study.
- 19 days ago arxiv.org
RLSD adjusts RLVR updates at the token level
A paper published on July 11, 2026 presents a training method that combines environmental rewards with self-distillation. Across five multimodal reasoning benchmarks, it reports higher average accuracy than the base model and GRPO.
- 19 days ago arxiv.org
RoLA proposes low-rank linear attention for Diffusion Transformers
Published on September 15, 2026, the paper presents RoLA, a method combining low-rank linear attention and rotary position encoding to address scaling bottlenecks in Diffusion Transformers.
- 19 days ago arxiv.org
RWKV combines parallel training with recurrent inference
An article published on September 20, 2026 describes an architecture combining Transformer-style parallel training with RNN-style recurrent inference, with memory and computation scaling linearly with sequence length.
Elastic sandboxes for training and evaluating agents
Published on September 23, 2026, the article describes stateful, elastic execution environments for large-scale agent training and evaluation, as well as defenses against agents exploiting training environments.
- 19 days ago arxiv.org
SearchEyes connects simulated worlds to agent training
The SearchEyes paper describes a simulated search world built on a typed knowledge graph for training multimodal agents. Published on July 8, 2026, it addresses gaps among training data, search environments, and reward signals in multi-hop reasoning.
- 19 days ago arxiv.org
On-Policy Self-Distillation for Reasoning in LLMs
A paper dated September 28, 2026 describes a training method that combines trajectories generated by the student with dense teacher supervision at each token.
- 19 days ago arxiv.org
LLM serving: cache and balancing after a year in production
A paper published on August 21, 2026 analyzes a year of production LLM traffic and examines prefix reuse, KV cache locality, and load balancing.
- 19 days ago arxiv.org
SFT in LLMs: tracking effects and deciding when to stop
A paper published on June 15, 2026 examines why supervised fine-tuning can benefit smaller models yet have inconsistent or harmful effects in LLMs.
- 19 days ago arxiv.org
SHIFT proposes gates for knowledge conflicts in RAG
Published on June 29, 2026, the paper presents an intervention in FFN activations to balance retrieved evidence and a model’s internal knowledge.
- 19 days ago arxiv.org
Softmax Linear Attention seeks better selection at linear cost
Published on September 16, 2026, the paper proposes applying softmax at the head level to restore global competition in linear attention while retaining linear complexity.
- 19 days ago arxiv.org
Speculative decoding may reduce inference steps
An article dated September 20, 2026 describes a technique that pairs proposals from an efficient model with parallel verification by a larger model to speed generation without changing the output distribution.
- 19 days ago arxiv.org
STEB compares style embeddings across 96 datasets
Published on July 1, 2026, the post presents an open-source benchmark evaluating style embeddings across 96 datasets and seven languages.
- 19 days ago arxiv.org
StreamDQ moves LLM dequantization into HBM memory
A paper published on July 13, 2026 presents an architecture that dequantizes LLM weights in real time on the HBM base die while preserving conventional GPU load semantics.
- 19 days ago arxiv.org
SVD-LLM proposes model compression with post-truncation updates
Published on September 26, 2026, the paper presents an SVD-based LLM compression method that updates compressed weights after truncating singular values.
- 19 days ago arxiv.org
Switch Attention dynamically combines full and windowed attention
A paper dated September 16, 2026 presents a dynamic, granular approach to combining full and sliding-window attention in hybrid transformers.
- 19 days ago arxiv.org
TAPA proposes token-aware phase attention for long contexts
Published on June 26, 2026, the paper presents Token-Aware Phase Attention (TAPA), a positional encoding method with a trainable phase function, and compares it with RoPE in long contexts.
- 19 days ago arxiv.org
Deep learning theory for analyzing model behavior
A book published on June 24, 2026, presents a theoretical framework linking deep neural network architecture and training to model outputs.
Transformer: from attention to implementation
Chapter 8 of Speech and Language Processing covers Transformer components from intuition to matrix formulation, providing a foundation for assessing how the architecture relates to business applications.
- 19 days ago arxiv.org
Transformers: attention and computational costs
An article published on August 2, 2026 explains the mathematical components of Transformers and discusses techniques to reduce attention processing and memory costs.
- 19 days ago arxiv.org
Looped Transformers: greater depth without more parameters
An article dated July 17, 2026 examines how looped Transformers reuse blocks to increase unfolded depth and how parameter sharing affects residual scaling.
- 19 days ago arxiv.org
Looped Transformers: reducing redundancy and cost
A paper published on September 26, 2026, identifies three types of computational redundancy across loops in looped Transformers. The authors report 1.65× lower latency and 6× lower memory use.
- 19 days ago arxiv.org
Recurrent Transformers as programmable computers
An article dated September 17, 2026 describes recurrent Transformer layers that emulate basic computing operations, including conditional branches and program counters.
- 19 days ago arxiv.org
Block-recurrent Transformers target linear scaling
An article published on September 17, 2026 presents a recurrent, block-based Transformer architecture with complexity linear in sequence length.
Three autonomy levels for agentic recommendations
A survey published on July 7, 2026 proposes a taxonomy of recommendation systems using LLM-based agents, organized by level of autonomy.
- 19 days ago arxiv.org
Trie execution plans may speed up retrieval pipeline experiments
An article published on July 2, 2026 presents a radix trie execution plan for PyTerrier that reuses shared prefixes and reports reductions of up to 26% in experiment time.
- 19 days ago arxiv.org
Tucker Attention frames attention variants through low-rank factorization
Published on September 16, 2026, the paper presents Tucker Attention as a tensor factorization approach that frames MHA, GQA, and MLA as specialized low-rank approximate attention methods.
- 19 days ago arxiv.org
Turn-PPO and training agents across multiple turns
An article published on June 21, 2026 presents Turn-PPO, a per-turn advantage estimation method for using PPO to reinforce-train agentic LLMs, and examines GRPO limitations in long-horizon tasks.
- 19 days ago arxiv.org
Universal Transformers reuse one layer recurrently
A post dated September 17, 2026 describes an architecture that repeatedly applies the same Transformer layer instead of stacking distinct layers, and discusses sequential and parallel computation in sequence models.
VeriHarness verifies agents on long-horizon tasks
A paper published on October 4, 2026 presents a method for checking agent claims against workspace evidence, without reference answers or evaluation rubrics during testing.
- 19 days ago arxiv.org
Vocabulary Transfer for improved sparse retrieval
A paper dated July 2, 2026, attributes advanced encoders’ weaker learned sparse retrieval performance to tokenizer vocabulary incompatibilities and proposes Vocabulary Transfer.
- 19 days ago github.com
WANDR: an environment for evaluating agentic search
In a post published on July 14, 2026, Perplexity says it is making available a reinforcement-learning and evaluation environment for agentic search, synthesized from production traces with weak human supervision.
WikiSkill: persistent knowledge for agent skills
Published on August 28, 2026, the WikiSkill description separates execution traces, a persistent knowledge wiki, and executable skills to reuse experience across iterations.
Let's assess your project
I reply with scope and a proposal.