AI and infrastructure news
Filter by type: Paper273
- 19 days ago arxiv.org
Sparse memory expands RNN state without raising compute per token
Published on July 11, 2026, the post describes Sparse Delta Memory, which combines an RNN with explicit sparse-addressed memory to increase state capacity while keeping compute per token constant.
Proactive memory for long-horizon agents
A paper describes a separate memory agent that maintains structured records and decides when to present reminders to an action agent. The approach was evaluated on Terminal-Bench 2.0 and tau-squared-Bench.
Recurrent memory for long-context agents
A paper published on September 29, 2026 identifies retention as a bottleneck in recurrent-memory agents and presents a training-free approach that selects one memory head and preserves the others.
- 19 days ago
arxiv.orgMemory Attention proposes token memory for calculating values
In a post published on September 25, 2026, Memory Attention combines layer-specific token memory with contextual keys to form attention values, with potential KV cache reduction and CPU offloading.
- 19 days ago arxiv.org
MiniCPM-SALA combines sparse and linear attention for long contexts
A paper presents a hybrid architecture with 9 billion parameters and reports good results on long-context benchmarks, with scalable inference up to 1 million tokens.
- 19 days ago arxiv.org
Mixture-of-Depths adjusts compute per token in Transformers
An article dated September 22, 2026 presents a method that allocates compute across sequence positions and limits token participation in each layer.
Byte and token models: scaling results up to 1 trillion bytes
A paper published on September 14, 2026 compares distilled byte and token models and reports a higher performance ceiling for byte models as compute increases.
- 19 days ago arxiv.org
Mol-JEPA explores multimodal data for molecular models
A post published on August 25, 2026 reports that Mol-JEPA learns from molecular structures, cell phenotypes and binding affinities, with out-of-distribution gains on small ADME datasets.
- 19 days ago arxiv.org
NapMem proposes active navigation of conversational memory
An article published on July 8, 2026 presents NapMem, a framework that organizes a user's history into a linked memory pyramid with multiple levels of granularity.
- 19 days ago arxiv.org
Native Hybrid Attention and the long-context trade-off
An article published on October 5, 2026 presents Native Hybrid Attention, an approach aimed at balancing speed in long contexts with detail retention. Companies can assess the proposal using benchmarks tied to their tasks.
- 19 days ago huggingface.co
NatureBench evaluates coding agents on research
Published on June 24, 2026, the post describes a benchmark with 90 tasks based on articles from the Nature family. The best agent exceeded the published reference result on 17.8% of tasks.
- 19 days ago arxiv.org
Hard negatives for two-tower retrieval
A paper published on July 2, 2026 proposes using LLM-derived item clusters to generate hard negatives in real time for two-tower recommendation models.
NOVA applies verified agents to recommendation systems
A description published on June 26, 2026 presents an agent harness for evolving recommendation architectures under domain, interface, resource, and serving constraints.
- 19 days ago huggingface.co
onPanda enables token annotation and model inspection
Published on September 22, 2026, the onPanda repository presents an open-source tool for LLM data annotation and model inspection.
- 19 days ago huggingface.co
OPD with one query: what the paper proposes
Dated September 29, 2026, the paper tests on-policy distillation with a single training query across math, coding, instruction following, and agentic tool use.
- 19 days ago arxiv.org
ORPO combines SFT and preference alignment in one stage
Published on September 19, 2026, the paper introduces ORPO, a method that combines supervised fine-tuning and preference optimization without a reference model.
Agent orchestration can reduce costs and latency
Published on July 13, 2026, the paper reports lower costs and latency with equivalent quality across 22 tasks and six models after changing orchestration.
PACE predicts agent performance with few tasks
A July 6, 2026 post presents a method to estimate agent results from a small task set at lower cost than a full evaluation.
- 19 days ago arxiv.org
PagedAttention and KV cache management in LLM serving
An article published on September 24, 2026 presents PagedAttention, a virtual-memory-inspired approach to reducing KV cache fragmentation and duplication, which can limit batch size.
- 19 days ago arxiv.org
Paper reports gains from redistributing capacity across layers
Research on language models reports that cosine-shaped capacity allocation improved perplexity without increasing parameters or FLOPs.
- 19 days ago arxiv.org
Paper evaluates LLM-as-a-Judge feedback in training
Published on September 19, 2026, the paper studies using the language model itself to provide feedback during training and compares this approach with frozen reward models trained on human preferences.
- 19 days ago arxiv.org
Phoenix-VAD: detecting the end of speech in voice dialogues
An article published on September 4, 2026 presents an LLM-based model for semantic end-of-speech detection in full-duplex streaming voice interactions, using sliding-window training.
PolaFormer++ proposes polarity-aware features for linear attention
The paper dated September 26, 2026 presents a theoretical criterion for feature-map spikiness and evaluates linear attention across six families of visual tasks.
- 19 days ago arxiv.org
Positional encodings: comparing alternatives to RoPE
A study published on September 29, 2026 examines RoPE’s bias toward nearby tokens and alternatives evaluated across different configurations.
- 19 days ago kawine.github.io
Post-training and real-world model adaptation
Slides from an introductory lecture, published August 21, 2026, present post-training methods and adaptation questions for agents in changing environments.
- 19 days ago arxiv.org
PPO and GRPO: choices for long-horizon training
A post published on June 21, 2026 discusses GRPO’s infrastructure constraints and the declining effect of group-based variance reduction on long sequences, compared with PPO.
- 19 days ago arxiv.org
Executable programs for studying Transformer attention heads
A paper published on June 29, 2026 proposes approximating deep-network components with executable programs, applying the method to attention heads in Transformer language models.
- 19 days ago arxiv.org
Pruning or training small LLMs from scratch?
The paper compares six pruning methods for Llama-3.1-8B with training from scratch under two token-budget scenarios; the advantage depends on how the budget is counted.
- 19 days ago arxiv.org
PTXBench measures performance of AI-generated GPU kernels
Published on September 29, 2026, PTXBench evaluates matrix multiplication and attention kernels in architecture-specific PTX on H100 and B200 GPUs, measuring correctness, speed, and whether requested instructions are actually executed.
- 19 days ago arxiv.org
QLoRA: fine-tuning a 65-billion-parameter model on a 48 GB GPU
A paper dated September 20, 2026, reports fine-tuning a 65-billion-parameter model on a single 48 GB GPU with QLoRA, preserving performance on 16-bit fine-tuning tasks.
- 19 days ago arxiv.org
Quiet-STaR: training models to reason before responding
A paper published on September 22, 2026 describes an approach for training language models to generate internal reasoning before visible text, based on the idea that text often contains implicit reasoning.
- 19 days ago arxiv.org
Verified reasoning without updating model weights
A post published on June 23, 2026 proposes keeping a language model frozen and retrieving verified reasoning from an external pool, with logical verification as a key challenge.
- 19 days ago arxiv.org
Production RAG: evaluating retrieval improvements
A paper published on June 30, 2026, evaluates RAG retrieval strategies and whether their benefits hold across collections with varied formats.
- 19 days ago arxiv.org
Tracing training data to LLM circuits
Published on July 7, 2026, the paper presents a method for linking training samples to interpretable units in language models and examines effects on attention heads and in-context learning.
ReContext: improving evidence use in long contexts
Published on July 6, 2026, the ReContext paper reports improved evidence use across eight 128K datasets, with three model backbones and no training.
Red Queen Gödel Machine coevolves agents and evaluators
A paper published June 29, 2026 describes a method that evolves agents and evaluators, keeping evaluators fixed per epoch and replacing them only after better performance on reserved ground truth.
Rejection sampling for on-policy LLM alignment
A paper published on July 7, 2026 proposes using rejection sampling to treat accepted off-policy tokens as on-policy and address the variance of accumulated per-token importance sampling ratios.
RESA: recovering context with sparse attention
Published on September 29, 2026, the RESA paper proposes using residual estimation to recover context ignored by sparse attention, with no additional computational cost, according to the paper.
- 19 days ago arxiv.org
ResidencyRL explores clinical training through simulated consultations
Published on August 10, 2026, the article describes reinforcement learning in simulated clinical environments to train reasoning beyond static medical benchmarks.
- 19 days ago arxiv.org
RetNet proposes recurrence as an alternative to attention
Published on September 24, 2026, the Retentive Network paper presents a language-model architecture that combines parallel training with recurrent inference.
Let's assess your project
I reply with scope and a proposal.