AI and infrastructure news
Filter by type: All169
- 18 days ago arxiv.org
Transformers: attention and computational costs
An article published on August 2, 2026 explains the mathematical components of Transformers and discusses techniques to reduce attention processing and memory costs.
- 18 days ago
arxiv.orgLooped Transformers: greater depth without more parameters
An article dated July 17, 2026 examines how looped Transformers reuse blocks to increase unfolded depth and how parameter sharing affects residual scaling.
- 18 days ago arxiv.org
Looped Transformers: reducing redundancy and cost
A paper published on September 26, 2026, identifies three types of computational redundancy across loops in looped Transformers. The authors report 1.65× lower latency and 6× lower memory use.
- 18 days ago arxiv.org
Recurrent Transformers as programmable computers
An article dated September 17, 2026 describes recurrent Transformer layers that emulate basic computing operations, including conditional branches and program counters.
- 18 days ago arxiv.org
Block-recurrent Transformers target linear scaling
An article published on September 17, 2026 presents a recurrent, block-based Transformer architecture with complexity linear in sequence length.
Recurrent training improves reasoning benchmark results
A post published on September 16, 2026, reports that local denoising objectives improved recurrent models on six benchmarks, including 58.8% accuracy on ARC-AGI-1.
- 18 days ago arxiv.org
Tucker Attention frames attention variants through low-rank factorization
Published on September 16, 2026, the paper presents Tucker Attention as a tensor factorization approach that frames MHA, GQA, and MLA as specialized low-rank approximate attention methods.
TWT reduces compute for Vision Transformers
Published on September 26, 2026, the post describes Transformer-Within-Transformer, which replaces groups of similar layers. For DINOv2, it reports about half the compute with a small drop in accuracy.
- 18 days ago arxiv.org
Universal Transformers reuse one layer recurrently
A post dated September 17, 2026 describes an architecture that repeatedly applies the same Transformer layer instead of stacking distinct layers, and discusses sequential and parallel computation in sequence models.
Let's assess your project
I reply with scope and a proposal.