AI and infrastructure news
Filter by type: Project10
μLattice: an alternative to attention for memory scaling, according to its author
An open-source project and paper describe μLattice, a memory design that grows with input without quadratic cost. The performance claims come from the author and have not been independently verified.
Triton kernels and PyTorch operators for AttnRes
Published on May 3, 2026, the repository provides Triton kernels and PyTorch operators for Block Attention Residuals (AttnRes).
microGPT in C explores low-level optimizations
Published on May 3, 2026, the repository presents a dependency-free implementation of microGPT in plain C, using AVX2 intrinsics, fixed-width dot-product kernels, and a fast approximation of the exponential function.
Nanochat outlines architecture and optimization choices to investigate
Published on February 1, 2026, the post about nanochat lists model architecture components and describes splitting optimization between Muon and AdamW.
Compact reasoning could lower inference costs
A post published on April 22, 2026 describes Latent Reasoning Tuning, a method that trains models to reason with compact internal representations and claims gains over other efficient reasoning methods on mathematics and general benchmarks.
Scaled dot-product attention: from mechanism to practice
A post published on February 1, 2026 explains how queries, keys, and values guide token relevance and presents a notebook for implementing LLM and NLP systems from first principles.
Continual learning in LLMs with iSDFT
A repository published on September 22, 2026 presents a self-distillation method for continual learning in LLMs, with training code, checkpoints, and evaluations.
Continuous learning on a notebook with 8 GB of VRAM
The mini-AGI repository describes a model trained from scratch on a notebook with 8 GB of VRAM, using batch size 1, disk-paged weights, and learning-rate adjustments to reduce forgetting.
Recurrent state could extend Transformer memory
The description of the Recurrent Looped Transformer, dated September 13, 2026, presents a recurrent decoder that preserves state between tokens to extend the temporal information path.
OpenMythos explores a recurrent-depth Transformer
Published on June 19, 2026, the repository presents an open-source theoretical reconstruction featuring recurrent depth, MoE routing, and adaptive halting per token.
Let's assess your project
I reply with scope and a proposal.