AI and infrastructure news
- Speech: TTS and ASRPost
FireRedChat: full-duplex voice for assistants
A post dated October 7, 2025 describes a modular voice interaction system with cascade and semi-cascade modes, interruption handling, end-of-speech detection, and real-time responses.
- RAG and retrievalPost
Baidu framework makes RAG more traceable
A Baidu post published on July 30, 2024 describes a framework that evaluates retrieved documents, selects evidence, and analyzes reasoning paths before answering.
- Training and fine-tuningPost
Using GEPA to detect malicious AI-generated code
A publication dated October 5, 2025 reports that prompts found with GEPA detected malicious behavior in AI-generated code, blocking 90% of malicious submissions with 1% of the audit budget.
- Attention and model architecturePaper
Gradient descent viewed as attention over examples
Published on February 6, 2023, the article interprets gradient-trained linear layers as memories of training examples and describes predictions as unnormalized attention over training experience.
- RAG and retrievalPost
Graphiti: temporal graphs for evolving data
Published on December 10, 2024, the Graphiti project describes dynamic knowledge graphs that represent relationships between entities over time and offer multiple search methods.
- Attention and model architecturePost
Grokking in modular addition: three stages to generalization
A post dated October 4, 2025 describes a paper proposing three stages of grokking in modular addition and a data threshold near n log n, with weight decay and moderate width supporting feature learning.
- Training and fine-tuningArticle
LLM post-training: methods and evaluation
A guide published on October 12, 2025, on adapting LLMs: from next-token prediction to instruction following, including training methods and evaluation.
- HardwarePost
High Bandwidth Flash: what to assess before 2026
In an article published on October 12, 2025, SanDisk proposes NAND-based memory with 8 to 16 times the capacity of HBM. First samples are expected in the second half of 2026.
- Speech: TTS and ASRProject
Hibiki: streaming speech translation
A repository published on February 6, 2025 presents a decoder-only model for simultaneous French-to-English and English-to-French speech translation, trained on synthetic data.
- RAG and retrievalPost
HijackRAG highlights risks in RAG systems
Published on December 1, 2024, the post describes an attack that uses injected texts, attention diversion, and instructions to manipulate RAG system outputs.
- SecurityPaper
HPAC-IDS: intrusion detection with hierarchical attention
An article published on January 14, 2025 describes a method that analyzes network-packet segments and reports promising results on CIC-IDS2017.
- LLM inferencePost
Concurrent Llama 3.1 8B inference on an RTX 3090
A Backprop post published on August 16, 2024 reports reasonable performance in a benchmark with more than 100 concurrent requests on a single GPU.
- LLM inferencePost
Layered inference on memory-limited GPUs
On September 25, 2024, the AirLLM publication described inference that loads one Transformer layer at a time from disk, as well as block quantization and compression support.
- LLM inferencePaper
InfLLM-V2 switches between dense and sparse attention
Published on October 13, 2025, the paper presents an attention system that switches between dense and sparse modes for short and long sequences.
- SecurityArticle
V8 Internals: Browser Security in Focus
Published on April 20, 2025, the article opens a three-part series on Chrome internals and browser exploitation, introducing V8 and JavaScript internals.
- LLM inferencePost
Jamba 1.5: 256K context and inference evaluation
On August 22, 2024, AI21 Labs announced the Jamba 1.5 Mini and Large models, combining Mamba and Joint Attention, with a 256K-token context window and features including ExpertInt8 and JSON output.
- Training and fine-tuningPost
JEPA-SCORE estimates density with JEPA encoders
A study published on October 8, 2025, presents a way to estimate sample probabilities using an encoder’s Jacobian, without retraining.
- CUDA and GPU programmingArticle
Matrix multiplication kernels on NVIDIA GPUs
An article published on October 1, 2025, on GPU architecture and kernel techniques for accelerating matrix multiplication.
- RAG and retrievalPost
LazyGraphRAG combines vector and graph search
A post published on November 29, 2024 describes an approach combining vector RAG with breadth-first and best-first iterative search, without index summarization.
- RAG and retrievalPaper
LightRAG combines graphs with two-level retrieval
A description published on October 13, 2024 presents a RAG framework combining graph-based text indexing with low- and high-level retrieval for entity-specific and conceptual information.
- HardwareProject
Linux-insides updates chapter on kernel boot
On September 24, 2025, the linux-insides project updated its fourth chapter on Linux kernel boot to cover modern kernel versions.
- LLM inferenceProject
LLM Compressor brings model compression techniques to vLLM
In a post published on August 16, 2024, Neural Magic introduced a library of compression techniques intended to reduce inference latency while preserving accuracy.
- Training and fine-tuningPost
LoRA adapts an MLP with about 9,000 trainable parameters
Published on July 30, 2024, the example applies LoRA to a 5-million-parameter MLP for classification and compares performance with the adapter enabled and disabled.
- Training and fine-tuningPost
LoRA for reinforcement learning fine-tuning
A Thinky Machines post published on September 29, 2025 describes LoRA configurations for reinforcement learning fine-tuning and claims rank 1 can match full fine-tuning performance when configured well.
- Speech: TTS and ASRPaper
MaskGCT: zero-shot speech synthesis and voice cloning
Published on October 24, 2024, the MaskGCT paper presents a text-to-speech model that the publication says can clone a voice from five seconds of speech.
- RAG and retrievalPost
Memory in AI agents: criteria for long-term projects
A survey published on August 11, 2025, by researchers at Renmin University of China and Huawei covers the design, evaluation, applications, limitations, and future directions of memory in LLM-based agents.
- AI agentsArticle
Meta presents open-weight 32-billion-parameter model for code research
On September 24, 2025, Meta presented Code World Model, a 32-billion-parameter model with open weights for research into code generation with world models.
- Speech: TTS and ASRPost
Mini-Omni-Reasoner combines reasoning and speech
A September 22, 2025 post describes an architecture that alternates silent reasoning with spoken tokens, alongside a new dataset and reported gains in arithmetic reasoning and contextual understanding.
- LLM inferenceProject
Mirage compiles LLMs into persistent megakernels
Published on June 19, 2025, the publication describes a compiler that transforms LLMs into persistent megakernels and claims latency reductions of 1.2 to 6.7 times.
- Attention and model architecturePaper
MoE-CL proposes continuous LLM adaptation
Published on October 6, 2025, the paper presents a continuous tuning method using LoRA experts and reports evaluations on the MTL5 and Tencent3 benchmarks, plus an A/B test.
- LLM inferenceArticle
Up to 72 TOPS of NPU capacity on Raspberry Pi 5
In a publication dated August 11, 2025, Sipeed introduced the Maix4-HAT, with an AXera AX650N NPU, up to 72 TOPS in INT4 or 18 TOPS in INT8, and 8 GB of LPDDR4x RAM for edge Transformer inference.
- Attention and model architecturePost
What attention sinks reveal about LLMs
Experiments on models from 410 million to 120 billion parameters link extreme activation norms of the initial token to compression valleys and attention sinks.
- RAG and retrievalProject
OpenScholar brings code, models and 45 million papers
On November 19, 2024, the OpenScholar repository presented code, model checkpoints, a collection of more than 45 million papers through October 2024, and the ScholarQABench benchmark.
- CUDA and GPU programmingArticle
Attention optimization: lessons from FlashAttention-4
In a post published on September 26, 2025, Modal describes reverse-engineering techniques for FlashAttention-4 and attributes a 20% performance gain to them.
- Speech: TTS and ASRPost
Parler-TTS: evaluating controllable voice models
Published on August 8, 2024, the announcement presents Mini and Large v1, models trained on 45,000 hours of speech, with voice controls set through text prompts.
- Training and fine-tuningPost
PEER retrieves over one million experts using product keys
The PEER architecture combines product-key routing and single-neuron MLP experts to sparsely select the components used for each input.
- CUDA and GPU programmingArticle
Penny and GPU communication: what to monitor
A post dated September 21, 2025, begins a development diary about Penny, a GPU communication library built manually.
- AI agentsPost
Research as configurable code: NVIDIA’s approach
On September 7, 2025, a NVIDIA post described a system that turns natural-language research strategies into executable functions, using search, interchangeable language models, and sandboxed execution.
- LLM inferencePost
PHLoRA extracts LoRA adapters from fine-tuned models
Published on September 17, 2025, the post describes a method for extracting low-rank adapters from fine-tuned checkpoints without training data or gradients.
- Speech: TTS and ASRPost
Piper offers local multilingual neural speech synthesis
Published on January 7, 2025, the text describes Piper, a local system that converts text to speech with multiple voices, languages, and customizable options.
arxiv.org
tokens-for-thoughts.notion.site
Let's assess your project
I reply with scope and a proposal.