AI and infrastructure news
- HardwareArticle
On-device AI in education: from investment to deployment
On November 18, 2025, a post reported that Kneron invested in Innovedeus to bring schools a Raspberry Pi-like platform for on-device ML and AI.
- SecurityArticle
Using AI to identify gaps in threat detection
In an article published on January 31, 2026, Microsoft describes an AI workflow that extracts TTPs from reports, compares them with existing detections, and flags potential gaps for human review.
- Reinforcement learningPost
AI may discover reinforcement learning algorithms
A Nature article published on October 28, 2025, reports autonomous discovery of reinforcement learning algorithms and points to a potential new approach for engineers.
- RAG and retrievalPost
Iceberg broadens vector search evaluation
On December 21, 2025, researchers from Alibaba and partners presented Iceberg, a suite with eight datasets that evaluates vector search through its effects on downstream tasks, beyond recall and latency.
- LLM inferenceArticle
Unlinkable inference to protect user identity
In a post published on February 26, 2026, OpenAnonymity describes a layer designed to prevent AI providers from linking inference calls to individual users.
- LLM inferenceArticle
Production AI inference requires end-to-end engineering
Published on March 3, 2026, Philip Kiely’s book covers the hardware, software, techniques, and infrastructure for production inference systems, focusing mainly on language models with some coverage of diffusion.
- LLM inferenceProject
Running Llama 70B on an RTX 3090 with ntransformer
On February 22, 2026, the ntransformer repository page says its C++/CUDA inference engine runs Llama 70B on an RTX 3090, transferring data from NVMe to the GPU without passing through the CPU.
- Speech: TTS and ASRProject
C inference enables integrated speech transcription
The antirez/qwen-asr repository provides C inference for 0.6B and 1.7B speech-recognition models, with potential use in transcription bots and voice-controlled interfaces.
- LLM inferencePost
Local inference: performance and power on an RTX 3090
Tests of eight local LLMs on one RTX 3090 compared throughput and energy efficiency at power limits ranging from 100 W to 450 W.
- CUDA and GPU programmingArticle
ONNX inference: comparing CPU and GPU
Published on March 11, 2026, the article compares ONNX Runtime inference on an NVIDIA L4 GPU and an AMD EPYC 9965 CPU. The author reports that OCaml and memory placement with libnuma made CPU inference faster than using scarce GPUs at TESSERA.
- Attention and model architecturePaper
Instability in low-precision Flash Attention
Published on March 15, 2026, the paper examines catastrophic loss spikes in low-precision training with Flash Attention, linking them to similar internal attention data patterns and accumulating biased rounding errors.
- C and C++Paper
Integer overflow in C and C++: controls to reduce risk
A 2012 University of Utah article discusses integer overflow in C and C++, a relevant risk when writing and reviewing code in these languages.
- HardwareArticle
Intel Arc Pro B60: evaluating 24 GB for workstations
An article dated November 6, 2025, says the Intel Arc Pro B60 workstation GPU, with 24 GB of memory, was expected to reach Europe between mid and late November, starting at €769.
- C and C++Paper
Intel updates the AVX10.2 specification to version 6.0
In a post published on March 11, 2026, Intel announced version 6.0 of the AVX10.2 specification and the renaming of AVX10_VNNI_INT to AVX10_V1_AUX.
- C and C++Article
Intel documents processor-instruction access through C functions
Published on February 17, 2026, Intel’s documentation presents C-style functions that provide access to processor instructions without writing assembly.
- HardwarePaper
Intel publishes HPC tuning guide for Xeon 6
On December 29, 2025, Intel released version 1.2a of its tuning and optimization guide for high-performance computing applications on Xeon 6 systems.
- Attention and model architecturePaper
Interleaved Head Attention explores communication between heads
A post published on April 11, 2026, explores communication between attention heads within a layer, focusing on reasoning with Interleaved Head Attention.
- SecurityArticle
AWS intrusion: exposed credentials lead from S3 to Bedrock
In an article published on February 4, 2026, Sysdig describes an AWS intrusion that began with credentials exposed in S3, moved through Lambda, and reached Bedrock access. The article highlights an Elastic rule for detecting external layers added to Lambda functions.
- SecurityPaper
Gradient inversion in Federated Learning: risks and defenses
Published on January 16, 2026, the post presents a study of gradient inversion attacks in Federated Learning. It says the most practical attack method remains unreliable and that the work proposes a three-stage defense.
- HardwarePost
IPUs and small batches: cost versus ease of use
A post published on December 29, 2025 describes Graphcore IPUs as massively parallel accelerators suited to small batches, and says used units may cost little but can be difficult to use.
- C and C++Docs
GCC jump threading: evaluating its effects on generated code
A Red Hat Developer article published on April 2, 2026, explains how jump threading aims to reduce dynamically executed jumps across different control-flow paths.
- Reinforcement learningPaper
JustRL tests simple RL on 1.5B reasoning models
Published on January 4, 2026, the paper describes single-stage training with fixed hyperparameters and reports average accuracies of 54.9% and 64.3% across two 1.5B reasoning models.
- Attention and model architecturePaper
KArAt proposes learnable attention without fixed softmax
Published on October 27, 2025, the paper presents KArAt, a learnable alternative to softmax attention. It reports mixed results in vision transformers and low-rank variants intended to reduce memory use.
- Attention and model architecturePost
Kascade speeds inference with sparse attention
Published on December 24, 2025, the paper describes a training-free method that reuses key patterns across layers. It reports up to 4.1× faster generation and 2.2× faster prefill on H100 GPUs.
- Attention and model architecturePaper
Kascade reduces attention computation in long contexts
Published on February 18, 2026, the article describes a technique that reuses attention results across Transformer layers to reduce inference work without retraining the model or changing its weights.
- CUDA and GPU programmingArticle
CUDA array-sum kernels: from example to practice
Published on November 16, 2025, Tensara’s article introduces beginners to writing a simple CUDA kernel for adding arrays. The example can be a starting point for evaluating GPU workloads.
- CUDA and GPU programmingArticle
CUDA/PTX matmul kernel reports 6% gain on B200
A post published on April 27, 2026 describes a pure CUDA/PTX matrix multiplication kernel that reportedly outperformed cuBLAS by 6% on a B200 with M=N=K=8192.
- C and C++Article
A minimal Zig time-sharing kernel for RISC-V
Published on May 12, 2026, the article describes building a minimal time-sharing kernel in Zig for RISC-V on OpenSBI, including S-mode traps and timer-triggered context switching.
- Attention and model architectureProject
Triton kernels and PyTorch operators for AttnRes
Published on May 3, 2026, the repository provides Triton kernels and PyTorch operators for Block Attention Residuals (AttnRes).
- HardwareArticle
Embedded kit combines modules and AI accelerators
Published on December 30, 2025, the article describes a kit combining computing modules and accelerators to evaluate LLM and VLM workloads on embedded systems.
- LLM inferenceArticle
KV cache: less recomputation in LLM inference
Published on May 9, 2026, the article explains how KV cache reuses previously computed values and why that matters for memory and computation during inference.
- Reinforcement learningPost
LaSeR explores last-token rewards for LLMs
In a post published on October 19, 2025, Tencent presents LaSeR, a reinforcement learning algorithm that aligns last-token scores with real rewards and uses one extra token inference.
- CUDA and GPU programmingDocs
CuTe layouts help explain shapes and strides
A NVIDIA post published on February 3, 2026 recommends layout algebra tutorials for understanding how CuTe represents tensors.
- SecurityProject
Leaker and passive enumeration of exposed credentials
Published on January 12, 2026, the repository describes a tool for finding credential leaks associated with email addresses through passive online sources.
- Training and fine-tuningPost
LeJEPA: SIGReg chunks cut VRAM use and runtime
A report published on March 4, 2026 says splitting 256 SIGReg slices into chunks of 32 used eight times less VRAM and finished faster on a 4090 MaxQ GPU.
- Training and fine-tuningArticle
LeJEPA combines latent prediction and regularization
Published on November 22, 2025, the post describes LeJEPA, which combines JEPA's latent-space prediction loss with SIGReg to shape embeddings into an isotropic Gaussian distribution.
- Speech: TTS and ASRArticle
LEMAS publishes a 150,000-hour speech dataset
A January 12, 2026 publication presents an audio dataset with 150,000 hours across 10 languages, word-level timestamps, and the generative models LEMAS-TTS and LEMAS-Edit.
- Training and fine-tuningPost
LeWorldModel: a compact JEPA trained from pixels
A May 9, 2026 publication describes a 15-million-parameter JEPA trained end to end on raw pixels and reports planning up to 48 times faster than foundation-model-based world models on control tasks.
- RAG and retrievalPaper
LightRetriever cuts the cost of LLM-based retrieval
A post published on March 5, 2026 reports query encoding 1,000× faster and 10× higher throughput with LightRetriever, while retaining 95% of benchmark performance.
- RAG and retrievalPaper
Geometric limits on compression in RAG
A NeurIPS 2026 paper studies compression in RAG through embedding geometry and says a spectral quantity defines a minimum limit for compression.
arxiv.org
Let's assess your project
I reply with scope and a proposal.