AI and infrastructure news
Filter by type: Article20
- 16 days ago arxivb.org
LayerRoPE: encoding depth in normalization weights for very deep Transformers
The paper proposes LayerRoPE, which encodes depth in normalization weights to contain the growth of hidden-state norm across layers, reporting the loss of a 1.3B Pre-Norm model with 3.4x less compute and stable scaling up to 512 layers.
- 17 days ago alphaxiv.org
H-JEPA: hierarchical world models for long-horizon visual planning
The article, guided by Yann LeCun, proposes H-JEPA, with JEPA world models stacked across multiple time scales. The author reports a rise from 18% to 73% on Visual AntMaze with this hierarchical approach.
- 17 days ago manning.com
GRPO with verifiable rewards: how to test the method
Published on January 31, 2026, a chapter of Build a Reasoning Model (From Scratch) introduces GRPO from the ground up for reinforcement learning with verifiable rewards.
Using GRPO to extract receipt data as JSON
An article published on May 4, 2026 presents a GRPO fine-tuning workflow for extracting data from receipts with OCR and structuring it as JSON.
- 17 days ago turingpost.com
LeJEPA combines latent prediction and regularization
Published on November 22, 2025, the post describes LeJEPA, which combines JEPA's latent-space prediction loss with SIGReg to shape embeddings into an isotropic Gaussian distribution.
- 17 days ago huggingface.co
RoPE and model architecture decisions
A post published on April 12, 2026 presents an article as an explanation of rotary positional embeddings (RoPE), for engineers studying positional encoding.
- 17 days ago deeplearning.ai
Text-to-LoRA generates adapters from descriptions
Published on October 21, 2025, Sakana AI's study presents a method for generating task-specific LoRA adapters from natural-language descriptions and evaluates its results across 479 tasks.
- 17 days ago
unsloth.aiUnsloth: fine-tuning embedding models
On January 22, 2026, Unsloth documentation presented a guide to fine-tuning embedding models, including a Colab notebook for a 300-million-parameter model.
- 17 days ago turingpost.com
LoRA variants: comparing parameter-efficient tuning strategies
Published on April 26, 2026, the Turing Post article presents more than 15 LoRA variants, including QLoRA, DoRA, and Mixture-of-LoRA-Experts, and traces the evolution of parameter-efficient fine-tuning for large language models.
Detecting basketball elements with computer vision
A September 24, 2025 publication describes fine-tuning a detection model to identify players, jersey numbers, referees, the ball, and the hoop in basketball footage.
LLM post-training: methods and evaluation
A guide published on October 12, 2025, on adapting LLMs: from next-token prediction to instruction following, including training methods and evaluation.
Classifying documents at scale with few examples
A post published on September 10, 2026 describes labeling 200 examples with an agent and classifying 191,724 documents for about US$0.70 in inference compute.
- 18 days ago aleph-alpha.com
Test training scale before expanding GPU capacity
In a report published on October 5, 2026, Aleph Alpha describes scaling a 30B-A3B MoE model from 16 to 512 B200 GPUs, reporting near-linear scalability and 35.3% MFU.
- 18 days ago books.vizuara.ai
What pretraining a MoE model demands in practice
Published on August 20, 2026, the account documents an attempt to pretrain a 1.02-billion-parameter MoE model on an H200, covering its data pipeline, failures, and throughput tuning.
PostTrainBench highlights model performance and reliability
On June 25, 2026, a post about PostTrainBench reported close ranking scores and zero failed runs in 84 attempts for GLM 5.2.
- 18 days ago alphaxiv.org
Projection Sampling adapts demonstrations for SFT
A paper published on October 4, 2026 presents a way to rewrite expert demonstrations as correct task trajectories that are more likely under the target model, then use them in standard supervised fine-tuning.
- 18 days ago doi.org
Physical neural network of metasurfaces trained without digital models
The Nature Communications paper describes a physical neural network built from four programmable metasurfaces that learns through electromagnetic interactions to focus waves, recognize objects and perform localization, without digital models.
- 18 days ago zhihu.com
Long-horizon RL may favor PPO
A Zhihu post published on June 24, 2026, argues that long, noisy agentic rollouts may make credit assignment harder for GRPO and make PPO's value model useful.
RL for beneficial behavior across domains
In a post published on June 18, 2026, OpenAI reports that behavior-focused reinforcement learning improved alignment across domains and resisted adversarial pressure.
- 18 days ago alphaxiv.org
Self-play for pretraining models without natural data
A paper published on September 26, 2026 examines pretraining with program-generated byte sequences and reports transfer to unseen modalities and tasks.
tokens-for-thoughts.notion.site
Let's assess your project
I reply with scope and a proposal.