AI and infrastructure news

All120

Training and fine-tuning

Training and fine-tuning Post
10/10/2615 days ago x.com
Social media post on RoboJEPA: predictable scaling in robotic world models
A social media post describes RoboJEPA, an 8B-parameter JEPA world model trained on 15,022 hours of robot video, and scaling-law fits that anticipate the results of the 4B and 8B models.
Training and fine-tuning Article
10/09/2616 days ago arxivb.org
LayerRoPE: encoding depth in normalization weights for very deep Transformers
The paper proposes LayerRoPE, which encodes depth in normalization weights to contain the growth of hidden-state norm across layers, reporting the loss of a 1.3B Pre-Norm model with 3.4x less compute and stable scaling up to 512 layers.
Training and fine-tuning Paper
10/09/2616 days ago arxiv.org
RoboJEPA: how scaling JEPA world models affects robotic planning
Meta presented RoboJEPA, a latent world model based on JEPA, scaled from 22M to 8B parameters and trained on data from 12 robotic embodiments. The paper tests whether latent prediction error predicts later planning performance.
Training and fine-tuning Article
10/08/2617 days ago alphaxiv.org
H-JEPA: hierarchical world models for long-horizon visual planning
The article, guided by Yann LeCun, proposes H-JEPA, with JEPA world models stacked across multiple time scales. The author reports a rise from 18% to 73% on Visual AntMaze with this hierarchical approach.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
Memory-Efficient Expert Routing for Distributed MoE Training
The arXiv paper proposes methods to reduce the memory peak in distributed Mixture-of-Experts training, where expert dispatch dominates memory use.
Training and fine-tuning Project
10/08/2617 days ago github.com
Unsloth notebooks for fine-tuning and RL across model types
The Unsloth repository gathers more than 250 notebooks for fine-tuning and RL on text, vision, audio, embedding and TTS models, serving as examples of training workflows.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
Anti-grokking: signs of generalization loss in model weights
A paper published on February 4, 2026 examines whether weight matrices can reveal late generalization collapse in two prolonged-training experiments.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
CE-GPPO aims to preserve gradients in LLM RL
Published on October 30, 2025, the paper presents CE-GPPO, a method addressing low-probability token gradients discarded by PPO clipping and coordinating policy entropy during training.
Training and fine-tuning Post
10/08/2617 days ago x.com
Gradient compression cuts network demand in pretraining
A post published on December 23, 2025, reports using gradient compression in pretraining while maintaining high efficiency with one quarter of the stated network bandwidth.
Training and fine-tuning Post
10/08/2617 days ago x.com
DAPO reports GRPO improvement on AIME, from 30% to 50%
In a paper published on March 1, 2026, DAPO reports that standard GRPO scored 30% on AIME in its experiments, while adding some techniques raised the result to 50%.
Training and fine-tuning Post
10/08/2617 days ago x.com
DroPE extends context without long-context fine-tuning
In a post published on January 12, 2026, DroPE removes positional embeddings after training and uses brief calibration to extend context. The post reports gains over baselines in 16k retrieval and question answering for models up to 7B.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
EB-JEPA explores world models through embedding predictions
Published on February 4, 2026, the article presents EB-JEPA, an open-source modular library for learning representations and world models with architectures that predict in representation space rather than pixel space.
Training and fine-tuning Article
10/08/2617 days ago manning.com
GRPO with verifiable rewards: how to test the method
Published on January 31, 2026, a chapter of Build a Reasoning Model (From Scratch) introduces GRPO from the ground up for reinforcement learning with verifiable rewards.
Training and fine-tuning Article
10/08/2617 days ago leoniemonigatti.com
Using GRPO to extract receipt data as JSON
An article published on May 4, 2026 presents a GRPO fine-tuning workflow for extracting data from receipts with OCR and structuring it as JSON.
Training and fine-tuning Post
10/08/2617 days ago x.com
GRPO: a parameter to investigate training stability
The INTELLECT-2 technical report, published on January 26, 2026, points to using delta=4.0 as a configuration to increase the stability of GRPO training.
Training and fine-tuning Post
10/08/2617 days ago x.com
Training-Free GRPO Uses Memory of Experiences
A post dated February 14, 2026 describes a method that keeps the model frozen and consults textual memory of successful and unsuccessful rollouts. It claims performance comparable to fine-tuning with 100 examples.
Training and fine-tuning Post
10/08/2617 days ago x.com
LeJEPA: SIGReg chunks cut VRAM use and runtime
A report published on March 4, 2026 says splitting 256 SIGReg slices into chunks of 32 used eight times less VRAM and finished faster on a 4090 MaxQ GPU.
Training and fine-tuning Article
10/08/2617 days ago turingpost.com
LeJEPA combines latent prediction and regularization
Published on November 22, 2025, the post describes LeJEPA, which combines JEPA's latent-space prediction loss with SIGReg to shape embeddings into an isotropic Gaussian distribution.
Training and fine-tuning Post
10/08/2617 days ago x.com
LeWorldModel: a compact JEPA trained from pixels
A May 9, 2026 publication describes a 15-million-parameter JEPA trained end to end on raw pixels and reports planning up to 48 times faster than foundation-model-based world models on control tasks.
Training and fine-tuning Post
10/08/2617 days ago x.com
LoRA as memory: evaluating data and rank
A post dated March 4, 2026 summarizes a study of how knowledge memory stored by LoRA scales and saturates with rank. QA and summaries produced more memory per token than raw passages.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
LoRA as knowledge memory: limits to evaluate
Published on March 3, 2026, the paper analyzes LoRA adapters for updating language-model knowledge and examines capacity limits, modular scaling, and interference when merging modules.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
Megatron-LM splits models across GPUs for training
A 2019 NVIDIA paper describes using model parallelism to train language models with billions of parameters by distributing weights across GPUs when they do not fit on one.
Training and fine-tuning Project
10/08/2617 days ago github.com
GRPO improvements in a PyTorch notebook
A notebook published on February 15, 2026 analyzes improvements to a GRPO implementation, including constrained policy ratios, a KL term, and format rewards.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
Ouroboros tests input-conditioned LoRA in recursive transformers
Published on April 25, 2026, the paper explores a compact controller that modulates a shared transformer block at each step. On Qwen2.5-3B, it reports lower training loss than a 17-layer baseline, with gains observed in-distribution.
Training and fine-tuning Post
10/08/2617 days ago x.com
South Korean program sets requirements for model training
Published on December 31, 2025, the article describes three requirements for teams in a South Korean model-training initiative: training from scratch, allowing some commercial use, and pursuing ambitious goals.
Training and fine-tuning Demo
10/08/2617 days ago huggingface.co
QED-Nano: what a 4B model can teach us about mathematical proofs
A post published on February 15, 2026 presents QED-Nano, a project to train a 4-billion-parameter model to prove olympiad problems, with material on training, datasets, and implementation.
Training and fine-tuning Post
10/08/2617 days ago x.com
QLoRA fine-tuning with 50 outputs from a teacher model
A post published on February 24, 2026 describes a local demonstration using 50 GPT-5.2 outputs to fine-tune a 3-billion-parameter Llama model with QLoRA.
Training and fine-tuning Docs
10/08/2617 days ago docs.primeintellect.ai
RL recipes to evaluate model training
Documentation published on February 25, 2026, presents reinforcement-learning training recipes for mathematical reasoning, code generation, and tool use.
Training and fine-tuning Post
10/08/2617 days ago x.com
Stepwise rewards for flow matching training
A description published on February 15, 2026 presents TurningPoint-GRPO, a framework that identifies steps with sign changes in local reward to capture long-term dependencies.
Training and fine-tuning Post
10/08/2617 days ago x.com
Ring-linear 2.0: hybrid attention and FP8 optimization
In a post published on October 27, 2025, Ant Group describes two models combining linear and softmax attention and claims improved training efficiency with FP8 operators.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
RobustMerge combines specialist MLLMs without retraining
Published on November 15, 2025, the post describes a method for combining multimodal models fine-tuned for specific tasks into a multitask model without further training.
Training and fine-tuning Article
10/08/2617 days ago huggingface.co
RoPE and model architecture decisions
A post published on April 12, 2026 presents an article as an explanation of rotary positional embeddings (RoPE), for engineers studying positional encoding.
Training and fine-tuning Post
10/08/2617 days ago x.com
SAM in mid-training may reduce forgetting during fine-tuning
A May 8, 2026 post says applying SAM during the final 10% of mid-training reduced forgetting by over 35% after fine-tuning or quantization.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
Lower SFT learning rates may preserve general capabilities
A study published on October 23, 2025, reports that lower learning rates may retain LLMs’ general capabilities while maintaining specialized performance; it also presents the TALR method.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
SHINE generates LoRA adapters from context in one pass
Published on April 6, 2026, the paper presents a hypernetwork that maps contexts to LoRA adapters while reusing a language model's frozen parameters.
Training and fine-tuning Post
10/08/2617 days ago x.com
TC-JEPA: captions guide visual learning
A post published on May 9, 2026 describes a self-supervised method that uses captions to guide the prediction of masked image patches. According to the post, the approach improves training stability and visual reasoning compared with contrastive methods.
Training and fine-tuning Article
10/08/2617 days ago deeplearning.ai
Text-to-LoRA generates adapters from descriptions
Published on October 21, 2025, Sakana AI's study presents a method for generating task-specific LoRA adapters from natural-language descriptions and evaluates its results across 479 tasks.
Training and fine-tuning Post
10/08/2617 days ago x.com
Tile Kernels optimize GPU operations for LLMs
On April 23, 2026, the company published GPU kernels developed with TileLang for LLM operations and said some are used in its internal training and inference.
Training and fine-tuning Article
10/08/2617 days ago unsloth.ai
Unsloth: fine-tuning embedding models
On January 22, 2026, Unsloth documentation presented a guide to fine-tuning embedding models, including a Colab notebook for a 300-million-parameter model.
Training and fine-tuning Article
10/08/2617 days ago turingpost.com
LoRA variants: comparing parameter-efficient tuning strategies
Published on April 26, 2026, the Turing Post article presents more than 15 LoRA variants, including QLoRA, DoRA, and Mixture-of-LoRA-Experts, and traces the evolution of parameter-efficient fine-tuning for large language models.

Let's assess your project

I reply with scope and a proposal.