AI and infrastructure news

Post46

Training and fine-tuning

Training and fine-tuning Post
10/10/2616 days ago x.com
Social media post on RoboJEPA: predictable scaling in robotic world models
A social media post describes RoboJEPA, an 8B-parameter JEPA world model trained on 15,022 hours of robot video, and scaling-law fits that anticipate the results of the 4B and 8B models.
Training and fine-tuning Post
10/08/2618 days ago x.com
Gradient compression cuts network demand in pretraining
A post published on December 23, 2025, reports using gradient compression in pretraining while maintaining high efficiency with one quarter of the stated network bandwidth.
Training and fine-tuning Post
10/08/2618 days ago x.com
DAPO reports GRPO improvement on AIME, from 30% to 50%
In a paper published on March 1, 2026, DAPO reports that standard GRPO scored 30% on AIME in its experiments, while adding some techniques raised the result to 50%.
Training and fine-tuning Post
10/08/2618 days ago x.com
DroPE extends context without long-context fine-tuning
In a post published on January 12, 2026, DroPE removes positional embeddings after training and uses brief calibration to extend context. The post reports gains over baselines in 16k retrieval and question answering for models up to 7B.
Training and fine-tuning Post
10/08/2618 days ago x.com
GRPO: a parameter to investigate training stability
The INTELLECT-2 technical report, published on January 26, 2026, points to using delta=4.0 as a configuration to increase the stability of GRPO training.
Training and fine-tuning Post
10/08/2618 days ago x.com
Training-Free GRPO Uses Memory of Experiences
A post dated February 14, 2026 describes a method that keeps the model frozen and consults textual memory of successful and unsuccessful rollouts. It claims performance comparable to fine-tuning with 100 examples.
Training and fine-tuning Post
10/08/2618 days ago x.com
LeJEPA: SIGReg chunks cut VRAM use and runtime
A report published on March 4, 2026 says splitting 256 SIGReg slices into chunks of 32 used eight times less VRAM and finished faster on a 4090 MaxQ GPU.
Training and fine-tuning Post
10/08/2618 days ago x.com
LeWorldModel: a compact JEPA trained from pixels
A May 9, 2026 publication describes a 15-million-parameter JEPA trained end to end on raw pixels and reports planning up to 48 times faster than foundation-model-based world models on control tasks.
Training and fine-tuning Post
10/08/2618 days ago x.com
LoRA as memory: evaluating data and rank
A post dated March 4, 2026 summarizes a study of how knowledge memory stored by LoRA scales and saturates with rank. QA and summaries produced more memory per token than raw passages.
Training and fine-tuning Post
10/08/2618 days ago x.com
South Korean program sets requirements for model training
Published on December 31, 2025, the article describes three requirements for teams in a South Korean model-training initiative: training from scratch, allowing some commercial use, and pursuing ambitious goals.
Training and fine-tuning Post
10/08/2618 days ago x.com
QLoRA fine-tuning with 50 outputs from a teacher model
A post published on February 24, 2026 describes a local demonstration using 50 GPT-5.2 outputs to fine-tune a 3-billion-parameter Llama model with QLoRA.
Training and fine-tuning Post
10/08/2618 days ago x.com
Stepwise rewards for flow matching training
A description published on February 15, 2026 presents TurningPoint-GRPO, a framework that identifies steps with sign changes in local reward to capture long-term dependencies.
Training and fine-tuning Post
10/08/2618 days ago x.com
Ring-linear 2.0: hybrid attention and FP8 optimization
In a post published on October 27, 2025, Ant Group describes two models combining linear and softmax attention and claims improved training efficiency with FP8 operators.
Training and fine-tuning Post
10/08/2618 days ago x.com
SAM in mid-training may reduce forgetting during fine-tuning
A May 8, 2026 post says applying SAM during the final 10% of mid-training reduced forgetting by over 35% after fine-tuning or quantization.
Training and fine-tuning Post
10/08/2618 days ago x.com
TC-JEPA: captions guide visual learning
A post published on May 9, 2026 describes a self-supervised method that uses captions to guide the prediction of masked image patches. According to the post, the approach improves training stability and visual reasoning compared with contrastive methods.
Training and fine-tuning Post
10/08/2618 days ago x.com
Tile Kernels optimize GPU operations for LLMs
On April 23, 2026, the company published GPU kernels developed with TileLang for LLM operations and said some are used in its internal training and inference.
Training and fine-tuning Post
10/08/2618 days ago x.com
The Stack v3 expands code dataset to 5 trillion tokens
A post published on July 23, 2026 presents an open dataset with about 5 trillion deduplicated and filtered code tokens across 713 languages, excluding code with restrictive licenses.
Training and fine-tuning Post
10/08/2618 days ago x.com
ACE: structured context that evolves with feedback
A post dated October 13, 2025 describes Agentic Context Engineering, an approach that incrementally accumulates and refines structured context using performance feedback as an alternative to iterative prompt rewriting.
Training and fine-tuning Post
10/08/2618 days ago x.com
Using GEPA to detect malicious AI-generated code
A publication dated October 5, 2025 reports that prompts found with GEPA detected malicious behavior in AI-generated code, blocking 90% of malicious submissions with 1% of the audit budget.
Training and fine-tuning Post
10/08/2618 days ago x.com
JEPA-SCORE estimates density with JEPA encoders
A study published on October 8, 2025, presents a way to estimate sample probabilities using an encoder’s Jacobian, without retraining.
Training and fine-tuning Post
10/08/2618 days ago x.com
LoRA adapts an MLP with about 9,000 trainable parameters
Published on July 30, 2024, the example applies LoRA to a 5-million-parameter MLP for classification and compares performance with the adapter enabled and disabled.
Training and fine-tuning Post
10/08/2618 days ago x.com
LoRA for reinforcement learning fine-tuning
A Thinky Machines post published on September 29, 2025 describes LoRA configurations for reinforcement learning fine-tuning and claims rank 1 can match full fine-tuning performance when configured well.
Training and fine-tuning Post
10/08/2618 days ago x.com
PEER retrieves over one million experts using product keys
The PEER architecture combines product-key routing and single-neuron MLP experts to sparsely select the components used for each input.
Training and fine-tuning Post
10/08/2618 days ago x.com
SWE-Swiss: a training recipe for software engineering
On August 12, 2025, researchers from Peking University, ByteDance Seed, and HKU presented a training recipe for software engineering tasks. The researchers report that SWE-Swiss-32B scored 60.2% on SWE-bench Verified.
Training and fine-tuning Post
10/08/2618 days ago x.com
Knowledge transfer between AI models
A repository published on July 29, 2024 presents a notebook for tuning a smaller model with domain knowledge generated by a larger model.
Training and fine-tuning Post
10/08/2618 days ago x.com
UserRL trains agents with simulated users
In a post published on October 8, 2025, Salesforce presents UserRL, a framework for training agents with standardized environments and simulated users.
Training and fine-tuning Post
10/07/2619 days ago x.com
ARGUS diagnoses slowdowns in large-scale training
A paper published on arXiv on June 19, 2026 describes a system combining CPU, framework, and GPU data to investigate training slowdowns, with continuous overhead below 2%.
Training and fine-tuning Post
10/07/2619 days ago x.com
Making model training more observable
On August 21, 2026, a publication about Marin described training data and metrics visualizations that can support model inspection.
Training and fine-tuning Post
10/07/2619 days ago x.com
On-policy distillation combines specialist models
A post published on October 6, 2026 describes MOPD, a method that trains a student model on its own rollouts with token-level supervision from domain specialists.
Training and fine-tuning Post
10/07/2619 days ago x.com
Token-level distillation blends teacher and model supervision
A post dated July 8, 2026, describes a paper on dual on-policy distillation, which dynamically selects a supervision source for each token.
Training and fine-tuning Post
10/07/2619 days ago x.com
Model distillation: the direction of per-token KL
A systematic study published on October 3, 2026, reports that the direction of per-token KL divergence matters more than rollout policy in on-policy versus off-policy distillation.
Training and fine-tuning Post
10/07/2619 days ago x.com
From REINFORCE to GRPO: policy gradient methods for LLMs
An article published on September 24, 2026, presents a progression of policy gradient methods used to train LLMs, from vanilla policy gradient and REINFORCE to PPO, GRPO, and GRPO variants.
Training and fine-tuning Post
10/07/2619 days ago x.com
Weight spectra may signal memorization in LLMs
A preliminary analysis published on September 24, 2026, found that the attention matrix K exponent α tracked memorization of random labels during LLM training.
Training and fine-tuning Post
10/07/2619 days ago x.com
GEPA adapts a model with prompts and labeled examples
A demonstration published on September 19, 2026 describes using prompts and labeled examples to adapt a model to specific decision criteria.
Training and fine-tuning Post
10/07/2619 days ago x.com
IW-OPD reweights tokens in on-policy distillation
A post published on June 30, 2026 presents IW-OPD, a method that assigns greater weight to early tokens in on-policy distillation and reports a 6.9-point gain over standard OPD on AIME-2025.
Training and fine-tuning Post
10/07/2619 days ago x.com
NoRA proposes normalizing A at LoRA initialization
A post published on September 2, 2026 describes an initialization that normalizes LoRA's A columns and reports higher average results than several adaptation methods in SFT and RLVR.
Training and fine-tuning Post
10/07/2619 days ago x.com
Ostris AI Toolkit adds LoRA training for MiniMax H3
On August 3, 2026, the Ostris AI Toolkit repository announced LoRA training support for MiniMax H3, limited to text-to-video and image-to-video workflows, using quantized Comfy weights.
Training and fine-tuning Post
10/07/2619 days ago x.com
Model pruning uses effective sample size to select components
A post published on July 1, 2026 presents Effective Model Pruning, a method that uses effective sample size to calculate how many components to retain in neural networks and language models.
Training and fine-tuning Post
10/07/2619 days ago x.com
Local branching to scale LLM inference
A paper published on July 5, 2026 proposes generating and selecting token branches during inference; the post reports better mathematical reasoning than three comparison approaches.
Training and fine-tuning Post
10/07/2619 days ago x.com
Post-training RL with fewer parameters changed
A post published on July 5, 2026 describes an approach that reinforces high-contribution intermediate layers during post-training RL. It may match or outperform RL across all parameters while changing fewer parameters.

Let's assess your project

I reply with scope and a proposal.