AI and infrastructure news

All120

Training and fine-tuning

Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
VL-JEPA proposes a different way to predict text for visual tasks
An article published on December 26, 2025 describes VL-JEPA, which predicts continuous target-text embeddings and aims to prioritize task meaning while reducing sensitivity to wording variations.
Training and fine-tuning Post
10/08/2617 days ago x.com
The Stack v3 expands code dataset to 5 trillion tokens
A post published on July 23, 2026 presents an open dataset with about 5 trillion deduplicated and filtered code tokens across 713 languages, excluding code with restrictive licenses.
Training and fine-tuning Post
10/08/2617 days ago x.com
ACE: structured context that evolves with feedback
A post dated October 13, 2025 describes Agentic Context Engineering, an approach that incrementally accumulates and refines structured context using performance feedback as an alternative to iterative prompt rewriting.
Training and fine-tuning Project
10/08/2617 days ago github.com
DeepSpeed Domino and distributed LLM training
A November 26, 2024 post describes a tensor-parallelism mechanism designed to reduce communication overhead in LLM training.
Training and fine-tuning Article
10/08/2617 days ago universe.roboflow.com
Detecting basketball elements with computer vision
A September 24, 2025 publication describes fine-tuning a detection model to identify players, jersey numbers, referees, the ball, and the hoop in basketball footage.
Training and fine-tuning Docs
10/08/2617 days ago huggingface.co
Reflective prompt optimization with DSPy GEPA
A cookbook published on October 16, 2025 presents reflective prompt optimization with DSPy GEPA and reports an 11% accuracy gain on NuminaMath-1.5 for less than US$0.50 in total.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
ESFT targets task-relevant experts in sparse LLMs
Published on July 5, 2024, the paper studies parameter-efficient fine-tuning for Mixture-of-Experts models and presents ESFT, which trains experts selected for their relevance to the task.
Training and fine-tuning Post
10/08/2617 days ago x.com
Using GEPA to detect malicious AI-generated code
A publication dated October 5, 2025 reports that prompts found with GEPA detected malicious behavior in AI-generated code, blocking 90% of malicious submissions with 1% of the audit budget.
Training and fine-tuning Article
10/08/2617 days ago tokens-for-thoughts.notion.site
LLM post-training: methods and evaluation
A guide published on October 12, 2025, on adapting LLMs: from next-token prediction to instruction following, including training methods and evaluation.
Training and fine-tuning Post
10/08/2617 days ago x.com
JEPA-SCORE estimates density with JEPA encoders
A study published on October 8, 2025, presents a way to estimate sample probabilities using an encoder’s Jacobian, without retraining.
Training and fine-tuning Post
10/08/2617 days ago x.com
LoRA adapts an MLP with about 9,000 trainable parameters
Published on July 30, 2024, the example applies LoRA to a 5-million-parameter MLP for classification and compares performance with the adapter enabled and disabled.
Training and fine-tuning Post
10/08/2617 days ago x.com
LoRA for reinforcement learning fine-tuning
A Thinky Machines post published on September 29, 2025 describes LoRA configurations for reinforcement learning fine-tuning and claims rank 1 can match full fine-tuning performance when configured well.
Training and fine-tuning Post
10/08/2617 days ago x.com
PEER retrieves over one million experts using product keys
The PEER architecture combines product-key routing and single-neuron MLP experts to sparsely select the components used for each input.
Training and fine-tuning Paper
10/08/2617 days ago arxiv.org
RLPT uses pretraining data for reinforcement learning
Published on September 25, 2025, the paper presents a way to apply reinforcement learning to pretraining data without new human annotations, and reports benchmark gains for Qwen3-4B-Base.
Training and fine-tuning Post
10/08/2617 days ago x.com
SWE-Swiss: a training recipe for software engineering
On August 12, 2025, researchers from Peking University, ByteDance Seed, and HKU presented a training recipe for software engineering tasks. The researchers report that SWE-Swiss-32B scored 60.2% on SWE-bench Verified.
Training and fine-tuning Post
10/08/2617 days ago x.com
Knowledge transfer between AI models
A repository published on July 29, 2024 presents a notebook for tuning a smaller model with domain knowledge generated by a larger model.
Training and fine-tuning Post
10/08/2617 days ago x.com
UserRL trains agents with simulated users
In a post published on October 8, 2025, Salesforce presents UserRL, a framework for training agents with standardized environments and simulated users.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
Continuous learning: evaluating models updated after deployment
A publication dated September 29, 2026 describes a 7B model updated with prediction-market data and evaluated in continuous backtests over 390 days.
Training and fine-tuning Post
10/07/2618 days ago x.com
ARGUS diagnoses slowdowns in large-scale training
A paper published on arXiv on June 19, 2026 describes a system combining CPU, framework, and GPU data to investigate training slowdowns, with continuous overhead below 2%.
Training and fine-tuning Article
10/07/2618 days ago danielvanstrien.xyz
Classifying documents at scale with few examples
A post published on September 10, 2026 describes labeling 200 examples with an agent and classifying 191,724 documents for about US$0.70 in inference compute.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
How reinforcement learning is distributed across LLM layers
A preprint published on July 15, 2026, examines how reinforcement learning adaptation is distributed across transformer layers during LLM post-training.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
Reducing training–inference gaps in RL for LLMs
An article published on June 30, 2026 discusses inconsistent probabilities between training and inference engines and describes TIS and checkpoint rejection when rewards diverge.
Training and fine-tuning Article
10/07/2618 days ago aleph-alpha.com
Test training scale before expanding GPU capacity
In a report published on October 5, 2026, Aleph Alpha describes scaling a 30B-A3B MoE model from 16 to 512 B200 GPUs, reporting near-linear scalability and 35.3% MFU.
Training and fine-tuning Post
10/07/2618 days ago x.com
Making model training more observable
On August 21, 2026, a publication about Marin described training data and metrics visualizations that can support model inspection.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
Lightweight compression reduces vectors in ColBERT systems
Published on July 8, 2026, the paper describes pooling-aware fine-tuning with k-means to reduce token vectors per document in ColBERT models, with no loss of accuracy according to the publication.
Training and fine-tuning Project
10/07/2618 days ago github.com
datatrove 0.10.0 runs pipelines on managed jobs
datatrove 0.10.0 adds pipeline execution on managed jobs, support for storage buckets as data folders, and preservation of reasoning outputs in inference results.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
Black-box LLM distillation with GAD
An article dated September 13, 2026 presents an on-policy method for distilling models using only the text outputs of a proprietary teacher.
Training and fine-tuning Post
10/07/2618 days ago x.com
On-policy distillation combines specialist models
A post published on October 6, 2026 describes MOPD, a method that trains a student model on its own rollouts with token-level supervision from domain specialists.
Training and fine-tuning Post
10/07/2618 days ago x.com
Token-level distillation blends teacher and model supervision
A post dated July 8, 2026, describes a paper on dual on-policy distillation, which dynamically selects a supervision source for each token.
Training and fine-tuning Post
10/07/2618 days ago x.com
Model distillation: the direction of per-token KL
A systematic study published on October 3, 2026, reports that the direction of per-token KL divergence matters more than rollout policy in on-policy versus off-policy distillation.
Training and fine-tuning Post
10/07/2618 days ago x.com
From REINFORCE to GRPO: policy gradient methods for LLMs
An article published on September 24, 2026, presents a progression of policy gradient methods used to train LLMs, from vanilla policy gradient and REINFORCE to PPO, GRPO, and GRPO variants.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
EnvACE rehearses the world to train agents
Published on August 9, 2026, the article describes a reinforcement learning method that replaces interactions with external environments with internal rehearsals during agent training.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
Learning-rate scaling in LLMs may be nonlinear
An article published on June 30, 2026, evaluates learning rates in GPT-2-style models ranging from 22 million to 707 million parameters, trained on 5 billion to 100 billion tokens.
Training and fine-tuning Post
10/07/2618 days ago x.com
Weight spectra may signal memorization in LLMs
A preliminary analysis published on September 24, 2026, found that the attention matrix K exponent α tracked memorization of random labels during LLM training.
Training and fine-tuning Post
10/07/2618 days ago x.com
GEPA adapts a model with prompts and labeled examples
A demonstration published on September 19, 2026 describes using prompts and labeled examples to adapt a model to specific decision criteria.
Training and fine-tuning Project
10/07/2618 days ago github.com
GuppyLM: an experiment with a 9-million-parameter model
In a post published on September 19, 2026, the author describes GuppyLM, a model with about 9 million parameters that, according to the author, can be trained from scratch in five minutes.
Training and fine-tuning Project
10/07/2618 days ago github.com
IFM-AI releases open tools for LLMs
On September 28, 2026, IFM-AI announced open-source tools for LLM pre-training, attention, checkpoint conversion, and serving.
Training and fine-tuning Post
10/07/2618 days ago x.com
IW-OPD reweights tokens in on-policy distillation
A post published on June 30, 2026 presents IW-OPD, a method that assigns greater weight to early tokens in on-policy distillation and reports a 6.9-point gain over standard OPD on AIME-2025.
Training and fine-tuning Paper
10/07/2618 days ago arxiv.org
JEPA-Anything explores predictive learning across seven domains
The post, published on October 6, 2026, presents JEPA-Anything as a domain-agnostic world-modeling framework based on orthogonal predictive factorization.
Training and fine-tuning Project
10/07/2618 days ago github.com
KohakuFA proposes an alternative for backward-pass precision
On October 3, 2026, a post presents the KohakuFA kernel as an alternative amid claims of precision issues in backward passes involving cuDNN, FA4, and FlexAttention.

Let's assess your project

I reply with scope and a proposal.