Isolated experiments with local models: reproducible setup in Codex CLI
The post describes a reproducible harness that runs local models in Codex CLI with tool-calling disabled, so results can be shared and compared.
By Wendelmaques Pereira · Updated today
Source: Execução de experimentos isolados com modelos locais via configuração do Codex CLI (github.com). Text prepared with AI from this source.
What happened and what to do
The post recommends running isolated experiments with local models through Codex CLI, using configuration files that disable tool-calling so that results are shareable. It points to a configuration file in the lm_understand-logs repository and shows a reproducible harness for evaluating local models without tool access that could affect the results.
In practice, a company evaluating local models needs a controlled benchmark where the only variable is the model. This can be built with versioned harness configurations, fixed prompts and parameters, logging of every run, and a dashboard that compares models before any adoption. During evaluation, tool access stays off; tool use can be tested later in a separate environment with explicit controls. This pipeline keeps results auditable and allows the evaluation to be repeated when the model or CLI version changes.
How the consultancy can help
Diagnosis of how your team evaluates local models today and design of a reproducible evaluation harness, with versioned configurations, run logs and a comparison dashboard. Implementation in your own environment and ongoing operation under an agreed scope.
Next step
Send a short description of your case: how models are tested today and which decisions depend on those results. You receive a proposal with a defined scope.
Let's assess your project
I reply with scope and a proposal.