Byte-level H-Nets: tokenizer-free language modeling, according to the author's post

What happened and what to do

The post announces a new version of H-Nets, a hierarchical architecture that processes raw bytes and learns its representation end to end, with no tokenization step. The author claims efficiency gains of 3-5x in scaling law experiments and parity with an open-source model trained on 11T tokens, using about 10x less data. These figures are the author's own claims about his experiments.

In practice, a company working with Portuguese or multilingual text can assess this direction with its own data. A first step is an internal benchmark comparing conventional tokenization with byte-level approaches, measuring cost per document, required context length, and quality on text with typos and mixed languages. The evaluation pipeline should use versioned datasets and run on private infrastructure, so sensitive documents stay inside the company. Until the results are reproduced on the client's case, the architecture should be treated as a research hypothesis, not a production decision.

How the consultancy can help

A diagnosis of how current tokenization affects the cost and quality of your text workflows and, where it makes sense, a private pilot comparing approaches, sized to your infrastructure and backed by reproducible metrics.

Next step

If your company processes Portuguese or multilingual text and wants to assess this direction with its own data, send a short description of your case. Wendelmaques will reply with a scoped proposal.