Local LLM benchmarks on a Ryzen AI Max+ PRO 495 and the decision to return the machine
Throughput analysis of four large open-weight models run with llama.cpp/Vulkan on a 192 GB unified-memory APU, reporting 18 to 58 tps and a return made for cost reasons.
Source: Benchmarks de LLM local no Ryzen AI Max+ PRO 495 e decisão de devolução (x.com). Text prepared with AI from this source.
What happened and what to do
The analysis tested four large open-weight models, including one with 120 billion parameters, on a Minisforum mini PC with a Ryzen AI Max+ PRO 495 and 192 GB of unified memory, using llama.cpp with the Vulkan backend. The author reports throughput between 18 and 58 tokens per second depending on the model, and states that the machine was returned because of cost.
For a company, the case shows that local inference performance depends on the model, quantization and runtime, and that total cost can rule out a purchase even when memory is sufficient. Before buying hardware, it is worth building a benchmark harness with real workloads: measuring tokens per second, tail latency, memory and energy use, and calculating cost per useful token including operation. That harness can feed a telemetry dashboard and clear decision criteria, comparing local private inference, dedicated servers in the company's own datacenter and rented capacity under the company's control.
How the consultancy can help
Capacity and cost diagnosis for local inference: measuring throughput, latency and memory with the company's real workload, comparing hardware and models, and deploying private inference with monitoring in the client's environment.
Next step
If your company is evaluating equipment to run language models locally, send a short description of the case and receive a proposal with a defined scope.
Let's assess your project
I reply with scope and a proposal.