ZML Launches Free LLMD Inference Server for Cross-Chip AI Workloads

ZML, a Paris-based AI startup founded by Steeve Morin, former VP of engineering at Zenly (acquired by Snapchat), launched LLMD on July 8, 2026, a free inference server designed to speed large language model workloads across multiple hardware platforms. It's an alpha release, not a production-ready 1.0, supporting Llama, Gemma, Qwen, and Mistral model families across Nvidia, AMD, Google TPU, Apple Metal, and Intel targets.

Last updated August 4, 2026: added that LLMD ships as an alpha release, specific model-family support and named technical features (DFlash, continuous batching, prefix caching) from the founder's own release notes, and honest context on where LLMD sits relative to the inference engines developers actually compare today (vLLM, TensorRT-LLM, SGLang).

What Happened: ZML Launches Free LLMD Server

ZML announced LLMD as a free inference server that lets developers run models across multiple hardware targets without rewriting code, abstracting away hardware-specific optimizations. Founder Steeve Morin described it on release as a "homegrown LLM server built on top of our homegrown high performance heterogeneous inference stack," shipping with support for DFlash decoding, continuous batching, and prefix caching, features aimed at real production inference workloads rather than a proof-of-concept demo. Unlike ZML's first machine learning framework from 2024, LLMD is closed-source.

The company, a lean team of about 20 people, has raised $20 million from investors including 20VC, LocalGlobe, and Kima Ventures, with Turing Award winner Yann LeCun and Hugging Face co-founders Clément Delangue and Julien Chaumond among its notable backers.

Where LLMD Actually Sits in the Inference-Engine Market

The inference-engine conversation developers actually have today centers on vLLM, TensorRT-LLM, and SGLang, tools with established benchmark comparisons, large user bases, and mature tooling. LLMD, as a July 2026 alpha release, doesn't yet appear in those comparisons; its differentiation isn't raw speed against those engines but breadth, running the same deployment across five different chip vendors, something none of the established engines do as their primary design goal. Whether LLMD's abstraction layer matches or trails vendor-specific engines like TensorRT-LLM on their home hardware is an open, unanswered benchmark question at this stage.

What the Named Features Actually Do

The three technical features ZML highlighted at launch each target a distinct inefficiency in production inference. Continuous batching lets the server add new requests to a batch already being processed, rather than waiting for a full batch to complete before starting the next one, which is what makes it possible to serve many concurrent users without idle GPU time between requests. Prefix caching stores the computed state for a prompt's shared beginning, useful when many requests start with the same system prompt or context, so the server doesn't redo that computation for every single request. DFlash decoding is ZML's approach to speculative decoding, predicting several tokens ahead and verifying them in parallel rather than generating strictly one token at a time, which is one of the more effective known techniques for cutting inference latency when it's implemented well. None of these techniques are unique to LLMD, vLLM and TensorRT-LLM implement versions of continuous batching and prefix caching too, but bundling all three across five hardware targets in a single free tool is the actual differentiation ZML is betting on.

Why It Matters

Hardware fragmentation is real: organizations that start on Nvidia GPUs often want AMD or Intel accelerators for cost or availability, and each hardware transition currently requires real engineering work. LLMD's free, cross-platform bet attacks that lock-in directly, but as an unbenchmarked alpha release from a 20-person team, it's a genuine attempt to solve a real problem rather than a proven solution yet.

The decision to make LLMD closed-source, unlike ZML's original 2024 framework, is also worth sitting with. Open-source inference tooling has generally won developer trust in this space precisely because engineers can inspect and modify the code running their production workloads; vLLM and SGLang are both open-source projects with community contributions driving much of their hardware support. ZML offering LLMD for free while keeping it closed suggests the company is treating the software itself, not just a hosted service around it, as the eventual monetization path, a bet that free-but-closed can still build the kind of trust and adoption that open-source tools have earned in this specific market.

What Happens Next

ZML will need independent benchmarks showing LLMD holds up against vendor-native tools on their own hardware, not just its cross-platform breadth, before it becomes a serious alternative to vLLM or TensorRT-LLM rather than just an interesting alpha project. The company has said it wants to see real usage patterns before deciding whether to eventually charge for it, a sequencing that suggests ZML is prioritizing adoption data over near-term revenue while it establishes whether LLMD's cross-platform approach actually holds up under real production load.

Final Takeaway

LLMD targets a genuine infrastructure problem with real technical ambition (DFlash, continuous batching, prefix caching across five chip vendors), but it launched as an alpha with no public benchmarks against the engines developers already use. The cross-platform pitch is differentiated; whether it's competitive is still an open question.

Key Points

ZML's Team and Backers

ZML was founded by Steeve Morin, who previously served as VP of engineering at Zenly, the location-sharing app Snapchat acquired for a nine-figure sum in 2017. The Paris-based company operates with a lean team of about 20 people and has raised $20 million to date from investors including 20VC, LocalGlobe, Kima Ventures, Kindred Capital, AALVC, Drysdale Ventures and Puzzle Ventures.

The company's cap table also includes several notable individual backers: Turing Award-winning AI researcher Yann LeCun, who is now with AMI Labs; Solomon Hykes, founder of Docker and Dagger; and Hugging Face co-founders Clément Delangue and Julien Chaumond. Their involvement lends credibility to ZML's technical ambitions, though investor backing alone does not guarantee that LLMD will match the performance of vendor-native inference tools.

FAQs