An engineering team that wants Kimi to work on private code used to have to choose among hosted options for K3.1 Pro. Now it can obtain the model itself. Moonshot AI’s repository on Hugging Face contains the parameters that give the model its learned behavior, split across 96 weight files. The repository totals about 1.56 terabytes. The release, reported by NYU Shanghai’s Research and Instructional Technology Services on November 5, follows the October launch of the hosted model.
The release lets an operator with suitable infrastructure place the model inside an environment it controls. It also gives a team a way to keep a particular checkpoint, the saved version of the model, when a hosted provider changes its offering. Getting a copy answers the first question. The team still has to figure out whether it can serve that version at a speed people will use, and whether the terms cover what it plans to offer.
Why 104 billion active parameters still means a 1.56 TB download
Moonshot describes Kimi K3.1 Pro as a model with 2.8 trillion parameters in total and about 104 billion active for each token. A token is a unit the model processes or produces, such as part of a word. Kimi uses a design called mixture of experts. At each step it routes work through a subset of the model’s specialized components. Different inputs can send work to different components. An operator has to account for the entire pool of weights even though a single step uses only part of it.
Moonshot has already reduced the storage burden. The vLLM project’s recipe says the routed experts’ weights use MXFP4, a compact four-bit format, while the rest use a higher-precision format. A team that sees a community build with a smaller file should check what changed in the conversion and whether the model still handles the team’s tasks well. Lower precision can reduce the footprint and alter the results. The file size alone tells you neither how much fast memory the system needs nor how quickly it can produce an answer.
The vLLM maintainers list one eight-GPU NVIDIA B300 node or two GB300 trays as examples for the published checkpoint. They also document a configuration on an AMD MI355X node. The NVIDIA baseline is a working reference without throughput tuning. It gives an infrastructure team a place to start estimating the cost of a trial. There may be other ways to get a version of the model running, with different compromises in precision, memory and speed.
Shane Downing’s November 8 report for Tom’s Hardware shows how far those compromises can go. A hobbyist reported running the older Kimi K3 on a CPU with secondhand Intel Optane memory. The result was roughly 0.85 generated tokens per second with a heavily compressed build. That demonstration is a useful challenge to the idea that a giant model always requires a bank of recent GPUs. It used a different checkpoint, and the pace points to a different kind of use. Someone exploring how a model fits in memory might happily wait for a reply. An engineer stopping work while an assistant generates a patch faces a different cost.
Fitting the model is only the first capacity test
Consider an internal tool that answers questions about large codebases. The model’s advertised context window of up to one million tokens sounds like enough space to hand it a substantial collection of files. But a session that fits can still take time to begin answering. The engine first works through the input. It also keeps a working memory of the input and the reply so far. That memory, called a cache, grows as sessions get longer. Several people asking long questions at once can put pressure on memory even when a single request runs comfortably.
The vLLM documentation shows that an operator can change how this cache is spread across devices. One B300 configuration holds substantially more cached tokens than the baseline but generates each step more slowly at the same batch size. The interesting choice is whether cache space or the speed of generating the next token is the tighter constraint. A team that runs many large, overlapping sessions might prefer the larger cache. A tool that answers short, interactive questions might care more about the time between generated tokens. Neither use can be sized reliably from the maximum context number on the model card.
A useful trial would measure how long the first piece of output takes to appear and how quickly the rest follows. It would also count how many people can use the system at once before either measure gets worse. The workload should include the long inputs the team expects in practice and the tools or output formats its application depends on. If an alternative deployment uses different precision, compare what the answers get right as well as how fast they arrive. A warm demonstration with one familiar prompt leaves out the arrival pattern and workload that determine what the service feels like on a busy day.
Who runs the service, and who gets access?
Together AI lists both serverless and dedicated access to Kimi K3.1 Pro. Its published serverless prices are $2 per million uncached input tokens, $0.20 per million cached input tokens and $10 per million output tokens. Those figures make a hosted trial easy to price from actual request logs, subject to the provider’s terms. They also give a team a baseline for comparison. A self-operated system adds hardware or rental costs, idle time, upgrades, monitoring and the work of restoring service after a failure. Where the workload has peaks, the team may have to keep capacity ready for hours when little of it is in use.
Price is one reason to make that comparison. Control can be another. An organization that has to keep a set of documents inside a particular environment can place the model there and manage access to it. It then has to account for the entire application. A model server can remain inside the boundary while the surrounding tool sends files to an external search service or copies prompts to a managed logging system. The weights give the organization control of one important piece. The endpoint, tools, logs and backups determine where information actually travels.
The license draws its own lines. It generally allows people to use, modify and distribute the files subject to its conditions, including retaining the required notices. Under section 2, if the licensee or an affiliate operates a model-as-a-service business and aggregate revenue across the licensee and its affiliates exceeds $20 million over any consecutive 12 months, the license calls for a separate agreement with Moonshot before commercial use. The text defines that business around giving third parties meaningful control over model inference or fine-tuning. It excludes model capabilities embedded solely in particular features and the mere relay of requests to models hosted elsewhere.
Section 3 sets a different threshold. A commercial product or service using the model or a derivative must display the Kimi K3.1-Pro name prominently in its interface if it has more than 100 million monthly active users or more than $20 million in monthly revenue. Both sections have exceptions for internal use as defined in the license and for use through Moonshot’s official products or certified inference partners. The internal-use definition matters. It looks at whether the software, outputs or underlying capabilities are made available to third parties. Running on hardware the company owns does not by itself make every use internal.
Those distinctions separate an engineering assistant for employees from a customer-facing feature that summarizes support cases, and from a service that lets customers send arbitrary prompts or fine-tune a model. All three could use the same weight files. Their users receive different kinds of access. The customer-facing feature can fall outside that definition when the model is embedded solely in that feature, while still raising the separate attribution question at sufficient scale. A company that also operates an inference business has to read the affiliate and revenue language at the company level. The public text gives each operator a different question to answer.
Where owning the checkpoint earns its keep
A team with intermittent traffic and permission to send its data to a provider can use a managed endpoint to see how well Kimi handles the work before committing to its own infrastructure. A team that needs to retain a fixed version for repeated evaluations, or process sensitive material in an environment it controls, gets a more immediate benefit from the download. It can validate the checkpoint against its own requirements and keep the same files available for a later audit or rerun. The comparison should include the cost of running the surrounding service and any work needed to match the application’s existing behavior.
The strongest case for taking on that work is a concrete requirement the hosted endpoint cannot meet. For the codebase assistant, that might mean keeping prompts and retrieved files on infrastructure the team administers, while still meeting a response-time target for several engineers working at once. The downloadable checkpoint gives the team a way to attempt that deployment. Success would be a service that meets those requirements on ordinary workdays and can be restored when it fails.