How to Run Hemmingway-1 Locally: llama.cpp, LM Studio, a Mac, and What We Know About Ollama

Hemmingway-1 has open weights. So you can run it on your own computer.

Then you read the makers’ instructions. One line: vllm serve. The download is 54.6 GB. That sounds like a job for a server.

It isn’t. Within days of the release, the community published small builds. The usual one is about 17 GB and fits one consumer GPU or an Apple Silicon Mac.

Here is the short version. Download the Q4_K_M GGUF file and run it with llama.cpp or LM Studio. On a Mac, you can use an MLX build instead. Ollama can pull the same file, but I have found nobody who confirms it works for this model.

One note before the steps. I have not run any of these routes. Everything below comes from the model cards and each tool’s documentation, and I say where a step is unconfirmed.

If you only want to see how it writes, the free browser demo answers that without a 17 GB download. Running it yourself is for keeping your text on your own machine, or for building on the weights.

Disclosure: I build StashBase, which runs that demo and offers Hemmingway as a hosted writing skill. This guide does not need StashBase.

What do the makers publish?

The full weights and very little about running them.

(1) The official weights are 54.64 GB.

BF16 safetensors in 13 files, on Hugging Face under Apache-2.0. The context is 262,144 tokens.

(2) The instructions are vLLM and Transformers.

Their write-up gives no hardware requirements. Then it suggests the hosted API.

(3) The local model is text-only.

The hosted API reads pictures. The open weights have no vision part.

Which file should you download?

Q4_K_M, 17.44 GB.

(4) It is the build’s own default.

From bartowski’s card: “Grab Q4_K_M (17.44GB) - usually a good mix of size and performance.”

Build	Size	For
GGUF Q4_K_M	17.44 GB	llama.cpp, LM Studio, Ollama
GGUF IQ4_XS	15.48 GB	A little smaller, “similar performance”
GGUF Q5_K_M	20.92 GB	”High quality”
GGUF Q6_K	23.86 GB	”Very high quality, near perfect”
MLX 4-bit	15.15 GB	Apple Silicon
MLX 6-bit	21.88 GB	Apple Silicon with more memory

Sizes are from the Hugging Face file listings on September 22, 2026.

(5) Do not go below 4-bit.

The card calls those builds “low quality.” You chose this model for its prose. That is a poor trade.

How much memory do you need?

More than the file.

(6) Leave 1 to 2 GB of headroom.

bartowski’s rule: pick a file “1-2GB smaller than your GPU’s total VRAM.” By that rule Q4_K_M wants a 24 GB GPU. No source publishes memory use or speed for this model yet, so I give no numbers for them.

(7) Shrink the context.

llama.cpp reads the context size from the model unless you set it. Here that is 262,144 tokens, and context costs memory. The commands below use 16,384, which is plenty for emails and short pieces.

The four routes

(8) llama.cpp is the documented route.

You need release b10964 or newer. The builds were made with that release, and the architecture is new. Install it from the llama.cpp README, then:

llama-server -hf bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M -c 16384

The first run downloads the file. Open http://localhost:8080 for the built-in chat page. The server also speaks the OpenAI API, so other tools can use it.

The card says adding --spec-type draft-mtp speeds up generation. The builds include the model’s multi-token-prediction layers.

(9) LM Studio is the easy route.

bartowski’s card lists it as compatible. Open Discover (Cmd+2 on a Mac, Ctrl+2 elsewhere), paste bartowski/Altworld_Hemmingway-1-GGUF, and pick a file. LM Studio marks the one it recommends for your hardware.

lms get https://huggingface.co/bartowski/Altworld_Hemmingway-1-GGUF@Q4_K_M

It needs an Apple Silicon Mac on macOS 14 or newer, or Windows or Linux with AVX2. Intel Macs are not supported.

(10) On a Mac, MLX is the native route.

hermitdave publishes 4-bit and 6-bit builds. The card’s quickstart:

pip install -U mlx-lm

python3 -m mlx_lm.generate \
  --model hermitdave/Hemmingway-1-MLX-4bit \
  --prompt "Write a text to your landlord about the broken boiler." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

python3 -m mlx_lm.chat --model hermitdave/Hemmingway-1-MLX-4bit gives you a chat in the terminal.

If the model is large for your memory, the mlx-lm README explains how to raise the wired-memory limit. That needs macOS 15 or newer.

(11) Ollama is the unconfirmed route.

There is no Hemmingway-1 in the Ollama library. Ollama can run GGUF files straight from Hugging Face, so the documented pattern gives:

ollama run hf.co/bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M

I have not seen this confirmed. bartowski’s list of compatible apps leaves Ollama out. Ollama’s tracker has several closed reports of Hugging Face files with this architecture failing to load, for example #14636.

Ollama does ship the base model as qwen3.8:27b, so the architecture itself is supported. Update Ollama first. If the pull fails, use llama.cpp or LM Studio.

Ollama also defaults to a small context, 4k tokens under 24 GiB of VRAM. If replies are cut short, run OLLAMA_CONTEXT_LENGTH=16384 ollama serve.

How do you turn the thinking down?

With a template setting. The model thinks before it answers, at its highest level by default, and that is slow on a desktop.

(12) The template takes two switches.

The chat template accepts reasoning_effort of xhigh, medium or low, and enable_thinking: false.

# llama.cpp: brief thinking for every request
llama-server -hf bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M -c 16384 \
  --chat-template-kwargs '{"reasoning_effort":"low"}'

# mlx-lm: no thinking
python3 -m mlx_lm.generate --model hermitdave/Hemmingway-1-MLX-4bit \
  --chat-template-config '{"enable_thinking": false}' --prompt "..."

(13) Start with low, not off.

On the hosted API, which our demo uses, turning thinking off gave a visibly weaker short story. low cost about four seconds before the reply started. That was the hosted model on September 22, 2026. A local 4-bit build may behave differently.

(14) Ollama and LM Studio have no documented way to pass these.

Ollama has --think=false. Whether it works with this template is unconfirmed.

What sampling settings should you use?

(15) The makers publish none.

Someone has asked. The only source is the generation_config.json shipped with the weights: temperature 1.0, top_p 0.95, top_k 20. The MLX quickstart uses the same temperature and top_p. Start there.

Are there other builds?

(16) Eighteen repositories on September 22, 2026.

mradermacher has static and imatrix GGUF sets. For serving on a GPU with vLLM there is a GPTQ Int4 build and an FP8 build. Here is the full list.

When is local not worth it?

When you lack the memory, or you only write a few messages a day.

The hosted options cost less than the electricity: the free demo, the makers’ API at $0.24 per million input tokens and $0.90 per million output, or the Hemmingway writing skill in StashBase, which works beside your own files.

With the hosted options, your text leaves your computer. With the local ones, it stays there.