Can You Run Hemmingway-1 Locally? Setup Guide (2026)

Yes. Download the Q4_K_M file (about 17 GB) and run it with llama.cpp or LM Studio. On a Mac, you can use an MLX build instead.

Plan for a 24 GB GPU, or an Apple Silicon Mac with enough memory to hold a 15 to 17 GB file and still run macOS. Less than that? Use the hosted options at the end.

Everything below comes from model cards and each tool’s docs. We have not run these routes ourselves, and where a step is unconfirmed, the guide says so.

Disclosure: we build StashBase, which runs the free Hemmingway-1 demo and offers Hemmingway as a hosted writing skill. You don’t need StashBase for anything in this guide.

What is Hemmingway-1, and why run it on your own machine?

Hemmingway-1 is a 27B open-weight model, fine-tuned from Qwen3.8-27B for writing: texts, emails and prose that reads like a person wrote it. Altworld published the weights on Hugging Face under Apache-2.0. For the full background, read what Hemmingway-1 is.

There are three good reasons to run it yourself:

• Privacy. Your text stays on your computer.
• Cost. No per-token fees once you have the file.
• Control. You choose the settings and can build on the weights. Apache-2.0 allows commercial use.

There are costs too. The download is big and you need a lot of memory. It will be slower than a hosted service, and you do the setup.

Not sure yet? Try the free browser demo first. It needs no sign-up and no download.

What do you need to run Hemmingway-1 locally?

Four things:

• Enough memory for the file you pick
• Enough disk space for that file (about 17 GB for the usual one)
• A recent runtime. For llama.cpp that means release b10964 or newer
• Text-only prompts. The open weights have no vision part. The hosted API reads pictures; the local model does not

How much memory does Hemmingway-1 need?

More than the file itself.

The official weights are 54.64 GB, BF16 files split into 13 parts. That is what the makers’ vLLM route downloads, and it is a server job. Their write-up gives no hardware requirements.

The community made smaller builds. The usual one is about 17 GB. bartowski’s card gives a simple rule: pick a file “1-2GB smaller than your GPU’s total VRAM.” By that rule, Q4_K_M points to a 24 GB card.

We found no published memory or speed measurements for this model, so this guide gives none.

Context costs memory too. More on that below.

Can you run Hemmingway-1 on 16 GB of RAM?

Not the good builds.

The 4-bit files run from about 15 to 17.5 GB. The file alone doesn’t fit in 16 GB, and your system needs memory as well.

Smaller files exist. mradermacher lists Q2_K at 11.0 GB and Q3_K_M at 13.6 GB. But the cards flag the files below 4-bit as lower quality. You chose this model for its prose. That is a poor trade.

With 16 GB, use the hosted demo instead.

Do you need a GPU to run Hemmingway-1?

Not strictly. llama.cpp can run on a CPU alone. But 27B is large for a CPU, and nobody has published CPU speed numbers for this model. If you only have a CPU, expect to wait.

Can you run Hemmingway-1 on a Mac?

Yes, on an Apple Silicon Mac. Memory is shared between the CPU and GPU, so the question is simply whether the file fits in your total memory with room to spare.

Two Mac-specific facts:

• LM Studio needs macOS 14 or newer. Intel Macs are not supported.
• MLX builds exist. The 4-bit one is 15.15 GB and the 6-bit one is 21.88 GB.

The MLX route has its own section below.

Can you run Hemmingway-1 on Windows or Linux?

Yes. llama.cpp installs with winget install llama.cpp on Windows and brew install llama.cpp on Linux or a Mac. LM Studio runs on Windows and Linux if your CPU supports AVX2. The same GGUF file works everywhere.

Which Hemmingway-1 file should you download?

Q4_K_M. From bartowski’s card: “Grab Q4_K_M (17.44GB) - usually a good mix of size and performance.”

Build	Size	Notes
GGUF Q4_K_M	17.44 GB	Usual pick. Works with llama.cpp, LM Studio
GGUF IQ4_XS	15.48 GB	A little smaller, “similar performance”
GGUF Q5_K_M	20.92 GB	”High quality”
GGUF Q6_K	23.86 GB	”Very high quality, near perfect”
MLX 4-bit	15.15 GB	Apple Silicon
MLX 6-bit	21.88 GB	Apple Silicon with more memory

Sizes are from bartowski’s and hermitdave’s Hugging Face listings, checked September 22, 2026.

What is a GGUF file?

A single-file model format. llama.cpp uses it, and so do the apps built on llama.cpp.

Big models store their numbers with lots of detail, which makes the files huge. Quantizing stores those numbers with fewer bits. The file gets smaller and quality drops a little.

What does Q4_K_M mean?

It is a 4-bit family. “K” is a mixed method, and “M” means medium.

These mixed types use more storage per parameter than the name suggests. bartowski’s card reports 5.04 bits per weight for its Q4_K_M, which is why a “4-bit” file of a 27B model is 17 GB and not 13.5 GB.

Why is the same quant a different size on different pages?

Different people quantize with different recipes. bartowski lists Q4_K_M at 17.44 GB; mradermacher lists it at 16.9 GB. Check the size on the file list before you download.

Are the “heretic” and other modified builds the same model?

No. Some repositories hold edited versions of Hemmingway-1. One card describes a build tuned to refuse less, and says that strict length and format adherence regressed and thinking mode became less reliable.

Those builds are a different model. If you want the original, pick a repository that says it holds quantizations of Altworld/Hemmingway-1, and read the card either way.

Are there other builds?

Yes. There were 34 on Hugging Face on September 24, 2026; here is the full list. mradermacher also has an imatrix set. For serving on a GPU with vLLM there is a GPTQ Int4 build and an FP8 build.

How do you run Hemmingway-1 with llama.cpp?

llama.cpp is the documented route. Follow these steps.

Step 1. Install it. On a Mac or Linux:

brew install llama.cpp

On Windows:

winget install llama.cpp

Or download a pre-built binary from the llama.cpp releases page.

Step 2. Check the version. You need b10964 or newer. The builds were made with that release, and the card says older releases may not load a newly supported architecture. Run llama-server --version to see what you have.

Step 3. Start the server.

llama-server -hf bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M -c 16384

The first run downloads the file. Be patient; it is 17 GB.

Step 4. Open the chat page. Go to http://localhost:8080. llama.cpp serves a built-in chat page there.

What does the -c 16384 flag do?

It sets the context size: how much text the model can hold at once. If you leave it out, llama.cpp reads the size from the model, which here is 262,144 tokens. A context that big eats memory.

16,384 tokens is plenty for emails and short pieces. Raise it only if you need it.

Can other apps talk to the llama.cpp server?

Yes. The server speaks the OpenAI API, so tools that accept a base URL can point at http://localhost:8080/v1. More on that near the end.

How do you run Hemmingway-1 in LM Studio?

LM Studio is the easy route: a window, a model browser and no terminal. bartowski’s card lists it as compatible.

1. Install LM Studio. You need an Apple Silicon Mac on macOS 14 or newer, or Windows or Linux with AVX2.
2. Open Discover. That is Cmd+2 on a Mac and Ctrl+2 elsewhere.
3. Paste bartowski/Altworld_Hemmingway-1-GGUF into the search box.
4. Pick a file. LM Studio marks the one it recommends for your hardware.
5. Download it, load it, and chat.

Prefer a command? This does the download for you:

lms get https://huggingface.co/bartowski/Altworld_Hemmingway-1-GGUF@Q4_K_M

LM Studio can also run a local server. It usually sits on port 1234.

How do you run Hemmingway-1 on a Mac with MLX?

MLX is Apple’s own framework for its chips. hermitdave publishes a 4-bit and a 6-bit build. The card’s quickstart:

pip install -U mlx-lm

python3 -m mlx_lm.generate \
  --model hermitdave/Hemmingway-1-MLX-4bit \
  --prompt "Write a text to your landlord about the broken boiler." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

For a chat in the terminal:

python3 -m mlx_lm.chat --model hermitdave/Hemmingway-1-MLX-4bit

For a local server, use mlx_lm.server with the same model. There is also an OptiQ 4-bit build from mlx-community that serves with optiq serve.

Short on memory for the model? The mlx-lm README explains how to raise the wired-memory limit. That needs macOS 15 or newer.

Does Hemmingway-1 work with Ollama?

We can’t confirm it. That is the honest answer.

• The Ollama library has no Hemmingway-1 entry.
• Ollama can run GGUF files straight from Hugging Face, so this should work:

ollama run hf.co/bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M

• Many Hugging Face pages for this model show that exact command. It is a template Hugging Face generates, not a test.
• bartowski’s list of compatible apps leaves Ollama out.
• Ollama’s tracker has closed reports of Hugging Face files with this architecture failing to load, for example #14636.
• The good sign: Ollama ships the base model as qwen3.8:27b, so it knows the architecture.

Update Ollama first. If the pull fails, use llama.cpp or LM Studio.

One trap: Ollama defaults to a small context, 4k tokens under 24 GiB of VRAM. If replies get cut short, start it like this:

OLLAMA_CONTEXT_LENGTH=16384 ollama serve

What is the makers’ route with vLLM?

vLLM is the route the makers document. It is a server for big GPUs:

vllm serve Altworld/Hemmingway-1 --max-model-len 262144

It downloads the full 54.64 GB. If you don’t need the full window, pass a smaller --max-model-len; that cuts memory use. Transformers and SGLang also work with the official weights. For most people at a desk, this is the wrong tool.

How do you make Hemmingway-1 faster?

Turn the thinking down first. Then shrink the context.

How do you turn down Hemmingway-1’s thinking?

The model thinks before it answers, at its highest level by default. On a desktop, that is slow.

The chat template accepts two switches: reasoning_effort, with values xhigh, medium or low, and enable_thinking: false.

# llama.cpp: brief thinking for every request
llama-server -hf bartowski/Altworld_Hemmingway-1-GGUF:Q4_K_M -c 16384 \
  --chat-template-kwargs '{"reasoning_effort":"low"}'

# mlx-lm: no thinking
python3 -m mlx_lm.generate --model hermitdave/Hemmingway-1-MLX-4bit \
  --chat-template-config '{"enable_thinking": false}' --prompt "..."

Start with low, not off. On the hosted API, which our demo uses, turning thinking off gave a visibly weaker short story in our tests, and low added about four seconds before the reply began. That was the hosted model on September 22, 2026. A local 4-bit build may behave differently.

Ollama and LM Studio have no documented way to pass these switches. Ollama has --think=false; whether it works with this template is unconfirmed.

What is MTP, and should you turn it on?

The GGUF builds include the model’s multi-token-prediction layers. In llama.cpp, add --spec-type draft-mtp to use them for faster generation, as bartowski’s card suggests.

One community build’s notes add a warning: the MTP path allocates its own cache, so a large context costs roughly double. Size your context to what you need.

What settings should you use for Hemmingway-1?

The makers publish no sampling settings. Someone has asked on the model page.

The only source is the generation_config.json shipped with the weights:

• Temperature: 1.0
• Top-p: 0.95
• Top-k: 20

The MLX quickstart uses the same temperature and top-p. Start there, and change one setting at a time so you know what did what.

How can you check that Hemmingway-1 is really running locally?

Do this once, after the download finishes.

1. Turn off your Wi-Fi or unplug the cable.
2. Send a prompt.
3. If you get a reply, the model is running on your machine.

Also look at the address. A local server lives at localhost. If your app talks to a website instead, you are using a hosted service.

What if Hemmingway-1 won’t load or runs badly?

Try these fixes in order:

• “Unknown architecture” or a load error. Update llama.cpp to b10964 or newer. Update Ollama or LM Studio if you use those.
• Out of memory. Use a smaller file or a smaller context. Close other big apps.
• Replies cut short in Ollama. Raise the context length.
• Very slow replies. Lower reasoning_effort to low. Shorten the context.
• A long wait on first run. It is downloading 17 GB. Check your connection and free disk space.
• Odd behaviour from a build. Check whether it is a modified version, and compare it with the original.

Can you use a local Hemmingway-1 in Obsidian, VS Code or Cursor?

Possibly. The local server speaks the OpenAI API, so tools that let you paste a base URL can use it.

Use Hemmingway-1 in Obsidian, VS Code or Cursor covers which tools accept a custom base URL and which don’t. Check it before you spend time wiring anything up.

Is running Hemmingway-1 locally worth it?

It is worth it if one of these is true:

• Your text must stay on your machine.
• You want to build on the open weights.
• You write a lot, and you already own the hardware.

It is not worth it if you lack the memory, or you write a few messages a day. For light use, the hosted options cost less than the electricity:

• The free browser demo needs no sign-up.
• The makers’ API costs $0.24 per million input tokens and $0.90 per million output.
• The Hemmingway writing skill in StashBase works beside your own files. StashBase keeps your files and index on your computer, but the model is not local: the text you send goes through StashBase’s hosted model gateway.

With the hosted options, your text leaves your computer. With the local ones, it stays.

Still deciding if it’s good enough to bother? See how Hemmingway-1 compares with Claude and ChatGPT for writing. Looking at other options that fit one GPU? Read the best open-weight models for writing.

Already running it? A good next step is a two-agent workflow: draft with Claude, finish with Hemmingway. If your real problem is text that sounds machine-made, see AI humanizer or a better model. And to keep your own voice in any AI draft, read how to write with AI without losing your voice.

Frequently asked questions

Is Hemmingway-1 free to use commercially?

The weights are released under Apache-2.0, which permits commercial use. Read the license text if your case is unusual.

Does Hemmingway-1 work offline?

Yes, once the model file and the runtime are on your machine. You need the internet only to download them.