Skip to the content.

nightsmith

Local AI that runs in the background on your own Mac, set up with one terminal command.

curl -fsSL https://nightsmith.sh/install | sh

Use cases

The model runs on your Mac, so the text that you give it stays on your Mac. That makes nightsmith a fit for text that you cannot paste into a cloud service: contracts, customer data, medical letters, private code. There is also no cost for each request and no rate limit.

One prompt can hold up to 40,000 tokens, which is roughly 30,000 words. The model does not look anything up, so do not use it as a source of facts. See Known limits.

What setup does

Nightsmith looks at your Mac and picks a model that fits. It installs the runtime (the program that runs the model) and downloads the model. Then it asks the model a real question, to prove that the model works.

To talk to the model, type nightsmith. You can also point any OpenAI-compatible client at http://127.0.0.1:8080/v1. The model server keeps running after you close the terminal.

Nightsmith installs to ~/.local/bin. If that directory is not in your PATH, nightsmith adds it and opens a fresh shell at the end. You do not source a file or restart the terminal. On a 16 GB M4, setup takes about five minutes and 8 GB of disk.

Nightsmith runs on Apple Silicon only (M1 or newer). On an Intel Mac, nightsmith refuses clearly. It does not fall back to a slow mode.

Using it

Type nightsmith to start a conversation. Nightsmith shows each answer as the model writes it. At nine words a second, you do not want to wait for the whole answer. Press Ctrl-C to stop an answer and stay in the conversation. Press Ctrl-D to leave. These commands work inside a conversation: /help /new /sessions /resume ID /context /paste /exit. Nightsmith writes each conversation to ~/.nightsmith/chats as it happens.

Command What it does
nightsmith Talk to the model. Before setup, this command runs setup. With no terminal, it reports status and uses the exit codes of status
nightsmith -p "…" One question, one answer. With no question, or with a lone -, it reads stdin
nightsmith -c Continue the last conversation
nightsmith -r [ID] Continue that conversation. With no ID, list all conversations
nightsmith start / nightsmith stop Turn the model server on and off
nightsmith status Ask the model a real question and show its memory use. Add --json for programs
nightsmith remove List everything that nightsmith installed, then delete it (uninstall also works)
nightsmith config check Show what your configuration costs, against what this Mac can give
nightsmith model list Show the models that fit on this Mac, with real sizes
nightsmith model use <repo> Switch model, and prove that the new model answers
nightsmith --yes Set up without questions, for a script
nightsmith -p "what's a spring tide?"
cat notes.txt | nightsmith -p "sum this up" -   # instruction, then material
cat invoice.txt | nightsmith -p "list the supplier, date and total as JSON" -
git diff | nightsmith -p "write a commit message for this diff" -

Standard output contains the answer and nothing else. Timings and warnings go to standard error, and only when standard error is a terminal.

For a program or an agent

The server speaks the OpenAI chat-completions API. Any program that accepts a custom base URL can use it. The base URL is http://127.0.0.1:8080/v1. If port 8080 was taken, nightsmith status prints the port in use. The model id is the Hugging Face repo as /v1/models lists it. You can also leave model out.

curl -fsSL https://nightsmith.sh/install | NIGHTSMITH_YES=1 sh   # unattended
nightsmith status --json    # 0 answered · 3 not running · 4 not answering
nightsmith start            # returns only once the model has answered

Know these five things before you send requests:

Errors use the OpenAI shape on every path (error.message, type, param, code). nightsmith stop lets answers in progress finish. That can take the time of one whole answer, about 2.5 minutes at the defaults. In Activity Monitor, the server is named nightsmith-model.

Measured on a base M4 / 16 GB

These numbers are for Gemma 4 12B (4-bit) on mlx_lm.server. Setup measures your Mac and reports its own numbers. It does not quote these.

What Measured
Writing ~12 tokens/s (about 9 words a second), flat over a 92-minute run
Reading a prompt ~131 tokens/s cold, ~1,900 warm · ~19 s per 2,048 tokens, so 40,000 tokens is six minutes before the first word
Memory 10.2 GB peak, 7.7 GB idle, against Metal’s 12.7 GB GPU limit
Endurance 953 requests over 3 hours, zero failures, memory flat
Context ceiling 38,009 tokens answered in 349 s · 43,389 in 406 s · 62,009 fatal. Capped at 40,000
Concurrency ceiling 3 × 8,000 tokens fine · 4 × 4,000 fatal · 5 × 2,000 fatal. Capped at 3
A conversation Gets faster as it grows. The first word takes 1.40 s on turn 1 and 0.71 s by turn 4, because the cache reuses the previous turn

Memory bandwidth sets the speed, not the amount of RAM. The same model is about 2× faster on an M-series Pro and 4× faster on a Max.

The cache (the saved work from the last prompt) has one slot. nightsmith status sends a real question of its own, and that question takes the slot. In the middle of a conversation, reuse drops from 157 tokens to 4. The next first word then takes 2.15 s, up from 0.71 s.

Known limits

Building

go test ./...
GOOS=darwin GOARCH=arm64 go build -o nightsmith .

License

MIT