In this article 12 sections
- 1. What you get, and what you do not
- 2. Choose the memory before anything else
- 3. Install Ollama and run a first model
- 4. Talk to it from a program
- 5. Make the Mac behave like a server
- 6. Three settings that suit an assistant
- 7. Reach it from your phone and laptop
- 8. Add a chat interface
- 9. Connect the rest of your tools
- 10. What stays private
- 11. When it misbehaves
- 12. Where a Mac mini stops
A Mac mini is small, silent and built to stay on, and its memory is shared between the processor and the graphics chip, which is exactly what a language model needs. That makes it a practical home for a private assistant: one you can ask anything without the question leaving your house. This guide sets one up from an empty Mac to a chat page on your phone.
1. What you get, and what you do not
A local model of the size a Mac mini can hold is good at summarizing, drafting, explaining, translating and working over your own notes. For the hardest reasoning and coding tasks, a hosted model is still stronger. Many people end up using both: the local one for anything private or routine, a hosted one for the rest.
2. Choose the memory before anything else
On Apple silicon, the model lives in the same memory as macOS and your apps, and that memory cannot be added later. It is the one decision you cannot undo.
Apple's specification page listed these options on the day this was written:
A workable rule of thumb: the model's download size should be no more than about half of the Mac's memory. The rest goes to macOS, the conversation's context and whatever else is open. Ollama's own quickstart gives one data point: for a model whose download is about 7.2 GB, it recommends 8 GB of available memory, and notes that larger context windows need more.
Storage matters too. Models are several gigabytes each and you will try more than one, so the smallest disk fills quickly.
3. Install Ollama and run a first model
Ollama is the program that downloads models and runs them. Its documentation asks for macOS Sonoma (version 14) or newer and an Apple M-series chip for GPU support.
- Download Ollama for macOS from ollama.com, open the disk image and drag the app to Applications.
- Open it. On first start it checks that the
ollama command is available in Terminal and offers to set it up.
- Open Terminal and download a model. This is the one from Ollama's quickstart:
ollama pull gemma4:e2b
ollama run gemma4:e2b
The second command opens a chat in the terminal. Ask something, then type /bye to leave. The model library on ollama.com lists other models with their sizes; apply the rule from step 2 when choosing.
Check that the model runs on the graphics chip and not only the processor:
ollama ps
The PROCESSOR column should read 100% GPU. If it shows a split with CPU, the model is too large for the free memory; close other apps or choose a smaller model.
4. Talk to it from a program
Ollama runs a local server on port 11434. It answers in the OpenAI format, so existing code needs only a different address:
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gemma4:e2b",
"messages": [{"role": "user", "content": "Say hello in one sentence."}]
}'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama") # the key is required but ignored
reply = client.chat.completions.create(
model="gemma4:e2b",
messages=[{"role": "user", "content": "Summarize in one line: the meeting moved to Friday at 3."}],
)
print(reply.choices[0].message.content)
The comment in that code is from Ollama's documentation, and it carries a warning: the local server does not check who is asking. Keep that in mind for step 7.
5. Make the Mac behave like a server
An assistant that is asleep when you need it is no assistant. In System Settings, open Energy:
- Turn on "Prevent automatic sleeping when the display is off". The screen can still switch off.
- Turn on "Wake for network access".
- Look for the option to start up automatically after a power failure, and turn it on.
Then make sure Ollama starts with the Mac: in System Settings, under General, check Login Items and add Ollama if it is not listed. Apple notes that preventing sleep can increase power use.
6. Three settings that suit an assistant
When Ollama runs as a Mac app, its documentation says to set these with launchctl and then restart the app:
launchctl setenv OLLAMA_CONTEXT_LENGTH 8192
launchctl setenv OLLAMA_KEEP_ALIVE -1
Quit Ollama from its menu bar icon and open it again. If the settings seem to be gone after a restart of the Mac, run the commands again; values set this way belong to the current login session.
Memory use grows with the context length. If ollama ps stops showing 100% GPU after you raise it, you went too far for this Mac.
7. Reach it from your phone and laptop
To let other devices on your home network connect, change the address Ollama listens on:
launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
Restart the Ollama app. Find the Mac's address in System Settings under Network; it looks like 192.168.1.20. From another computer on the same network, repeat the request from step 4 with that address in place of localhost.
Now the warning from step 4 applies. Anyone who can reach that port can use your model, and there is no password.
- Do this only on a network you control. Not on shared or public Wi-Fi.
- Do not forward port 11434 on your router to the internet.
- To use the assistant away from home, connect to your home network through a private VPN. Do not expose the port.
- Give the Mac a fixed address in your router's settings, so the address you saved on your phone keeps working.
8. Add a chat interface
A terminal is fine for testing. For daily use, and for anyone else at home, you want a chat page. Open WebUI is an open-source one that connects to Ollama. Its documentation offers a way to run it without Docker. Install the uv tool as that page describes, then:
DATA_DIR=~/.open-webui uvx --python 3.12 open-webui@latest serve
It runs at http://localhost:8080. The first start takes a minute. The first screen offers "Create Admin Account"; that first account is the administrator, and sign-up closes once it exists. To let family members join, the documentation says to turn on new sign-ups in the admin settings, where new accounts then wait for your approval.
If your models do not appear, open the avatar menu, then Settings, Admin, Connections, and set the Ollama address to http://localhost:11434.
From your phone, open http:// followed by the Mac's address and :8080. If the page does not load from another device, start the command with --host 0.0.0.0 added. Add the page to your home screen and it behaves much like an app.
Unlike the Ollama port, this page has accounts and passwords. If you use it, you can set OLLAMA_HOST back to its default so that only the chat page is reachable from the network.
9. Connect the rest of your tools
- Scripts and shortcuts: anything that can send a web request can use the API from step 4. A daily summary of a notes folder is a good first project.
- Desktop apps: Ollama's quickstart describes an Apps section in the Mac app that connects supported desktop assistants to your local models.
- Coding agents: the same page lists launch commands such as
ollama launch claude that start a coding agent against a model you choose. Expect less from a small local model than from a hosted one on large coding tasks.
10. What stays private
Ollama's FAQ states that when you run locally, it does not see your prompts or data. Ollama also offers cloud-hosted models through the same app, and for those the FAQ says prompts are processed to provide the service. For private work, use a model you downloaded with ollama pull, and confirm with ollama ps that it is loaded on this Mac while you chat.
The Mac itself is now the thing to protect. Turn on FileVault, use a strong login password, and remember that chat history in Open WebUI is stored in the data folder you set.
11. When it misbehaves
Ollama's logs are in ~/.ollama/logs; server.log is the one to read when a model fails to load.
12. Where a Mac mini stops
This setup is for one person or one household. By default Ollama answers one request at a time for each model; its OLLAMA_NUM_PARALLEL setting can raise that, with memory use growing to match. When a whole team needs the same model at once, the tool changes: a GPU server running vLLM batches many requests together. That is the subject of the series Run your own LLM server with vLLM.
Sources
This article reflects public information available when it was written. Versions, defaults and prices can change; check the linked official documentation.
Commands and defaults come from the Ollama, Open WebUI and Apple documentation as read on November 12, 2026. No Mac was available, so nothing in this guide was run for it, and no speeds are quoted. The memory guidance in section 2 is a rule of thumb, not a measurement. Written for this site with AI assistance, from the sources above. For errors or additional information, please request a correction. Editorial policy