Not only do I run and share it. I've built an entire service to allow everyone else to do so safely. https://aihorde.net/

Yes, I have a desktop with a radeon 9700 ai pro with 32GB of VRAM. Its more than doubled since I bought it but its basically a down clocked 9070xt with double the VRAM. I also have a little arc b50 in my server.

Mostly yeah use it for privacy reasons. I run llama.CPP directly and access via open webui. I host a searx instance for it to use as web search. Mostly run qwen3.6/3.8 or gemma4 on the radeon and almost exclusively gptoss 20b on the B50 along with a second small model for tags and titles in open webui. Yeah I just use it mostly as a basic chatbot or as an agent in obsidian using the engram plugin. Helps search and summarize years of notes and works pretty well.

I've tried to use it as a coding assistant in zed without much luck so usually just post my code into open webui for feedback and it works alot better for me.

Yeah I can access it with other machines on my network including phone with conduit app. I can get to it on the go by connecting to home network with wireguard and sending a magic packet to wakeup desktop.

I havent done any image gen nor have much use for it outside of maybe using it to make app icons in the style of papirus icons for programs they don't and won't support.

It definitely feels more comprable to first ai wave than current frontier models but the privacy and price trade off is worth it to me. I'd say using it in obsidian has been the best use of it so far and one I certainly wouldn't trust to an external AI.

Hardware:

2 x ASUS GX10 128GB(each)

2 x Mac Mini M4 Pro 24GB(each)

The GX10s are clustered and run DeepSeek-V4-Flash-0731-Abliterated. The M4 Pros run various utility LLMs such as Qwen 3.8

Software:

OpenWebUI

Hermes Agent

LiteLLM

Llama.cpp

vLLM

StoryBinder - custom web app that I use for story/script writing when in the mood

LASM - Local AI Stack Manger - custom web interface to manage pretty much all aspects of my local stack. Agents run on the 4 nodes as well as my NAS(Download GGUF/HF snapshots once and copy out to the nodes)

Have yet to connect my Home Assistant into the stack.

For mobile access I use Conduit https://github.com/cogwheel0/conduit

Yes, I run Ollama on a NVIDIA 1080 Ti at home. I run granite4:3b for my Home Assistant assistant for my voice assistant, paperless-ngx, OpenWebUI for chat, and a few other things. For speech to text, I run a custom-built hybrid speech-to-phrase + parakeetTTS (requests that match speech-to-phrase take priority over parakeet.) I run GOModel to route requests to my local Ollama instance, or DeepInfra (cloud-based for larger models), or Tinfoil (cloud-based for encrypted LLM chats.)

As much as I love to use local-everything, only my smarthome+basic chats go local. Trying to fit both the STT and LLM models on the same GPU at the same time while also using this computer as a media console is challenging. I've recently started looking at the Hailo m.2 accelerator board to see if I can shift the STT to it.

Yes, I installed an LLM at home. I use it for many useful things:

1. reading novels by translating them from English to my language on the fly via machine
2. for communicating on lemmy by translating posts and comments
3. For HomeAssistant
4. And very rarely for development and searching the internet

I have a RX9070XT and llama-swap to switch models on the fly. I'm thinking about installing a second RX9070XT so that I don't take away the translation capability from my lemmy instance while I stress the other neural network with other tasks.

Yeah, I have a fairly useable setup although it's only accessible on my lan. I have a web server but it doesn't host the llm as it's not powerful enough.

Rx7600 8gb vram GPU 5700x CPU 64gb 3600mts ram

Hosting ornith 35b a3b iq4_xs Llama.cpp server Pi harness or llama.cpp webui Searxng web search Coding sandbox Restricted filesystem access

Get around 40 tok / sec and it's really useable. Not as good as Claude sonnet, but good enough for most things.

Ollama, openwebui, running the ~20GB Qwen models on a 3090

Epyc 7551? 256gb ddr4, r9700 + mi50. Running two models depending on what I’m doing . Llama.cpp + comfyUI or two llama.cpp for model and advisor in OMP. Main model is Qwen 3.8 27b abliterated

What a timely comment.

I recently gave my kid a gaming PC for his birthday...which I used as an excuse to upgrade my GPU from a 1660 Super to a 5060ti 16GB

Combined with a 5600x3d and 32gb of system memory.

I just started messing with local LLMs yesterday. "Getting started" is rather easy. pacman -S lm-studio (I use arch, btw), launch it, download a model that your hardware supports, and start talking.

Right now I'm working on getting it to talk to vscode.

I self-host my own neural network, all within the confines of my own skull.

24GB GPU

Qwen 3.8 ( 2 variants )

Pangolin>OpenWebUI>lama.cpp for general use and available outside the homelab

Zoo Code>OpenWebUI>lama.cpp for coding , mostly scripts and html

Qwen 3.8 was a game changer for me.

I'm self-hosting immich (with ML-based facial recognition, semantic search and OCR) and psiTTSa, an offline text-to-speech engine.

(Prob not the answer you were looking for, but a reminder that "AI" is more than just LLMs.)

Not me. Call me old-fashioned, but I prefer organic.

Running Qwen 3.8 27B dense on some R9700s, and Qwen 3.8 flash next on dgx sparks. I use it for code gen outside work, including an automated Dev pipeline that rubs on its own. When I did the math comparing to token API pricing of Opus 4.8, it covered its own costs in a few months. With the sparks I get about 40tg/s, 1500pp/s, but can get concurrency up to 5 stably. With the R9700s, I get ~100tg/s, 3000pp/s and can also get up to 5 concurrency.

I access it over wireguard to the network that it is on, and use termux to ssh into a build machine for dev over CLI using https://pi.dev/

I also started building https://fcli.dev/ to make integrations easier; the idea being that LLMs should just implement simple reusable functions instead of repeatedly making complex curl requests to integrate with a service.

I also use LLMs heavily for work; Qwen 3.8 seems almost on par to Opus 4.8 IMO. I also don't have to worry about token costs, and it often responds faster than Claude would because there usually isn't a queue on my machines. At this point, I wouldn't complain if my work provided me 2x B6000 instead of a Claude subscription; I'd still be able to get all my work done.