LocalLLaMA

localllama@sh.itjust.works

5,580 readers

3,020 active

138 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago

moderators

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hosted (www.theregister.com)

submitted 21 hours ago* by Spite_By_Design_LLC@lemmy.dbzer0.com to c/localllama@sh.itjust.works

15 comments fedilink hide all child comments

*for certain definition of self hosted.

Developed by Chinese open weights model dev Z.ai, GLM 5.1 isn't exactly a small model. At roughly 754 billion total parameters (40 billion activated for each token generated), the model requires roughly 380 GB of GPU memory just to hold the weights and another 6-26 GB per user to maintain context depending on precision.

To serve GLM-5.3 at scale, Revolut deployed 56 Nvidia B200 GPUs, equivalent in GPU count to seven eight-GPU DGX B200 systems. But as Abitbol noted, the greatest performance gains didn't come from adding GPUs, but rather tuning the model's hyperparameters along with switching to NVFP4 quantization, implementing disaggregated prefill and decode, and tuning context lengths.

you are viewing a single comment's thread
view the rest of the comments

[–] Spite_By_Design_LLC@lemmy.dbzer0.com [S] 1 point 20 hours ago*

BRB, just getting some scratch lotto tickets.

14.3 kw. That's fucking obscene.

And by all rights, the Fintech datacentre is what...small? medium?

GLM again:

One DGX B200 flat-out: 14.3 kW × 24h = ~343 kWh/day ≈ what a typical UK house uses in a month

Datacentre size: a full rack of AI gear like this is ~50–100kW. Classification-wise:

Your average office server closet: 5–20 kW
Small datacentre: <1 MW
Mid-size / enterprise: 1–10 MW
Hyperscale (AWS/Google/Microsoft): 50–500+ MW per campus

Revolut's 56 GPUs ≈ one dense rack or two — well under 1 MW, so firmly "small," and that's exactly why they colocate/rent space ("a US-based service") rather than owning a facility. Self-hosting the GPUs doesn't mean self-hosting the building. You don't put 100kW of fridges in your London office — you rent rack space in a proper colo where the power and cooling already exist.

Enjoy the lotto. If you win, you can buy an eighth of a DGX.

Fuuuuuuuuuck

permalink

fedilink

source

parent