Technology

technology@lemmy.world

88,348 readers

14,662 active

4,940 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

L3s@lemmy.world

enu@lemmy.world

technopagan@lemmy.world

L4s@lemmy.world

L3s@hackingne.ws

OpenAI Hoarding Tens Of Thousands Of Apple Mac mini And Mac Studio Devices, As ASUS And MSI Burn Through Their Entire First Batch Of NVIDIA RTX Spark Chip And Beg For More. (wccftech.com)

submitted 1 month ago by eicker@lemmy.world to c/technology@lemmy.world

79 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] Lydia_K@lemmy.world 7 points 1 month ago (3 children)

https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant

I'm running gwen 3.6 with 131k context window on a 3090, it's fast enough and about as good as pay to play Claude at work.

permalink

fedilink

source

parent

hideshow 3 child comments

[–] ArchAengelus@lemmy.dbzer0.com 3 points 1 month ago (1 child)

Upgrade that to 3.8 as soon as your hardware allows (and your use case makes sense). 3.8 is quite a bit more rational.

permalink

fedilink

source

parent

hideshow 1 child comment

[–] Lydia_K@lemmy.world 1 point 1 month ago

I plan to once there is a version with turboquant and MTP as that huge context window is key.

permalink

fedilink

source

parent

[–] boonhet@sopuli.xyz 1 point 1 month ago

A used 3090 is like 2-3k though, IF you can find one :| A month of claude is like 20 EUR. A month of opencode go is half that, but you get less usage.

permalink

fedilink

source

parent