Greentext

hideshow 4 child comments

parent

[–] webghost0101@sopuli.xyz 13 points 2 years ago

Much depends on how you prompt. I have definitely encounters this but some basic default instructions around it can tweak it for any usecase.

When i prep a gpt i give it a list of information and steps it is supposed to follow and the further context about the job and my system. Stuff it can assume and what it should never assume.

Creating a good gpt and prompt are alot like programming itself, i can definitely see how if you just want to get some work done that having to do that feels like wasting time but you once a gpt is working its ready to be used any time.

parent

[–] UnRelatedBurner@sh.itjust.works 5 points 2 years ago (2 children)

every AI is getting bad. I'm using some off-brand (some might say indie) ones, and they're slower dumber and have more payed options by the day.

ChatGPT if not for cheating in school I wouldn't use it at all probly. Phind.com was my go-to favorite, now can't search the web for shit gpt option is payed and it's just dumb. You.com my friend used this, straight ignored my request yesterday.

do other's have the same experience? If yes, follow-up question are they dumbing down AI, and keeping the power to themselves? 2nd follow-up how can I selfhost a good AI and what do I need for it to work?

hideshow 4 child comments

parent

[–] WaxedWookie@lemmy.world 4 points 2 years ago (2 children)

Look into ollama and mixtral variants... You'll be limited by things like GPU memory though.

hideshow 4 child comments

parent

[–] miss_brainfarts@lemmy.blahaj.zone 3 points 2 years ago (1 child)

Mixtral 8x7B seems fairly capable to me so far. It's just that I need to wait a few minutes for it to reply, given I'm running it on a 1st gen Ryzen...

Any models you'd reccommend that fit into 4GB of VRAM?

I've tried Deepseek Coder, and it certainly works well for quickly churning out bash scripts for whatever purpose I can possibly think of

parent

[–] WaxedWookie@lemmy.world 2 points 2 years ago (1 child)

Similar story here with an old 10xx GPU. I've just started tinkering with dolphin-mixtral-8x7b, but it's early days.

parent

[–] miss_brainfarts@lemmy.blahaj.zone 3 points 2 years ago

Dolphin is just peak amusement to me. I've asked so many weird things of it that it's basically just cowering in a corner now, afraid of answering any question for fear of getting another kitten killed.

parent

[–] UnRelatedBurner@sh.itjust.works 1 point 2 years ago (1 child)

what'd you say a good GPU is for it?

parent

[–] WaxedWookie@lemmy.world 2 points 2 years ago (1 child)

An nvidia card with as much memory as possible - the newer the better.

I'm also beginning to look into the dedicated accelerators like the coral, but at first wash, it looks like lack of onboard memory will be a massive bottleneck.

parent