▲ 240 ▼ Torvalds: AI Is the New Compiler, Not the New Programmer (pbxscience.com) submitted 1 month ago* by inari@piefed.zip to c/technology@lemmy.world 103 comments fedilink hide all child comments
[–] brucethemoose@lemmy.world 17 points 1 month ago* (3 children) Technically, LLMs (and most ML models) are deterministic with the same input and same seed. I get what you mean though. permalink fedilink source parent hideshow 3 child comments replies: [–] Scrollone@feddit.it 8 points 1 month ago* (2 children) So does ChatGPT intentionally change seed at every interaction so it always spits two different outputs given the same input? permalink fedilink source parent hideshow 2 child comments replies: [–] Scipitie@lemmy.dbzer0.com 16 points 1 month ago In short: yes. You can tune these values when self hosting - it basically changes the b chance which tokens will be used under which circumstances. permalink fedilink source parent [–] brucethemoose@lemmy.world 5 points 1 month ago* Like Scipitie said, sort of. In the ChatGPT app, there's tons of shuffling in the background, like context being injected, maybe sampling changed, agenic action, quantization... its "far" from the actual LLM, opaque by design, and as a result certainly not deterministic. You can sometimes get deterministic output with the OpenAI API, but it's also dependent on nothing changing on their end. And their end changes a lot. But self hosting or using a more consistent provider will give you deterministic output. permalink fedilink source parent
[–] Scrollone@feddit.it 8 points 1 month ago* (2 children) So does ChatGPT intentionally change seed at every interaction so it always spits two different outputs given the same input? permalink fedilink source parent hideshow 2 child comments replies: [–] Scipitie@lemmy.dbzer0.com 16 points 1 month ago In short: yes. You can tune these values when self hosting - it basically changes the b chance which tokens will be used under which circumstances. permalink fedilink source parent [–] brucethemoose@lemmy.world 5 points 1 month ago* Like Scipitie said, sort of. In the ChatGPT app, there's tons of shuffling in the background, like context being injected, maybe sampling changed, agenic action, quantization... its "far" from the actual LLM, opaque by design, and as a result certainly not deterministic. You can sometimes get deterministic output with the OpenAI API, but it's also dependent on nothing changing on their end. And their end changes a lot. But self hosting or using a more consistent provider will give you deterministic output. permalink fedilink source parent
[–] Scipitie@lemmy.dbzer0.com 16 points 1 month ago In short: yes. You can tune these values when self hosting - it basically changes the b chance which tokens will be used under which circumstances. permalink fedilink source parent
[–] brucethemoose@lemmy.world 5 points 1 month ago* Like Scipitie said, sort of. In the ChatGPT app, there's tons of shuffling in the background, like context being injected, maybe sampling changed, agenic action, quantization... its "far" from the actual LLM, opaque by design, and as a result certainly not deterministic. You can sometimes get deterministic output with the OpenAI API, but it's also dependent on nothing changing on their end. And their end changes a lot. But self hosting or using a more consistent provider will give you deterministic output. permalink fedilink source parent