this post was submitted on 10 Feb 2024
1034 points (100.0% liked)
196
16484 readers
1775 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Sure but what is their purpose other than to create convincing sounding sentences? And use a lot of computer resources to do so?
Sure if you want to reduce it down to it's most basic elements. Anything sounds useless like that. Video games are just dots changing color on a screen and use a lot of electricity. Banking software is just tracking changes to a number and also is extremely inefficient and resource intensive.
This is not a convincing argument to say that other things can be reduced down arbitrarily, than explaining the usefulness of it.
Well I can personally say it has cut down the amount of busy work at my job down by a lot. Boilerplate code is easy and near instantaneous. I am also working on a project that can bridge the communication gap between highly technical text and non technical readers. It works extremely well at this task even with a non fine tuned model, which is my current step in development.
In all reality this tokenization and llm language processing is useful for all sorts of things which can be mathematically somewhat modeled similarly to language. Using them for shitty web searches is not ideal.