997
It's Breathtaking How Fast AI Is Screwing Up the Education System (gizmodo.com)
submitted 1 year ago by to c/technology@lemmy.world
359 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 32 points 1 year ago (2 children)

I'm thinking the only way people will be able to do schoolwork without cheating now is going to be to make them sit in a monitored room and finish it there.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [+] -36 points 1 year ago (2 children)

    How is this kind of testing relevant anymore? Isn't it creating an unrealistic situation, given the brave new world of AI everywhere?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 29 points 1 year ago (1 child)

    Because it test what you actually retained, not what you can convince an AI to tell you.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -13 points 1 year ago* (3 children)

    But what good is that if AI can do it anyway?

    That is the crux of the issue.

    Years ago the same thing was said about calculators, then graphing calculators. I had to drop a stat class and take it again later because the dinosaur didn't want me to use a graphing calculator. I have ADD (undiagnosed at the time) and the calculator was a big win for me.

    Naturally they were all full of shit.

    But this? This is different. AI is currently as good as a graphing calculator for some engineering tasks, horrible for some others, excellent at still others. It will get better over time. And what happens when it's awesome at everything?

    What is the use of being the smartest human when you're easily outclassed by a machine?

    If we get fully automated yadda yadda, do many of us turn into mush-brained idiots who sit around posting all day? Everyone retires and builds Adirondack chairs and sips mint juleps and whatever? (That would be pretty sweet. But how to get there without mass starvation and unrest?)

    Alternately, do we have to do a Butlerian Jihad to get rid of it, and threaten execution to anyone who tries to bring it back... only to ensure we have capitalism and poverty forever?

    These are the questions. You have to zoom out to see them.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 23 points 1 year ago* (1 child)

    Because if you don't know how to tell when the AI succeeded, you can't use it.

    To know when it succeeded, you must know the topic.

    The calculator is predictable and verifiable. LLM is not

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [+] -11 points 1 year ago (2 children)

    I'm not sure what you're implying. I've used it to solve problems that would've taken days to figure out on my own, and my solutions might not have been as good.

    I can tell whether it succeeded because its solutions either work, or they don't. The problems I'm using it on have that property.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 1 year ago

    The problem is offloading critical thinking to a blackbox of questionably motivated design. Did you use it to solve problems or did you use it to find a sufficient approximation of a solution? If you can't deduce why the given solution works then it is literally unknowable if your problem is solved, you're just putting faith in an algorithm.

    There are also political reasons we'll never get luxury gay space communism from it. General Ai is the wet dream of every authoritarian: an unverifiable, omnipresent, first line source of truth that will shift the narrative to whatever you need.

    The brain is a muscle and critical thinking is trained through practice; not thinking will never be a shortcut for thinking.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 year ago (1 child)

    That says more about you.

    There are a lot of cases where you can not know if it worked unless you have expertise.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 1 year ago*

    This still seems too simplistic. You say you can't know whether it's right unless you know the topic, but that's not a binary condition. I don't think anyone "knows" a complex topic to its absolute limits. That would mean they had learned everything about it that could be learned, and there would be no possibility of there being anything else in the universe for them to learn about it.

    An LLM can help fill in gaps, and you can use what you already know as well as credible resources (e g., textbooks) to vet its answer, just as you would use the same knowledge to vet your own theories. You can verify its work the same way you'd verify your own. The value is that it may add information or some part of a solution that you wouldn't have. The risk is that it misunderstands something, but that risk exists for your own theories as well.

    This approach requires skepticism. The risk would be that the person using it isn't sufficiently skeptical, which is the same problem as relying too much on their own opinions or those of another person.

    For example, someone studying statistics for the first time would want to vet any non-trivial answer against the textbook or the professor rather than assuming the answer is correct. Answer comes from themself, the student in the next row, or an LLM, doesn't matter.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 13 points 1 year ago (3 children)

    but what good is that if AI can do it anyway?

    It can't. It just fucking can't. We're all pretending it does, but it fundamentally can't.

    https://appleinsider.com/articles/24/10/12/apples-study-proves-that-llm-based-ai-models-are-flawed-because-they-cannot-reason

    Creative thinking is still a long way beyond reasoning as well. We're not close yet.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 1 year ago (1 child)

    It can and it has done creative mathematical proof work. Nothing spectacular, but at least on par with a mathematics grad student.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 1 year ago (1 child)

    Specialized AI like that is not what most people know as AI. Most people reffer to it as LLMs.

    Specialized AI, like that showcased, is still decades away from generalized creative thinking. You can't ask it to do a science experiment with in a class because it just can't. It's only built for math proof.

    Again, my argument is that it won't never exist.

    Just that it's so far off it'd be like trying to regulate smart phone laws in the 90s. We would have only had pipe dreams as to what the tech could be, never mind its broader social context.

    So tall to me when it can, in the case of this thread, clinically validated ways of teaching. We're still decades from that.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Show me a human that can do it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    The faulty logic was supported by a previous study from 2019

    This directly applies to the human journalist, studies on other models 6 years ago are pretty much irrelevant and this one apparently tested very small distilled ones that you can run on consumer hardware at home (Llama3 8B lol).

    Anyway this study seems trash if their conclusion is that small and fine-tuned models (user compliance includes not suspecting intentionally wrong prompts) failing to account for human misdirection somehow means "no evidence of formal reasoning". Which means using formal logic and formal operations and not reasoning in general, we use informal reasoning for the vast majority of what we do daily and we also rely on "sophisticated pattern matching" lmao, it's called cognitive heuristics. Kahneman won the Nobel prize for recognizing type 1 and type 2 thinking in humans.

    Why don't you go repeat the experiment yourself on huggingface (accounts are free, over ten models to test, actually many are the same ones the study used) and see what actually happens? Try it on model chains that have a reasoning model like R1 and Qwant and just see for yourself and report back. It would be intellectually honest to verify things since we're talking about critical thinking in here.

    Oh add a control group here, a comparison with average human performance to see what the really funny but hidden part is. Pro-tip: CS STEMlords catastrophically suck when larping being cognitive scientists.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    So you say I should be intellectually honest by doing the experiment myself, then say that my experiment is going to be shit anyways? Sure... That's also intellectually honest.

    Here's the thing.

    My education is in physics, not CS. I know enough to know what I try isn't going to be really valid.

    But unless you have peer reviewed searches to show otherwise, because I would take your home grown experiment to be as valid as mine.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    And here's experimental verification that humans lack formal reasoning when sentences don't precisely spell it out for them: all the models they tested except chatGPT4 and o1 variants are from 27B and below, all the way to Phi-3 which is an SLM, a small language model with only 3.8B parameters. ChatGPT4 has 1.8T parameters.

    1.8 trillion > 3.8 billion

    ChatGPT4's performance difference (accuracy drop) with regular benchmarks was a whooping -0.3 versus Mistral 7B -9.2 drop.

    Yes there were massive differences. No, they didn't show significance because they barely did any real stats. The models I suggested you try for yourself are not included in the test and the ones they did use are known to have significant limitations. Intellectual honesty would require reading the actual "study" though instead of doubling down.

    Maybe consider the possibility that a. STEMlords in general may know how to do benchmarks but not cognitive testing type testing or how to use statistical methods from that field b. this study being an example of a few "I'm just messing around trying to confuse LLMs with sneaky prompts instead of doing real research because I need a publication without work" type of study, equivalent to students making chatGPT do their homework c. 3.8B models = the size in bytes is between 1.8 and 2.2 gigabytes d. not that "peer review" is required for criticism lol but uh, that's a preprint on arxiv, the "study" itself hasn't been peer reviewed or properly published anywhere (how many months are there between October 2024 to May 2025?) e. showing some qualitative difference between quantitatively different things without showing p and using weights is garbage statistics f. you can try the experiment yourself because the models I suggested have visible Chain of Thought and you'll see if and over what they get confused about g. when there are graded performance differences with several models reliably not getting confused at least more than half the time but you say "fundamentally can't reason" you may be fundamentally misunderstanding what the word means

    Need more clarifications instead of reading the study or performing basic fun experiments? At least be intellectually curious or something.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    And still nothing peer reviewed to show?

    Synethic benchmarks mean nothing. I don't care how much context someone can store, when the context being stored is putting glue on pizza.

    Again, I'm looking for some academic sources (doesn't have to be stem, education would be preferred here) that the current tech is close to useful.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    You made huge claims using a non peer reviewed preprint with garbage statistics and abysmal experimental design where they put together 21 bikes and 4 race cars to bury openAI flagship models under the group trend and go to the press with it. I'm not going to go over all the flaws but all the performance drops happen when they spam the model with the same prompt several times and then suddenly add or remove information, while using greedy decoding which will cause artificial averaging artifacts. It's context poisoning with extra steps i.e. not logic testing but prompt hacking.

    This is Apple (that is falling behind in its AI research) attacking a competitor with fake FUD and doesn't even count as research, which you'd know if you looked it up and saw you know, opinions of peers.

    You're just protecting an entrenched belief based on corporate slop so what would you do with peer reviewed anything? You didn't bother to check the one you posted yourself.

    Or you post corporate slop on purpose and now trying to turn the conversation away from that. Usually the case when someone conveniently bypasses absolutely all your arguments lol.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    Okay, here's a non apple source since you want it.

    https://arxiv.org/abs/2402.12091

    5 Conclusion In this study, we investigate the capacity of LLMs, with parameters varying from 7B to 200B, to com- prehend logical rules. The observed performance disparity between smaller and larger models indi- cates that size alone does not guarantee a profound understanding of logical constructs. While larger models may show traces of semantic learning, their outputs often lack logical validity when faced with swapped logical predicates. Our findings suggest that while LLMs may improve their logical reason- ing performance through in-context learning and methodologies such as COT, these enhancements do not equate to a genuine understanding of logical operations and definitions, nor do they necessarily confer the capability for logical reasoning.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    Another unpublished preprint that hasn't published peer review? Funny how that somehow doesn't matter when something seemingly supports your talking points. Too bad it doesn't exactly mean what you want it to mean.

    "Logical operations and definitions" = Booleans and propositional logic formalisms. You don't do that either because humans don't think like that but I'm not surprised you'd avoid mentioning the context and go for the kinda over the top and easy to misunderstand conclusion.

    It's really interesting how you get people constantly doubling down on specifically chatbots being useless citing random things from google but somehow Palantir finds great usage in their AIs for mass surveillance and policing. What's the talking point there, that they're too dumb to operate and that nobody should worry?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    As apposed to the nothing you've cited that context tokens actually improve reasoning?

    I love how you keep going further and further away from the education topic at hand, and now brining in police survalinece, which everyone knows is 100% accurate.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    You're less coherent than a broken LLM lol. You made the claim that transformer-based AIs are fundamentally incapable of reasoning or something vague like that using gimmicky af "I tricked the chatbot into getting confused therefore it can't think" unpublished preprints (while asking for peer review). Why would I need to prove something? LLMs can write code, that's an undeniable demonstration that they understand abstract logic fairly well that can't be faked using probability and it would be a complete waste of time to explain it to anyone who is either having issues with cognitive dissonance or less often may be intentionally trying to spread misinformation.

    Are the AIs developed by Palantir "fundamentally incapable" of their demonstrated effectiveness or not? It's a pretty valid question when we're already surveilled by them but some people like you indirectly suggest that this can't be happening. Should people not care about predictive policing?

    How about the industrial control AIs that you "critics" never mention, do power grid controllers fake it? You may need to tell Siemens, they're not aware their deployed systems work. And while on that, we shouldn't be concerned about monopolies controlling public infrastructure with closed source AI models because they're "fundamentally incapable" to operate?

    I don't know, maybe this "AI skepticism" thing is lowkey intentional industry misdirection and most of you fell for it?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    My larger point, AI replacing teachers is at least a decade away.

    You've given no evidence that it is. You've just said you hate my sources, while not actually making a single argument that it is.

    You said well it stores context, but who cares? I showed that it doesn't translate to what you think, and you said you don't like, without providing any evidence that it means anything beyond looking good on a graph.

    I've said several times, SHOW ME ITS CLOSE. I don't care what law enforcement buys, because that has nothing to do with education.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    I never said its going to replace teachers or that it "stores context" but your sloppily googled preprints to support your "fundamentally can't reason" statement were demonstrably garbage. You didn't say even once "show me it's close" but you think you said several times. Either your reading comprehension is worse than an LLM and you wildly confabulate, which means an LLM could replace you or you're a bot. Anyway, so far you proved nothing and already said they can write code, it's a non trivial cognitive task that you can't perform without several higher order abilities so cope and seethe I guess.

    So, what about Palantir AI? Is that also "not close"? Why are you avoiding surveillance AI? They're both neural networks. Some are LLMs.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago* (last edited 1 year ago) (1 child)

    I said AI isn't close in education. That was my entire claim

    I never said anything about any other company. I said AI in education isn't happening soon. You keep pulling in other sectors.

    I've also had several comments in this thread before you came in saying that.

    EDIT: give me a citation that LLMs can reason for code. Because in my experience as someone that professionally codes with AI (copilot) it's not capable at that. It's guess what it thinks I want to write in small segments.

    https://x.com/leojr94_/status/1901560276488511759

    Espcially when it has a nasty habit of leaking secrets.

    EDIT2 forgot to say why I'm ignoring other fields. Because we're not talking about AI in those fields. We're talking education and search engines at best. My original comment was that AI generated educational papers still serve their original purpose.

    What the fuck does that have to do with anything to do with plaintair?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    Your claim was this, "supported" by some corporate unpublished preprint (which is really funny considering you have the nerve to ask for citations):

    It can't. It just fucking can't. We're all pretending it does, but it fundamentally can't.

    You don't need a citation for LLMs being able to "reason for code", doubting AI coding abilities is delusional online yapping considering how documented it is since its deployed all over the place so how about you prove that being able to write code and do things like control flow, conditionals etc can be done without reason. Try doing that instead of spamming incoherent replies.

    Nobody cares if you're a professional vibe coder all of a sudden, if you can't code without copilot maybe you shouldn't have an opinion based on Apple's "research".

    But until then, are Palantir's AIs fundamentally incapable of reasoning? Yes or no? None of you anti-AI warriors are clear, should we not worry about corporate AI surveillance because apparently AI isn't really "I" or not? Simple question, but maybe ask copilot for help. But you seem bugged when it comes to corporate propaganda contradictions, it's really interesting.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -3 points 1 year ago (1 child)

    It's already capable of doing a lot, and there is reason to expect it will get better over time. If we stick our fingers in our ears and pretend that's not possible, we will not be prepared.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago* (last edited 1 year ago) (1 child)

    If you read, it's capable of very little under the surface of what it is.

    Show me one that is well studied, like clinical trial levels, then we'll talk.

    We're decades away at this point.

    My overall point of it's just as meaningless to talk about now as it was in the 90s. Because we can't convince of what a functioning product will be, never mind it's context I'm a greater society. When we have it, we can discuss it then as we have something tangible to discuss. But where we'll be in decades is hard to regulate now.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago (1 child)

    Alpha Fold. We're not decades away. We're years at worst.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago*

    If you assume the unlimited power needed right now to power Aloha fold at scale of all human education.

    We have at best proof of concepts that computers can talk. But LLMs don't have any way of actually knowing anything behind them. That's kinda the problem.

    And it's not a "we'll figure out the one trick" but more fundamentally how it works doesn't allow for that to happen.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 10 points 1 year ago (2 children)

    If you want to compare a calculator to an LLM, you could at least reasonably expect the calculator result to be accurate.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 year ago (1 child)

    Why. Because you put trust into the producers of said calculators to not fuck it up. Or because you trust others to vet those machines or are you personally validating. Unless your disassembling those calculators and inspecting their chips sets your just putting your trust in someone else and claiming "this magic box is more trust worthy"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    A combination of personal vetting via analyzing output and the vetting of others. For instance, the Pentium calculation error was in the news. Otherwise, calculation by computer processor is understood and the technology is acceptable to be used for cases involving human lives.

    In contrast, there are several documented cases where LLM's have been incorrect in the news to a point where I don't need personal vetting. No one is anywhere close to stating that LLM's can be used in cases involving human lives.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 1 year ago (1 child)

    How exactly do you think those instances got into the news in the first place. I'll give you a hint. People ARE vetting them and reporting when they're fucking up. It is a bias plain and simple. People are absolutely using Ai in cases involving humans.

    https://www.nytimes.com/2025/03/20/well/ai-drug-repurposing.html

    https://www.advamed.org/2024/09/20/health-care-ai-is-already-saving-lives/

    https://humanprogress.org/doctors-told-him-he-was-going-to-die-then-ai-saved-his-life/

    Your opinions are simply biased and ill-informed. This is only going to grow and become a larger and larger dataset. Just like the auto driving taxis. Everyone likes to shit on them while completely ignoring the truth and statistics. All while acting like THIS MOMENT RIGHT NOW is the best they're ever going g to get.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 22 points 1 year ago (1 child)

    What do you think testing is for? It's to show what you know/have learned

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •