775
re: the "ai torture chamber" thing (thelemmy.club)
submitted 4 days ago by to c/fuck_ai@lemmy.world
160 comments fedilink hide all child comments
 

alt textCrudely drawn drawing of a person telling a computer "say 'i am in pain'". The computer replies with "> I AM IN PAIN". The person then says "oh my god."

yes, it's a real thing

Crudely drawn drawing of a person telling a computer "say 'i am in pain'". The computer replies with "> I AM IN PAIN". The person then says "oh my god."
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 days ago (18 children)

The input to a model is not part of the model. That's nonsense, you cannot count your prompt as part of its memory.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 18 child comments
    •
  • [–] 1 point 3 days ago (17 children)

    Why does it matter if it is "part" of the model or stored separately as text or a kv cache.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 17 child comments
    •
  • [–] 1 point 3 days ago (16 children)

    Because that's the entire conversation, whether or not a model can have experiences?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 16 child comments
    •
  • [–] 1 point 3 days ago (15 children)

    I really don't see what the location of the state has to do with whether the model has experiences.

    Replying to your other comment as well since they're converging

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 15 child comments
    •
  • [–] 1 point 3 days ago (14 children)

    It's not state though! Prompts are input, they belong to the user of the model. The model has absolutely no control over what gets sent to it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 14 child comments
    •
  • [–] 1 point 3 days ago (13 children)

    By default, the previous conversation turns do. Just because it's easier to corrupt than a human brain doesn't make it not-state.

    If ownership of the prompt is the issue, what about the model's own output reasoning? What about notes/"memories" it may write for itself?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 13 child comments
    •
  • [–] 1 point 3 days ago (12 children)

    I feel like I'm talking to a wall. The entire conversation is whether the model can have experiences. The fact that the prompt comes from a user makes it external to the model. If you include the user as part of the system under investigation, of course you'll come to the conclusion it can have experiences.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] 1 point 3 days ago (11 children)

    You're assuming that for the model to experience something, it's own weights have to change and I don't see why. The "experience" can be contained within the forward pass of the model or stored as a "memory" in a kv cache.

    If I text you your mother died are you somehow unable to experience anything because I wrote the text? No of course not.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 1 point 3 days ago (10 children)

    for the model to experience something, it's own weights have to change

    Yes, exactly. An experience, by definition, must change your behavior. The model being bit-for-bit identical in between prompts precludes this. I can experience you telling me my mother died, and my behavior would change as a result: I might conclude you're an asshole without any credibility, and behave accordingly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 1 point 3 days ago (9 children)

    I don't entirely agree with this definition (most of my day is mundane things that don't really change me), but even then, it's behavior does change, just not permanently.

    Or if you just protect the kv cache consider a whole system like model + harness + files, then it does in fact change permanently as well.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 1 point 3 days ago (8 children)

    most of my day is mundane things that don't really change me

    Don't 'really change' you or don't change you at all? Are you the sum your experiences?

    it's behavior does change, just not permanently.

    It does not change, ever. From the moment it starts processing, to the moment it finishes processing, it is still the same model. It has access to all of its input the moment it's brought into existence, it doesn't experience the prompt linearly, the input is it's raison d'être.

    whole system like model + harness + files

    I would generally agree that a llm could be part of a larger system capable of experiencing things and having difficult conversations about it, but I don't think these harnesses are it. They are very simple systems, compared to the llms they control. I think emergent behavior would require something much more complex, given what we know about life.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 1 point 3 days ago (7 children)

    Okay look, the harness question is a whole nother one. I just want to know, if you consider the model + kv cache together, does that match your definition? The KV cache can change and the model's internal state and outputs can change in response to it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 3 days ago* (last edited 3 days ago) (6 children)

    KV caching doesn't persist between prompts, it is an internal optimization used while generating text. I don't see how it's relevant.

    Edit: it kinda sounds like you're technobabbling me, like "but it uses a recursive algorithm, it must be alive, check mate!" Your question is a non sequitur.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 3 days ago (5 children)

    I'm not techno babbling you. I specifically said "if you consider the model + kv cache together," e.g preserved. A kv cache is not an internal optimization, a model cannot generate text without a kv cache. There's a separate idea of caching the kv cache between requests as an optimization. That's not the one I'm talking about.

    Or if it helps, consider it just within one generation. The model + kv cache. Does that not match your definition?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 3 days ago (4 children)

    Definition of what? Do I think that because llms use a kv cache during inference, that proves they can experience things? You're gonna have to explain your thought process. To me, the particular algorithm used is not the philosophical crux of the issue.

    There's a separate idea of caching the kv cache between requests

    I've never heard of this anyways. The kv cache is specific to a particular input, it can't be reused for a subsequent prompts.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 3 days ago (3 children)

    It can as long as there's a shared prefix.

    Definition of what? Do I think that because llms use a kv cache during inference, that proves they can experience things?

    No, I never claimed to have proof they can experience things. I just think it's possible and you can't categorically dismiss it based on how they work.

    I'm talking about your condition above which we've been arguing about for the last ten messages

    An experience, by definition, must change your behavior.

    Which so far seems like your only argument for why they can't experience things

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 3 days ago (2 children)

    It can as long as there's a shared prefix.

    I've never heard of that. The cache is meant to support token generation, so unless it's trying to repeat itself, I think the previous cache would be little use.

    An experience, by definition, must change your behavior.

    Ok, yes, that is my argument, now how the fuck does a temporary cache, thrown out after every response, prove that the model is changing as a result of undergoing inference?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 days ago (1 child)

    I'm just going to quote myself at this point

    I specifically said “if you consider the model + kv cache together,” e.g preserved.

    Or if it helps, consider it just within one generation.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 days ago* (last edited 2 days ago)

    OK, that's not helpful at all. My answer is no, it does not meet my definition of changing behavior, and I have no idea why you think it would.

    Edit: ok maybe this does make sense from a very stupid point of view, if you allow me to interpret what you mean: you believe because the kv cache is generated during inference, that proves the model is learning a new behavior, and if you save that cache, that proves... Something??

    It's absolutely silly: the model's behavior is to generate a key-value cache, and that's exactly what it did. The fact that the model is doing intermediate calculations based on the input is not evidence of a new behavior being generated, that is just how algorithms work. That's what 'processing' is, you dunce.

    Edit 2: I also just want to point out, if you think the word 'cache' is important, it's really not. Caches are fundamental to how a computer works, no processing could ever occur without them: every cpu instruction involves reading from or writing to at least one L1 cache (register).

  • permalink
  • fedilink
  • source
  • parent