375
Leaked Documents Show OpenAI Has a Very Clear Definition of ‘AGI’ (gizmodo.com)
submitted 2 years ago by to c/technology@lemmy.world
181 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 2 years ago* (31 children)

Its a definition, but not an effective one in the sense that we can test and recognize it. Can we list all cognitive tasks a human can do? To avoid testing a probably infinite list, we should instead understand what are the basic cognitive abilities of humans that compose all other cognitive abilities we have, if thats even possible. Like the equivalent of a turing machine, but for human cognition. The Turing machine is based on a finite list of mechanisms and it is considered as the ultimate computer (in the classical sense of computing, but with potentially infinite memory). But we know too little about whether the limits of the turing machine are also limits of human cognition.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 31 child comments
    •
  • [–] 1 point 2 years ago*

    As with many things, it’s hard to pinpoint the exact moment when narrow AI or pre-AGI transitions into true AGI. However, the definition is clear enough that we can confidently look at something like ChatGPT and say it’s not AGI - nor is it anywhere close. There’s likely a gray area between narrow AI and true AGI where it’s difficult to judge whether what we have qualifies, but once we truly reach AGI, I think it will be undeniable.

    I doubt it will remain at "human level" for long. Even if it were no more intelligent than humans, it would still process information millions of times faster, possess near-infinite memory, and have access to all existing information. A system like this would almost certainly be so obviously superintelligent that there would be no question about whether it qualifies as AGI.

    I think this is similar to the discussion about when a fetus becomes a person. It may not be possible to pinpoint a specific moment, but we can still look at an embryo and confidently say that it’s not a person, just as we can look at a newborn baby and say that it definitely is. In this analogy, the embryo is ChatGPT, and the baby is AGI.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 years ago* (28 children)

    But we know too little about whether the limits of the turing machine are also limits of human cognition.

    Erm, no. Humans can manually step interpreters of Turing-complete languages so we're TC ourselves. There is no more powerful class of computation, we can compute any computable function and our silicon computers can do it as well (given infinite time and scratch space yadayada theoretical wibbles)

    The question isn't "whether", the answer to that is "yes of course", the question is first and foremost "what" and then "how", as in "is it fast and efficient enough".

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 28 child comments
    •
  • [–] 1 point 2 years ago* (27 children)

    No, you misread what I said. Of course humans are at least as powerful as a turing machine, im not questioning that. What is unkonwn is if turing machines are as powerful as human cognition. Who says every brain operation is computable (in the classical sense)? Who is to say the brain doesnt take advantage of some weird physical phenomenon that isnt classically computable?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 27 child comments
    •
  • [–] 0 points 2 years ago* (last edited 2 years ago) (26 children)

    Who is to say the brain doesnt take advantage of some weird physical phenomenon that isnt classically computable?

    Logic, from which follows the incompleteness theorem, reified in material reality as cause and effect. Instead of completeness you could throw out soundness (that is, throw out cause and effect) but now the physicists are after you because you made them fend off even more Boltzmann brains. There is theory on hypercomputation but all it really boils down to is "if incomputable inputs are allowed, then we can compute the incomputable". It should be called reasoning modulo oracles.

    Or, put bluntly: Claiming that brains are legit hypercomputers amounts to saying that humanity is supernatural, as in aphysical. Even if that were the case, what would hinder an AI from harnessing the same supernatural phenomenon? The gods?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 26 child comments
    •
  • [–] 1 point 2 years ago (25 children)

    You say an incompleteness theorem implies that brains are computable? Then you consider the possibility of them being hypercomputers? What is this?

    Im not saying brains are hypercomputers, just that we dont know if thats the case. If you think that would be "supernatural", ok, i dont mind. And i dont object to the possibility of eventually having AI on hypercomputers. All I said is that the plain old Turing machine wouldn't be the adequate model for human cognitive capacity in this scenario.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 25 child comments
    •
  • [–] 1 point 2 years ago* (last edited 2 years ago) (24 children)

    You say an incompleteness theorem implies that brains are computable?

    No, I'm saying that incompleteness implies that either cause and effect does not exist, or there exist incomputable functions. That follows from considering the universe, or its collection of laws, as a logical system, which are all bound by the incompleteness theorem once they reach a certain expressivity.

    All I said is that the plain old Turing machine wouldn’t be the adequate model for human cognitive capacity in this scenario.

    Adequate in which sense? Architecturally, of course not, and neither would be lambda calculus or other common models. I'm not talking about specific abstract machines, though, but Turing-completeness, that is, the property of the set of all abstract machines that are as computationally powerful as Turing machines, and can all simulate each other. Those are a dime a gazillion.

    Or, see it this way: Imagine a perfect, virtual representation of a human brain stored on an ordinary computer. That computer is powerful enough to simulate all physical laws relevant to the functioning of a human brain... it might take a million years to simulate a second of brain time, but so be it. Such a system would be AGI (for ethically dubious values of "artificial"). That is why I say the "whether" is not the question: We know it is possible. We've in fact done it for simpler organisms. The question is how to do it with reasonable efficiency, and that requires an understanding of how the brain does the computations it does so we can mold it directly into silicon instead of going via several steps of one machine simulating another machine, each time incurring simulation overhead from architectural mismatch.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 24 child comments
    •
  • [–] 1 point 2 years ago* (23 children)

    No,

    Ok. So nothing you said backs the claim that "logic" implies that the brain cannot be using some uncomputable physical phenomenon, and so be uncomputable.

    I'm not sure about what you mean by "cause and effect" existing. Does it mean that the universe follows a set of laws? If cause and effect exists, the disjunction you said is implied by the incompleteness theorem entails that there are uncomputable functions, which I take to mean that there are uncomputable oracles in the physical world. But i still find suspicious your use of incompleteness. We take the set of laws governing the universe and turn it into a formal system. How? Does the resulting formal system really meet all conditions of the incompleteness theorem? Expressivity is just one of many conditions. Even then, the incompleteness theorem says we can't effectively axiomatize the system... so what?

    Adequate in which sense?

    I dont mean just architecturally, the turing machine wouldnt be adequate to model the brain in the sense that the brain, in that hypothetical scenario, would be a hypercomputer, and so by definition could not be simulated by a turing machine. As simple as that. My statement there was almost a tautology.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 23 child comments
    •
  • [–] 0 points 2 years ago (22 children)

    entails that there are uncomputable functions, which I take to mean that there are uncomputable oracles in the physical world.

    It means that there are functions that are not computable. You cannot, for example, write a program that decides, in finite time, whether an arbitrary program halts on a particular input. If you doubt that, have an easy-going explanation of the proof by diagonalisation.

    We take the set of laws governing the universe and turn it into a formal system. How?

    Ask a physicist, that's their department not mine. Also I'd argue that the universe itself is a formal system, and lots of physicists would agree they're onto the whole computability and complexity theory train. They may or may not agree to the claim that computer science is more fundamental than physics, we're still working on that one.

    Does the resulting formal system really meet all conditions of the incompleteness theorem?

    Easily, because it will have to express the natural numbers. Have a Veritasium video on the whole thing. The two results (completeness and in computability) are fundamentally linked.

    he turing machine wouldnt be adequate to model the brain in the sense that the brain, in that hypothetical scenario, would be a hypercomputer,

    If the brain is a hypercomputer then, as already said, you're not talking physics any more, you're in the realms of ex falso quodlibet.

    Hypercomputers are just as impossible as a village barber who shaves everyone in the village who does not shave themselves: If the barber shaves himself, then he doesn't shave himself. If he shaves himself, then he doesn't shave himself. Try to imagine a universe in which that's not a paradox, that's the kind of universe you're claiming we're living in when you're claiming that hypercomputers exist.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 22 child comments
    •
  • [–] 1 point 2 years ago (21 children)

    you mention a lot of theory that does exist, but your arguments make no sense. You might want to study the incompleteness theorems more in depth before continuing to cite them like that. The book Godels proof by Nagel and Newman is a good start to go beyond these youtube expositions.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 21 child comments
    •
  • [–] 1 point 2 years ago (20 children)

    youtube expositions

    Dude any uni you go to likely has lectures that are worse than the Arsdigita ones. If you want to save face, act less like a philosopher next time and don't assume that you know things better than someone who actually studied it. I ended up linking veritasium because I realised that you have no idea about the mathematical fundamentals or you wouldn't say silly things such as

    entails that there are uncomputable functions, which I take to mean that there are uncomputable oracles in the physical world.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 20 child comments
    •
  • [–] 1 point 2 years ago (19 children)

    I took this interpretation to the "existence of uncomputable functions" because of course they exist mathematically, but we were talking about the physical world, so another meaning of existence was probably being used.

    You say you studied, but still your arguments linking incompleteness and the physical world did not make sense. To the point that you say things like the universe already is a formal system to which we can apply the incompleteness theorem. Again, expressivity of arithmetic isnt the only condition for using incompleteness. The formal system must be similar to first order logic, as the sentences must be finite, the inference rules must be computable and their set must be recursively enumerable, ... among others. When I asked this, you only mentioned being able to express natural numbers. But can the formal system express them in the specific sense that we need here to use incompleteness?

    Then, what do you do with the fact that you cant effectively axiomatize the laws of the universe? (which would be the conclusion taken from using incompleteness theorem here, if you could) What's the point of using incompleteness here? How do you relate this to the computability of brain operations?

    These are all giant holes you skipped, which suggest to me that you brushed over these topics somewhere and started to extrapolate unrigorous conclusions from them.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 19 child comments
    •
  • [–] 1 point 2 years ago (18 children)

    To the point that you say things like the universe already is a formal system to which we can apply the incompleteness theorem.

    And that is contentious, why? If the laws of the universe are formalisable, then the universe is isomorphic to that formalisation and as such also a formal system. We're not talking being and immanence, here, we're talking transcendent properties.

    When I asked this, you only mentioned being able to express natural numbers. But can the formal system express them in the specific sense that we need here to use incompleteness?

    How do you express them in ways that do not trigger incompleteness? Hint: You can't. It's a sufficient condition, there's equivalent ones, if I'm not mistaken an infinite number of them, but that doesn't matter because they're all equivalent.

    These are all giant holes you skipped

    These are all things you would understand if I didn't have to remind you of basic computability and complexity theory literally every time you reply. As said: Stop the philosophising. The maths are way more watertight than you think. We're in "God can't make a triangle with four sides" territory, here, just that computability is a wee bit less intuitive than triangles.

    If you want to attack my line of reasoning you could go for solipsism, you could come up with something theological ("god chooses to hide certain aspects of the universe from machines" or whatever). I'm aware of the limits. I didn't come up with this stuff yesterday and my position isn't out of the ordinary, either.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 18 child comments
    •
  • [–] 1 point 2 years ago (17 children)

    its not a "god cant make a triangle of four sides" discussion. Disregarding the mysterious formal system that "obviously" expresses arithmetic, you always skip my question: then what? how does the laws of the universe being not axiomatizable relate to the brain not using uncomputable functions? This was always the main point of the argument and you keep avoiding giving me an answer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 17 child comments
    •
  • [–] 1 point 2 years ago* (last edited 2 years ago) (16 children)

    how does the laws of the universe being not axiomatizable

    ...I never said they are not.

    relate to the brain not using uncomputable functions?

    That is unspecific: Do you mean it is using external oracles? It cannot use use them because they cannot exist because they're four-sided triangles. If you mean that it is considering uncomputable functions, then it can do so symbolically, but it cannot evaluate them, not in finite time that is: The brain can consider the notion of four-sided triangles, but it cannot calculate the lengths of those sides given, say, an area and an angle or such. What would that even mean.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 16 child comments
    •
  • [–] 1 point 2 years ago (15 children)

    ...I never said they are not.

    The incompleteness theorem says that a consistent axiomatic formal system satisfying some conditions cannot be complete, so the universe as a formal system (supposed consistent, complete, expressive enough, ...) cannot be axiomatized.

    external oracles

    What do you mean external?

    The possibility of using physical phenomena as oracles for solving classically uncomputable problems in the real world is an open question. If you think this is logically as impossible as a four sided triangle you should give sources for this claim, not just some vague statements involving the incompleteness theorem. Prove this logical impossibility or give sources, thats all im asking.

    Who says you cant take a first order logic sentence, codify it as a particular arrangement of certain particles and determine if the sentence was valid by observing how the particles behave? Some undiscovered physical phenomenon might make this possible... who knows. It would make possible the making of a real world machine that surpasses the turing machine in computability, no? How is this like a four sided triangle? The four sided triangle is logically impossible, but a hypercomputer is logically possible. The question is whether it is also physically possible, which is an open question.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 15 child comments
    •
  • [–] 1 point 2 years ago* (14 children)

    The incompleteness theorem says that a consistent axiomatic formal system satisfying some conditions cannot be complete, so the universe as a formal system (supposed consistent, complete, expressive enough, …) cannot be axiomatized.

    It can also be axiomisable but inconsistent. In principle, that is, but as said you'd annoy a lot of physicists.

    What do you mean external?

    As in the previously mentioned summation of the results of theoretical hypercomputation: "If uncomputable inputs are permitted, then uncomputable outputs can be produced". Those oracles would be the input.

    The possibility of using physical phenomena as oracles for solving classically uncomputable problems in the real world is an open question.

    If they exist, then they can be used. We do that all the time in the sense that we're pretending they exist, it's useful to e.g. prove that an algorithm is optimal: We compare an implementable algorithm it with one that can e.g. see the future, can magically make all the right choices, etc. But they don't exist.

    If you think this is logically as impossible as a four sided triangle you should give sources for this claim

    I already pointed you to an easy-going explanation of the proof by diagonalization. I'm not going to sit here and walk you through your homework. In fact I have given up explaining it to you because you're not putting in the work, hence why I resorted to an analogy, the four-sided triangle.

    Some undiscovered physical phenomenon might make this possible… who knows.

    Are all thinkable phenomena possible? Can there be four-sided triangles?

    The four sided triangle is logically impossible, but a hypercomputer is logically possible.

    That is an assertion without substantiation, and for what it's worth you're contradicting the lot of Computer Science. A hypercomputer is a more involved, not as intuitive, four-sided triangle.

    If you think that it's logically possible, go back to that proof I pointed you to. I will not do so again.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 14 child comments
    •
  • [–] 1 point 2 years ago (12 children)

    I know diagonalization proofs, they dont prove what you say they prove. Cite any computer science source stating that the existence of hypercomputers are logically impossible. If you keep saying it follows from some diagonalization argument without showing how or citing sources ill move on from this.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] 1 point 2 years ago* (11 children)

    I know diagonalization proofs, they dont prove what you say they prove.

    Not proofs, plural, not the category. This specific one. The details involve a method to enumerate all programs which is the hard part. IIRC the lecturer doesn't actually get into that, though. Read the original papers if you want nobody found issue with them in nearly 100 years.

    Cite any computer science source stating that the existence of hypercomputers are logically impossible.

    Church-Turing is a fundamental result of CS, arguably its founding one, and I will not suffer any more denial of it. It's like asking a physicist to provide a citation for the non-existence of telekinesis: You fucking move something with your mind, then we'll talk. In the meantime, I'm going to judge you to be nuts.

    Feel free to have a look at the criticism section of Wikipedia's hypercomputation article, though. Feel free to read everything about it but don't pester me with that nonsense. Would you even have known about it if I didn't mention off-hand that it was bunk, serves me right I guess.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 1 point 2 years ago* (10 children)

    church-turing is a a thesis, not a logical theorem. You pointed me to a proof that the halting problem is unsolvable by a Turing Machine, not that hypercomputers are impossible.

    The critic Martin Davis mentioned in wikipedia has an article criticizing a kind of attempt at showing the feasibility of hypercomputers. Thats fine. If there was a well-known logical proof of its unfeasibility, his task would be much simpler though. The purely logical argument hasnt been made as far as i know and as far as you were able to show.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 1 point 2 years ago (9 children)

    You would need to invent a complexity class larger than R, one that contains more than countably infinite programs. Those, too, can be diagonalised, there would still be incomputable functions. Our whole argument would repeat with that complexity class instead of R. Rinse and repeat. By induction, nothing changes, Q.E.D.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 1 point 2 years ago* (8 children)

    A hypercomputer has its own class of unsolvable problems, I agree. That doesnt mean that a hypercomputer cannot exist.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 1 point 2 years ago (7 children)

    You know what? I'm going to plant a nuke under your ass: Turing machines can't exist, either. Any finite machine can be expressed as a DFA. We're nothing but a bunch of complicated regexen.

    This whole time we were talking about potentiality, not reality; in terms that are convenient theory, not physics. I see no reason to extend potentiality to uncountable infinities when we can't even exploit countable infinity.

    Side note, and this might actually change my mind on things regarding "Is R all that we'll ever need": If people manage to get an asymptotic speedup out of quantum computers. The question is whether the parallelism inherent in operations on qbits is eaten up by noise, more or less the more states you load onto the qbits, the more fuzzy the results get, because the universe has a maximum amount of computational oomph it spends on a particle or per unit volume or whatever the right measure is. Of course, before we'd need to move past R we'd first have to load an actually infinite number of states into a qbit and I don't really see that happening. A gazillion? Doubtful, but thinkable. An infinite number in finite time? Not while we have fat fingers typing away on macroscopic keyboards.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 2 years ago (6 children)

    Turing machines can’t exist, either.

    Oh no! You got me there!

    Why do you need uncountable infinities for hypercomputers, though?. I see that Martin Davis criticism has to do with that approach, and I agree this approach seems silly. But, it doesnt seem to cover all potential fronts for hypercomputers. Im not talking about current approaches to quantum computing either. What if some yet unknown physical law makes arrangements of particles somehow solve the first order logic validity problem, which is also not in R? Doesnt involve uncountable infinity at all. Again, im not saying its possible, just that theres no purely logical proof of impossibility, thats all.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 2 years ago (5 children)

    first order logic validity problem

    Validity is RE (semidecidable), Satisfiability is undecidable.

    How do we figure out that your fancy new law is actually the oracle you claim it is? It could be lying to us, to establish the thing as an oracle we'd have to be able to either inspect it or unit-test it over the whole infinite range.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 2 years ago (4 children)

    Right, validity is semidecidable, just like the halting problem.

    We might never know for certain that any natural law is true, we might never be certain that that oracle actually solves validity. But that doesnt prevent the oracle from working. My point is that its existence is possible, not that we will ever be able to trust it.

    Besides, we dont know that the physical laws we work with today are true, but we nevetheless use them for practical purpuses all the time.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 years ago (3 children)

    I mean if the point is that we know that we know nothing then I'll agree.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 2 years ago (2 children)

    Not my point... and you know it. My point is that even if we consider that proven theorems are known facts, we still dont know if hypercomputers are infeasible. We know, however, that i'll never write python code that decides Validity because it is not (classically) decidable. But we have no theorems on the impossibility of hypercomputation.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    The diagonalization argument you pointed me to is about the uncomputability of the halting problem. I know about it, but it just proves that no turing machine can solve the halting problem. Hypercomputers are supposed to NOT be turing machines, so theres no proof of the impossibility of hypercomputers to be found there.

  • permalink
  • fedilink
  • source
  • parent