194
The first minds to be controlled by generative AI will live inside video games (www.cnbc.com)
submitted 2 years ago by to c/technology@lemmy.world
76 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 years ago (3 children)

Unless you want to call your predictive text on your keyboard a mind you really can't call an LLM a mind. It is nothing more than a linear progression from that. Mathematically proven to not show any form of emergent behavior.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 4 points 2 years ago (1 child)

    No such thing has been "mathematically proven." The emergent behavior of ML models is their notable characteristic. The whole point is that their ability to do anything is emergent behavior.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 years ago* (1 child)

    Here's a white paper explicitly proving:

    1. No emergent properties (illusory due to bad measures)
    2. Predictable linear progress with model size

    https://arxiv.org/abs/2304.15004

    The field changes fast, I understand it is hard to keep up

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    Sure, if you define "emergent abilities" just so. It's obvious from context that this is not what I described.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 2 years ago* (1 child)

    I do not think that it is “linear” progression. ANN by definition is nonlinear. Neither I think anything is “mathematically proven”. If I am wrong, please provide a link.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Sure thing: here's a white paper explicitly proving:

    1. No emergent properties (illusory due to bad measures)
    2. Predictable linear progress with model size

    https://arxiv.org/abs/2304.15004

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago*

    Thank you. This paper though does not state that there are no emergent abilities. It only states that one can introduce a metric with respect to which the emergent ability behaves smoothly and not threshold-like. While interesting, it only suggests that things like intelligence are smooth functions, but so what? Some other metrics show exponential or threshold dependence and whether the metric is right depends only how one will use it. And there is no law that emerging properties have to be threshold like. Quite the opposite - nearly all examples in physics that I know, the emergence appears gradually.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 2 years ago (1 child)

    It is obvious that you do not know what either "mathematical proof" or "emergence" mean. Unfortunately, you are misrepresenting the facts.

    I don't mean to criticize your religious (or philosophical) convictions. There is a reason people mostly try to keep faith and science separate.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 years ago (1 child)

    Here's a white paper explicitly proving:

    No emergent properties (illusory due to bad measures)

    Predictable linear progress with model size

    https://arxiv.org/abs/2304.15004

    The field changes fast, I understand it is hard to keep up

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 2 years ago (1 child)

    As I said, you do not understand what these 2 terms mean. As such, you are incapable of understanding that paper.

    Perhaps your native language is Italian, so here are links to the .it Wikipedia.

    https://it.wikipedia.org/wiki/Comportamento_emergente

    https://it.wikipedia.org/wiki/Dimostrazione_matematica

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    1. Emergence is the whole being greater than the sum of its parts. That's the original meaning of emergent properties, which is laid out in the first paragraph of the article. It's the scholarly usage as well, and what the claims of observed emergence are using as the base of their claim.

    2. The article very explicitly demonstrated that only about 10% of any of the measures for LLMs displayed any emergence and that illusory emergence was the result of overly rigid metrics. Swapping to edit distance as an approximately close metric causes the sharp spikes to disappear for obvious reasons: no longer having a sharp yes/no allows for linear progression to reappear. It was always there, merely masked by flawed statistics.

    If you can't be bothered to read here's a very easy to understand video by one of the authors: https://www.youtube.com/watch?v=ypKwNrmuuPM

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •