• azimir@lemmy.ml
      link
      fedilink
      arrow-up
      9
      ·
      2 months ago

      We’re entering the ‘blockchain for every need’ stage. Expect massive money to flow into scams, poor ideas, and outright dangerous uses for a few years .

      Before Blockchain we had ‘the web’ itself in the dot com era. Before that? I saw it in basic computing as a solution to everything.

    • Sl00k@programming.dev
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      6
      ·
      2 months ago

      Curious why your perspective is they’re are more of a scam when by all metrics they’ve only improved in accuracy?

        • Sl00k@programming.dev
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          3
          ·
          2 months ago

          Olympic Arena analysis OpenAI analyses

          Compare the GPT increase from their V2 GPT4o model to their reasoning o1 preview model. The jumps from last years GPT 3.5 -> GPT 4 were also quite large. Secondly if you want to take OpenAI’s own research into account that’s in the second image.

          • TootSweet@lemmy.world
            link
            fedilink
            English
            arrow-up
            6
            ·
            2 months ago

            if you want to take OpenAI’s own research into account

            No thank you.

            OlympicArena validation set (text-only)

            “Our extensive evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy (28.67% for mathematics and 29.71% for physics)”

            • The OlympicArena analysis that you cited.
            • Sl00k@programming.dev
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              4
              ·
              2 months ago

              The jump from GPT-4o -> o1 (preview not full release) was a 20% cumulative knowledge jump. If that’s not an improvement in accuracy I’m not sure what is.

              • Aceticon@lemmy.world
                link
                fedilink
                arrow-up
                3
                ·
                edit-2
                2 months ago

                One of the first things they teach you in Experimental Physics is that you can’t derive a curve from just 2 data points.

                You can just as easilly fit an exponential growth curve to 2 points like that one 20% above the other, as you can a a sinusoidal curve, a linear one, an inverse square curve (that actually grows to a peak and then eventually goes down again) and any of the many curves were growth has ever diminishing returns and can’t go beyond a certain point (literally “with a limit”)

                I think the point that many are making is that LLM growth in precision is the latter kind of curve: growing but ever slower and tending to a limit which is much less than 100%. It might even be like more like the inverse square one (in that it might actually go down) if the output of LLM models ends up poluting the training sets of the models, which is a real risk.

                You showing that there was some growth between two versions of GPT (so, 2 data points, a before and an after) doesn’t disprove this hypotesis. I doesn’t prove it either: as I said, 2 data points aren’t enough to derive a curve.

                If you do look at the past growth of precision for LLMs, whilst improvement is still happening, the rate of improvement has been going down, which does support the idea that there is a limit to how good they can get.

                • Sl00k@programming.dev
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  2 months ago

                  which does support the idea that there is a limit to how good they can get.

                  I absolutely agree, im not necessarily one to say LLMs will become this incredible general intelligence level AIs. I’m really just disagreeing with people’s negative sentiment about them becoming worse / scams is not true at the moment.

                  I doesn’t prove it either: as I said, 2 data points aren’t enough to derive a curve

                  Yeah only reason I didn’t include more is because it’s a pain in the ass pulling together multiple research papers / results over the span of GPT 2, 3, 3.5, 4, 01 etc.