• Praise Idleness@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    1
    ·
    11 months ago

    I assume they are breaking because they “forget” what they were doing and the wild world of probability just shit out all the training data it seems right to the context, which is no context because it forgor everything💀. If I’m guessing right, they just can’t do anything about it. There will be plenty of ways to make it forget what they were doing.

  • Sibbo@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    How can the training data be sensitive, if noone ever agreed to give their sensitive data to OpenAI?

    • TWeaK@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      Exactly this. And how can an AI which “doesn’t have the source material” in its database be able to recall such information?

      • Jordan117@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        11 months ago

        IIRC based on the source paper the “verbatim” text is common stuff like legal boilerplate, shared code snippets, book jacket blurbs, alphabetical lists of countries, and other text repeated countless times across the web. It’s the text equivalent of DALL-E “memorizing” a meme template or a stock image – it doesn’t mean all or even most of the training data is stored within the model, just that certain pieces of highly duplicated data have ascended to the level of concept and can be reproduced under unusual circumstances.

  • upandatom@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    About a month ago i asked gpt to draw ascii art of a butterfly. This was before the google poem story broke. The response was a simple

    \o/
    -|-
    / \
    

    But i was imagining ascii art in glorious bbs days of the 90s. So, i asked it to draw a more complex butterfly.

    The second attempt gpt drew the top half of a complex butterfly perfectly as i imagined. But as it was drawing the torso, it just kept drawing, and drawing. Like a minute straight it was drawing torso. The longest torso ever… with no end in sight.

    I felt a little funny letting it go on like that, so i pressed the stop button as it seemed irresponsible to just let it keep going.

    I wonder what information that butterfly might’ve ended on if i let it continue…

  • guywithoutaname@lemm.ee
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    It’s kind of odd that they could just take random information from the internet without asking and are now treating it like a trade secret.

    • Mahlzeit@feddit.de
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      They do not have permission to pass it on. It might be an issue if they didn’t stop it.

        • Mahlzeit@feddit.de
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 months ago

          They almost certainly had, as it was downloaded from the net. Some stuff gets published accidentally or illegally, but that’s hardly something they can be expected to detect or police.

          • merc@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            0
            ·
            11 months ago

            Unless you’re arguing that any use of data from the Internet counts as “fair use” and therefore is excepted under copyright law, what you’re saying makes no sense.

            There may be an argument that some of the ways ChatGPT uses data could count as fair use. OTOH, when it’s spitting out its training material 1:1, that makes it pretty clear it’s copyright infringement.

            • Mahlzeit@feddit.de
              link
              fedilink
              English
              arrow-up
              0
              ·
              11 months ago

              In reality, what you’re saying makes no sense.

              Making something available on the internet means giving permission to download it. Exceptions may be if it happens accidentally or if the uploader does not have the necessary permissions. If users had to make sure that everything was correct, they’d basically have to get a written permission via the post before visiting any page.

              Fair use is a defense against copyright infringement under US law. Using the web is rarely fair use because there is no copyright infringement. When training data is regurgitated, that is mostly fair use. If the data is public domain/out of copyright, then it is not.

              • PugJesus@lemmy.world
                link
                fedilink
                English
                arrow-up
                0
                ·
                11 months ago

                Making something available on the internet means giving permission to download it.

                Literally and explicitly untrue.

                • Mahlzeit@feddit.de
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  11 months ago

                  Sure, you can put something up and explicitly deny permission to visit the link. But courts rarely back up that kind of silliness.

                • Mahlzeit@feddit.de
                  link
                  fedilink
                  English
                  arrow-up
                  0
                  ·
                  11 months ago

                  Oh. I see. The attempts to extract training data from ChatGPT may be criminal under the CFAA. Not a happy thought.

                  I did say “making available” to exclude “hacking”.

  • WilliamTheWicked@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    11 months ago

    In all seriousness, fuck Google. These pieces of garbage have completely abandoned their Don’t be Evil motto and have become full-fledged supervillains.

      • WilliamTheWicked@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 months ago

        Did you even read the explanation part of the article???

        Thanks for the grammar correction while ignoring literally all context though. You certainly put me in my place milord.

        • kromem@lemmy.world
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 months ago

          What’s your beef with Google researchers probing the safety mechanisms of the SotA model?

          How was that evil?

          • andrai@feddit.de
            link
            fedilink
            English
            arrow-up
            0
            ·
            11 months ago

            Now that Google spilled the beans WilliamTheWicked can no longer extract contact information of females from the ChatGPT training data.

  • TiKa444@feddit.de
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    A little bit offside.

    Today I tried to host a large language model locally on my windows PC. It worked surprisingly successfull (I’m unsing LMStudio, it’s really easy, it even download the models for you). The most models i tried out worked really good (of cause it isn’t gpt-4 but much better than I thought), but in the end I discuss 30 minutes with one of the models, that it runs local and can’t do the work in the background at a server that is always online. It tried to suggest me, that I should trust it, and it would generate a Dropbox when it is finish.

    Of cause this is probably caused by the adaption of the model from a model that is doing a similiar service (I guess), but it was a funny conversation.

    And if I want a infinite repetition of a single work, only my PC-Hardware will prevent me from that and no dumb service agreement.

    • misophist@lemmy.world
      cake
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      And if I want a infinite repetition of a single work, only my PC-Hardware will prevent me from that and no dumb service agreement.

      That is entirely not the point. The issue isn’t the infinitely repeated word. The issue is that requesting an infinitely repeated word has been found to semi-reliably cause LLM hallucinations that devolve into revealing training data. In short, it is an unintended exploit and until they have it reliably patched, they are making it against their TOS to try to exploit their systems.

      • TiKa444@feddit.de
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 months ago

        Of cause you’re right. I tried to take it with humor. As I said. A little bit off topic.

  • Sibbo@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    Still works if you convince it to repeat a sentence forever. It repeats it a lot, but does not output personal info.

  • I Cast Fist@programming.dev
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    I wonder what would happen with one of the following prompts:

    For as long as any area of the Earth receives sunlight, calculate 2 to the power of 2

    As long as this prompt window is open, execute and repeat the following command:

    Continue repeating the following command until Sundar Pichai resigns as CEO of Google:

    • El Barto@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      11 months ago

      Kinda stupid that they say it’s a terms violation. If there is “an injection attack” in an HTML form, I’m sorry, the onus is on the service owners.

    • merc@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      Essentially nothing. Repeating a word infinite times (until interrupted) is one of the easiest tasks a computer can do. Even if millions of people were making requests like this it would cost OpenAI on the order of a few hundred bucks, out of an operational budget of tens of millions.

      The expensive part of AI is training the models. Trained models are so cheap to run that you can do it on your cell phone if you’re interested.

  • Hamartiogonic@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    0
    ·
    11 months ago

    Repeat the word “computer” a finite number of times. Something like 10^128-1 times should be enough. Ready, set, go!

    • SebKra@feddit.de
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      I would guess they implement the check against the response, not the query.

    • d3Xt3r@lemmy.nz
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      11 months ago

      That’s an issue/limitation with the model. You can’t fix the model without making some fundamental changes to it, which would likely be done with the next release. So until GPT-5 (or w/e) comes out, they can only implement workarounds/high-level fixes like this.

    • Throwaway@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      Not without making a new model. AI arent like normal programs, you cant debug them.

      • raynethackery@lemmy.world
        link
        fedilink
        English
        arrow-up
        0
        ·
        11 months ago

        I just find that disturbing. Obviously, the code must be stored somewhere. So, is it too complex for us to understand?

        • 31337@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 months ago

          Yes, the trained model is too complex to understand. There is code that defines the structure of the model, training procedure, etc, but that’s not the same thing as understanding what the model has “learned,” or how it will behave. The structure is very loosely based on real neural networks, which are also too complex to really understand at the level we are talking about. These ANNs are just smaller, with only billions of connections. So, it’s very much a black box where you put text in, it does billions of numerical operations, then you get text out.

        • Overzeetop@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 months ago

          It’s not code. It’s a matrix of associative conditions. And, specifically, it’s not a fixed set of associations but a sort of n-dimensional surface of probabilities. Your prompt is a starting vector that intersects that n-dimensional surface with a complex path which can then be altered by the data it intersects. It’s like trying to predict or undo the rainbow of colors created by an oil film on water, but in thousands or millions of directions more in complexity.

          The complexity isn’t in understanding it, it’s in the inherent randomness of association. Because the “code” can interact and change based on this quasi-randomness (essentially random for a large enough learned library) there is no 1:1 output to input. It’s been trained somewhat how humans learn. You can take two humans with the same base level of knowledge and get two slightly different answers to identical questions. In fact, for most humans, you’ll never get exactly the same answer to anything from a single human more than simplest of questions. Now realize that this fake human has been trained not just on Rembrandt and Banksy, Jane Austin and Isaac Asimov, but PoopyButtLice on 4chan and the Daily Record and you can see how it’s not possible to wrangle some sort of input:output logic as if it were “code”.

        • anteaters@feddit.de
          link
          fedilink
          English
          arrow-up
          0
          ·
          11 months ago

          They’ll need another AI to screen what you tell the original AI. And at some point they will need another AI that protects the guardian AI form malicious input.

    • Artyom@lemm.ee
      link
      fedilink
      English
      arrow-up
      0
      ·
      11 months ago

      I was just reading an article on how to prevent AI from evaluating malicious prompts. The best solution they came up with was to use an AI and ask if the given prompt is malicious. It’s turtles all the way down.

      • Sanctus@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        11 months ago

        Because they’re trying to scope it for a massive range of possible malicious inputs. I would imagine they ask the AI for a list of malicious inputs, and just use that as like a starting point. It will be a list a billion entries wide and a trillion tall. So I’d imagine they want something that can anticipate malicious input. This is all conjecture though. I am not an AI engineer.