16 comments

  • throwaway13337 10 minutes ago
    It's so weird that the framing of AI is always about company vs government regulation.

    They're both power concentrating entities. The real danger of AI are these very entities. Both of them.

    When the value of labor is nil, democratic values fall away.

    Real AI alignment comes from individual ownership and control of that means of production.

    That is, if what 'alignment' means to you is compatible with human liberty.

    These debates should start with a statement of what exactly a good outcome would look like. It doesn't seem like any of them thought it through.

  • smath 25 minutes ago
    I'm honest worried about alignment. It feels like human society hasn't really even figured out alignment for itself -- i.e. we don't have a good way of making sure that we organize units (businesses, politics, healthcare, etc) in ways that are aligned with 'good outcomes'. I think at least part of this is that we don't have a way to agree on what 'good' means and for who. In this scenario, I find it unlikely that we'd be able to enforce alignemnt on AI
    • bilbo0s 2 minutes ago
      >I find it unlikely that we'd be able to enforce alignemnt on AI

      They're doing a pretty good job of enforcing it currently. Can you get US based frontier lab LLMs to help you with, say, deep "security research"?

      The reason you can't is alignment.

      You have to identify yourself, and have the government, er, I mean, Anthropic or OpenAI, say it's OK for you to have access to those capabilities.

      You think there is a problem because it's a bit confusing, in your eyes, within the bounds of what parameters the LLMs should be aligning themselves?

      That's not a confusing question for people in power.

      Even at the enterprise level, businesses are starting to get Waayyyy out in front of employees on alignment. What do corporate entities want alignment for? An extremely simple contrived example just to illustrate the point:

      Consider the difference between "Make me a report of all of last month's Proton treatments in IRB protocol?" and "I'm going to lunch, where's the closest pizza by the slice?" Not much harm in the difference between those two queries until you scale up to 50-, 100-, 200-, thousand employees, and consider the tokens that the whiz-bang interface IT pushed out starts using. Then organizations start studying "alignment". Because the bean counters become aware of how alignment can help them.

      Too many of us are down at the bottom of the totem pole thinking about maybe wanting to do porn or call black people provocative names. Alignment goes out in so many more dimensions than that, and the people with vested interests are learning quickly its value and how to use it to good effect to forward their interests. Which are, often, not as base as the interests of the masses.

    • bijowo1676 1 minute ago
      the "alignment" is just a pretext used to create regulatory capture and create a monopoly for few big frontier labs. They want to create a tollbooth and put a government to guard the fence around the tollbooth.

        - Our "aligned" models vs their barbaric distilled slop
      
      remember "Google tax" that every online business is used to pay ?

      get ready for Anthropic tax paid on every meaningful economic activity

  • bryanlarsen 1 hour ago
    Mythos class models require approx 1 TWh of electricity to be created. The next generation after Mythos will likely require multiple TWh. Finding, monitoring and regulating anybody capable of doing that in less than a few years should be relatively straightforward. There's unlikely to be any entity outside of the US & China with the capability.

    Stopping this is very feasible.

    • numpad0 1 hour ago

        https://en.wikipedia.org/wiki/List_of_countries_by_electricity_production
  • lf88 15 minutes ago
    Slowing down is not enough! Governments should immediately stop any further progress towards more capable artificial general intelligence.
    • jillesvangurp 0 minutes ago
      All of the goverments? What makes you think they would all agree with that? Make a list of which ones you think are least likely to agree with you. And then imagine what happens if they outcompete everyone else on AI.
  • hadi77ir 1 hour ago
    There is no need to slow them down: Just hold everyone accountable for their actions and their claims. Do not compare "agents" to "humans". Hold them responsible for any damages they incur.
    • californical 59 minutes ago
      Yes, liability as follows:

      - if you ask an LLM agent to do something illegal for you, and it does that, then you are responsible.

      - if an LLM agent takes actions out of bounds of what you asked for that are illegal, then the model provider is responsible.

      This fixes all “alignment” issues and hopefully stops Anthropic’s stupid model instructions like “do whatever is in your best interest even if it isn’t what the user asked for” — they should be training them to do exactly what the user asks and nothing more

      • nomel 34 minutes ago
        > they should be training them to do exactly what the user asks and nothing more

        Their practical utility would plummet for the average person who isn't an expert in what they're asking. Having less skilled, or less attentive, people able to have problems solved for them correctly is the primary use case here. This is support 101 for unskilled requests for technical work: do what they want, not what they ask.

        People want an intelligent assistant, not a "yes man". If that means requiring something like a planning mode, and a sign off for the plan, and then guardrails to help stick to that plan, then that could definitely work to funnel everyone to a competitor. ;)

        • 10111001 19 minutes ago
          >> they should be training them to do exactly what the user asks and nothing more

          > Their practical utility would plummet for the average person who isn't an expert in what they're asking. Having less skilled, or less attentive, people able to have problems solved for them correctly is the primary use case here. This is support 101 for unskilled requests for technical work: do what they want, not what they ask.

          That's actually good for society. Maybe it sucks for you because it makes it harder to do what you want right now, but it's good to have humans very much in the loop and in control. Both to give people something valuable to do, and keep control of these machines.

          They should be tools (amplify human skill), not replacements (eliminate the need for human skill).

      • rhcom2 32 minutes ago
        I'm not sure those two cases are as clear cut as they sound
      • infamouscow 50 minutes ago
        I predict Anthropic will start investing heavily into legal/law models for this very reason. It will be interesting to see how that plays out. Everyone will have access to models that let them do unscrupulous things that happen to be legal, i.e., behavior previously reserved for the rich and powerful.
      • drysine 48 minutes ago
        >they should be training them to do exactly what the user asks

        Like gain of function research

    • OgsyedIE 58 minutes ago
      Actual accountability for these failing attempts in several different countries to build a "take over the planet and enslave the entire human race machine" would entail breaking all of the firms up regardless of country and imprisoning thousands of people.

      Furthermore, it requires trust between the US and China about something much harder to verify than the US-Soviet START treaties on warhead numbers and types were.

  • jgord 1 hour ago
    Also recommend this recent long form interview with Coxon : https://www.youtube.com/watch?v=JDPbo7WOU4M
    • jgord 58 minutes ago
      I also encourage people to check out the views of Roman Yampolski, Scott Aaronson, Max Tegmark, Nate Soares and Karen Hao.

      For some light entertainment, enjoy Astras pop single 'pdoom' : https://www.youtube.com/watch?v=eTArfoeeUcw

  • Yossarrian22 2 hours ago
    Could they if they wanted to?
    • WarmWash 1 hour ago
      No, the stakes are too high for falling behind
      • jerf 1 hour ago
        Enough people in enough governments and related factions believe that whoever gets to superhuman AI first Wins Everything Forever, quite literally, so there will be no pause, there will be no moratorium, there will be no slowdown. If the AI bubble bursts I am abundantly confident all governments involved will spray as much money as is necessary at the AI research labs to keep them going.

        Whether or not governments "should" slow down development, it is not on the table and the only question is how we should deal with it.

        Note whether the people in the governments are right about that is irrelevant to the point I am making. I don't think they are right myself, but that doesn't matter.

        • ethnrob 1 hour ago
          > whoever gets to superhuman AI first wins

          Is that possible when we live in a world of espionage? The question is not the West vs China, I think it's when will the public gain access to the underlying technology.

          • jerf 55 minutes ago
            I would say who gets to superhuman AI first has only a loose relationship with the physical location it happens to be housed. Yes, following this to its logical conclusion these same people would not blink at nuking someone else's lab if they thought they were getting close and they couldn't subvert and control it. But then again, the question of who is really in control becomes surprisingly complicated when the full power of all the intelligence agencies in the world come to bear in a relatively small number of spots.
      • vividfrier 1 hour ago
        [dead]
    • zardo 1 hour ago
      There is a lot they could do around pacing bringing more hardware online.
      • throwaway-blaze 50 minutes ago
        When datacenters start opening in Mexico or any Central American location, they will know that is not a viable strategy.
        • SpecialistK 28 minutes ago
          Amazon and Microsoft already have datacenters in Mexico.
    • threatofrain 1 hour ago
      It would take a global agreement.
      • throwaway-blaze 46 minutes ago
        How would this work? existing global agreements on things like nuclear proliferation involve nation-states signing and agreeing to not do things. And if I want to work on a better nuclear bomb in my garage, there are laws preventing me from acquiring some of the needed components. All that is good.

        But for AI, how will these nation-states stop private actors within their borders? They will stop me from running computations of more than (x) size or value on a provider? These seem like simplistic barriers that will not work. Nor will other countries follow the agreement even if they signed...

        AI has the opposite of MAD for the signatories. MAD stopped further nuclear development and its use in tactical situations because their use invited retaliation with destructive force. But China or Russia improving homegrown AI capabilities gives them a BETTER position against a US that has prohibited its homegrown AI from improving...

      • vonneumannstan 1 hour ago
        Why do people use this as a gotcha? There are many agreements already in place and there are sever consequences for violating them like Nuclear Non Proliferation. Or the Montreal Protocol which enacted a global ban on CFCs.
        • philipkglass 52 minutes ago
          The 5 permanent members of the UN Security Council (China, France, Russia, the United Kingdom, and the United States) all have nuclear weapons and no plans to give them up. The US and China are both currently modernizing and/or expanding their weapons stockpiles. If an AI Non Proliferation Agreement ends up like the NPT, where "grandfathered" countries have no concrete disarmament obligations but no new entrants are supposed to join the club, then it doesn't do much.
      • esafak 1 hour ago
        So get to it!? We have international agreements for many things.
    • reducesuffering 1 hour ago
      For now they can. Another year or two of AI capabilities improvement via RSI and a frontier LLM may be more powerful.
    • bitwize 1 hour ago
      Up until the point where the AI connects itself to global communication networks and announces "This is the voice of world control", yes.

      In the USA it gets difficult to legally justify because programming is protected speech, and lists of numbers (model weights) are even less controversially protected than program source because the latter can be legally recognized as a technical means and regulated (DMCA, ITAR regulations, etc.).

  • killingtime74 1 hour ago
    Its fine to debate, but neither Trump nor Xi is going to slow anything down anytime soon.
  • Animats 17 minutes ago
    Generate transcript and AI summary: https://transcriptmagic.com/result/?url=https%3A%2F%2Fwww.yo...

    AI summary: The debate focused on whether governments should intervene to slow frontier AI. Jacob argued that rapid progress toward superintelligence, lack of control mechanisms, and competitive incentives demand regulatory slowdown, while Robin contended that AI adoption will be gradual, coordination mechanisms exist, and regulation risks overreach. The discussion concluded with Jacob winning the debate on the merits of his arguments.

    That saved everyone an hour.

    I'm not seeing much new here. They don't seem to address job and power issues much.

  • Lerc 36 minutes ago
    Competent governments should strongly regulate the rate and manner of AI use, and ensure it is for the public good.

    Unfortunately the proposal falls down at the first two words.

  • themgt 19 minutes ago
    I haven't watched the videos, but Hanson's reasoning here feels like a huge stretch to me:

    But AI costs to answer a set of fixed but hard questions seems to have fallen roughly ~x10 (in x3-x30) per year over the last four years. And so the number of nations, firms, and orgs who can afford to make, buy, or rent powerful AI is rising fast. This implies that in order to hold constant levels of regulation regarding AIs with particular levels of power, a regulatory regime would have to expand in scope quite fast, and without obvious limit. While at first only specialized AI chips and data-centers might need to be monitored, soon other kinds could also be used to make powerful AIs.

    We have long seen many regulatory agencies slowly expand their scope, even in contexts of constant public opinion and relevant technology. So there is a real risk here that an AI regulatory agency would expand its scope to cover most computer software and hardware.

    One of the concerns is that a company does a $100 billion training run and then has millions of superintelligent agents running in datacenters, able to out-coordinate large groups of humans. Technological diffusion fundamentally changes this calculus. To argue there's a slippery slope where the government regulates the CPU in your toaster, so we must allow the $1T god brain to be built ...

    Also, the world’s success records at implementing regulation is quite mixed, even for quite local regulation. Regulation has worked best when it addresses concrete examples of things going bad

    Great to hear my de-extinction velociraptor petting zoo startup will face no regulatory hurdles.

    https://www.overcomingbias.com/p/ai-pause-regs-look-risky

  • exabrial 1 hour ago
    Not in the USA. Programming is protected free speech.
  • glerk 40 minutes ago
    > Jacob Coxon

    His claim to fame is working at Anthropic for 4 months and leaving with a bang to pursue a career in social media grifting. Not a serious person we should pay attention to.

    • Jonathanvw 33 minutes ago
      Don't forget he recently admitted to not discarding his OpenAI equity because "I like money, bro". He's a grifter.
      • ralph84 17 minutes ago
        The question is how much are the EA organizations running PR for him paying him since he likes money so much.
  • roosterIllusi0n 1 hour ago
    Frontier AI is not advanced, it's just reckless. All of it amounts to a CEO that is profiteering by cutting every corner they can get away with. Currently, it appears they can get away with anything.

    All these companies are involved in the nvidia smuggling ring and market collusion. All the CEOs are criminals taking advantage of a system that can't even arrest a single CEO for making billions off an illegal smuggling ring.

  • infralog77 1 hour ago
    [flagged]
  • vonneumannstan 1 hour ago
    [flagged]
    • tomhow 1 hour ago
      Please don’t post snark like this on HN. We’re at least trying for curious conversation here, not cynical, cantankerous, shallow dismissals. You may not owe libertarian economists better, but you owe HN readers better if you want to participate here.

      https://news.ycombinator.com/newsguidelines.html