Half a Second – a book about the XZ backdoor

(half-second.com)

44 points | by zvr 13 hours ago

8 comments

  • this_user 10 hours ago
    Well, the "About the Author" section should probably just be a link to claude.ai.
    • jllyhill 7 hours ago
      For anyone wondering, here's the author confirming it

      https://news.ycombinator.com/item?id=48966159

      And here's their Claude skill for writing

      https://github.com/AdrianMastronardi/bookwright

    • sevg 10 hours ago
      Yeah the (annoyingly) excessive use of colons feels like recent Claude models to me.

      Looks like author posted this themselves earlier, and even used Claude for the HN comment:

      https://news.ycombinator.com/item?id=48958457

      • infinite_spin 10 hours ago
        I've always used a lot of semi-colons in my writing, especially in my technical writing. So I did a search on this pdf for semi colons, and there are 260 in a 264 page document. I then repeated this for the last book I read, Candide by Voltaire, and there were 507 semi colons in a 189 page book.

        This seems like a witch hunt.

        • rwmj 10 hours ago
          I read parts of it, being first hand involved in all this, and it seems like it was written by AI to me. If you used an AI to write it or help with it, why not just admit to it?
          • infinite_spin 10 hours ago
            the argument was that use of semi-colons indicates AI usage..

            what do you mean by "being first hand involved in all this"? what part did you take if you don't mind sharing, this was an incredibly interesting turn of events.

            • rwmj 9 hours ago
              You'd know that if you'd done any research at all.

              Compare this to Jeff Guo (NPR) or Henry van Dyck (Veritasium) or my contact at the WSJ. They all investigated this story, interviewed key people (more people were interviewed for the Veritasium video than appeared), and fact-checked everything with subject matter experts. The NPR story took about 3 months of work and the Veritasium video took 5 months. I was interviewed in total for over 6 hours across all of them.

              These are real journalists who worked on these stories and they produced novel work and new findings. I have huge respect for them, especially after being involved with it and seeing what work it took to add something new to the story. At the same time they managed to tell that story to a lay audience which is another skill in itself.

              • infinite_spin 9 hours ago
                > You'd know that if you'd done any research at all.

                I'm not in the habit of researching the people who respond to me before I ask them about what their role was in something they seem proud of. I thought I was being polite by asking about it.

                That's really cool that you were interviewed, which of the interviews do you think most represents your hand in this? I wouldn't mind checking it out. I also really like Veritasium's channel, that's pretty awesome.

                • bigDinosaur 8 hours ago
                  In this case all it required was to follow the link to their blog on their HN profile just FYI. Although your question was reasonable either way.
        • jstanley 9 hours ago
          I pasted the introduction into Pangram and it said 100% AI.
          • infinite_spin 9 hours ago
            well also I read the parent's comment wrong, they just said colons, not semi-colons
        • sevg 9 hours ago
          You know I said colons right? Not semi colons :)
          • infinite_spin 9 hours ago
            ah my mistake, those are a bit uncommon in my writing, I feel like they seem pretentious. i just use the semi-colons because it's closer to the way I pause, and alternate, in my thoughts.
      • progbits 10 hours ago
        In last month or two I noticed semicolons getting used where previously it would be em-dash, in both cases excessively and often incorrectly.

        I assumed some of my coworkers added "replace all em-dashes with semicolons" into their CLAUDE.md as a really crappy attempt at hiding their inability to write a single sentence without assistance.

        • tux3 10 hours ago
          Come on now, you can't also take away my semicolons. What am I supposed to do now; I barely have any punctuation left.
          • dag100 8 hours ago
            Embrace the inner Cormac McCarthy in you and abandon punctuation altogether
          • progbits 4 hours ago
            Not taking it away.

            I hate those arguments "it has emdash therefore AI", of course humans also write that way.

            But poor and excessive usage of them is a pretty strong red flag, also together with other tells.

      • eth0up 10 hours ago
        "free book" "no paywall and nothing to buy."

        Might this not actually be a reasonable purpose for AI? I can tolerate the quirky style and AI signatures for something honest and free.

        • rwmj 9 hours ago
          But prompting an AI adds nothing. Journalism is about research, interviewing the people involved, finding new facts, and then presenting that to your audience in a way they can understand. Getting an AI to write about something is just diluting the signal for future researchers.
          • jersmtpd 9 hours ago
            You can have an AI help write it while still having done research yourself, they're not mutually exclusive. I'm not claiming whether he actually did that or not.

            Like, for me personally, I'm terrible at writing, so I'll gladly have some AI throw together a draft, which I then verify and edit.

            • eth0up 9 hours ago
              exactly. If I am writing a free book with solid information, well, I am not writing it, because I am not writing free books -- I don't have time. But if AI can arrange my data, thoughts, perspective, and articulate it effectively, then I will let AI write the free book. If the book contains new, or previously unarticulated data and insight, then why not? But I see the very thought is hotly contested here.
        • bandrami 9 hours ago
          Why not just post the prompt you used? I have access to LLMs too.
          • rpdillon 6 hours ago
            You think if he gave you the "prompt" you'd be able to produce that book?

            Heck, I should put money on this. Start a competition: I give folks a prompt, and tell them to write a book about a subject with AI. I'm the sole judge of quality. Winner gets $2000. Then let's examine the variance in the entries.

            In short: using AI is a skill. It's silly to pretend otherwise.

            • eth0up 1 hour ago
              Are you the only reasonable person on HN today? Others are framing this author as some official journalist who must be held to rigorous standards and has violated every rule of the trade, with wanton contempt. The author has failed to produce new data. The author is bad. This seems deranged and extraordinarily unfair. The front page explicitly states the book is for general readers and is non-technical. I do not see anything claiming official original research or boasting academic credentials. No ads. Free. No download registration. Just there. The result of work, tokens, time -- didn't 'just happen'.

              The Gladwell-style of conceptual-synthesis is the foundation for half the popular-science books in the world, and the bedrock for the majority of general-reader material out there. And this book happens to have a truly fascinating and extremely relevant and rich theme at its core, sufficiently rich that any motivated idiot could churn out half-decent content with. Yet it gets flagged, insulted and framed as abuse of AI, a tool specifically designed as a force multiplier and tool. Damned if you do, damned if you don't, and how dare anyone generate something of value with it.

              I sincerely find this all highly suspicious. The only explanation is hypersensitive AI allergy in a highly unstable immune system, or the topic itself is making some nervous and, ironically, insecure...

              I could totally understand disagreement, and complaining. But a comment that simply states acceptance of the book as possibly of value, being flagged? Who not hiding something trounces on a friendly comment to flag it? It's gray already, but that's not dead enough? Kill it til it's dead!? I say this because on a completely different post, I mentioned Jia Tan and several of the very concepts highlighted in this book, and I was flagged for it. I have said some stupid shit around here and not been flagged. But this subject really seems to bring out seriously inspired opponents. One would think, gee wiz, security, potentially many lives or billions of dollars or FOSS at stake -- there's no such thing as too much security, by all means, obsess on it, stay focused and talk about it, there are real world dangers here. But no.

              Something seems fishy.

              Anyway, glad someone had the courage to question the outrage. And I will promptly warn my author friends of the coming witch-hunt and to be very quiet about the tools they're using.

          • eth0up 9 hours ago
            I have over 6gb of material resulting from interactions with AI. Even a micro-essay of mediocre quality requires dozens of prompts. Why post dozens and dozens of prompts if the final product is cleaner, and easier to process? I see utterly zero reason.

            Edit: One of the things I learned quickly while working with LLMs, is that the quality of the user, the input, determines the quality of the output. Not everyone's input is of equal quality.

  • OldMatey 11 hours ago
    Given the time and effort that went into this, and the luck that one diligent person noticed, investigated and discovered what was going on before it could get further... it seems very likely to me that this has happened already in other libraries without being discovered.
    • VladVladikoff 9 hours ago
      I wonder if the nation state actors are doing people profiling on owners of important packages to find the most vulnerable for an attack.
    • miramba 11 hours ago
      Exactly, and I wonder since then: How closely did people in comparable situations look? Since nothing similar has been reported, I suspect not very close…
    • IshKebab 7 hours ago
      The effort of gaining trust over an existing project isn't even really required. All you need to do is monitor when popular GitHub repos get archived. That's usually when the original authors don't want to work on it any more. Then just quickly make a fork to continue the project (think Phabricator -> Phorge), and if you're quick enough and authoritative sounding enough, boom control of the project!

      Maintain it for a bit so people switch to your version, and job done.

    • akimbostrawman 8 hours ago
      Anybody running opensnitch would notice
  • skippyfish 8 hours ago
    Here's the tool developed by the author that was almost certainly used to generate this book in its entirety - "A structured pipeline for writing long-form nonfiction, packaged as a Claude Code skill":

    https://github.com/AdrianMastronardi/bookwright

    There's nowhere near enough public information about the xz vuln to be worth turning into a book, so the merits of AI-generated text aside, this is just a very inefficient way to learn about the topic.

    • eth0up 6 hours ago
      "There's nowhere near enough public information about the xz vuln to be worth turning into a book," As an actual person, who reads, and having glanced at this book, I do not agree. I can already see part of the purpose is to put perspective on the crazy reality of backdoors/exploits and the undermined implications thereof. Jia Tan alone could warrant a book or film. It also says "for the general reader" and non-technical, which is where I myself think the importance really is. Maybe the AI is catching blindspots.

      I downloaded the book. It's not fake. Perhaps no masterpiece, but far from worthless. Also, not "enough public information" really sells inference and imagination short. There is a rich amount of material to work with here. More than enough.

      I am going to go ahead and say that flagging this was more information suppression than crankiness about AI. A lot of folks get strange when Jia Tan or similar subjects come up. I guess it's wiser to just wait until the grid goes down, or the water supply gets a bit more chlorinated....

  • dhx 9 hours ago
    See [1] for April 2024 Clickhouse Github activity analysis of the xz backdoor.

    I haven't seen anyone write up a proper analysis that includes consideration of:

    - GitHub activity (e.g. all API actions on GitHub side including replying to comments) _and_ mailing list activity _and_ other public facing activity all considered together.

    - Complexity of public actions e.g. was there a queue of code changes that would have taken 20 hours effort to put together that were all committed at once? Were there any long streaks of high activity where it might reveal how many people were involved?

    - Latency of public actions e.g. if an issue was raised by some random person, how long did it take for the attacker to respond, and later resolve/commit a patch? Similar to the complexity of public actions, it might reveal how many people were involved by estimation of the time needed for an experienced developer to fix an issue vs. actual time taken, both in terms of level of effort and duration.

    - International dispersement of a team in different timezones with some core hours for collaboration, review and public facing activity.

    - Public holidays, country/region-specific work habits, etc--e.g. consideration of "summer holiday" periods or similar common holiday periods, consideration of unusual days of no/low activity versus snow days, power outages, etc which might have been experienced by the attacker.

    Distribution of actions from Github indicates the attacker used a 6 day work week excluding Sunday, and almost all activity conducted between UTC 12:00-16:00. Within these 6 days, activity was uneven at 0.5, 1, 1, 1, 1, 0.5 effort per day. There are low activity periods too that line up with summer solstice (southern hemisphere) or winter solstice (northern hemisphere).

    There are interesting patterns in the data not yet publicly analysed (I think?) that seemingly would reveal the true location of attackers, particularly because attacker actions are anchored to uncontrollable events such as a known-good contributor (such as Linux distro maintainer) raising a Github issue against a repository and the attacker replying an hour later. For such events with low latency of reply, it'd be well worth considering when a reply was made quickly, and when it wasn't, across a few years of data points.

    [1] https://news.ycombinator.com/item?id=39905375

  • kreyenborgi 10 hours ago
    > The catch is where the book begins, not what it is about.
    • edblair 9 hours ago
      Half expecting the next few paragraphs to contain, verbatim, "The smoking gun was a performance issue in a development build of Debian. It's was a sharp observation, and sharper than you may think."
  • pflenker 7 hours ago
    It’s great to have the entire topic brought together into a cohesive book - but wow, I find it very annoying to read.
    • ptx 7 hours ago
      Probably because LLM output only gives the appearance of cohesive writing, but the more you try to understand the author's intent and meaning, the more confusing and annoying it gets, as there is no author, no intent and no meaning there to understand.
  • tryauuum 10 hours ago
    So Microsoft did something good? I thought they are too busy keeping my personal data in a prison and writing tight bash loops wasting 100 percent of a core
    • akimbostrawman 8 hours ago
      no, someone who just happens to work for microsoft doing something at home did something good.
  • lucasRW 11 hours ago
    It's only going to recycle old news. The missing piece is attribution. Had it been the usuals (DPRK, Russia, China), the attribution would have been made publicly. The fact that is has not points as a friendly - especially when Microsoft (who owns Github) had all that telemetry and very likely has the means to find out. Some serious OSINT (consistency of timezone across months if not years of commits) pointed to the Middle East. An obvious Unit name comes to mind.
    • diogocp 10 hours ago
      Fun fact: the guy who reported it (Andres Freund) works for Microsoft.

      Another fun fact: Moscow is in the same time zone as the Middle East.

    • anonreplier 9 hours ago
      "all that telemetry" doesn't count for much if it's behind a VPN