Fooling around with encrypted reasoning blobs

3 comments

• glitchc 9 minutes ago
  Very interesting. The state management is the really insightful find here.

  I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.
• Reubend 51 minutes ago
  Super cool side channel attack. I tend to agree that it's pretty impractical, but it's such a fun discovery!
• Retr0id 1 hour ago
  Very cool idea to use thinking duration (either in tokens or in wall time) as a side-channel!