8 comments

  • erelong 4 minutes ago
    I was trying Ornith 9B locally (it's up on Ollama) which claims:

    > Ornith-1.0-9B, which can be easily deployed on edge devices, matches or exceeds the performance of much larger models such as Gemma 4-31B and Qwen 3.6 35B.

    https://deep-reinforce.com/ornith_1_0.html

    Only tried it so much so far; it did a little better than Qwen 9B

    • liuliu 2 minutes ago
      Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).
  • syntaxing 4 minutes ago
    For those curious about their demo, I’m pretty sure it’s using Locally AI (iOS only) that lmstudio acquired/aquihired a couple months ago.
  • liuliu 36 minutes ago
    The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases.
    • liuliu 26 minutes ago
      You also need to pay close attention to BFCLv3 multi-turn result, that helps you to get a sense how frequently these quants will be in a doom loop.
  • simonw 22 minutes ago
    The models themselves are showing up on Hugging Face here: https://huggingface.co/prism-ml/models
  • alvatech 43 minutes ago
    TIL that 1 bit models are actually 1.58 bit with three values +1, 0 and -1
    • NitpickLawyer 21 minutes ago
      There's two variants of this (or, as the joke goes, for very big values of bit):

      Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling, giving a true 1.71 effective bits per weight.

      1-bit Bonsai 27B uses binary {−1, +1} weights with the same group-wise scaling, giving 1.125 effective bits per weight.

    • bensyverson 33 minutes ago
      Yeah, it's an unfortunate convention from the very first "1 bit" model. But to be clear, Bonsai comes in both ternary and actual 1-bit variants.
  • xyzsparetimexyz 15 minutes ago
    That's awesome. What's the largest model that could fit onto a single 16gb gpu at 1.125 effects bits per weight?
  • Havoc 23 minutes ago
    This must be some sort of unpublished app?

    I can just see their image tool on the app store

  • ai_fry_ur_brain 26 minutes ago
    [dead]