• 5 Posts
  • 454 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle
  • Not sure about AI lab (although I also use podman, prefer Llama-swap), but pretty much everything uses llama-cpp under the hood, which usually takes a day or three to setup for a new architecture. Although I seem to recall them being ready for Qwen3.5 day one due to collaboration.

    I find giving it a week or so for the dust to settle (even if it ‘works’, best parameters, quantization bugs etc take a while to shake out) unless there’s a huge motivation.

    Also benches are more like guidelines than actual rules, best to do your own on your own use cases.


  • Crank it up to 11 12, there’s profit to be made.

    ETA Another likely scenario:

    George: Hey, Ron, you know that maintenance we’ve been putting off because line must go up ?

    Ron: Yeah…

    George: Wouldn’t it be terrible if something happened and the government had to pay for it to be fixed real fast…

    Ron: Heh heh heh.

    The best punchline would be if the government nationalized it as well due to negligence of critical infrastructure…

    Would also send a great message to the power companies about line maintenance. But I’m (probably) dreaming, I’ll send it out into the aether anyway.














  • And avoid fans, spin them fast and they’re generators, with a good chance of reverse polarity to the intended. In fact, judicious canned air is probably better than a vacuum, but you use what you’ve got, it’s a thinkpad, it’ll probably be fine.

    +1 to Isopropanol, (unused) makeup brushes can also be useful.



  • Nope, not top, but apparently not awful (seems to be her husband coding). Some interesting things happening currently with memory systems for local AI which seem to truly enhance (make smarter) smaller thinking models, meaning you can use more, and more coherent context.

    As with most of this stuff I’m waiting for a bit more maturity before looking deeply at it, but there’s definitely some excitement at the moment, and it has the potential to make models that fit on say a 16GB video card capable of many more use cases than previously.