• 2 Posts
  • 637 Comments
Joined 3 years ago
cake
Cake day: August 15th, 2023

help-circle













  • The training is sophisticated, but inference is unfortunately really a text prediction machine. Technically token prediction, but you get the idea.

    For every single token/word. You input your system prompt, context, user input, then the output starts.

    The

    Feed the entire context back in and add the reply “The” at the end.

    The capital

    Feed everything in again with “The capital”

    The capital of

    Feed everything in again…

    The capital of Austria

    It literally works like that, which sounds crazy :)

    The only control you as a user can have is the sampling, like temperature, top-k and so on. But that’s just to soften and randomize how deterministic the model is.

    Edit: I should add that tool and subagent use makes this approach a bit more powerful nowadays. But it all boils down to text prediction again. Even the tools are described per text for what they are for.


  • I’m very satisfied with my S25 so far. But I wanted a smaller phone, not a big fan of the tablet sized monsters people use.

    Camera could be a bit better, but it’s decent enough. I’m using the stock ROM, unfortunately if you tinker certain banking and government apps no longer work :-/

    The only other option would be an iPhone at that size, which I’m not a fan of.

    Though there’s plenty of cheaper mid-range phones nowadays if you want to save some money. Like the OnePlus Nord.


  • Decent sized for what?

    Creative writing and roleplay? Plenty, but I try to fit it into my 16 GB VRAM as otherwise it’s too slow for my liking.

    Coding/complex tasks? No, that would need 128GB and upwards and it would still be awfully slow. Except you use a Mac with unified memory.

    For image and video generation you’d want to fit it into GPU VRAM again, system RAM would be way too slow.