The llama.cpp CPU backend can use the CPU's matrix-instructions (if you have them, cmake detects the CPU capabilities) and that can make them quite fast. E.g. on the Snapdragon X Elite, its modern 12-core CPU is faster for LLM-operations than its not-so-modern GPU.

E.g. llama-bench llama 7b Q4_0 on Snapdragon X 12-core: CPU-pp512: ~180 token/s, GPU/openCL-pp512: ~130 token/s. But a fast, integrated, low-power GPU like on the NVIDIA Jeton Orin NX16 SoC even gets a GPU/CUDA-pp512: ~510 token/s (OK, it has a 2x faster memory-bandwidth than your system, but less than the Snapdragon X), and does this while only consuming ~15W.

In general LLM prompt-processing/"fill" is mostly compute-horsepower bound - so having faster computing (GPU) accelerates. Token-generation/"generate" is mostly memory-bandwidth bound - most modern CPUs can saturate their memory-bus, GPUs idle a lot during this phase. LLMs have to calculate the entire compute-graph with its billions of parameters and the KV-cache (which gets larger quadratically with more context) for each and every token - so you transfer GBs of data for each token from RAM to the chip (CPU+GPU, their on-chip caches are to small to hold all). Prompt-processing can do batches of tokens, so it's not this much during that phase (much better on-chip cache use), and therefore compute-horsepower makes a difference there.

Your system probably has approx. 50GB/s memory bandwidth, the Snapdragon X has 135GB/s. llama-bench llama 7b Q4_0 Snapdragon X 12-core has CPU-tg128: ~24 token/s (in-line with the memory-bandwidth difference).

One of the key features of the new Ryzen AI Max+ is it's 256GB/s RAM - so it's probably even 5x faster than your system on token-generation. And Token-generation speed is especially interesting for the new reasoner-models, which require additional "think" token-generation.

Hope this helps/clarifys.