Im encountering an issue, while transmitting large data (eg. files, ~50kb) over MQTT. The first transmission works flawlessly, but then the ESP has problems to allocate enough heap for further large transmissions in receiving. Blog posts depicts that the problem could be, that the heap is fragmentet after some allocations.

My solution here was to setting up a max chunk size for the read() command, e.g. 1024 byte (multiple of four, to retain the possibility to decode base64 encoded data).

max_size = 1024
left_size = sz
while left_size > 0:
    r = min(max_size, left_size)
    msg = self.sock.read(r)
    left_size -= r
    self.cb(topic, msg, (r<max_size))

The additional bool argument for the callback indicates the completeness of the transmission.

Peter Hinch already mentioned to me (in general) a better solution: send multiple short messages or use an device with SPIRAM. Im not entirely sure if this would fit in my situation, where i publish files via MQTT to update these in the filesystem. So at least I wanted to share my results here once again.

Inadvertent i've initially opened this issue under the mqtt repo of @peterhinch (whoops, that was probably a case of mental derangement, im sorry for that). Here was the intended place.