[projectgus]

Working LoRa drivers for:

• SX1276, SX1277, SX1278, SX1279 modems
• SX1261, SX1262 modems.

Eventual goal (not in this PR) is to have enough support for MicroPython to be a LoRaWAN Class A device with a fully async API available.

How to Test this PR

The lora README added in the PR branch has instructions for how to use the LoRa drivers.

However, until the PR is merged you need some additional steps to install the lora MicroPython packages.

Installing with mip

The --index argument of mip install is needed to install packages from a PR branch.

Example:

$ mpremote connect /dev/ttyUSB0 mip install --index https://projectgus.github.io/micropython-lib/mip/feature/lora lora-sx127x lora-sync

(Replace /dev/ttyUSB0 with the serial port you are using, and replace lora-sx127x and lora-sync with the appropriate packages for your LoRa hardware and programming style - see the lora README for explanation.)

Or from a REPL:

import mip
mip.install("lora-sync", index="https://projectgus.github.io/micropython-lib/mip/feature/lora")

(Replace the package name as needed.)

Freezing lora into the firmware

If your board doesn't support mip, you can freeze the lora modules into the firmware.

• First, you need to be able to build the MicroPython firmware for your board locally, with the micropython source cloned from the github repo.
• Next, switch the branch of the micropython-lib git submodule, as found in the lib/micropython-lib directory of the micropython source, to the branch for this PR. GitHub CLI client is the easiest way to check out a PR branch from the command line: Running commands similar to cd lib/micropython; gh pr checkout 501 should do it.
• Update the board manifest file to add the lora packages you need.
• Rebuild the firmware.

TODOs

• Examples are currently hardcoded to the LoRa modems I was using for testing. I think the best solution here is to just make them say modem = # TODO put your modem config here, but haven't made this change yet.
• Confirm the API & AsyncModem wrapper design is a reasonable approach
• Support ThreadSafeFlag.clear() in MP
• Figure out why the RxTimeout interrupt line is toggling during TX (currently using _irq_en as a workaround)
• Split out the single configure() function to allow fast runtime configuration of parameters changed on the fly by the LoRAWAN MAC layer. (Done by having configure() keep the current value for any missing keys, instead of setting defaults.)
• Decide about support for interleaving rx,tx as managed by the driver - i.e. it would be nice if one task could be continuously (or arbitrarily) calling receive() while another task sometimes called transmit(), and have the modem driver handle the state changes internally without the caller needing to do anything. This isn't necessary for LoRAWAN, but it would make some other types of application logic much simpler. (Implemented.)
• Decide about support for rx_continuous mode. If the previous dot point can be implemented then it might not be needed to explicitly support putting the modem into continuous RX mode. (Implemented.)
• Review use of assert in the code - some places are probably ValueErrors, some places may be unnecessary.
• Add support for a second modem apart from SX1276. (Done, SX1262.)
• Resolve issue where SX1276 invert_iq_tx setting seems to be inverted from all other modems (this may be a hardware compatibility issue, but it's unusual!)
• Check TODOs in the code, mostly these are things that could be configurable but currently aren't.
• Proper packaging layout
• Documentation

Follow-up work

Intended for future PRs:

• ✔️ Document the ant_sw feature that allows callbacks for switching off-modem antenna switches for RX, TX.
• Document calibrate() and calibrate_image() functions.
• ✔️ Document the modem crc_errors counter field, and maybe break out CRC checking to its own doc section.
• Add some packages for common modules and boards (some have fixed pins, others have specific antenna switch configs). This will make it a lot easier to get up and running with a LoRa development board or module.
• ✔️ SX126x radios with DIO2 configured as "enable TCXO" line currently take around 18ms to start the TCXO, which can be made configurable. We can also add a configuration option to not automatically stop the TCXO (and go to "RC Standby") when a receive or transmit ends, which allows tighter receive/transmit timing.
• STM32WL5x is a special case of the SX126x which requires hardware support in MicroPython, have moved to micropython/lora: Add stm32wl5 modem driver #572.
• OCP limit is not currently configured either automatically or manually by the driver, and possibly should be.
• Add a function to read the instantaneous RSSI at any time during receive.
• Add a documentation section about common hardware issues - clock drift, symptoms of incorrect antenna pin setting, known modem bugs, etc.

Example API usage

This PR adds two examples, each with synchronous and async API versions:

• simple_rxtx example that just transmits and receives in a loop
• reliable_delivery which implements a relatively simple "ACK or retry" scheme to receive packets reliably from low power nodes.

This work was funded through GitHub Sponsors.

[beyonlo]

Hello @projectgus

Great. This is a great lib for the MicroPython system!

1. This lib support all features of the SX1276?
2. This lib is really specific to the SX1276 or support other models?

Thank you very much.

[beyonlo]

Sorry, I see above now that is specific for the SX1276.

[beyonlo]

I was questioning that because I would like to use in the feature the 2.4Ghz LORA https://www.semtech.com/products/wireless-rf/lora-24ghz for indoor and outdoor location - It has ~3 meters accuracy, like as the SX1280

[projectgus]

@beyonlo The idea is to have a generic LoRA modem API that can be extended to other modem models in the future, such as the SX1280. However at least for now the focus is on messaging and supporting LoRAWAN nodes, rather than ToF range measurement.

[beyonlo]

@projectgus No problem. To have a Micropython driver for messaging on LORA nodes, will be great :)

[dpgeorge]

I've had a play with this driver using 2x SX1276 modules connected to a PYBLITEv1.0 and a PYBD-SF2. Then they run the rxtx_sync.py demo code. I didn't wire up the D0/D1 or RST pins, so it's just using SPI and CS.

It works! Both devices can send and receive to/from each other.

For the power consumption, it seems quite high. I measure about 13mA when in RX mode (on top of the pyboard current), and about 30-45mA when transmitting a message (again, on top of pyboard current). Is that expected?

[dpgeorge]

Decide about support for interleaving rx,tx as managed by the driver - i.e. it would be nice if one task could be continuously (or arbitrarily) calling receive() while another task sometimes called transmit(), and have the modem driver handle the state changes internally without the caller needing to do anything.

I think this would be very useful, for the driver to automatically manage switching between TX and RX.

For LoRaWAN, or for a custom LoRa protocol, is the idea to have RX constantly on (except when transmitting), or for RX only to be active periodically to reduce power consumption?

[projectgus]

For the power consumption, it seems quite high. I measure about 13mA when in RX mode (on top of the pyboard current), and about 30-45mA when transmitting a message (again, on top of pyboard current). Is that expected?

That sounded high to me also, but SX1276 datasheet says "Supply current in receive mode" between 10.8 and 11.5mA for AU915 depending on which antenna is used. Transmit current 20-120mA depending on transmit power level.

So probably to be expected.

Decide about support for interleaving rx,tx as managed by the driver - i.e. it would be nice if one task could be continuously (or arbitrarily) calling receive() while another task sometimes called transmit(), and have the modem driver handle the state changes internally without the caller needing to do anything.
For LoRaWAN, or for a custom LoRa protocol, is the idea to have RX constantly on (except when transmitting), or for RX only to be active periodically to reduce power consumption?

LoRaWAN specifies particular time slices when the node has to receive.

Given how much of a power drain constant receiving is, it won't be desirable for battery-powered devices. It's likely most custom LoRa setups will have at least one end on battery power, so it's probably not that useful to have seamless RX/TX switching in the driver. Will pull it out.

[dpgeorge]

So probably to be expected.

Yeah, I came to the same conclusion reading the SX datasheet.

Will pull it out.

OK. We can always put it back in when we have more experience with this driver. For now, best to keep it simple/minimal.

[projectgus]

@dpgeorge Thanks for the review, I've updated based on your comments and did some additional memory usage optimisations (especially in sx1262.py).

The stm32wl5 support has now been split into an sx1262.py driver, tested with a local module, and I've moved the small amount of additional changes needed for the stm32wl5 into another branch for a separate PR. This means this PR no longer requires any MicroPython changes, and @jimmo your modules should now work properly with it!

[dpgeorge]

Looking good!

One thing that's needed is a manifest.py for packaging. Also to consider how to organise the files in this package, and whether they should be split out into separate packages or components within the one package. For example, as a user I probably only want either the SX1276 or SX1262 driver, but not both. And I probably only want to use AsyncModem or Modem, but not both.

Probably the directory micropython/lora should have subdirs in it for the actual packages, like micropython/lora/sx1276 and micropython/lora/lora-modem etc. See aioble for a good example of how to structure things. @jimmo will probably have opinions here 😄

[projectgus]

@dpgeorge Can do! Sorry, I did know I needed to do this but I hadn't put it on the TODO list. Will crib from aioble and come up with something soon.

Modem is actually an abstract base class, so everyone will need that[*] - I will rename it to BaseModem or something and make sure everything is laid out to make this clearer too.

[*] It does mostly implement the synchronous methods that using the AbstractModem() wrapper removes the need for, but I can't think of a better hierarchy that doesn't include multiple inheritance. And I think probably best to avoid multiple inheritance!

[tve]

Nice code overall!

I'm having a hard time picturing how to use this driver, there are a bunch of things missing and few that make usage difficult. If this driver is supposed to be a low level driver that can be used for different high-level LoRa uses then I would expect it to look somewhat different:

• break all sub-parts of config() out into separate functions so they can be called when needed, examples are changing BW/SF and TX power in response to ACKs (or lack thereof) or changing channels (freq) (see https://github.com/lemariva/uPyLoRaWAN/blob/master/sx127x.py for example where the init is a collection of calls to these broken out functions)
• ability to have continuous RX as the default mode as opposed to switching back to standby (that's what I tend to use for powered devices)
• provide functionality to check for RX in-progress before TX
• provide a function to calculate packet duration (with LoRa it's not that trivial and often needed since air time can be quite long)
• provide timing information, in particular when a packet was received (I find this essential to sync nodes e.g. to define an ACK window when there is no other shared clock, such as GPS) I take that back, there's no significant difference between calling ticks_ms in the driver irq handler vs. the upper layer's irq handler, except for the fact that the upper layer doesn't immediately know why it's called
• provide functions to enter and leave sleep mode, incl. setting all pins to high impedance
• provide a function to read the FEI (frequency error indication), I find it pretty useful to tune the frequency of sensor nodes when using FSK mode; I don't know whether LoRa is more robust WRT freq deviation (hmmm, certainly not in the very narrow band modes) and the FEI is an easy way to ensure the nodes agree with a central gateway (I'm not using LoRaWAN)

I find the way the IRQ flags are managed rather fragile. All the masking/unmasking only works if start_xx/end_xx are always correctly called in the right sequence. If they're not, one is guaranteed odd and difficult to troubleshoot results. I'm used to clear all flags before switching to TX/RX mode so I'm guaranteed I'm starting from a known state for that phase. Also, I've never used the IRQ masks on the semtech chips and rather do a dispatch in the IRQ handler to figure out what happened. You seem to experience some spurious interrupt according tot he comments and I wouldn't be surprised if it had to do with changing mode, changing IRQ masks, and changing DIO Mapping in back-to-back SPI transactions. There's a wimpy little uC in that sx1276 that has to keep up and do the right thing kepping in mind that the mode change may take way longer than the subsequent SPI transactions (we know that little uC has bugs...)

WRT polling for RX I would not provide any examples or high level code that polls the radio for RX complete. It's a noise disaster in my experience. (Maybe different in stm32wle.) I believe the only appropriate non-interrupt usage pattern for RX is to have an RX window, start the RX at the beginning of the window, wait, and then check whether something arrived once at the end of the window.

I did not see high-level code to operate the driver in always-RX mode and I believe it's rather tricky. Specifically because the IRQ handler gets no info about why it got called and there's an issue with calling RX when no RX-complete indication has been provided (e.g. because one wants to TX).

Documentation???

[tve]

[Oh weird, looks like github submitted a second copy of my comments just because I added another comment... Ignore...]

Nice code overall!

I'm having a hard time picturing how to use this driver, there are a bunch of things missing and few that make usage difficult. If this driver is supposed to be a low level driver that can be used for different high-level LoRa uses then I would expect it to look somewhat different:

• break all sub-parts of config() out into separate functions so they can be called when needed, examples are changing BW/SF and TX power in response to ACKs (or lack thereof) or changing channels (freq) (see https://github.com/lemariva/uPyLoRaWAN/blob/master/sx127x.py for example where the init is a collection of calls to these broken out functions)
• ability to have continuous RX as the default mode as opposed to switching back to standby (that's what I tend to use for powered devices)
• provide functionality to check for RX in-progress before TX
• provide a function to calculate packet duration (with LoRa it's not that trivial and often needed since air time can be quite long)
• provide timing information, in particular when a packet was received (I find this essential to sync nodes e.g. to define an ACK window when there is no other shared clock, such as GPS)
• provide functions to enter and leave sleep mode, incl. setting all pins to high impedance~

I find the way the IRQ flags are managed rather fragile. All the masking/unmasking only works if start_xx/end_xx are always correctly called in the right sequence. If they're not, one is guaranteed odd difficult to troubleshoot results. I'm more used to clear all flags before switching to TX/RX mode so I'm guaranteed I'm starting from a known state for that phase. Also, I've never used the IRQ masks on the semtech chips and rather do a dispatch in the IRQ handler to figure out what happened. You seem to experience some spurious interrupt according tot he comments and I wouldn't be surprised if it had to do with changing mode, changing IRQ masks, and changing DIO Mapping in back-to-back SPI transactions. There's a wimpy little uC in that sx1276 that has to keep up and do the right thing kepping in mind that the mode change may take way longer than the subsequent SPI transactions (we know that little uC has bugs...)

WRT polling for RX I would not provide any examples or high level code that polls the radio for RX complete. It's a noise disaster in my experience. (Maybe different in stm32wle.) I believe the only appropriate non-interrupt usage pattern for RX is to have an RX window, start the RX at the beginning of the window, wait, and then check whether something arrived once at the end of the window.

I did not see high-level code to operate the driver in always-RX mode and I believe it's rather tricky. Specifically because the IRQ handler gets no info about why it got called and there's an issue with calling RX when no RX-complete indication has been provided (e.g. because one wants to TX).

Documentation???

[dpgeorge]

One thing @jimmo and I just discussed was splitting out the sync-modem methods (namely BaseModem.transmit, BaseModem.receive and BaseModem._sync_wait) into a separate class called SyncModem. That would take a particular SX12xx instance in its constructor, essentially mirroring AsyncModem. It would live in lora-sync/lora/sync_modem.py.

Benefits:

• a clear separation of layers (BaseModem -> SX12xx -> SyncModem/AsyncModem)
• no need to have any of the sync modem code if you only use AsyncModem

Drawbacks:

• users must now always install (at least) two packages: one for their hardware, eg sx126x, and one for the top-level driver (sync or async)

@projectgus what are your thoughts on that?

[projectgus]

@dpgeorge

One thing @jimmo and I just discussed was splitting out the sync-modem methods (namely BaseModem.transmit, BaseModem.receive and BaseModem._sync_wait) into a separate class called SyncModem. That would take a particular SX12xx instance in its constructor, essentially mirroring AsyncModem. It would live in lora-sync/lora/sync_modem.py.

@tve also asked about the possibility of a change like this, so it's clearly a logical layout for folks.

I agree about the benefits, it's a more symmetrical set of relationships. The reasons I avoided it until now was a general idea of "keep the simple case - Sync - as simple as possible", but also that I don't know how to logically describe these classes if we split them into two types of "thing":

• SyncModem & AsyncModem are one type of thing ("modem interface"?)
• SX1276, SX1262, BaseModem, etc are a different type of a thing ("modem implementation"?)

... not knowing what names to give them makes me think it's a code smell.

(I guess the current code has this problem as well regarding AsyncModem, and maybe if keeping the current structure then AsyncModem should be called AsyncAdapter or something to disambiguate... this is all starting to feel very Java-y!)

The other way around this would be to use multiple inheritance, so each "public" class in the package inherits from either SyncModem or AsyncModem and also from a chip-specific modem base (SX1276, SX1262, etc.), meaning you can do something like from lora.sync import SX1276. I generally avoid multiple inheritance (and I'm not even sure how well MP supports it) but actually this might be the most intuitive class structure to go with?

[projectgus]

Method renames have been implemented!

One thing @jimmo and I just discussed was splitting out the sync-modem methods (namely BaseModem.transmit, BaseModem.receive and BaseModem._sync_wait) into a separate class called SyncModem. That would take a particular SX12xx instance in its constructor, essentially mirroring AsyncModem. It would live in lora-sync/lora/sync_modem.py.

@dpgeorge @jimmo Please take a look at the head commit in this branch. I'm not 100% sure this is a good idea, but I think it's still preferable to having SyncModem/AsyncModem wrappers that take chip-specific objects as arguments.

I put some reasons in the previous comment on this PR, but the final challenge was telling the user which object to call which method on. For example, do you call modem.configure(...) or modem.modem.configure(...)? The old AsyncModem wrapper had some hacky workarounds for this like def configure(self, ...): self.modem.configure(...), but this is not a great solution either.

If you think the "mixin-like class" approach in that commit is too fiddly then I fully understand though, I'm not 100% convinced on it myself - the best I could say for it is that it might be the "least bad" way. Am happy to drop thjs commit and try something else instead if you're not convinced.

[projectgus]

(Rebased, tweaked some wording in the README a tiny bit.)

[projectgus]

Rebased, pushed a bugfix with CRC as reported by Barry on Discord and another bug fix with async wait timeout on send (only relevant if IRQ doesn't trigger as expected).

[dpgeorge]

Please make it configure the pins in the constructor (using hasattr(pin, init)). See sdcard.py for example.

Also rename u-module to module.

[projectgus]

@dpgeorge @jimmo Thanks for the review of a large and complex PR! Have applied the requested changes (see 30214ee, 247597b), re-tested the two major modem models with sync and async code, squashed commits and rebased on master.

EDIT: Oh well, looks like when I rebased my individual fix commits were lost from GitHub. But the fixes are in there. :)

[dpgeorge]

I tested the simple_rxtx code, both sync and async versions, on a pair of SX1276's. It works!

(I had to fix the simple_rxtx_async.py to rename uasyncio to asyncio, and I pushed that here.)

[dpgeorge]

Thanks for the huge effort on this @projectgus !

[mattytrentini]

Congrats and thanks to everyone involved!