diff --git a/DRIVERS.md b/DRIVERS.md deleted file mode 100644 index 3359123..0000000 --- a/DRIVERS.md +++ /dev/null @@ -1,309 +0,0 @@ -# 1. Introduction - -Drivers for switches and pushbuttons are provided, plus a retriggerable delay -class. The switch and button drivers support debouncing. The switch driver -provides for running a callback or launching a coroutine (coro) on contact -closure and/or opening. - -The pushbutton driver extends this to support long-press and double-click -events. - -# 2. Modules - - 1. `aledflash.py` Flashes the four Pyboard LED's asynchronously for 10s. The - simplest uasyncio demo. Import it to run. - 2. `aswitch.py` This provides classes for interfacing switches and pushbuttons - and also a software retriggerable delay object. Pushbuttons are a - generalisation of switches providing logical rather than physical status along - with double-clicked and long pressed events. - 3. `astests.py` Test/demonstration programs for `aswitch.py`. - -# 3. Module aswitch.py - -This module provides the following classes: - - * `Switch` This supports debouncing a normally open switch connected between - a pin and ground. Can run callbacks or schedule coros on contact closure - and/or opening. - * `Pushbutton` A generalisation of `Switch` to support normally open or - normally closed switches connected to ground or 3V3. Can run callbacks or - schedule coros on double-click or long press events. - * `Delay_ms` A class providing a retriggerable delay measured in ms. Can be - used to run a callback or to schedule a coro. Its state can be tested by any - coro. - -The module `astests.py` provides examples of usage. In the following text the -term **function** implies a Python `callable`: namely a function, bound method, -coroutine or bound coroutine interchangeably. - -### Timing - -The `Switch` class relies on millisecond-level timing: callback functions must -be designed to terminate rapidly. This applies to all functions in the -application; coroutines should yield regularly. If these constraints are not -met, switch events can be missed. - -## 3.1 Switch class - -This assumes a normally open switch connected between a pin and ground. The pin -should be initialised as an input with a pullup. A **function** may be -specified to run on contact closure or opening; where the **function** is a -coroutine it will be scheduled for execution and will run asynchronously. -Debouncing is implicit: contact bounce will not cause spurious execution of -these functions. - -Constructor argument (mandatory): - - 1. `pin` The initialised Pin instance. - -Methods: - - 1. `close_func` Args: `func` (mandatory) a **function** to run on contact - closure. `args` a tuple of arguments for the **function** (default `()`) - 2. `open_func` Args: `func` (mandatory) a **function** to run on contact open. - `args` a tuple of arguments for the **function** (default `()`) - 3. `__call__` Call syntax e.g. `myswitch()` returns the physical debounced - state of the switch i.e. 0 if grounded, 1 if connected to `3V3`. - -Methods 1 and 2 should be called before starting the scheduler. - -Class attribute: - 1. `debounce_ms` Debounce time in ms. Default 50. - -```python -from pyb import LED -from machine import Pin -import uasyncio as asyncio -from aswitch import Switch - -async def pulse(led, ms): - led.on() - await asyncio.sleep_ms(ms) - led.off() - -async def my_app(): - await asyncio.sleep(60) # Dummy application code - -pin = Pin('X1', Pin.IN, Pin.PULL_UP) # Hardware: switch to gnd -red = LED(1) -sw = Switch(pin) -sw.close_func(pulse, (red, 1000)) # Note how coro and args are passed -loop = asyncio.get_event_loop() -loop.run_until_complete(my_app()) # Run main application code -``` - -## 3.2 Pushbutton class - -This can support normally open or normally closed switches, connected to `gnd` -(with a pullup) or to `3V3` (with a pull-down). The `Pin` object should be -initialised appropriately. The assumption is that on initialisation the button -is not pressed. - -The Pushbutton class uses logical rather than physical state: a button's state -is considered `True` if pressed, otherwise `False` regardless of its physical -implementation. - -**function** instances may be specified to run on button press, release, double -click or long press events; where the **function** is a coroutine it will be -scheduled for execution and will run asynchronously. - -Please see the note on timing in section 3. - -Constructor arguments: - - 1. `pin` Mandatory. The initialised Pin instance. - 2. `suppress` Default `False`. See 3.2.1 below. - -Methods: - - 1. `press_func` Args: `func` (mandatory) a **function** to run on button push. - `args` a tuple of arguments for the **function** (default `()`). - 2. `release_func` Args: `func` (mandatory) a **function** to run on button - release. `args` a tuple of arguments for the **function** (default `()`). - 3. `long_func` Args: `func` (mandatory) a **function** to run on long button - push. `args` a tuple of arguments for the **function** (default `()`). - 4. `double_func` Args: `func` (mandatory) a **function** to run on double - push. `args` a tuple of arguments for the **function** (default `()`). - 5. `__call__` Call syntax e.g. `mybutton()` Returns the logical debounced - state of the button (`True` corresponds to pressed). - 6. `rawstate()` Returns the logical instantaneous state of the button. There - is probably no reason to use this. - -Methods 1 - 4 should be called before starting the scheduler. - -Class attributes: - 1. `debounce_ms` Debounce time in ms. Default 50. - 2. `long_press_ms` Threshold time in ms for a long press. Default 1000. - 3. `double_click_ms` Threshold time in ms for a double click. Default 400. - -```python -from pyb import LED -from machine import Pin -import uasyncio as asyncio -from aswitch import Pushbutton - -def toggle(led): - led.toggle() - -async def my_app(): - await asyncio.sleep(60) # Dummy - -pin = Pin('X1', Pin.IN, Pin.PULL_UP) # Pushbutton to gnd -red = LED(1) -pb = Pushbutton(pin) -pb.press_func(toggle, (red,)) # Note how function and args are passed -loop = asyncio.get_event_loop() -loop.run_until_complete(my_app()) # Run main application code -``` - -An alternative Pushbutton class with lower RAM usage is available -[here](https://github.com/kevinkk525/pysmartnode/blob/dev/pysmartnode/utils/abutton.py). - -### 3.2.1 The suppress constructor argument - -When the button is pressed `press_func` runs immediately. This minimal latency -is ideal for applications such as games, but does imply that in the event of a -long press, both `press_func` and `long_func` run: `press_func` immediately and -`long_func` if the button is still pressed when the timer has elapsed. Similar -reasoning applies to the double click function. - -There can be a need for a **function** which runs if a button is pressed but -only if a doubleclick or long press function does not run. The soonest that the -absence of a long press can be detected is on button release. The absence of a -double click can only be detected when the double click timer times out without -a second press occurring. - -This **function** is the `release_func`. If the `suppress` constructor arg is -set, `release_func` will be launched as follows: - 1. If `double_func` does not exist on rapid button release. - 2. If `double_func` exists, after the expiration of the doubleclick timer. - 3. If `long_func` exists and the press duration causes `long_func` to be - launched, `release_func` will not be launched. - 4. If `double_func` exists and a double click occurs, `release_func` will not - be launched. - -## 3.3 Delay_ms class - -This implements the software equivalent of a retriggerable monostable or a -watchdog timer. It has an internal boolean `running` state. When instantiated -the `Delay_ms` instance does nothing, with `running` `False` until triggered. -Then `running` becomes `True` and a timer is initiated. This can be prevented -from timing out by triggering it again (with a new timeout duration). So long -as it is triggered before the time specified in the preceeding trigger it will -never time out. - -If it does time out the `running` state will revert to `False`. This can be -interrogated by the object's `running()` method. In addition a **function** can -be specified to the constructor. This will execute when a timeout occurs; where -the **function** is a coroutine it will be scheduled for execution and will run -asynchronously. - -Constructor arguments (defaults in brackets): - - 1. `func` The **function** to call on timeout (default `None`). - 2. `args` A tuple of arguments for the **function** (default `()`). - 3. `can_alloc` Boolean, default `True`. See below. - 4. `duration` Integer, default 1000ms. The default timer period where no value - is passed to the `trigger` method. - -Methods: - - 1. `trigger` optional argument `duration=0`. A timeout will occur after - `duration` ms unless retriggered. If no arg is passed the period will be that - of the `duration` passed to the constructor. See Class variable below. - 2. `stop` No argument. Cancels the timeout, setting the `running` status - `False`. The timer can be restarted by issuing `trigger` again. - 3. `running` No argument. Returns the running status of the object. - 4. `__call__` Alias for running. - -Class variable: - - 1. `verbose=False` If `True` a warning will be printed if a running timer is - retriggered with a time value shorter than the time currently outstanding. - Such an operation has no effect owing to the design of `uasyncio`. - -If the `trigger` method is to be called from an interrupt service routine the -`can_alloc` constructor arg should be `False`. This causes the delay object -to use a slightly less efficient mode which avoids RAM allocation when -`trigger` runs. - -In this example a 3 second timer starts when the button is pressed. If it is -pressed repeatedly the timeout will not be triggered. If it is not pressed for -3 seconds the timeout triggers and the LED lights. - -```python -from pyb import LED -from machine import Pin -import uasyncio as asyncio -from aswitch import Pushbutton, Delay_ms - -async def my_app(): - await asyncio.sleep(60) # Run for 1 minute - -pin = Pin('X1', Pin.IN, Pin.PULL_UP) # Pushbutton to gnd -red = LED(1) -pb = Pushbutton(pin) -d = Delay_ms(lambda led: led.on(), (red,)) -pb.press_func(d.trigger, (3000,)) # Note how function and args are passed -loop = asyncio.get_event_loop() -loop.run_until_complete(my_app()) # Run main application code -``` - -# 4. Module astests.py - -This provides demonstration/test functions for the `Switch` and `Pushbutton` -classes. They assume a switch or button wired between pin X1 and gnd. Tests may -be terminated by grounding X2. - -## 4.1 Function test_sw() - -This will flash the red LED on switch closure, and the green LED on opening -and demonstrates the scheduling of coroutines. See section 5 for a discussion -of its behaviour if the switch is toggled rapidly. - -## 4.2 Function test_swcb() - -Demonstrates the use of callbacks to toggle the red and green LED's. - -## 4.3 Function test_btn(lpmode=False) - -This will flash the red LED on button push, and the green LED on release. A -long press will flash the blue LED and a double-press the yellow one. - -Test the launching of coroutines and also the `suppress` constructor arg. - -It takes three optional positional boolean args: - 1. `Suppresss=False` If `True` sets the `suppress` constructor arg. - 2. `lf=True` Declare a long press coro. - 3. `df=true` Declare a double click coro. - -The note below on race conditions applies. - -## 4.4 Function test_btncb() - -Demonstrates the use of callbacks. Toggles the red, green, yellow and blue -LED's on press, release, double-press and long press respectively. - -# 5 Race conditions - -Note that in the tests such as test_sw() where coroutines are scheduled by -events and the switch is cycled rapidly the LED behaviour may seem surprising. -This is because each time the switch is closed a coro is launched to flash the -red LED; on each open event one is launched for the green LED. With rapid -cycling a new coro instance will commence while one is still running against -the same LED. This type of conflict over a resource is known as a race -condition: in this instance it leads to the LED behaving erratically. - -This is a hazard of asynchronous programming. In some situations it is -desirable to launch a new instance on each button press or switch closure, even -if other instances are still incomplete. In other cases it can lead to a race -condition, leading to the need to code an interlock to ensure that the desired -behaviour occurs. The programmer must define the desired behaviour. - -In the case of this test program it might be to ignore events while a similar -one is running, or to extend the timer to prolong the LED illumination. -Alternatively a subsequent button press might be required to terminate the -illumination. The "right" behaviour is application dependent. - -A further consequence of scheduling new coroutine instances when one or more -are already running is that the `uasyncio` queue can fill causing an exception. diff --git a/FASTPOLL.md b/FASTPOLL.md deleted file mode 100644 index 2439342..0000000 --- a/FASTPOLL.md +++ /dev/null @@ -1,556 +0,0 @@ -# fast_io: A modified version of uasyncio - -This version is a "drop in" replacement for official `uasyncio`. Existing -applications should run under it unchanged and with essentially identical -performance except that task cancellation and timeouts are expedited "soon" -rather than being deferred until the task is next scheduled. - -"Priority" features are only enabled if the event loop is instantiated with -specific arguments. - -This version has the following features relative to official V2.0: - * Timeouts and task cancellation are handled promptly, rather than being - deferred until the coroutine is next scheduled. - * I/O can optionally be handled at a higher priority than other coroutines - [PR287](https://github.com/micropython/micropython-lib/pull/287). - * Tasks can yield with low priority, running when nothing else is pending. - * Callbacks can similarly be scheduled with low priority. - * A [bug](https://github.com/micropython/micropython/pull/3836#issuecomment-397317408) - whereby bidirectional devices such as UARTS can fail to handle concurrent - input and output is fixed. - * It is compatible with `rtc_time.py` for micro-power applications documented - [here](./lowpower/README.md). This is a Pyboard-only extension (including - Pyboard D). - * An assertion failure is produced if `create_task` or `run_until_complete` - is called with a generator function - [PR292](https://github.com/micropython/micropython-lib/pull/292). This traps - a common coding error which otherwise results in silent failure. - * The presence and version of the `fast_io` version can be tested at runtime. - * The presence of an event loop instance can be tested at runtime. - * `run_until_complete(coro())` now returns the value returned by `coro()` as - per CPython - [micropython-lib PR270](https://github.com/micropython/micropython-lib/pull/270). - * The `StreamReader` class now has a `readinto(buf, n=0)` method to enable - allocations to be reduced. - -Note that priority device drivers are written by using the officially supported -technique for writing stream I/O drivers. Code using such drivers will run -unchanged under the `fast_io` version. Using the fast I/O mechanism requires -adding just one line of code. This implies that if official `uasyncio` acquires -a means of prioritising I/O other than that in this version, application code -changes should be minimal. - -#### Changes incompatible with prior versions - -V0.24 -The `version` bound variable now returns a 2-tuple. - -Prior versions. -The high priority mechanism formerly provided in `asyncio_priority.py` was a -workround based on the view that stream I/O written in Python would remain -unsupported. This is now available so `asyncio_priority.py` is obsolete and -should be deleted from your system. The facility for low priority coros -formerly provided by `asyncio_priority.py` is now implemented. - -###### [Main README](./README.md) - -# Contents - - 1. [Installation](./FASTPOLL.md#1-installation) - 1.1 [Benchmarks](./FASTPOLL.md#11-benchmarks) Benchmark and demo programs. - 2. [Rationale](./FASTPOLL.md#2-rationale) - 2.1 [Latency](./FASTPOLL.md#21-latency) - 2.2 [Timing accuracy](./FASTPOLL.md#22-timing-accuracy) - 2.3 [Polling in uasyncio](./FASTPOLL.md#23-polling-in-usayncio) - 3. [The modified version](./FASTPOLL.md#3-the-modified-version) - 3.1 [Fast IO](./FASTPOLL.md#31-fast-io) - 3.2 [Low Priority](./FASTPOLL.md#32-low-priority) - 3.3 [Other Features](./FASTPOLL.md#33-other-features) - 3.3.1 [Version](./FASTPOLL.md#331-version) - 3.3.2 [Check event loop status](./FASTPOLL.md#332-check-event-loop-status) - 3.3.3 [StreamReader readinto method](./FASTPOLL.md#333-streamreader-readinto-method) - 3.4 [Low priority yield](./FASTPOLL.md#34-low-priority-yield) - 3.4.1 [Task Cancellation and Timeouts](./FASTPOLL.md#341-task-cancellation-and-timeouts) - 3.5 [Low priority callbacks](./FASTPOLL.md#35-low-priority-callbacks) - 4. [ESP Platforms](./FASTPOLL.md#4-esp-platforms) - 5. [Background](./FASTPOLL.md#4-background) - 6. [Performance](./FASTPOLL.md#6-performance) - -# 1. Installation - -The basic approach is to install and test `uasyncio` on the target hardware. -Replace `core.py` and `__init__.py` with the files in the `fast_io` directory. - -The current MicroPython release build (1.10) has `uasyncio` implemented as a -frozen module. The following options for installing `fast_io` exist: - - 1. Use a daily build, install `uasyncio` as per the tutorial then replace the - above files. - 2. Build the firmware with the `fast_io` version implemented as frozen - bytecode. - 3. Use a release build. Install as in 1. above. Then change the module search - order by modifying `sys.path`. The initial entry `''` specifies frozen - bytecode. If this is deleted and appended to the end, frozen files will only - be found if there is no match in the filesystem. - -```python -import sys -sys.path.append(sys.path.pop(0)) # Prefer modules in filesystem -``` - -See [ESP Platforms](./FASTPOLL.md#6-esp-platforms) for general comments on the -suitability of ESP platforms for systems requiring fast response. - -## 1.1 Benchmarks - -The following files demonstrate the performance gains offered by prioritisation -and the improvements to task cancellation and timeouts. They also show the use -of these features. Documentation is in the code. - -Tests and benchmarks to run against the official and `fast_io` versions: - * `benchmarks/latency.py` Shows the effect on latency with and without low - priority usage. - * `benchmarks/rate.py` Shows the frequency with which uasyncio schedules - minimal coroutines (coros). - * `benchmarks/rate_esp.py` As above for ESP32 and ESP8266. - * `fast_io/ms_timer.py` An I/O device driver providing a timer with higher - precision timing than `wait_ms()` when run under the `fast_io` version. - * `fast_io/ms_timer_test.py` Test/demo program for above. - * `fast_io/pin_cb.py` An I/O device driver which causes a pin state change to - trigger a callback. This is a driver, not an executable test program. - * `fast_io/pin_cb_test.py` Demo of above driver: illustrates performance gain - under `fast_io`. - -Tests requiring the current version of the `fast_io` fork: - * `benchmarks/rate_fastio.py` Measures the rate at which coros can be scheduled - if the fast I/O mechanism is used but no I/O is pending. - * `benchmarks/call_lp.py` Demo of low priority callbacks. - * `benchmarks/overdue.py` Demo of maximum overdue feature. - * `benchmarks/priority_test.py` Cancellation of low priority coros. - * `fast_io/fast_can_test.py` Demo of cancellation of paused tasks. - * `fast_io/iorw_can.py` Cancellation of task waiting on I/O. - * `fast_io/iorw_to.py` Timeouts applies to tasks waiting on I/O. - -# 2. Rationale - -MicroPython firmware now enables device drivers for stream devices to be -written in Python, via `uio.IOBase`. This mechanism can be applied to any -situation where a piece of hardware or an asynchronously set flag needs to be -polled. Such polling is efficient because it is handled in C using -`select.poll`, and because the coroutine accessing the device is descheduled -until polling succeeds. - -Unfortunately official `uasyncio` polls I/O with a relatively high degree of -latency. - -Applications may need to poll a hardware device or a flag set by an interrupt -service routine (ISR). An overrun may occur if the scheduling of the polling -coroutine (coro) is subject to excessive latency. Fast devices with interrupt -driven drivers (such as the UART) need to buffer incoming data during any -latency period. Lower latency reduces the buffer size requirement. - -Further, a coro issuing `await asyncio.sleep_ms(t)` may block for much longer -than `t` depending on the number and design of other coros which are pending -execution. Delays can easily exceed the nominal value by an order of magnitude. - -This variant mitigates this by providing a means of scheduling I/O at a higher -priority than other coros: if an I/O queue is specified, I/O devices are polled -on every iteration of the scheduler. This enables faster response to real time -events and also enables higher precision millisecond-level delays to be -realised. - -The variant also enables coros to yield control in a way which prevents them -from competing with coros which are ready for execution. Coros which have -yielded in a low priority fashion will not be scheduled until all "normal" -coros are waiting on a nonzero timeout. The benchmarks show that the -improvement in the accuracy of time delays can exceed two orders of magnitude. - -## 2.1 Latency - -Coroutines in uasyncio which are pending execution are scheduled in a "fair" -round-robin fashion. Consider these functions: - -```python -async def foo(): - while True: - yield - # code which takes 4ms to complete - -async def handle_isr(): - global isr_has_run - while True: - if isr_has_run: - # read and process data - isr_has_run = False - yield -``` - -Assume a hardware interrupt handler sets the `isr_has_run` flag, and that we -have ten instances of `foo()` and one instance of `handle_isr()`. When -`handle_isr()` issues `yield`, its execution will pause for 40ms while each -instance of `foo()` is scheduled and performs one iteration. This may be -unacceptable: it may be necessary to poll and respond to the flag at a rate -sufficient to avoid overruns. - -In this version `handle_isr()` would be rewritten as a stream device driver -which could be expected to run with latency of just over 4ms. - -Alternatively this latency may be reduced by enabling the `foo()` instances to -yield in a low priority manner. In the case where all coros other than -`handle_isr()` are low priority the latency is reduced to 300μs - a figure -of about double the inherent latency of uasyncio. - -The benchmark latency.py demonstrates this. Documentation is in the code; it -can be run against both official and priority versions. This measures scheduler -latency. Maximum application latency, measured relative to the incidence of an -asynchronous event, will be 300μs plus the worst-case delay between yields of -any one competing task. - -### 2.1.1 I/O latency - -The official version of `uasyncio` has even higher levels of latency for I/O -scheduling. In the above case of ten coros using 4ms of CPU time between zero -delay yields, the latency of an I/O driver would be 80ms. - -###### [Contents](./FASTPOLL.md#contents) - -## 2.2 Timing accuracy - -Consider these functions: - -```python -async def foo(): - while True: - await asyncio.sleep(0) - # code which takes 4ms to complete - -async def fast(): - while True: - # Code omitted - await asyncio.sleep_ms(15) - # Code omitted -``` - -Again assume ten instances of `foo()` and one of `fast()`. When `fast()` -issues `await asyncio.sleep_ms(15)` it will not see a 15ms delay. During the -15ms period `foo()` instances will be scheduled. When the delay elapses, -`fast()` will compete with pending `foo()` instances. - -This results in variable delays up to 55ms (10 tasks * 4ms + 15ms). A -`MillisecTimer` class is provided which uses stream I/O to achieve a relatively -high precision delay: - -```python -async def timer_test(n): - timer = ms_timer.MillisecTimer() - while True: - await timer(30) # More precise timing - # Code -``` - -The test program `fast_io/ms_timer_test.py` illustrates three instances of a -coro with a 30ms nominal timer delay, competing with ten coros which yield with -a zero delay between hogging the CPU for 10ms. Using normal scheduling the 30ms -delay is actually 300ms. With fast I/O it is 30-34ms. - -###### [Contents](./FASTPOLL.md#contents) - -## 2.3 Polling in uasyncio - -The asyncio library provides various mechanisms for polling a device or flag. -Aside from a polling loop these include awaitable classes and asynchronous -iterators. If an awaitable class's `__iter__()` method simply returns the state -of a piece of hardware, there is no performance gain over a simple polling -loop. - -This is because uasyncio schedules tasks which yield with a zero delay, -together with tasks which have become ready to run, in a "fair" round-robin -fashion. This means that a task waiting on a zero delay will be rescheduled -only after the scheduling of all other such tasks (including timed waits whose -time has elapsed). - -The `fast_io` version enables awaitable classes and asynchronous iterators to -run with lower latency by designing them to use the stream I/O mechanism. The -program `fast_io/ms_timer.py` provides an example. - -Practical cases exist where the `foo()` tasks are not time-critical: in such -cases the performance of time critical tasks may be enhanced by enabling -`foo()` to submit for rescheduling in a way which does not compete with tasks -requiring a fast response. In essence "slow" operations tolerate longer latency -and longer time delays so that fast operations meet their performance targets. -Examples are: - - * User interface code. A system with ten pushbuttons might have a coro running - on each. A GUI touch detector coro needs to check a touch against sequence of - objects. Both may tolerate 100ms of latency before users notice any lag. - * Networking code: a latency of 100ms may be dwarfed by that of the network. - * Mathematical code: there are cases where time consuming calculations may - take place which are tolerant of delays. Examples are statistical analysis, - sensor fusion and astronomical calculations. - * Data logging. - -###### [Contents](./FASTPOLL.md#contents) - -# 3. The modified version - -The `fast_io` version adds `ioq_len=0` and `lp_len=0` arguments to -`get_event_loop`. These determine the lengths of I/O and low priority queues. -The zero defaults cause the queues not to be instantiated, in which case the -scheduler operates as per the official version. If an I/O queue length > 0 is -provided, I/O performed by `StreamReader` and `StreamWriter` objects is -prioritised over other coros. If a low priority queue length > 0 is specified, -tasks have an option to yield in such a way to minimise their competition with -other tasks. - -Arguments to `get_event_loop()`: - 1. `runq_len=16` Length of normal queue. Default 16 tasks. - 2. `waitq_len=16` Length of wait queue. - 3. `ioq_len=0` Length of I/O queue. Default: no queue is created. - 4. `lp_len=0` Length of low priority queue. Default: no queue. - -###### [Contents](./FASTPOLL.md#contents) - -## 3.1 Fast IO - -Device drivers which are to be capable of running at high priority should be -written to use stream I/O: see -[Writing streaming device drivers](./TUTORIAL.md#64-writing-streaming-device-drivers). - -The `fast_io` version will schedule I/O whenever the `ioctl` reports a ready -status. This implies that devices which become ready very soon after being -serviced can hog execution. This is analogous to the case where an interrupt -service routine is called at an excessive frequency. - -This behaviour may be desired where short bursts of fast data are handled. -Otherwise drivers of such hardware should be designed to avoid hogging, using -techniques like buffering or timing. - -###### [Contents](./FASTPOLL.md#contents) - -## 3.2 Low Priority - -The low priority solution is based on the notion of "after" implying a time -delay which can be expected to be less precise than the asyncio standard calls. -The `fast_io` version adds the following awaitable instances: - - * `after(t)` Low priority version of `sleep(t)`. - * `after_ms(t)` Low priority version of `sleep_ms(t)`. - -It adds the following event loop methods: - - * `loop.call_after(t, callback, *args)` - * `loop.call_after_ms(t, callback, *args)` - * `loop.max_overdue_ms(t=None)` This sets the maximum time a low priority task - will wait before being scheduled. A value of 0 corresponds to no limit. The - default arg `None` leaves the period unchanged. Always returns the period - value. If there is no limit and a competing task runs a loop with a zero delay - yield, the low priority yield will be postponed indefinitely. - -See [Low priority callbacks](./FASTPOLL.md#35-low-priority-callbacks) - -###### [Contents](./FASTPOLL.md#contents) - -## 3.3 Other Features - -### 3.3.1 Version - -Variable: - * `version` Returns a 2-tuple. Current contents ('fast_io', '0.25'). Enables - the presence and realease state of this version to be determined at runtime. - -### 3.3.2 Check event loop status - -The way `uasyncio` works can lead to subtle bugs. The first call to -`get_event_loop` instantiates the event loop and determines the size of its -queues. Hence the following code will not behave as expected: -```python -import uasyncio as asyncio -bar = Bar() # Constructor calls get_event_loop() -# and renders these args inoperative -loop = asyncio.get_event_loop(runq_len=40, waitq_len=40) -``` -CPython V3.7 provides a function `get_running_loop` which enables the current -loop to be retrieved, raising a `RuntimeError` if one has not been -instantiated. This is provided in `fast_io`. In the above sample the `Bar` -constructor can call `get_running_loop` to avoid inadvertently instantiating an -event loop with default args. - -Function: - * `get_running_loop` No arg. Returns the event loop or raises a `RuntimeError` - if one has not been instantiated. - -Function: - * `got_event_loop()` No arg. Returns a `bool`: `True` if the event loop has - been instantiated. This is retained for compatibility: `get_running_loop` is - preferred. - -### 3.3.3 StreamReader readinto method - -The purpose of this asynchronous method is to be a non-allocating complement to -the `StreamReader.read` method, enabling data to be read into a pre-existing -buffer. It assumes that the device driver providing the data has a `readinto` -method. - -`StreamReader.readinto(buf, n=0)` args: -`buf` the buffer to read into. -`n=0` the maximum number of bytes to read - default the buffer size. - -The method will pause (allowing other coros to run) until data is available. - -Available data will be placed in the buffer. The return value is the number of -bytes read. The default maximum number of bytes is limited to the buffer size, -otherwise to the value of `n`. - -This method calls the synchronous `readinto` method of the data source. This -may take one arg (the buffer) or two (the buffer followed by the maximum number -of bytes to read). If `StreamReader.readinto` is launched with a single arg, -the `readinto` method will receive that one arg. - -It is the reponsibility of the device `readinto` method to validate the args, -to populate the buffer and to return the number of bytes read. It should return -"immediately" with as much data as is available. It will only be called when -the `ioctl` method indicates that read data is ready. - -###### [Contents](./FASTPOLL.md#contents) - -## 3.4 Low priority yield - -Consider this code fragment: - -```python -import uasyncio as asyncio -loop = asyncio.get_event_loop(lp_len=16) - -async def foo(): - while True: - # Do something - await asyncio.after(1.5) # Wait a minimum of 1.5s - # code - await asyncio.after_ms(20) # Wait a minimum of 20ms -``` - -These `await` statements cause the coro to suspend execution for the minimum -time specified. Low priority coros run in a mutually "fair" round-robin fashion. -By default the coro will only be rescheduled when all "normal" coros are waiting -on a nonzero time delay. A "normal" coro is one that has yielded by any other -means. - -This behaviour can be overridden to limit the degree to which they can become -overdue. For the reasoning behind this consider this code: - -```python -import uasyncio as asyncio -loop = asyncio.get_event_loop(lp_len=16) - -async def foo(): - while True: - # Do something - await asyncio.after(0) -``` - -By default a coro yielding in this way will be re-scheduled only when there are -no "normal" coros ready for execution i.e. when all are waiting on a nonzero -delay. The implication of having this degree of control is that if a coro -issues: - -```python -while True: - await asyncio.sleep(0) - # Do something which does not yield to the scheduler -``` - -low priority tasks will never be executed. Normal coros must sometimes wait on -a non-zero delay to enable the low priority ones to be scheduled. This is -analogous to running an infinite loop without yielding. - -This behaviour can be modified by issuing: - -```python -loop = asyncio.get_event_loop(lp_len = 16) -loop.max_overdue_ms(1000) -``` - -In this instance a task which has yielded in a low priority manner will be -rescheduled in the presence of pending "normal" tasks if they cause a low -priority task to become overdue by more than 1s. - -### 3.4.1 Task Cancellation and Timeouts - -Tasks which yield in a low priority manner may be subject to timeouts or be -cancelled in the same way as normal tasks. See [Task cancellation](./TUTORIAL.md#521-task-cancellation) -and [Coroutines with timeouts](./TUTORIAL.md#522-coroutines-with-timeouts). - -###### [Contents](./FASTPOLL.md#contents) - -## 3.5 Low priority callbacks - -The following `EventLoop` methods enable callback functions to be scheduled -to run when all normal coros are waiting on a delay or when `max_overdue_ms` -has elapsed: - -`call_after(delay, callback, *args)` Schedule a callback with low priority. -Positional args: - 1. `delay` Minimum delay in seconds. May be a float or integer. - 2. `callback` The callback to run. - 3. `*args` Optional comma-separated positional args for the callback. - -The delay specifies a minimum period before the callback will run and may have -a value of 0. The period may be extended depending on other high and low -priority tasks which are pending execution. - -A simple demo of this is `benchmarks/call_lp.py`. Documentation is in the -code. - -`call_after_ms(delay, callback, *args)` Call with low priority. Positional -args: - 1. `delay` Integer. Minimum delay in millisecs before callback runs. - 2. `callback` The callback to run. - 3. `*args` Optional positional args for the callback. - -###### [Contents](./FASTPOLL.md#contents) - -# 4. ESP Platforms - -It should be noted that the response of the ESP8266 to hardware interrupts is -remarkably slow. This also appears to apply to ESP32 platforms. Consider -whether a response in the high hundreds of μs meets project requirements; also -whether a priority mechanism is needed on hardware with such poor realtime -performance. - -# 5. Background - -This has been discussed in detail -[in issue 2989](https://github.com/micropython/micropython/issues/2989). - -A further discussion on the subject of using the ioread mechanism to achieve -fast scheduling took place -[in issue 2664](https://github.com/micropython/micropython/issues/2664). - -Support was finally [added here](https://github.com/micropython/micropython/pull/3836). - -# 6. Performance - -The `fast_io` version is designed to enable existing applications to run -unchanged and to minimise the effect on raw scheduler performance in cases -where the priority functionality is unused. - -The benchmark `rate.py` measures the rate at which tasks can be scheduled; -`rate_fastio` is identical except it instantiates an I/O queue and a low -priority queue. The benchmarks were run on a Pyboard V1.1 under official -`uasyncio` V2 and under the current `fast_io` version V0.24. Results were as -follows: - -| Script | Uasyncio version | Period (100 coros) | Overhead | PBD | -|:------:|:----------------:|:------------------:|:--------:|:---:| -| rate | Official V2 | 156μs | 0% | 123μs | -| rate | fast_io | 162μs | 3.4% | 129μs | -| rate_fastio | fast_io | 206μs | 32% | 181μs | - -The last column shows times from a Pyboard D SF2W. - -If an I/O queue is instantiated I/O is polled on every scheduler iteration -(that is its purpose). Consequently there is a significant overhead. In -practice the overhead will increase with the number of I/O devices being -polled and will be determined by the efficiency of their `ioctl` methods. - -Timings for current `fast_io` V0.24 and the original version were identical. diff --git a/HD44780/alcdtest.py b/HD44780/alcdtest.py deleted file mode 100644 index 2899c31..0000000 --- a/HD44780/alcdtest.py +++ /dev/null @@ -1,19 +0,0 @@ -# alcdtest.py Test program for LCD class -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license -# runs for 20s -import uasyncio as asyncio -import utime as time -from alcd import LCD, PINLIST - -lcd = LCD(PINLIST, cols = 16) - -async def lcd_task(): - for secs in range(20, -1, -1): - lcd[0] = 'MicroPython {}'.format(secs) - lcd[1] = "{:11d}uS".format(time.ticks_us()) - await asyncio.sleep(1) - - -loop = asyncio.get_event_loop() -loop.run_until_complete(lcd_task()) diff --git a/PRIMITIVES.md b/PRIMITIVES.md deleted file mode 100644 index ca147d4..0000000 --- a/PRIMITIVES.md +++ /dev/null @@ -1,792 +0,0 @@ -# 1. The asyn.py library - -This provides some simple synchronisation primitives, together with an API for -task monitoring and cancellation. Task cancellation requires usayncio V 1.7.1 -or higher. At the time of writing (7th Jan 2018) it requires a daily build of -MicroPython firmware or one built from source. - -The library is too large to run on the ESP8266 except as frozen bytecode. An -obvious workround is to produce a version with unused primitives removed. - -###### [Main README](./README.md) - -# Contents - - 1. [The asyn.py library](./PRIMITIVES.md#1-the-asyn.py-library) - 1.1 [Synchronisation Primitives](./PRIMITIVES.md#11-synchronisation-primitives) - 1.2 [Task control and monitoring](./PRIMITIVES.md#12-task-control-and-monitoring) - 2. [Modules](./PRIMITIVES.md#2-modules) - 3. [Synchronisation Primitives](./PRIMITIVES.md#3-synchronisation-primitives) - 3.1 [Function launch](./PRIMITIVES.md#31-function-launch) Launch a function or a coro interchangeably. - 3.2 [Class Lock](./PRIMITIVES.md#32-class-lock) Ensure exclusive access to a shared resource. - 3.2.1 [Definition](./PRIMITIVES.md#321-definition) - 3.3 [Class Event](./PRIMITIVES.md#33-class-event) Pause a coro until an event occurs. - 3.3.1 [Definition](./PRIMITIVES.md#331-definition) - 3.4 [Class Barrier](./PRIMITIVES.md#34-class-barrier) Pause multiple coros until all reach a given point. - 3.5 [Class Semaphore](./PRIMITIVES.md#35-class-semaphore) Limit number of coros which can access a resource. - 3.5.1 [Class BoundedSemaphore](./PRIMITIVES.md#351-class-boundedsemaphore) - 3.6 [Class Condition](./PRIMITIVES.md#36-class-condition) Control access to a shared reource. - 3.6.1 [Definition](./PRIMITIVES.md#361-definition) - 3.7 [Class Gather](./PRIMITIVES.md#37-class-gather) Synchronise and collect results from multiple coros. - 3.7.1 [Definition](./PRIMITIVES.md#371-definition) - 3.7.2 [Use with timeouts and cancellation](./PRIMITIVES.md#372-use-with-timeouts-and-cancellation) Demo of advanced usage of Gather. - 4. [Task Cancellation](./PRIMITIVES.md#4-task-cancellation) Methods of cancelling tasks and groups of tasks. - 4.1 [Coro sleep](./PRIMITIVES.md#41-coro-sleep) sleep() with reduced exception handling latency. - 4.2 [Class Cancellable](./PRIMITIVES.md#42-class-cancellable) Register tasks for cancellation. - 4.2.1 [Groups](./PRIMITIVES.md#421-groups) Group sets of tasks for cancellation. - 4.2.2 [Custom cleanup](./PRIMITIVES.md#422-custom-cleanup) - 4.3 [Class NamedTask](./PRIMITIVES.md#43-class-namedtask) Associate tasks with names for cancellation. - 4.3.1 [Latency and Barrier objects](./PRIMITIVES.md#431-latency-and-barrier-objects) - 4.3.2 [Custom cleanup](./PRIMITIVES.md#432-custom-cleanup) - -## 1.1 Synchronisation Primitives - -There is often a need to provide synchronisation between coros. A common -example is to avoid what are known as "race conditions" where multiple coros -compete to access a single resource. An example is provided in the `aswitch.py` -program and discussed in [the docs](./DRIVERS.md). Another hazard is the "deadly -embrace" where two coros wait on the other's completion. - -In simple applications these are often addressed with global flags. A more -elegant approach is to use synchronisation primitives. The module `asyn.py` -offers "micro" implementations of `Lock`, `Event`, `Barrier`, `Semaphore` and -`Condition` primitives, and a lightweight implementation of `asyncio.gather`. - -Another synchronisation issue arises with producer and consumer coros. The -producer generates data which the consumer uses. Asyncio provides the `Queue` -object. The producer puts data onto the queue while the consumer waits for its -arrival (with other coros getting scheduled for the duration). The `Queue` -guarantees that items are removed in the order in which they were received. As -this is a part of the uasyncio library its use is described in the [tutorial](./TUTORIAL.md). - -###### [Contents](./PRIMITIVES.md#contents) - -## 1.2 Task control and monitoring - -`uasyncio` does not implement the `Task` and `Future` classes of `asyncio`. -Instead it uses a 'micro' lightweight means of task cancellation. The `asyn.py` -module provides an API to simplify its use and to check on the running status -of coroutines which are subject to cancellation. - -# 2. Modules - -The following modules are provided: - * `asyn.py` The main library. - * `asyntest.py` Test/demo programs for the primitives. - * `asyn_demos.py` Minimal "get started" task cancellation demos. - * `cantest.py` Task cancellation tests. Examples of intercepting `StopTask`. - Intended to verify the library against future `uasyncio` changes. - -Import `asyn_demos.py` or `cantest.py` for a list of available tests. - -###### [Contents](./PRIMITIVES.md#contents) - -# 3. Synchronisation Primitives - -The primitives are intended for use only with `uasyncio`. They are `micro` in -design. They are not thread safe and hence are incompatible with the `_thread` -module and with interrupt handlers. - -## 3.1 Function launch - -This function accepts a function or coro as an argument, along with a tuple of -args. If the function is a callback it is executed with the supplied argumets. -If it is a coro, it is scheduled for execution. - -args: - * `func` Mandatory. a function or coro. These are provided 'as-is' i.e. not - using function call syntax. - * `tup_args` Optional. A tuple of arguments, default `()`. The args are - upacked when provided to the function. - -## 3.2 Class Lock - -This has now been superseded by the more efficient official version. - -At time of writing (18th Dec 2017) the official `Lock` class is not complete. -If a coro is subject to a [timeout](./TUTORIAL.md#44-coroutines-with-timeouts) -and the timeout is triggered while it is waiting on a lock, the timeout will be -ineffective. It will not receive the `TimeoutError` until it has acquired the -lock. - -The implementation in `asyn.py` avoids this limitation but at the cost of lower -efficiency. The remainder of this section describes this version. - -A lock guarantees unique access to a shared resource. The preferred way to use it -is via an asynchronous context manager. In the following code sample a `Lock` -instance `lock` has been created and is passed to all coros wishing to access -the shared resource. Each coro issues the following: - -```python -async def bar(lock): - async with lock: - # Access resource -``` - -While the coro `bar` is accessing the resource, other coros will pause at the -`async with lock` statement until the context manager in `bar()` is complete. - -Note that MicroPython had a bug in its implementation of asynchronous context -managers. This is fixed: if you build from source there is no problem. Alas the -fix was too late for release build V1.9.4. If using that build a `return` -statement should not be issued in the `async with` block. See note at end of -[this section](./TUTORIAL.md#43-asynchronous-context-managers). - -### 3.2.1 Definition - -Constructor: Optional argument `delay_ms` default 0. Sets a delay between -attempts to acquire the lock. In applications with coros needing frequent -scheduling a nonzero value will reduce the `Lock` object's CPU overhead at the -expense of latency. -Methods: - - * `locked` No args. Returns `True` if locked. - * `release` No args. Releases the lock. - * `acquire` No args. Coro which pauses until the lock has been acquired. Use - by executing `await lock.acquire()`. - -###### [Contents](./PRIMITIVES.md#contents) - -## 3.3 Class Event - -This provides a way for one or more coros to pause until another one flags them -to continue. An `Event` object is instantiated and passed to all coros using -it. Coros waiting on the event issue `await event`. Execution pauses -until a coro issues `event.set()`. `event.clear()` must then be issued. An -optional data argument may be passed to `event.set()` and retrieved by -`event.value()`. - -In the usual case where a single coro is awaiting the event this can be done -immediately after it is received: - -```python -async def eventwait(event): - await event - event.clear() -``` - -The coro raising the event may need to check that it has been serviced: - -```python -async def foo(event): - while True: - # Acquire data from somewhere - while event.is_set(): - await asyncio.sleep(1) # Wait for coro to respond - event.set() -``` - -If multiple coros are to wait on a single event, consider using a `Barrier` -object described below. This is because the coro which raised the event has no -way to determine whether all others have received it; determining when to clear -it down requires further synchronisation. One way to achieve this is with an -acknowledge event: - -```python -async def eventwait(event, ack_event): - await event - ack_event.set() -``` - -Example of this are in `event_test` and `ack_test` in asyntest.py. - -### 3.3.1 Definition - -Constructor: takes one optional integer argument. - * `delay_ms` default 0. While awaiting an event an internal flag is repeatedly - polled. Setting a finite polling interval reduces the task's CPU overhead at - the expense of increased latency. - -Synchronous Methods: - * `set` Initiates the event. Optional arg `data`: may be of any type, - sets the event's value. Default `None`. May be called in an interrupt context. - * `clear` No args. Clears the event, sets the value to `None`. - * `is_set` No args. Returns `True` if the event is set. - * `value` No args. Returns the value passed to `set`. - -Asynchronous Method: - * `wait` For CPython compatibility. Pause until event is set. The CPython - Event is not awaitable. - -The optional data value may be used to compensate for the latency in awaiting -the event by passing `loop.time()`. - -###### [Contents](./PRIMITIVES.md#contents) - -## 3.4 Class Barrier - -This enables multiple coros to rendezvous at a particular point. For example -producer and consumer coros can synchronise at a point where the producer has -data available and the consumer is ready to use it. At that point in time the -`Barrier` can optionally run a callback before releasing the barrier and -allowing all waiting coros to continue. - -Constructor. -Mandatory arg: -`participants` The number of coros which will use the barrier. -Optional args: -`func` Callback to run. Default `None`. -`args` Tuple of args for the callback. Default `()`. - -Public synchronous methods: - * `busy` No args. Returns `True` if at least one coro is waiting on the - barrier, or if at least one non-waiting coro has not triggered it. - * `trigger` No args. The barrier records that the coro has passed the critical - point. Returns "immediately". - -The callback can be a function or a coro. In most applications a function will -be used as this can be guaranteed to run to completion beore the barrier is -released. - -Participant coros issue `await my_barrier` whereupon execution pauses until all -other participants are also waiting on it. At this point any callback will run -and then each participant will re-commence execution. See `barrier_test` and -`semaphore_test` in `asyntest.py` for example usage. - -A special case of `Barrier` usage is where some coros are allowed to pass the -barrier, registering the fact that they have done so. At least one coro must -wait on the barrier. That coro will pause until all non-waiting coros have -passed the barrier, and all waiting coros have reached it. At that point all -waiting coros will resume. A non-waiting coro issues `barrier.trigger()` to -indicate that is has passed the critical point. - -This mechanism is used in the `Cancellable` and `NamedTask` classes to register -the fact that a coro has responded to cancellation. Using a non-waiting barrier -in a looping construct carries a fairly obvious hazard and is normally to be -avoided. - -###### [Contents](./PRIMITIVES.md#contents) - -## 3.5 Class Semaphore - -A semaphore limits the number of coros which can access a resource. It can be -used to limit the number of instances of a particular coro which can run -concurrently. It performs this using an access counter which is initialised by -the constructor and decremented each time a coro acquires the semaphore. - -Constructor: Optional arg `value` default 1. Number of permitted concurrent -accesses. - -Synchronous method: - * `release` No args. Increments the access counter. - -Asynchronous method: - * `acquire` No args. If the access counter is greater than 0, decrements it - and terminates. Otherwise waits for it to become greater than 0 before - decrementing it and terminating. - -The easiest way to use it is with a context manager: - -```python -async def foo(sema): - async with sema: - # Limited access here -``` - -There is a difference between a `Semaphore` and a `Lock`. A `Lock` -instance is owned by the coro which locked it: only that coro can release it. A -`Semaphore` can be released by any coro which acquired it. - -### 3.5.1 Class BoundedSemaphore - -This works identically to the `Semaphore` class except that if the `release` -method causes the access counter to exceed its initial value, a `ValueError` -is raised. - -###### [Contents](./PRIMITIVES.md#contents) - -## 3.6 Class Condition - -A `Condition` instance enables controlled access to a shared resource. In -typical applications a number of tasks wait for the resource to be available. -Once this occurs access can be controlled both by the number of tasks and by -means of a `Lock`. - -A task waiting on a `Condition` instance will pause until another task issues -`condition.notify(n)` or `condition.notify_all()`. If the number of tasks -waiting on the condition exceeds `n`, only `n` tasks will resume. A `Condition` -instance has a `Lock` as a member. A task will only resume when it has acquired -the lock. User code may release the lock as required by the application logic. - -Typical use of the class is in a synchronous context manager: - -```python - with await cond: - cond.notify(2) # Notify 2 tasks -``` - -```python - with await cond: - await cond.wait() - # Has been notified and has access to the locked resource - # Resource has been unocked by context manager -``` -### 3.6.1 Definition - -Constructor: Optional arg `lock=None`. A `Lock` instance may be specified, -otherwise the `Condition` instantiates its own. - -Synchronous methods: - * `locked` No args. Returns the state of the `Lock` instance. - * `release` No args. Release the `Lock`. A `RuntimeError` will occur if the - `Lock` is not locked. - * `notify` Arg `n=1`. Notify `n` tasks. The `Lock` must be acquired before - issuing `notify` otherwise a `RuntimeError` will occur. - * `notify_all` No args. Notify all tasks. The `Lock` must be acquired before - issuing `notify_all` otherwise a `RuntimeError` will occur. - -Asynchronous methods: - * `acquire` No args. Pause until the `Lock` is acquired. - * `wait` No args. Await notification and the `Lock`. The `Lock` must be - acquired before issuing `wait` otherwise a `RuntimeError` will occur. The - sequence is as follows: - The `Lock` is released. - The task pauses until another task issues `notify`. - It continues to pause until the `Lock` has been re-acquired when execution - resumes. - * `wait_for` Arg: `predicate` a callback returning a `bool`. The task pauses - until a notification is received and an immediate test of `predicate()` - returns `True`. - -###### [Contents](./PRIMITIVES.md#contents) - -## 3.7 Class Gather - -This aims to replicate some of the functionality of `asyncio.gather` in a -'micro' form. The user creates a list of `Gatherable` tasks and then awaits a -`Gather` object. When the last task to complete terminates, this will return a -list of results returned by the tasks. Timeouts may be assigned to individual -tasks. - -```python -async def foo(n): - await asyncio.sleep(n) - return n * n - -async def bar(x, y, rats): # Example coro: note arg passing - await asyncio.sleep(1) - return x * y * rats - -gatherables = [asyn.Gatherable(foo, n) for n in range(4)] -gatherables.append(asyn.Gatherable(bar, 7, 8, rats=77)) -gatherables.append(asyn.Gatherable(rats, 0, timeout=5)) -res = await asyn.Gather(gatherables) -``` - -The result `res` is a 6 element list containing the result of each of the 6 -coros. These are ordered by the position of the coro in the `gatherables` list. -This is as per `asyncio.gather()`. - -See `asyntest.py` function `gather_test()`. - -### 3.7.1 Definition - -The `Gatherable` class has no user methods. The constructor takes a coro by -name followed by any positional or keyword arguments for the coro. If an arg -`timeout` is provided it should have an integer or float value: this is taken -to be the timeout for the coro in seconds. Note that timeout is subject to the -latency discussed in [Coroutines with timeouts](./TUTORIAL.md#44-coroutines-with-timeouts). -A way to reduce this is to use `asyn.sleep()` in such coros. - -The `Gather` class has no user methods. The constructor takes one mandatory -arg: a list of `Gatherable` instances. - -`Gather` instances are awaitable. An `await` on an instance will terminate when -the last member task completes or times out. It returns a list whose length -matches the length of the list of `Gatherable` instances. Each element contains -the return value of the corresponding `Gatherable` instance. Each return value -may be of any type. - -### 3.7.2 Use with timeouts and cancellation - -The following complete example illustrates the use of `Gather` with tasks which -are subject to cancellation or timeout. - -```python -import uasyncio as asyncio -import asyn - -async def barking(n): - print('Start normal coro barking()') - for _ in range(6): - await asyncio.sleep(1) - print('Done barking.') - return 2 * n - -async def foo(n): - print('Start timeout coro foo()') - try: - while True: - await asyncio.sleep(1) - n += 1 - except asyncio.TimeoutError: - print('foo timeout.') - return n - -@asyn.cancellable -async def bar(n): - print('Start cancellable bar()') - try: - while True: - await asyncio.sleep(1) - n += 1 - except asyn.StopTask: - print('bar stopped.') - return n - -async def do_cancel(): - await asyncio.sleep(5.5) - await asyn.Cancellable.cancel_all() - -async def main(loop): - bar_task = asyn.Cancellable(bar, 70) # Note args here - gatherables = [asyn.Gatherable(barking, 21), - asyn.Gatherable(foo, 10, timeout=7.5), - asyn.Gatherable(bar_task)] - loop.create_task(do_cancel()) - res = await asyn.Gather(gatherables) - print('Result: ', res) # Expect [42, 17, 75] - -loop = asyncio.get_event_loop() -loop.run_until_complete(main(loop)) -``` - -###### [Contents](./PRIMITIVES.md#contents) - -# 4. Task Cancellation - -All current `uasyncio` versions have a `cancel(coro)` function. This works by -throwing an exception to the coro in a special way: cancellation is deferred -until the coro is next scheduled. This mechanism works with nested coros. - -There is a limitation with official `uasyncio` V2.0. In this version a coro -which is waiting on a `sleep()` or `sleep_ms()` or pending I/O will not get the -exception until it is next scheduled. This means that cancellation can take a -long time: there is often a need to be able to verify when this has occurred. - -This problem can now be circumvented in two ways both involving running -unofficial code. The solutions fix the problem by ensuring that the cancelled -coro is scheduled promptly. Assuming `my_coro` is coded normally the following -will ensure that cancellation is complete, even if `my_coro` is paused at the -time of cancellation: -```python -my_coro_instance = my_coro() -loop.add_task(my_coro_instance) -# Do something -asyncio.cancel(my_coro_instance) -await asyncio.sleep(0) -# The task is now cancelled -``` -The unofficial solutions are: - * To run the `fast_io` version of `uasyncio` presented her, with official - MicroPython firmware. - * To run [Paul Sokolovsky's Pycopy firmware fork](https://github.com/pfalcon/pycopy) - plus `uasyncio` V2.4 from - [Paul Sokolovsky's library fork](https://github.com/pfalcon/micropython-lib) - -The following describes workrounds for those wishing to run official code (for -example the current realease build which includes `uasyncio` V2.0). There is -usually a need to establish when cancellation has occured: the classes and -decorators described below facilitate this. - -If a coro issues `await uasyncio.sleep(secs)` or `await uasyncio.sleep_ms(ms)` -scheduling will not occur until the time has elapsed. This introduces latency -into cancellation which matters in some use-cases. Other potential sources of -latency take the form of slow code. `uasyncio` V2.0 has no mechanism for -verifying when cancellation has actually occurred. The `asyn.py` library -provides solutions in the form of two classes. - -These are `Cancellable` and `NamedTask`. The `Cancellable` class allows the -creation of named groups of tasks which may be cancelled as a group; this -awaits completion of cancellation of all tasks in the group. - -The `NamedTask` class enables a task to be associated with a user supplied -name, enabling it to be cancelled and its status checked. Cancellation -optionally awaits confirmation of completion. - -For cases where cancellation latency is of concern `asyn.py` offers a `sleep` -function which provides a delay with reduced latency. - -## 4.1 Coro sleep - -Pause for a period as per `uasyncio.sleep` but with reduced exception handling -latency. - -The asynchronous `sleep` function takes two args: - * `t` Mandatory. Time in seconds. May be integer or float. - * `granularity` Optional integer >= 0, units ms. Default 100ms. Defines the - maximum latency. Small values reduce latency at cost of increased scheduler - workload. - -This repeatedly issues `uasyncio.sleep_ms(t)` where t <= `granularity`. - -## 4.2 Class Cancellable - -This class provides for cancellation of one or more tasks where it is necesary -to await confirmation that cancellation is complete. `Cancellable` instances -are anonymous coros which are members of a named group. They are capable of -being cancelled as a group. A typical use-case might take this form: - -```python -async def comms(): # Perform some communications task - while True: - await initialise_link() - try: - await do_communications() # Launches Cancellable tasks - except CommsError: - await asyn.Cancellable.cancel_all() - # All sub-tasks are now known to be stopped. They can be re-started - # with known initial state on next pass. -``` - -A `Cancellable` task is declared with the `@cancellable` decorator: - -```python -@asyn.cancellable -async def print_nums(num): - while True: - print(num) - num += 1 - await sleep(1) # asyn.sleep() allows fast response to exception -``` - -Positional or keyword arguments for the task are passed to the `Cancellable` -constructor as below. Note that the coro is passed not using function call -syntax. `Cancellable` tasks may be awaited or placed on the event loop: - -```python -await asyn.Cancellable(print_nums, 5) # single arg to print_nums. -loop = asyncio.get_event_loop() -loop.create_task(asyn.Cancellable(print_nums, 42)()) # Note () syntax. -``` -**NOTE** A coro declared with `@asyn.cancellable` must only be launched using -the above syntax options. Treating it as a conventional coro will result in -`tuple index out of range` errors or other failures. - -The following will cancel any tasks still running, pausing until cancellation -is complete: - -```python -await asyn.Cancellable.cancel_all() -``` - -Constructor mandatory args: - * `task` A coro passed by name i.e. not using function call syntax. - -Constructor optional positional args: - * Any further positional args are passed to the coro. - -Constructor optional keyword args: - * `group` Any Python object, typically integer or string. Default 0. See - Groups below. - * Further keyword args are passed to the coro. - -Public class method: - * `cancel_all` Asynchronous. - Optional args `group` default 0, `nowait` default `False`. - The `nowait` arg is for use by the `NamedTask` derived class. The default - value is assumed below. - The method cancels all instances in the specified group and awaits completion. - See Groups below. - The `cancel_all` method will complete when all `Cancellable` instances have - been cancelled or terminated naturally before `cancel_all` was launched. - Each coro will receive a `StopTask` exception when it is next scheduled. If - the coro is written using the `@cancellable` decorator this is handled - automatically. - It is possible to trap the `StopTask` exception: see 'Custom cleanup' below. - -Public bound method: - * `__call__` This returns the coro and is used to schedule the task using the - event loop `create_task()` method using function call syntax. - -The `asyn.StopTask` exception is an alias for `usayncio.CancelledError`. In my -view the name is more descriptive of its function. - -A complete minimal, example: -```python -import uasyncio as asyncio -import asyn - -@asyn.cancellable -async def print_nums(num): - while True: - print(num) - num += 1 - await asyn.sleep(1) # asyn.sleep() allows fast response to exception - -async def main(loop): - loop.create_task(asyn.Cancellable(print_nums, 42)()) # Note () syntax - await asyncio.sleep(5) - await asyn.Cancellable.cancel_all() - print('Task cancelled: delay 3 secs to prove it.') - await asyncio.sleep(3) - -loop = asyncio.get_event_loop() -loop.run_until_complete(main(loop)) -``` - -### 4.2.1 Groups - -`Cancellable` tasks may be assigned to groups, identified by a user supplied -Python object, typically an integer or string. By default tasks are assigned to -group 0. The `cancel_all` class method cancels all tasks in the specified -group. The 0 default ensures that this facility can be ignored if not required, -with `cancel_all` cancelling all `Cancellable` tasks. - -### 4.2.2 Custom cleanup - -A task created with the `cancellable` decorator can intercept the `StopTask` -exception to perform custom cleanup operations. This may be done as below: -```python -@asyn.cancellable -async def foo(): - while True: - try: - await sleep(1) # Main body of task - except asyn.StopTask: - # perform custom cleanup - return # Respond by quitting -``` -The following example returns `True` if it ends normally or `False` if -cancelled. -```python -@asyn.cancellable -async def bar(): - try: - await sleep(1) # Main body of task - except asyn.StopTask: - return False - else: - return True -``` -A complete minimal example: -```python -import uasyncio as asyncio -import asyn - -@asyn.cancellable -async def print_nums(num): - try: - while True: - print(num) - num += 1 - await asyn.sleep(1) # asyn.sleep() allows fast response to exception - except asyn.StopTask: - print('print_nums was cancelled') - -async def main(loop): - loop.create_task(asyn.Cancellable(print_nums, 42)()) # Note () syntax - await asyncio.sleep(5) - await asyn.Cancellable.cancel_all() - print('Task cancelled: delay 3 secs to prove it.') - await asyncio.sleep(3) - -loop = asyncio.get_event_loop() -loop.run_until_complete(main(loop)) -``` - -###### [Contents](./PRIMITIVES.md#contents) - -## 4.3 Class NamedTask - -A `NamedTask` instance is associated with a user-defined name such that the -name may outlive the task: a coro may end but the class enables its state to be -checked. It is a subclass of `Cancellable` and its constructor disallows -duplicate names: each instance of a coro must be assigned a unique name. - -A `NamedTask` coro is defined with the `@cancellable` decorator. - -```python -@cancellable -async def foo(arg1, arg2): - await asyn.sleep(1) - print('Task foo has ended.', arg1, arg2) -``` - -The `NamedTask` constructor takes the name, the coro, plus any user positional -or keyword args. The resultant instance can be scheduled in the usual ways: - -```python -await asyn.NamedTask('my foo', foo, 1, 2) # Pause until complete or killed -loop = asyncio.get_event_loop() # Or schedule and continue: -loop.create_task(asyn.NamedTask('my nums', foo, 10, 11)()) # Note () syntax. -``` - -Cancellation is performed with: - -```python -await asyn.NamedTask.cancel('my foo') -``` - -When cancelling a task there is no need to check if the task is still running: -if it has already completed, cancellation will have no effect. - -NamedTask Constructor. -Mandatory args: - * `name` Names may be any immutable type capable of being a dictionary index - e.g. integer or string. A `ValueError` will be raised if the name is already - assigned by a running coro. If multiple instances of a coro are to run - concurrently, each should be assigned a different name. - * `task` A coro passed by name i.e. not using function call syntax. - - Optional positional args: - * Any further positional args are passed to the coro. - - Optional keyword only args: - * `barrier` A `Barrier` instance may be passed. See below. - * Further keyword args are passed to the coro. - -Public class methods: - * `cancel` Asynchronous. - Mandatory arg: a coro name. - Optional boolean arg `nowait` default `True` - By default it will return soon. If `nowait` is `False` it will pause until the - coro has completed cancellation. - The named coro will receive a `StopTask` exception the next time it is - scheduled. If the `@namedtask` decorator is used this is transparent to the - user but the exception may be trapped for custom cleanup (see below). - `cancel` will return `True` if the coro was cancelled. It will return `False` - if the coro has already ended or been cancelled. - * `is_running` Synchronous. Arg: A coro name. Returns `True` if coro is queued - for scheduling, `False` if it has ended or been cancelled. - -Public bound method: - * `__call__` This returns the coro and is used to schedule the task using the - event loop `create_task()` method using function call syntax. - -### 4.3.1 Latency and Barrier objects - -It is possible to get confirmation of cancellation of an arbitrary set of -`NamedTask` instances by instantiating a `Barrier` and passing it to the -constructor of each member. This enables more complex synchronisation cases -than the normal method of using a group of `Cancellable` tasks. The approach is -described below. - -If a `Barrier` instance is passed to the `NamedTask` constructor, a task -performing cancellation can pause until a set of cancelled tasks have -terminated. The `Barrier` is constructed with the number of dependent tasks -plus one (the task which is to wait on it). It is passed to the constructor of -each dependent task and the cancelling task waits on it after cancelling all -dependent tasks. Each task being cancelled terminates 'immediately' subject -to latency. - -See examples in `cantest.py` e.g. `cancel_test2()`. - -### 4.3.2 Custom cleanup - -A coroutine to be used as a `NamedTask` can intercept the `StopTask` exception -if necessary. This might be done for cleanup or to return a 'cancelled' status. -The coro should have the following form: - -```python -@asyn.cancellable -async def foo(): - try: - await asyncio.sleep(1) # User code here - except asyn.StopTask: - return False # Cleanup code - else: - return True # Normal exit -``` - -###### [Contents](./PRIMITIVES.md#contents) diff --git a/README.md b/README.md index db305bb..31e643c 100644 --- a/README.md +++ b/README.md @@ -1,237 +1,24 @@ -# 1. Asynchronous programming in MicroPython +# Asynchronous programming in MicroPython CPython supports asynchronous programming via the `asyncio` library. -MicroPython provides `uasyncio` which is a subset of this, optimised for small +MicroPython provides `asyncio` which is a subset of this, optimised for small code size and high performance on bare metal targets. This repository provides -documentation, tutorial material and code to aid in its effective use. It also -contains an optional `fast_io` variant of `uasyncio`. +documentation, tutorial material and code to aid in its effective use. -Damien has completely rewritten `uasyncio`. Its release is likely to be -imminent, see -[PR5332](https://github.com/micropython/micropython/pull/5332) and [below](./README.md#31-the-new-version). +# asyncio version 3 -## The fast_io variant +Damien has completely rewritten `asyncio` which was released as V3.0. This is +incorporated in all recent firmware builds. The resources in this repo may be found in the +`v3` directory. These include a tutorial, synchronisation primitives, drivers, +applications and demos. -This comprises two parts. - 1. The [fast_io](./FASTPOLL.md) version of `uasyncio` is a "drop in" - replacement for the official version providing bug fixes, additional - functionality and, in certain respects, higher performance. - 2. An optional extension module enabling the [fast_io](./FASTPOLL.md) version - to run with very low power draw. This is Pyboard-only including Pyboard D. +# Concurrency -## Resources for users of all versions +Other documents provide hints on asynchronous programming techniques including +threading and multi-core coding. - * [A tutorial](./TUTORIAL.md) An introductory tutorial on asynchronous - programming and the use of the `uasyncio` library. - * [Asynchronous device drivers](./DRIVERS.md). A module providing drivers for - devices such as switches and pushbuttons. - * [Synchronisation primitives](./PRIMITIVES.md). Provides commonly used - synchronisation primitives plus an API for task cancellation and monitoring. - * [A driver for an IR remote control](./nec_ir/README.md) This is intended as - an example of an asynchronous device driver. It decodes signals received from - infra red remote controls using the popular NEC protocol. - * [A driver for the HTU21D](./htu21d/README.md) temperature and humidity - sensor. This is intended to be portable across platforms and is another - example of an asynchronous device driver. - * [A driver for character LCD displays](./HD44780/README.md). A simple - asynchronous interface to displays based on the Hitachi HD44780 chip. - * [A driver for GPS modules](./gps/README.md) Runs a background task to read - and decode NMEA sentences, providing constantly updated position, course, - altitude and time/date information. - * [Communication using I2C slave mode.](./i2c/README.md) Enables a Pyboard to - to communicate with another MicroPython device using stream I/O. The Pyboard - achieves bidirectional communication with targets such as an ESP8266. - * [Communication between devices](./syncom_as/README.md) Enables MicroPython - boards to communicate without using a UART. This is hardware agnostic but - slower than the I2C version. - * [Under the hood](./UNDER_THE_HOOD.md) A guide to help understand the - `uasyncio` code. For scheduler geeks and those wishing to modify `uasyncio`. - -# 2. Version and installation of uasyncio +### [Go to V3 docs](./v3/README.md) -Paul Sokolovsky (`uasyncio` author) has released versions of `uasyncio` which -supercede the official version. His latest version is that on PyPi and requires -his [Pycopy](https://github.com/pfalcon/micropython) fork of MicroPython -firmware. His `uasyncio` code may also be found in -[his fork of micropython-lib](https://github.com/pfalcon/micropython-lib). +# uasyncio version 2 -I support only the official build of MicroPython. The library code guaranteed -to work with this build is in [micropython-lib](https://github.com/micropython/micropython-lib). -Most of the resources in here should work with Paul's forks (most work with -CPython). - -Most documentation and code in this repository assumes the current official -version of `uasyncio`. This is V2.0 from -[micropython-lib](https://github.com/micropython/micropython-lib). -It is recommended to use MicroPython firmware V1.11 or later. On many platforms -`uasyncio` is incorporated and no installation is required. - -Some examples illustrate features of the `fast_io` fork and therefore require -this version. - -See [tutorial](./TUTORIAL.md#installing-uasyncio-on-bare-metal) for -installation instructions where `uasyncio` is not pre-installed. - -# 3. uasyncio development state - -## 3.1 The new version - -This complete rewrite of `uasyncio` supports CPython 3.8 syntax. A design aim -is that it should be be a compatible subset of `asyncio`. Many applications -using the coding style advocated in the tutorial will work unchanged. The -following features will involve minor changes to application code: - - * Task cancellation: `cancel` is now a method of a `Task` instance. - * Event loop methods: `call_at`, `call_later`, `call_later_ms` and - `call_soon` are no longer supported. In CPython docs these are - [lightly deprecated](https://docs.python.org/3/library/asyncio-eventloop.html#preface) - in application code; there are simple workrounds. - * `yield` in coroutines should be replaced by `await asyncio.sleep_ms(0)`: - this is in accord with CPython where `yield` will produce a syntax error. - * Awaitable classes: currently under discussion. The `__iter__` method works - but `yield` should be replaced by `await asyncio.sleep_ms(0)`. As yet I have - found no way to write an awaitable class compatible with the new `uasyncio` - and which does not throw syntax errors under CPython 3.8/`asyncio`. - -### 3.1.1 Implications for this repository - -It is planned to retain V2 under a different name. The new version fixes bugs -which have been outstanding for a long time. In my view V2 is best viewed as -deprecated. I will retain V2-specific code and docs in a separate directory, -with the rest of this repo being adapted for the new version. - -#### 3.1.1.1 Tutorial - -This requires only minor changes. - -#### 3.1.1.2 Fast I/O - -The `fast_io` fork is incompatible and will be relegated to the V2 directory. - -The new version's design greatly simplifies the implementation of fast I/O: -I therefore hope the new `uasyncio` will include it. The other principal aims -were to provide workrounds for bugs now fixed. If `uasyncio` includes fast I/O -there is no reason to fork the new version; other `fast_io` features will be -lost unless Damien sees fit to implement them. The low priority task option is -little used and arguably is ill-conceived: I will not be advocating for its -inclusion. - -#### 3.1.1.3 Synchronisation Primitives - -The CPython `asyncio` library supports these synchronisation primitives: - * `Lock` - already incorporated in new `uasyncio`. - * `Event` - already incorporated. - * `gather` - already incorporated. - * `Semaphore` and `BoundedSemaphore`. My classes work under new version. - * `Condition`. Works under new version. - * `Queue`. This was implemented by Paul Sokolvsky in `uasyncio.queues`. - -Incorporating these will produce more efficient implementations; my solutions -were designed to work with stock `uasyncio` V2. - -The `Event` class in `asyn.py` provides a nonstandard option to supply a data -value to the `.set` method and to retrieve this with `.value`. It is also an -awaitable class. I will support these by subclassing the native `Event`. - -The following work under new and old versions: - * `Barrier` (now adapted). - * `Delay_ms` (this and the following in aswitch.py) - * `Switch` - * `Pushbutton` - -The following were workrounds for bugs and omissions in V2 which are now fixed. -They will be removed. - * The cancellation decorators and classes (cancellation works as per CPython). - * The nonstandard support for `gather` (now properly supported). - -## 3.2 The current version V2.0 - -These notes are intended for users familiar with `asyncio` under CPython. - -The MicroPython language is based on CPython 3.4. The `uasyncio` library -supports a subset of the CPython 3.4 `asyncio` library with some V3.5 -extensions. In addition there are non-standard extensions to optimise services -such as millisecond level timing and task cancellation. Its design focus is on -high performance and scheduling is performed without RAM allocation. - -The `uasyncio` library supports the following Python 3.5 features: - - * `async def` and `await` syntax. - * Awaitable classes (using `__iter__` rather than `__await__`). - * Asynchronous context managers. - * Asynchronous iterators. - * Event loop methods `call_soon` and `call_later`. - * `sleep(seconds)`. - -It supports millisecond level timing with the following: - - * Event loop method `call_later_ms` - * uasyncio `sleep_ms(time)` - -`uasyncio` V2 supports coroutine timeouts and cancellation. - - * `wait_for(coro, t_secs)` runs `coro` with a timeout. - * `cancel(coro)` tags `coro` for cancellation when it is next scheduled. - -Classes `Task` and `Future` are not supported. - -## 3.2.1 Asynchronous I/O - -Asynchronous I/O (`StreamReader` and `StreamWriter` classes) support devices -with streaming drivers, such as UARTs and sockets. It is now possible to write -streaming device drivers in Python. - -## 3.2.2 Time values - -For timing asyncio uses floating point values of seconds. The `uasyncio.sleep` -method accepts floats (including sub-second values) or integers. Note that in -MicroPython the use of floats implies RAM allocation which incurs a performance -penalty. The design of `uasyncio` enables allocation-free scheduling. In -applications where performance is an issue, integers should be used and the -millisecond level functions (with integer arguments) employed where necessary. - -The `loop.time` method returns an integer number of milliseconds whereas -CPython returns a floating point number of seconds. `call_at` follows the -same convention. - -# 4. The "fast_io" version. - -Official `uasyncio` suffers from high levels of latency when scheduling I/O in -typical applications. It also has an issue which can cause bidirectional -devices such as UART's to block. The `fast_io` version fixes the bug. It also -provides a facility for reducing I/O latency which can substantially improve -the performance of stream I/O drivers. It provides other features aimed at -providing greater control over scheduling behaviour. - -To take advantage of the reduced latency device drivers should be written to -employ stream I/O. To operate at low latency they are simply run under the -`fast_io` version. The [tutorial](./TUTORIAL.md#64-writing-streaming-device-drivers) -has details of how to write streaming drivers. - -The current `fast_io` version 0.24 fixes an issue with task cancellation and -timeouts. In `uasyncio` version 2.0, where a coroutine is waiting on a -`sleep()` or on I/O, a timeout or cancellation is deferred until the coroutine -is next scheduled. This introduces uncertainty into when the coroutine is -stopped. This issue is also addressed in Paul Sokolovsky's fork. - -## 4.1 A Pyboard-only low power module - -This is documented [here](./lowpower/README.md). In essence a Python file is -placed on the device which configures the `fast_io` version of `uasyncio` to -reduce power consumption at times when it is not busy. This provides a means of -using `uasyncio` in battery powered projects. - -# 5. The asyn.py library - -This library ([docs](./PRIMITIVES.md)) provides 'micro' implementations of the -`asyncio` synchronisation primitives. -[CPython docs](https://docs.python.org/3/library/asyncio-sync.html) - -It also supports a `Barrier` class to facilitate coroutine synchronisation. - -Coroutine cancellation is performed in an efficient manner in `uasyncio`. The -`asyn` library uses this, further enabling the cancelling coro to pause until -cancellation is complete. It also provides a means of checking the 'running' -status of individual coroutines. - -A lightweight implementation of `asyncio.gather` is provided. +This is obsolete: code and docs have been removed. diff --git a/TUTORIAL.md b/TUTORIAL.md deleted file mode 100644 index e0ba9ec..0000000 --- a/TUTORIAL.md +++ /dev/null @@ -1,2029 +0,0 @@ -# Application of uasyncio to hardware interfaces - -This tutorial is intended for users having varying levels of experience with -asyncio and includes a section for complete beginners. - -# Contents - - 0. [Introduction](./TUTORIAL.md#0-introduction) - 0.1 [Installing uasyncio on bare metal](./TUTORIAL.md#01-installing-uasyncio-on-bare-metal) - 1. [Cooperative scheduling](./TUTORIAL.md#1-cooperative-scheduling) - 1.1 [Modules](./TUTORIAL.md#11-modules) - 2. [uasyncio](./TUTORIAL.md#2-uasyncio) - 2.1 [Program structure: the event loop](./TUTORIAL.md#21-program-structure-the-event-loop) - 2.2 [Coroutines (coros)](./TUTORIAL.md#22-coroutines-coros) - 2.2.1 [Queueing a coro for scheduling](./TUTORIAL.md#221-queueing-a-coro-for-scheduling) - 2.2.2 [Running a callback function](./TUTORIAL.md#222-running-a-callback-function) - 2.2.3 [Notes](./TUTORIAL.md#223-notes) Coros as bound methods. Returning values. - 2.3 [Delays](./TUTORIAL.md#23-delays) - 3. [Synchronisation](./TUTORIAL.md#3-synchronisation) - 3.1 [Lock](./TUTORIAL.md#31-lock) - 3.1.1 [Locks and timeouts](./TUTORIAL.md#311-locks-and-timeouts) - 3.2 [Event](./TUTORIAL.md#32-event) - 3.2.1 [The event's value](./TUTORIAL.md#321-the-events-value) - 3.3 [Barrier](./TUTORIAL.md#33-barrier) - 3.4 [Semaphore](./TUTORIAL.md#34-semaphore) - 3.4.1 [BoundedSemaphore](./TUTORIAL.md#341-boundedsemaphore) - 3.5 [Queue](./TUTORIAL.md#35-queue) - 3.6 [Other synchronisation primitives](./TUTORIAL.md#36-other-synchronisation-primitives) - 4. [Designing classes for asyncio](./TUTORIAL.md#4-designing-classes-for-asyncio) - 4.1 [Awaitable classes](./TUTORIAL.md#41-awaitable-classes) - 4.1.1 [Use in context managers](./TUTORIAL.md#411-use-in-context-managers) - 4.1.2 [Awaiting a coro](./TUTORIAL.md#412-awaiting-a-coro) - 4.2 [Asynchronous iterators](./TUTORIAL.md#42-asynchronous-iterators) - 4.3 [Asynchronous context managers](./TUTORIAL.md#43-asynchronous-context-managers) - 5. [Exceptions timeouts and cancellation](./TUTORIAL.md#5-exceptions-timeouts-and-cancellation) - 5.1 [Exceptions](./TUTORIAL.md#51-exceptions) - 5.2 [Cancellation and Timeouts](./TUTORIAL.md#52-cancellation-and-timeouts) - 5.2.1 [Task cancellation](./TUTORIAL.md#521-task-cancellation) - 5.2.2 [Coroutines with timeouts](./TUTORIAL.md#522-coroutines-with-timeouts) - 6. [Interfacing hardware](./TUTORIAL.md#6-interfacing-hardware) - 6.1 [Timing issues](./TUTORIAL.md#61-timing-issues) - 6.2 [Polling hardware with a coroutine](./TUTORIAL.md#62-polling-hardware-with-a-coroutine) - 6.3 [Using the stream mechanism](./TUTORIAL.md#63-using-the-stream-mechanism) - 6.3.1 [A UART driver example](./TUTORIAL.md#631-a-uart-driver-example) - 6.4 [Writing streaming device drivers](./TUTORIAL.md#64-writing-streaming-device-drivers) - 6.5 [A complete example: aremote.py](./TUTORIAL.md#65-a-complete-example-aremotepy) - A driver for an IR remote control receiver. - 6.6 [Driver for HTU21D](./TUTORIAL.md#66-htu21d-environment-sensor) A - temperature and humidity sensor. - 7. [Hints and tips](./TUTORIAL.md#7-hints-and-tips) - 7.1 [Program hangs](./TUTORIAL.md#71-program-hangs) - 7.2 [uasyncio retains state](./TUTORIAL.md#72-uasyncio-retains-state) - 7.3 [Garbage Collection](./TUTORIAL.md#73-garbage-collection) - 7.4 [Testing](./TUTORIAL.md#74-testing) - 7.5 [A common error](./TUTORIAL.md#75-a-common-error) This can be hard to find. - 7.6 [Socket programming](./TUTORIAL.md#76-socket-programming) - 7.6.1 [WiFi issues](./TUTORIAL.md#761-wifi-issues) - 7.7 [Event loop constructor args](./TUTORIAL.md#77-event-loop-constructor-args) - 8. [Notes for beginners](./TUTORIAL.md#8-notes-for-beginners) - 8.1 [Problem 1: event loops](./TUTORIAL.md#81-problem-1:-event-loops) - 8.2 [Problem 2: blocking methods](./TUTORIAL.md#8-problem-2:-blocking-methods) - 8.3 [The uasyncio approach](./TUTORIAL.md#83-the-uasyncio-approach) - 8.4 [Scheduling in uasyncio](./TUTORIAL.md#84-scheduling-in-uasyncio) - 8.5 [Why cooperative rather than pre-emptive?](./TUTORIAL.md#85-why-cooperative-rather-than-pre-emptive) - 8.6 [Communication](./TUTORIAL.md#86-communication) - 8.7 [Polling](./TUTORIAL.md#87-polling) - -###### [Main README](./README.md) - -# 0. Introduction - -Most of this document assumes some familiarity with asynchronous programming. -For those new to it an introduction may be found -[in section 7](./TUTORIAL.md#8-notes-for-beginners). - -The MicroPython `uasyncio` library comprises a subset of Python's `asyncio` -library. It is designed for use on microcontrollers. As such it has a small RAM -footprint and fast context switching with zero RAM allocation. This document -describes its use with a focus on interfacing hardware devices. The aim is to -design drivers in such a way that the application continues to run while the -driver is awaiting a response from the hardware. The application remains -responsive to events and to user interaction. - -Another major application area for asyncio is in network programming: many -guides to this may be found online. - -Note that MicroPython is based on Python 3.4 with minimal Python 3.5 additions. -Except where detailed below, `asyncio` features of versions >3.4 are -unsupported. As stated above it is a subset; this document identifies supported -features. - -This tutorial aims to present a consistent programming style compatible with -CPython V3.5 and above. - -## 0.1 Installing uasyncio on bare metal - -It is recommended to use MicroPython firmware V1.11 or later. On many platforms -no installation is necessary as `uasyncio` is compiled into the build. Test by -issuing -```python -import uasyncio -``` -at the REPL. - -The following instructions cover cases where modules are not pre-installed. The -`queues` and `synchro` modules are optional, but are required to run all the -examples below. - -#### Hardware with internet connectivity - -On hardware with an internet connection and running firmware V1.11 or greater -installation may be done using `upip`, which is pre-installed. After ensuring -that the device is connected to your network issue: -```python -import upip -upip.install('micropython-uasyncio') -upip.install('micropython-uasyncio.synchro') -upip.install('micropython-uasyncio.queues') -``` -Error meesages from `upip` are not too helpful. If you get an obscure error, -double check your internet connection. - -#### Hardware without internet connectivity (micropip) - -On hardware which lacks an internet connection (such as a Pyboard V1.x) the -easiest way is to run `micropip.py` on a PC to install to a directory of your -choice, then to copy the resultant directory structure to the target hardware. -The `micropip.py` utility runs under Python 3.2 or above and runs under Linux, -Windows and OSX. It may be found -[here](https://github.com/peterhinch/micropython-samples/tree/master/micropip). - -Typical invocation: -```bash -$ micropip.py install -p ~/rats micropython-uasyncio -$ micropip.py install -p ~/rats micropython-uasyncio.synchro -$ micropip.py install -p ~/rats micropython-uasyncio.queues -``` - -#### Hardware without internet connectivity (copy source) - -If `micropip.py` is not to be used the files should be copied from source. The -following instructions describe copying the bare minimum of files to a target -device, also the case where `uasyncio` is to be frozen into a compiled build as -bytecode. For the latest release compatible with official firmware -files must be copied from the official -[micropython-lib](https://github.com/micropython/micropython-lib). - -Clone the library to a PC with -```bash -$ git clone https://github.com/micropython/micropython-lib.git -``` -On the target hardware create a `uasyncio` directory (optionally under a -directory `lib`) and copy the following files to it: - * `uasyncio/uasyncio/__init__.py` - * `uasyncio.core/uasyncio/core.py` - * `uasyncio.synchro/uasyncio/synchro.py` - * `uasyncio.queues/uasyncio/queues.py` - -The `uasyncio` modules may be frozen as bytecode in the usual way, by placing -the `uasyncio` directory and its contents in the port's `modules` directory and -rebuilding. - -###### [Main README](./README.md) - -# 1. Cooperative scheduling - -The technique of cooperative multi-tasking is widely used in embedded systems. -It offers lower overheads than pre-emptive scheduling and avoids many of the -pitfalls associated with truly asynchronous threads of execution. - -###### [Contents](./TUTORIAL.md#contents) - -## 1.1 Modules - -The following modules are provided which may be copied to the target hardware. - -**Libraries** - - 1. [asyn.py](./asyn.py) Provides synchronisation primitives `Lock`, `Event`, - `Barrier`, `Semaphore`, `BoundedSemaphore`, `Condition` and `gather`. Provides - support for task cancellation via `NamedTask` and `Cancellable` classes. - 2. [aswitch.py](./aswitch.py) Provides classes for interfacing switches and - pushbuttons and also a software retriggerable delay object. Pushbuttons are a - generalisation of switches providing logical rather than physical status along - with double-clicked and long pressed events. - -**Demo Programs** - -The first two are the most immediately rewarding as they produce visible -results by accessing Pyboard hardware. - - 1. [aledflash.py](./aledflash.py) Flashes the four Pyboard LEDs asynchronously - for 10s. The simplest uasyncio demo. Import it to run. - 2. [apoll.py](./apoll.py) A device driver for the Pyboard accelerometer. - Demonstrates the use of a coroutine to poll a device. Runs for 20s. Import it - to run. Requires a Pyboard V1.x. - 3. [astests.py](./astests.py) Test/demonstration programs for the - [aswitch](./aswitch) module. - 4. [asyn_demos.py](./asyn_demos.py) Simple task cancellation demos. - 5. [roundrobin.py](./roundrobin.py) Demo of round-robin scheduling. Also a - benchmark of scheduling performance. - 6. [awaitable.py](./awaitable.py) Demo of an awaitable class. One way of - implementing a device driver which polls an interface. - 7. [chain.py](./chain.py) Copied from the Python docs. Demo of chaining - coroutines. - 8. [aqtest.py](./aqtest.py) Demo of uasyncio `Queue` class. - 9. [aremote.py](./aremote.py) Example device driver for NEC protocol IR remote - control. - 10. [auart.py](./auart.py) Demo of streaming I/O via a Pyboard UART. - 11. [auart_hd.py](./auart_hd.py) Use of the Pyboard UART to communicate with a - device using a half-duplex protocol. Suits devices such as those using the - 'AT' modem command set. - 12. [iorw.py](./iorw.py) Demo of a read/write device driver using the stream - I/O mechanism. - -**Test Programs** - - 1. [asyntest.py](./asyntest.py) Tests for the synchronisation primitives in - [asyn.py](./asyn.py). - 2. [cantest.py](./cantest.py) Task cancellation tests. - -**Utility** - - 1. [check_async_code.py](./check_async_code.py) A Python3 utility to locate a - particular coding error which can be hard to find. See - [para 7.5](./TUTORIAL.md#75-a-common-error). - -**Benchmarks** - -The `benchmarks` directory contains scripts to test and characterise the -uasyncio scheduler. See [this doc](./FASTPOLL.md). - -###### [Contents](./TUTORIAL.md#contents) - -# 2. uasyncio - -The asyncio concept is of cooperative multi-tasking based on coroutines, -referred to in this document as coros or tasks. - -###### [Contents](./TUTORIAL.md#contents) - -## 2.1 Program structure: the event loop - -Consider the following example: - -```python -import uasyncio as asyncio -async def bar(): - count = 0 - while True: - count += 1 - print(count) - await asyncio.sleep(1) # Pause 1s - -loop = asyncio.get_event_loop() -loop.create_task(bar()) # Schedule ASAP -loop.run_forever() -``` - -Program execution proceeds normally until the call to `loop.run_forever`. At -this point execution is controlled by the scheduler. A line after -`loop.run_forever` would never be executed. The scheduler runs `bar` -because this has been placed on the scheduler's queue by `loop.create_task`. -In this trivial example there is only one coro: `bar`. If there were others, -the scheduler would schedule them in periods when `bar` was paused. - -Most embedded applications have an event loop which runs continuously. The event -loop can also be started in a way which permits termination, by using the event -loop's `run_until_complete` method; this is mainly of use in testing. Examples -may be found in the [astests.py](./astests.py) module. - -The event loop instance is a singleton, instantiated by a program's first call -to `asyncio.get_event_loop()`. This takes two optional integer args being the -lengths of the two coro queues. Typically both will have the same value being -at least the number of concurrent coros in the application. The default of 16 -is usually sufficient. If using non-default values see -[Event loop constructor args](./TUTORIAL.md#77-event-loop-constructor-args). - -If a coro needs to call an event loop method (usually `create_task`), calling -`asyncio.get_event_loop()` (without args) will efficiently return it. - -###### [Contents](./TUTORIAL.md#contents) - -## 2.2 Coroutines (coros) - -A coro is instantiated as follows: - -```python -async def foo(delay_secs): - await asyncio.sleep(delay_secs) - print('Hello') -``` - -A coro can allow other coroutines to run by means of the `await coro` -statement. A coro must contain at least one `await` statement. This causes -`coro` to run to completion before execution passes to the next instruction. -Consider these lines of code: - -```python -await asyncio.sleep(delay_secs) -await asyncio.sleep(0) -``` - -The first causes the code to pause for the duration of the delay, with other -coros being scheduled for the duration. A delay of 0 causes any pending coros -to be scheduled in round-robin fashion before the following line is run. See -the `roundrobin.py` example. - -###### [Contents](./TUTORIAL.md#contents) - -### 2.2.1 Queueing a coro for scheduling - - * `EventLoop.create_task` Arg: the coro to run. The scheduler queues the - coro to run ASAP. The `create_task` call returns immediately. The coro - arg is specified with function call syntax with any required arguments passed. - * `EventLoop.run_until_complete` Arg: the coro to run. The scheduler queues - the coro to run ASAP. The coro arg is specified with function call syntax with - any required arguments passed. The `run_until_complete` call returns when - the coro terminates: this method provides a way of quitting the scheduler. - * `await` Arg: the coro to run, specified with function call syntax. Starts - the coro ASAP. The awaiting coro blocks until the awaited one has run to - completion. - -The above are compatible with CPython. Additional uasyncio methods are -discussed in 2.2.3 below. - -###### [Contents](./TUTORIAL.md#contents) - -### 2.2.2 Running a callback function - -Callbacks should be Python functions designed to complete in a short period of -time. This is because coroutines will have no opportunity to run for the -duration. - -The following `EventLoop` methods schedule callbacks: - - 1. `call_soon` Call as soon as possible. Args: `callback` the callback to - run, `*args` any positional args may follow separated by commas. - 2. `call_later` Call after a delay in secs. Args: `delay`, `callback`, - `*args` - 3. `call_later_ms` Call after a delay in ms. Args: `delay`, `callback`, - `*args`. - -```python -loop = asyncio.get_event_loop() -loop.call_soon(foo, 5) # Schedule callback 'foo' ASAP with an arg of 5. -loop.call_later(2, foo, 5) # Schedule after 2 seconds. -loop.call_later_ms(50, foo, 5) # Schedule after 50ms. -loop.run_forever() -``` - -###### [Contents](./TUTORIAL.md#contents) - -### 2.2.3 Notes - -A coro can contain a `return` statement with arbitrary return values. To -retrieve them issue: - -```python -result = await my_coro() -``` - -Coros may be bound methods. A coro must contain at least one `await` statement. - -###### [Contents](./TUTORIAL.md#contents) - -## 2.3 Delays - -Where a delay is required in a coro there are two options. For longer delays and -those where the duration need not be precise, the following should be used: - -```python -async def foo(delay_secs, delay_ms): - await asyncio.sleep(delay_secs) - print('Hello') - await asyncio.sleep_ms(delay_ms) -``` - -While these delays are in progress the scheduler will schedule other coros. -This is generally highly desirable, but it does introduce uncertainty in the -timing as the calling routine will only be rescheduled when the one running at -the appropriate time has yielded. The amount of latency depends on the design -of the application, but is likely to be on the order of tens or hundreds of ms; -this is discussed further in [Section 6](./TUTORIAL.md#6-interfacing-hardware). - -Very precise delays may be issued by using the `utime` functions `sleep_ms` -and `sleep_us`. These are best suited for short delays as the scheduler will -be unable to schedule other coros while the delay is in progress. - -###### [Contents](./TUTORIAL.md#contents) - -# 3 Synchronisation - -There is often a need to provide synchronisation between coros. A common -example is to avoid what are known as "race conditions" where multiple coros -compete to access a single resource. An example is provided in the -[astests.py](./astests.py) program and discussed in [the docs](./DRIVERS.md). -Another hazard is the "deadly embrace" where two coros each wait on the other's -completion. - -In simple applications communication may be achieved with global flags or bound -variables. A more elegant approach is to use synchronisation primitives. The -module -[asyn.py](https://github.com/peterhinch/micropython-async/blob/master/asyn.py) -offers "micro" implementations of `Event`, `Barrier`, `Semaphore` and -`Condition` primitives. These are for use only with asyncio. They are not -thread safe and should not be used with the `_thread` module or from an -interrupt handler except where mentioned. A `Lock` primitive is provided which -is an alternative to the official implementation. - -Another synchronisation issue arises with producer and consumer coros. The -producer generates data which the consumer uses. Asyncio provides the `Queue` -object. The producer puts data onto the queue while the consumer waits for its -arrival (with other coros getting scheduled for the duration). The `Queue` -guarantees that items are removed in the order in which they were received. -Alternatively a `Barrier` instance can be used if the producer must wait -until the consumer is ready to access the data. - -The following provides a brief overview of the primitives. Full documentation -may be found [here](./PRIMITIVES.md). - -###### [Contents](./TUTORIAL.md#contents) - -## 3.1 Lock - -This describes the use of the official `Lock` primitive. - -This guarantees unique access to a shared resource. In the following code -sample a `Lock` instance `lock` has been created and is passed to all coros -wishing to access the shared resource. Each coro attempts to acquire the lock, -pausing execution until it succeeds. - -```python -import uasyncio as asyncio -from uasyncio.synchro import Lock - -async def task(i, lock): - while 1: - await lock.acquire() - print("Acquired lock in task", i) - await asyncio.sleep(0.5) - lock.release() - -async def killer(): - await asyncio.sleep(10) - -loop = asyncio.get_event_loop() - -lock = Lock() # The global Lock instance - -loop.create_task(task(1, lock)) -loop.create_task(task(2, lock)) -loop.create_task(task(3, lock)) - -loop.run_until_complete(killer()) # Run for 10s -``` - -### 3.1.1 Locks and timeouts - -At time of writing (5th Jan 2018) the official `Lock` class is not complete. -If a coro is subject to a [timeout](./TUTORIAL.md#522-coroutines-with-timeouts) -and the timeout is triggered while it is waiting on a lock, the timeout will be -ineffective. It will not receive the `TimeoutError` until it has acquired the -lock. The same observation applies to task cancellation. - -The module [asyn.py](./asyn.py) offers a `Lock` class which works in these -situations [see docs](./PRIMITIVES.md#32-class-lock). It is significantly less -efficient than the official class but supports additional interfaces as per the -CPython version including context manager usage. - -###### [Contents](./TUTORIAL.md#contents) - -## 3.2 Event - -This provides a way for one or more coros to pause until another flags them to -continue. An `Event` object is instantiated and made accessible to all coros -using it: - -```python -import asyn -event = asyn.Event() -``` - -Coros waiting on the event issue `await event` whereupon execution pauses until -another issues `event.set()`. [Full details.](./PRIMITIVES.md#33-class-event) - -This presents a problem if `event.set()` is issued in a looping construct; the -code must wait until the event has been accessed by all waiting coros before -setting it again. In the case where a single coro is awaiting the event this -can be achieved by the receiving coro clearing the event: - -```python -async def eventwait(event): - await event - event.clear() -``` - -The coro raising the event checks that it has been serviced: - -```python -async def foo(event): - while True: - # Acquire data from somewhere - while event.is_set(): - await asyncio.sleep(1) # Wait for coro to respond - event.set() -``` - -Where multiple coros wait on a single event synchronisation can be achieved by -means of an acknowledge event. Each coro needs a separate event. - -```python -async def eventwait(event, ack_event): - await event - ack_event.set() -``` - -An example of this is provided in the `event_test` function in `asyntest.py`. -This is cumbersome. In most cases - even those with a single waiting coro - the -Barrier class below offers a simpler approach. - -An Event can also provide a means of communication between an interrupt handler -and a coro. The handler services the hardware and sets an event which is tested -in slow time by the coro. - -###### [Contents](./TUTORIAL.md#contents) - -### 3.2.1 The event's value - -The `event.set()` method can accept an optional data value of any type. A -coro waiting on the event can retrieve it by means of `event.value()`. Note -that `event.clear()` will set the value to `None`. A typical use for this -is for the coro setting the event to issue `event.set(utime.ticks_ms())`. Any -coro waiting on the event can determine the latency incurred, for example to -perform compensation for this. - -###### [Contents](./TUTORIAL.md#contents) - -## 3.3 Barrier - -This has two uses. Firstly it can cause a coro to pause until one or more other -coros have terminated. - -Secondly it enables multiple coros to rendezvous at a particular point. For -example producer and consumer coros can synchronise at a point where the -producer has data available and the consumer is ready to use it. At that point -in time the `Barrier` can run an optional callback before the barrier is -released and all waiting coros can continue. [Full details.](./PRIMITIVES.md#34-class-barrier) - -The callback can be a function or a coro. In most applications a function is -likely to be used: this can be guaranteed to run to completion before the -barrier is released. - -An example is the `barrier_test` function in `asyntest.py`. In the code -fragment from that program: - -```python -import asyn - -def callback(text): - print(text) - -barrier = asyn.Barrier(3, callback, ('Synch',)) - -async def report(): - for i in range(5): - print('{} '.format(i), end='') - await barrier -``` - -multiple instances of `report` print their result and pause until the other -instances are also complete and waiting on `barrier`. At that point the -callback runs. On its completion the coros resume. - -###### [Contents](./TUTORIAL.md#contents) - -## 3.4 Semaphore - -A semaphore limits the number of coros which can access a resource. It can be -used to limit the number of instances of a particular coro which can run -concurrently. It performs this using an access counter which is initialised by -the constructor and decremented each time a coro acquires the semaphore. -[Full details.](./PRIMITIVES.md#35-class-semaphore) - -The easiest way to use it is with a context manager: - -```python -import asyn -sema = asyn.Semaphore(3) -async def foo(sema): - async with sema: - # Limited access here -``` -An example is the `semaphore_test` function in `asyntest.py`. - -###### [Contents](./TUTORIAL.md#contents) - -### 3.4.1 BoundedSemaphore - -This works identically to the `Semaphore` class except that if the `release` -method causes the access counter to exceed its initial value, a `ValueError` -is raised. [Full details.](./PRIMITIVES.md#351-class-boundedsemaphore) - -###### [Contents](./TUTORIAL.md#contents) - -## 3.5 Queue - -The `Queue` class is officially supported and the sample program `aqtest.py` -demonstrates its use. A queue is instantiated as follows: - -```python -from uasyncio.queues import Queue -q = Queue() -``` - -A typical producer coro might work as follows: - -```python -async def producer(q): - while True: - result = await slow_process() # somehow get some data - await q.put(result) # may pause if a size limited queue fills -``` - -and the consumer works along these lines: - -```python -async def consumer(q): - while True: - result = await(q.get()) # Will pause if q is empty - print('Result was {}'.format(result)) -``` - -The `Queue` class provides significant additional functionality in that the -size of queues may be limited and the status may be interrogated. The behaviour -on empty status and (where size is limited) the behaviour on full status may be -controlled. Documentation of this is in the code. - -###### [Contents](./TUTORIAL.md#contents) - -## 3.6 Other synchronisation primitives - -The [asyn.py](./asyn.py) library provides 'micro' implementations of CPython -capabilities, namely the [Condition class](./PRIMITIVES.md#36-class-condition) -and the [gather](./PRIMITIVES.md#37-class-gather) method. - -The `Condition` class enables a coro to notify other coros which are waiting on -a locked resource. Once notified they will access the resource and release the -lock in turn. The notifying coro can limit the number of coros to be notified. - -The CPython `gather` method enables a list of coros to be launched. When the -last has completed a list of results is returned. This 'micro' implementation -uses different syntax. Timeouts may be applied to any of the coros. - -###### [Contents](./TUTORIAL.md#contents) - -# 4 Designing classes for asyncio - -In the context of device drivers the aim is to ensure nonblocking operation. -The design should ensure that other coros get scheduled in periods while the -driver is waiting for the hardware. For example a task awaiting data arriving -on a UART or a user pressing a button should allow other coros to be scheduled -until the event occurs.. - -###### [Contents](./TUTORIAL.md#contents) - -## 4.1 Awaitable classes - -A coro can pause execution by waiting on an `awaitable` object. Under CPython -a custom class is made `awaitable` by implementing an `__await__` special -method. This returns a generator. An `awaitable` class is used as follows: - -```python -import uasyncio as asyncio - -class Foo(): - def __await__(self): - for n in range(5): - print('__await__ called') - yield from asyncio.sleep(1) # Other coros get scheduled here - return 42 - - __iter__ = __await__ # See note below - -async def bar(): - foo = Foo() # Foo is an awaitable class - print('waiting for foo') - res = await foo # Retrieve result - print('done', res) - -loop = asyncio.get_event_loop() -loop.run_until_complete(bar()) -``` - -Currently MicroPython doesn't support `__await__` -[issue #2678](https://github.com/micropython/micropython/issues/2678) and -`__iter__` must be used. The line `__iter__ = __await__` enables portability -between CPython and MicroPython. Example code may be found in the `Event`, -`Barrier`, `Cancellable` and `Condition` classes in [asyn.py](./asyn.py). - -### 4.1.1 Use in context managers - -Awaitable objects can be used in synchronous or asynchronous CM's by providing -the necessary special methods. The syntax is: - -```python -with await awaitable as a: # The 'as' clause is optional - # code omitted -async with awaitable as a: # Asynchronous CM (see below) - # do something -``` - -To achieve this the `__await__` generator should return `self`. This is passed -to any variable in an `as` clause and also enables the special methods to work. -See `asyn.Condition` and `asyntest.condition_test`, where the `Condition` class -is awaitable and may be used in a synchronous CM. - -###### [Contents](./TUTORIAL.md#contents) - -### 4.1.2 Awaiting a coro - -The Python language requires that `__await__` is a generator function. In -MicroPython generators and coroutines are identical, so the solution is to use -`yield from coro(args)`. - -This tutorial aims to offer code portable to CPython 3.5 or above. In CPython -coroutines and generators are distinct. CPython coros have an `__await__` -special method which retrieves a generator. This is portable: - -```python -up = False # Running under MicroPython? -try: - import uasyncio as asyncio - up = True # Or can use sys.implementation.name -except ImportError: - import asyncio - -async def times_two(n): # Coro to await - await asyncio.sleep(1) - return 2 * n - -class Foo(): - def __await__(self): - res = 1 - for n in range(5): - print('__await__ called') - if up: # MicroPython - res = yield from times_two(res) - else: # CPython - res = yield from times_two(res).__await__() - return res - - __iter__ = __await__ - -async def bar(): - foo = Foo() # foo is awaitable - print('waiting for foo') - res = await foo # Retrieve value - print('done', res) - -loop = asyncio.get_event_loop() -loop.run_until_complete(bar()) -``` - -Note that, in `__await__`, `yield from asyncio.sleep(1)` is allowed by CPython. -I haven't yet established how this is achieved. - -###### [Contents](./TUTORIAL.md#contents) - -## 4.2 Asynchronous iterators - -These provide a means of returning a finite or infinite sequence of values -and could be used as a means of retrieving successive data items as they arrive -from a read-only device. An asynchronous iterable calls asynchronous code in -its `next` method. The class must conform to the following requirements: - - * It has an `__aiter__` method defined with `async def`and returning the - asynchronous iterator. - * It has an ` __anext__` method which is a coro - i.e. defined with - `async def` and containing at least one `await` statement. To stop - iteration it must raise a `StopAsyncIteration` exception. - -Successive values are retrieved with `async for` as below: - -```python -class AsyncIterable: - def __init__(self): - self.data = (1, 2, 3, 4, 5) - self.index = 0 - - async def __aiter__(self): - return self - - async def __anext__(self): - data = await self.fetch_data() - if data: - return data - else: - raise StopAsyncIteration - - async def fetch_data(self): - await asyncio.sleep(0.1) # Other coros get to run - if self.index >= len(self.data): - return None - x = self.data[self.index] - self.index += 1 - return x - -async def run(): - ai = AsyncIterable() - async for x in ai: - print(x) -``` - -###### [Contents](./TUTORIAL.md#contents) - -## 4.3 Asynchronous context managers - -Classes can be designed to support asynchronous context managers. These are CM's -having enter and exit procedures which are coros. An example is the `Lock` -class described above. This has an `__aenter__` coro which is logically -required to run asynchronously. To support the asynchronous CM protocol its -`__aexit__` method also must be a coro, achieved by including -`await asyncio.sleep(0)`. Such classes are accessed from within a coro with -the following syntax: - -```python -async def bar(lock): - async with lock: - print('bar acquired lock') -``` - -As with normal context managers an exit method is guaranteed to be called when -the context manager terminates, whether normally or via an exception. To -achieve this the special methods `__aenter__` and `__aexit__` must be -defined, both being coros waiting on a coro or `awaitable` object. This example -comes from the `Lock` class: - -```python - async def __aenter__(self): - await self.acquire() # a coro defined with async def - return self - - async def __aexit__(self, *args): - self.release() # A conventional method - await asyncio.sleep_ms(0) -``` - -If the `async with` has an `as variable` clause the variable receives the -value returned by `__aenter__`. - -To ensure correct behaviour firmware should be V1.9.10 or later. - -###### [Contents](./TUTORIAL.md#contents) - -# 5 Exceptions timeouts and cancellation - -These topics are related: `uasyncio` enables the cancellation of tasks, and the -application of a timeout to a task, by throwing an exception to the task in a -special way. - -## 5.1 Exceptions - -Where an exception occurs in a coro, it should be trapped either in that coro -or in a coro which is awaiting its completion. This ensures that the exception -is not propagated to the scheduler. If this occurred the scheduler would stop -running, passing the exception to the code which started the scheduler. -Consequently, to avoid stopping the scheduler, coros launched with -`loop.create_task()` must trap any exceptions internally. - -Using `throw` or `close` to throw an exception to a coro is unwise. It subverts -`uasyncio` by forcing the coro to run, and possibly terminate, when it is still -queued for execution. - -There is a "gotcha" illustrated by this code sample. If allowed to run to -completion it works as expected. - -```python -import uasyncio as asyncio -async def foo(): - await asyncio.sleep(3) - print('About to throw exception.') - 1/0 - -async def bar(): - try: - await foo() - except ZeroDivisionError: - print('foo was interrupted by zero division') # Happens - raise # Force shutdown to run by propagating to loop. - except KeyboardInterrupt: - print('foo was interrupted by ctrl-c') # NEVER HAPPENS - raise - -async def shutdown(): - print('Shutdown is running.') # Happens in both cases - await asyncio.sleep(1) - print('done') - -loop = asyncio.get_event_loop() -try: - loop.run_until_complete(bar()) -except ZeroDivisionError: - loop.run_until_complete(shutdown()) -except KeyboardInterrupt: - print('Keyboard interrupt at loop level.') - loop.run_until_complete(shutdown()) -``` - -However issuing a keyboard interrupt causes the exception to go to the event -loop. This is because `uasyncio.sleep` causes execution to be transferred to -the event loop. Consequently applications requiring cleanup code in response to -a keyboard interrupt should trap the exception at the event loop level. - -###### [Contents](./TUTORIAL.md#contents) - -## 5.2 Cancellation and Timeouts - -As stated above, these features work by throwing an exception to a task in a -special way, using a MicroPython specific coro method `pend_throw`. The way -this works is version dependent. In official `uasyncio` V2.0 the exception is -not processed until the task is next scheduled. This imposes latency if the -task is waiting on a `sleep` or on I/O. Timeouts may extend beyond their -nominal period. Task cancelling other tasks cannot determine when cancellation -is complete. - -There is currently a wokround and two solutions. - * Workround: the `asyn` library provides means of waiting on cancellation of - tasks or groups of tasks. See [Task Cancellation](./PRIMITIVES.md#4-task-cancellation). - * [Paul Sokolovsky's library fork](https://github.com/pfalcon/micropython-lib) - provides `uasyncio` V2.4, but this requires his - [Pycopy](https://github.com/pfalcon/micropython) firmware. - * The [fast_io](./FASTPOLL.md) fork of `uasyncio` solves this in Python (in a - less elegant manner) and runs under official firmware. - -The exception hierarchy used here is `Exception-CancelledError-TimeoutError`. - -## 5.2.1 Task cancellation - -`uasyncio` provides a `cancel(coro)` function. This works by throwing an -exception to the coro using `pend_throw`. This works with nested coros. Usage -is as follows: -```python -async def foo(): - while True: - # do something every 10 secs - await asyncio.sleep(10) - -async def bar(loop): - foo_instance = foo() # Create a coroutine instance - loop.create_task(foo_instance) - # code omitted - asyncio.cancel(foo_instance) -``` -If this example is run against `uasyncio` V2.0, when `bar` issues `cancel` it -will not take effect until `foo` is next scheduled. There is thus a latency of -up to 10s in the cancellation of `foo`. Another source of latency would arise -if `foo` waited on I/O. Where latency arises, `bar` cannot determine whether -`foo` has yet been cancelled. This matters in some use-cases. - -Using the Paul Sokolovsky fork or `fast_io` a simple `sleep(0)` suffices: -```python -async def foo(): - while True: - # do something every 10 secs - await asyncio.sleep(10) - -async def bar(loop): - foo_instance = foo() # Create a coroutine instance - loop.create_task(foo_instance) - # code omitted - asyncio.cancel(foo_instance) - await asyncio.sleep(0) - # Task is now cancelled -``` -This would also work in `uasyncio` V2.0 if `foo` (and any coros awaited by -`foo`) never issued `sleep` or waited on I/O. - -Behaviour which may surprise the unwary arises when a coro to be cancelled is -awaited rather than being launched by `create_task`. Consider this fragment: - -```python -async def foo(): - while True: - # do something every 10 secs - await asyncio.sleep(10) - -async def foo_runner(foo_instance): - await foo_instance - print('This will not be printed') - -async def bar(loop): - foo_instance = foo() - loop.create_task(foo_runner(foo_instance)) - # code omitted - asyncio.cancel(foo_instance) -``` -When `foo` is cancelled it is removed from the scheduler's queue; because it -lacks a `return` statement the calling routine `foo_runner` never resumes. It -is recommended always to trap the exception in the outermost scope of a -function subject to cancellation: -```python -async def foo(): - try: - while True: - await asyncio.sleep(10) - await my_coro - except asyncio.CancelledError: - return -``` -In this instance `my_coro` does not need to trap the exception as it will be -propagated to the calling coro and trapped there. - -**Note** It is bad practice to issue the `close` or `throw` methods of a -de-scheduled coro. This subverts the scheduler by causing the coro to execute -code even though descheduled. This is likely to have unwanted consequences. - -###### [Contents](./TUTORIAL.md#contents) - -## 5.2.2 Coroutines with timeouts - -Timeouts are implemented by means of `uasyncio` methods `.wait_for()` and -`.wait_for_ms()`. These take as arguments a coroutine and a timeout in seconds -or ms respectively. If the timeout expires a `TimeoutError` will be thrown to -the coro using `pend_throw`. This exception must be trapped, either by the coro -or its caller. This is for the reason discussed above: if a coro times out it -is descheduled. Unless it traps the error and returns the only way the caller -can proceed is by trapping the exception itself. - -Where the exception is trapped by the coro, I have experienced obscure failures -if the exception is not trapped in the outermost scope as below: -```python -import uasyncio as asyncio - -async def forever(): - try: - print('Starting') - while True: - await asyncio.sleep_ms(300) - print('Got here') - except asyncio.TimeoutError: - print('Got timeout') # And return - -async def foo(): - await asyncio.wait_for(forever(), 5) - await asyncio.sleep(2) - -loop = asyncio.get_event_loop() -loop.run_until_complete(foo()) -``` -Alternatively it may be trapped by the caller: -```python -import uasyncio as asyncio - -async def forever(): - print('Starting') - while True: - await asyncio.sleep_ms(300) - print('Got here') - -async def foo(): - try: - await asyncio.wait_for(forever(), 5) - except asyncio.TimeoutError: - pass - print('Timeout elapsed.') - await asyncio.sleep(2) - -loop = asyncio.get_event_loop() -loop.run_until_complete(foo()) -``` - -#### Uasyncio V2.0 note - -This does not apply to the Paul Sokolovsky fork or to `fast_io`. - -If the coro issues `await asyncio.sleep(t)` where `t` is a long delay, the coro -will not be rescheduled until `t` has elapsed. If the timeout has elapsed -before the `sleep` is complete the `TimeoutError` will occur when the coro is -scheduled - i.e. when `t` has elapsed. In real time and from the point of view -of the calling coro, its response to the `TimeoutError` will be delayed. - -If this matters to the application, create a long delay by awaiting a short one -in a loop. The coro `asyn.sleep` [supports this](./PRIMITIVES.md#41-coro-sleep). - -###### [Contents](./TUTORIAL.md#contents) - -# 6 Interfacing hardware - -At heart all interfaces between `uasyncio` and external asynchronous events -rely on polling. Hardware requiring a fast response may use an interrupt. But -the interface between the interrupt service routine (ISR) and a user coro will -be polled. For example the ISR might trigger an `Event` or set a global flag, -while a coroutine awaiting the outcome polls the object each time it is -scheduled. - -Polling may be effected in two ways, explicitly or implicitly. The latter is -performed by using the `stream I/O` mechanism which is a system designed for -stream devices such as UARTs and sockets. At its simplest explicit polling may -consist of code like this: - -```python -async def poll_my_device(): - global my_flag # Set by device ISR - while True: - if my_flag: - my_flag = False - # service the device - await asyncio.sleep(0) -``` - -In place of a global, an instance variable, an `Event` object or an instance of -an awaitable class might be used. Explicit polling is discussed -further [below](./TUTORIAL.md#62-polling-hardware-with-a-coroutine). - -Implicit polling consists of designing the driver to behave like a stream I/O -device such as a socket or UART, using `stream I/O`. This polls devices using -Python's `select.poll` system: because the polling is done in C it is faster -and more efficient than explicit polling. The use of `stream I/O` is discussed -[here](./TUTORIAL.md#63-using-the-stream-mechanism). - -Owing to its efficiency implicit polling benefits most fast I/O device drivers: -streaming drivers can be written for many devices not normally considered as -streaming devices [section 6.4](./TUTORIAL.md#64-writing-streaming-device-drivers). - -###### [Contents](./TUTORIAL.md#contents) - -## 6.1 Timing issues - -Both explicit and implicit polling are currently based on round-robin -scheduling. Assume I/O is operating concurrently with N user coros each of -which yields with a zero delay. When I/O has been serviced it will next be -polled once all user coros have been scheduled. The implied latency needs to be -considered in the design. I/O channels may require buffering, with an ISR -servicing the hardware in real time from buffers and coroutines filling or -emptying the buffers in slower time. - -The possibility of overrun also needs to be considered: this is the case where -something being polled by a coroutine occurs more than once before the coro is -actually scheduled. - -Another timing issue is the accuracy of delays. If a coro issues - -```python - await asyncio.sleep_ms(t) - # next line -``` - -the scheduler guarantees that execution will pause for at least `t`ms. The -actual delay may be greater depending on the system state when `t` expires. -If, at that time, all other coros are waiting on nonzero delays, the next line -will immediately be scheduled. But if other coros are pending execution (either -because they issued a zero delay or because their time has also elapsed) they -may be scheduled first. This introduces a timing uncertainty into the `sleep()` -and `sleep_ms()` functions. The worst-case value for this overrun may be -calculated by summing, for every other coro, the worst-case execution time -between yielding to the scheduler. - -The [fast_io](./FASTPOLL.md) version of `uasyncio` in this repo provides a way -to ensure that stream I/O is polled on every iteration of the scheduler. It is -hoped that official `uasyncio` will adopt code to this effect in due course. - -###### [Contents](./TUTORIAL.md#contents) - -## 6.2 Polling hardware with a coroutine - -This is a simple approach, but is most appropriate to hardware which may be -polled at a relatively low rate. This is primarily because polling with a short -(or zero) polling interval may cause the coro to consume more processor time -than is desirable. - -The example `apoll.py` demonstrates this approach by polling the Pyboard -accelerometer at 100ms intervals. It performs some simple filtering to ignore -noisy samples and prints a message every two seconds if the board is not moved. - -Further examples may be found in `aswitch.py` which provides drivers for -switch and pushbutton devices. - -An example of a driver for a device capable of reading and writing is shown -below. For ease of testing Pyboard UART 4 emulates the notional device. The -driver implements a `RecordOrientedUart` class, where data is supplied in -variable length records consisting of bytes instances. The object appends a -delimiter before sending and buffers incoming data until the delimiter is -received. This is a demo and is an inefficient way to use a UART compared to -stream I/O. - -For the purpose of demonstrating asynchronous transmission we assume the -device being emulated has a means of checking that transmission is complete -and that the application requires that we wait on this. Neither assumption is -true in this example but the code fakes it with `await asyncio.sleep(0.1)`. - -Link pins X1 and X2 to run. - -```python -import uasyncio as asyncio -from pyb import UART - -class RecordOrientedUart(): - DELIMITER = b'\0' - def __init__(self): - self.uart = UART(4, 9600) - self.data = b'' - - def __iter__(self): # Not __await__ issue #2678 - data = b'' - while not data.endswith(self.DELIMITER): - yield from asyncio.sleep(0) # Necessary because: - while not self.uart.any(): - yield from asyncio.sleep(0) # timing may mean this is never called - data = b''.join((data, self.uart.read(self.uart.any()))) - self.data = data - - async def send_record(self, data): - data = b''.join((data, self.DELIMITER)) - self.uart.write(data) - await self._send_complete() - - # In a real device driver we would poll the hardware - # for completion in a loop with await asyncio.sleep(0) - async def _send_complete(self): - await asyncio.sleep(0.1) - - def read_record(self): # Synchronous: await the object before calling - return self.data[0:-1] # Discard delimiter - -async def run(): - foo = RecordOrientedUart() - rx_data = b'' - await foo.send_record(b'A line of text.') - for _ in range(20): - await foo # Other coros are scheduled while we wait - rx_data = foo.read_record() - print('Got: {}'.format(rx_data)) - await foo.send_record(rx_data) - rx_data = b'' - -loop = asyncio.get_event_loop() -loop.run_until_complete(run()) -``` - -###### [Contents](./TUTORIAL.md#contents) - -## 6.3 Using the stream mechanism - -This can be illustrated using a Pyboard UART. The following code sample -demonstrates concurrent I/O on one UART. To run, link Pyboard pins X1 and X2 -(UART Txd and Rxd). - -```python -import uasyncio as asyncio -from pyb import UART -uart = UART(4, 9600) - -async def sender(): - swriter = asyncio.StreamWriter(uart, {}) - while True: - await swriter.awrite('Hello uart\n') - await asyncio.sleep(2) - -async def receiver(): - sreader = asyncio.StreamReader(uart) - while True: - res = await sreader.readline() - print('Received', res) - -loop = asyncio.get_event_loop() -loop.create_task(sender()) -loop.create_task(receiver()) -loop.run_forever() -``` - -The supporting code may be found in `__init__.py` in the `uasyncio` library. -The mechanism works because the device driver (written in C) implements the -following methods: `ioctl`, `read`, `readline` and `write`. See -[Writing streaming device drivers](./TUTORIAL.md#64-writing-streaming-device-drivers) -for details on how such drivers may be written in Python. - -A UART can receive data at any time. The stream I/O mechanism checks for pending -incoming characters whenever the scheduler has control. When a coro is running -an interrupt service routine buffers incoming characters; these will be removed -when the coro yields to the scheduler. Consequently UART applications should be -designed such that coros minimise the time between yielding to the scheduler to -avoid buffer overflows and data loss. This can be ameliorated by using a larger -UART read buffer or a lower baudrate. Alternatively hardware flow control will -provide a solution if the data source supports it. - -### 6.3.1 A UART driver example - -The program [auart_hd.py](./auart_hd.py) illustrates a method of communicating -with a half duplex device such as one responding to the modem 'AT' command set. -Half duplex means that the device never sends unsolicited data: its -transmissions are always in response to a command from the master. - -The device is emulated, enabling the test to be run on a Pyboard with two wire -links. - -The (highly simplified) emulated device responds to any command by sending four -lines of data with a pause between each, to simulate slow processing. - -The master sends a command, but does not know in advance how many lines of data -will be returned. It starts a retriggerable timer, which is retriggered each -time a line is received. When the timer times out it is assumed that the device -has completed transmission, and a list of received lines is returned. - -The case of device failure is also demonstrated. This is done by omitting the -transmission before awaiting a response. After the timeout an empty list is -returned. See the code comments for more details. - -###### [Contents](./TUTORIAL.md#contents) - -## 6.4 Writing streaming device drivers - -The `stream I/O` mechanism is provided to support I/O to stream devices. Its -typical use is to support streaming I/O devices such as UARTs and sockets. The -mechanism may be employed by drivers of any device which needs to be polled: -the polling is delegated to the scheduler which uses `select` to schedule the -handlers for any devices which are ready. This is more efficient than running -multiple coros each polling a device, partly because `select` is written in C -but also because the coroutine performing the polling is descheduled until the -`poll` object returns a ready status. - -A device driver capable of employing the stream I/O mechanism may support -`StreamReader`, `StreamWriter` instances or both. A readable device must -provide at least one of the following methods. Note that these are synchronous -methods. The `ioctl` method (see below) ensures that they are only called if -data is available. The methods should return as fast as possible with as much -data as is available. - -`readline()` Return as many characters as are available up to and including any -newline character. Required if you intend to use `StreamReader.readline()` -`read(n)` Return as many characters as are available but no more than `n`. -Required to use `StreamReader.read()` or `StreamReader.readexactly()` - -A writeable driver must provide this synchronous method: -`write` Args `buf`, `off`, `sz`. Arguments: -`buf` is the buffer to write. -`off` is the offset into the buffer of the first character to write. -`sz` is the requested number of characters to write. -It should return immediately. The return value is the number of characters -actually written (may well be 1 if the device is slow). The `ioctl` method -ensures that this is only called if the device is ready to accept data. - -All devices must provide an `ioctl` method which polls the hardware to -determine its ready status. A typical example for a read/write driver is: - -```python -import io -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL_WR = const(4) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -class MyIO(io.IOBase): - # Methods omitted - def ioctl(self, req, arg): # see ports/stm32/uart.c - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - if hardware_has_at_least_one_char_to_read: - ret |= MP_STREAM_POLL_RD - if arg & MP_STREAM_POLL_WR: - if hardware_can_accept_at_least_one_write_character: - ret |= MP_STREAM_POLL_WR - return ret -``` - -The following is a complete awaitable delay class: - -```python -import uasyncio as asyncio -import utime -import io -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -class MillisecTimer(io.IOBase): - def __init__(self): - self.end = 0 - self.sreader = asyncio.StreamReader(self) - - def __iter__(self): - await self.sreader.readline() - - def __call__(self, ms): - self.end = utime.ticks_add(utime.ticks_ms(), ms) - return self - - def readline(self): - return b'\n' - - def ioctl(self, req, arg): - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - if utime.ticks_diff(utime.ticks_ms(), self.end) >= 0: - ret |= MP_STREAM_POLL_RD - return ret -``` - -which may be used as follows: - -```python -async def timer_test(n): - timer = ms_timer.MillisecTimer() - await timer(30) # Pause 30ms -``` - -With official `uasyncio` this confers no benefit over `await asyncio.sleep_ms()`. -Using [fast_io](./FASTPOLL.md) it offers much more precise delays under the -common usage pattern where coros await a zero delay. - -It is possible to use I/O scheduling to associate an event with a callback. -This is more efficient than a polling loop because the coro doing the polling -is descheduled until `ioctl` returns a ready status. The following runs a -callback when a pin changes state. - -```python -import uasyncio as asyncio -import io -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -class PinCall(io.IOBase): - def __init__(self, pin, *, cb_rise=None, cbr_args=(), cb_fall=None, cbf_args=()): - self.pin = pin - self.cb_rise = cb_rise - self.cbr_args = cbr_args - self.cb_fall = cb_fall - self.cbf_args = cbf_args - self.pinval = pin.value() - self.sreader = asyncio.StreamReader(self) - loop = asyncio.get_event_loop() - loop.create_task(self.run()) - - async def run(self): - while True: - await self.sreader.read(1) - - def read(self, _): - v = self.pinval - if v and self.cb_rise is not None: - self.cb_rise(*self.cbr_args) - return b'\n' - if not v and self.cb_fall is not None: - self.cb_fall(*self.cbf_args) - return b'\n' - - def ioctl(self, req, arg): - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - v = self.pin.value() - if v != self.pinval: - self.pinval = v - ret = MP_STREAM_POLL_RD - return ret -``` - -Once again with official `uasyncio` latency can be high. Depending on -application design the [fast_io](./FASTPOLL.md) version can greatly reduce -this. - -The demo program `iorw.py` illustrates a complete example. Note that, at the -time of writing there is a bug in `uasyncio` which prevents this from working. -See [this GitHub thread](https://github.com/micropython/micropython/pull/3836#issuecomment-397317408). -There are two solutions. A workround is to write two separate drivers, one -read-only and the other write-only. Alternatively the -[fast_io](./FASTPOLL.md) version addresses this. - -In the official `uasyncio` I/O is scheduled quite infrequently. See -[see this GitHub RFC](https://github.com/micropython/micropython/issues/2664). -The [fast_io](./FASTPOLL.md) version addresses this issue. - -###### [Contents](./TUTORIAL.md#contents) - -## 6.5 A complete example: aremote.py - -See [aremote.py](./nec_ir/aremote.py) documented [here](./nec_ir/README.md). -The demo provides a complete device driver example: a receiver/decoder for an -infra red remote controller. The following notes are salient points regarding -its `asyncio` usage. - -A pin interrupt records the time of a state change (in μs) and sets an event, -passing the time when the first state change occurred. A coro waits on the -event, yields for the duration of a data burst, then decodes the stored data -before calling a user-specified callback. - -Passing the time to the `Event` instance enables the coro to compensate for -any `asyncio` latency when setting its delay period. - -###### [Contents](./TUTORIAL.md#contents) - -## 6.6 HTU21D environment sensor - -This chip provides accurate measurements of temperature and humidity. The -driver is documented [here](./htu21d/README.md). It has a continuously running -task which updates `temperature` and `humidity` bound variables which may be -accessed "instantly". - -The chip takes on the order of 120ms to acquire both data items. The driver -works asynchronously by triggering the acquisition and using -`await asyncio.sleep(t)` prior to reading the data. This allows other coros to -run while acquisition is in progress. - -# 7 Hints and tips - -###### [Contents](./TUTORIAL.md#contents) - -## 7.1 Program hangs - -Hanging usually occurs because a task has blocked without yielding: this will -hang the entire system. When developing it is useful to have a coro which -periodically toggles an onboard LED. This provides confirmation that the -scheduler is running. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.2 uasyncio retains state - -When running programs using `uasyncio` at the REPL, issue a soft reset -(ctrl-D) between runs. This is because `uasyncio` retains state between runs -which can lead to confusing behaviour. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.3 Garbage Collection - -You may want to consider running a coro which issues: - -```python - gc.collect() - gc.threshold(gc.mem_free() // 4 + gc.mem_alloc()) -``` - -This assumes `import gc` has been issued. The purpose of this is discussed -[here](http://docs.micropython.org/en/latest/pyboard/reference/constrained.html) -in the section on the heap. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.4 Testing - -It's advisable to test that a device driver yields control when you intend it -to. This can be done by running one or more instances of a dummy coro which -runs a loop printing a message, and checking that it runs in the periods when -the driver is blocking: - -```python -async def rr(n): - while True: - print('Roundrobin ', n) - await asyncio.sleep(0) -``` - -As an example of the type of hazard which can occur, in the `RecordOrientedUart` -example above the `__await__` method was originally written as: - -```python - def __await__(self): - data = b'' - while not data.endswith(self.DELIMITER): - while not self.uart.any(): - yield from asyncio.sleep(0) - data = b''.join((data, self.uart.read(self.uart.any()))) - self.data = data -``` - -In testing this hogged execution until an entire record was received. This was -because `uart.any()` always returned a nonzero quantity. By the time it was -called, characters had been received. The solution was to yield execution in -the outer loop: - -```python - def __await__(self): - data = b'' - while not data.endswith(self.DELIMITER): - yield from asyncio.sleep(0) # Necessary because: - while not self.uart.any(): - yield from asyncio.sleep(0) # timing may mean this is never called - data = b''.join((data, self.uart.read(self.uart.any()))) - self.data = data -``` - -It is perhaps worth noting that this error would not have been apparent had -data been sent to the UART at a slow rate rather than via a loopback test. -Welcome to the joys of realtime programming. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.5 A common error - -If a function or method is defined with `async def` and subsequently called as -if it were a regular (synchronous) callable, MicroPython does not issue an -error message. This is [by design](https://github.com/micropython/micropython/issues/3241). -It typically leads to a program silently failing to run correctly: - -```python -async def foo(): - # code -loop.create_task(foo) # Case 1: foo will never run -foo() # Case 2: Likewise. -``` - -I have [a PR](https://github.com/micropython/micropython-lib/pull/292) which -proposes a fix for case 1. The [fast_io](./FASTPOLL.md) version implements -this. - -The script [check_async_code.py](./check_async_code.py) attempts to locate -instances of questionable use of coros. It is intended to be run on a PC and -uses Python3. It takes a single argument, a path to a MicroPython sourcefile -(or `--help`). It is designed for use on scripts written according to the -guidelines in this tutorial, with coros declared using `async def`. - -Note it is somewhat crude and intended to be used on a syntactically correct -file which is silently failing to run. Use a tool such as `pylint` for general -syntax checking (`pylint` currently misses this error). - -The script produces false positives. This is by design: coros are first class -objects; you can pass them to functions and can store them in data structures. -Depending on the program logic you may intend to store the function or the -outcome of its execution. The script can't deduce the intent. It aims to ignore -cases which appear correct while identifying other instances for review. -Assume `foo` is a coro declared with `async def`: - -```python -loop.run_until_complete(foo()) # No warning -bar(foo) # These lines will warn but may or may not be correct -bar(foo()) -z = (foo,) -z = (foo(),) -foo() # Will warn: is surely wrong. -``` - -I find it useful as-is but improvements are always welcome. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.6 Socket programming - -There are two basic approaches to socket programming under `uasyncio`. By -default sockets block until a specified read or write operation completes. -`uasyncio` supports blocking sockets by using `select.poll` to prevent them -from blocking the scheduler. In most cases it is simplest to use this -mechanism. Example client and server code may be found in the `client_server` -directory. The `userver` application uses `select.poll` explicitly to poll -the server socket. The client sockets use it implicitly in that the `uasyncio` -stream mechanism employs it. - -Note that `socket.getaddrinfo` currently blocks. The time will be minimal in -the example code but if a DNS lookup is required the blocking period could be -substantial. - -The second approach to socket programming is to use nonblocking sockets. This -adds complexity but is necessary in some applications, notably where -connectivity is via WiFi (see below). - -At the time of writing (March 2019) support for TLS on nonblocking sockets is -under development. Its exact status is unknown (to me). - -The use of nonblocking sockets requires some attention to detail. If a -nonblocking read is performed, because of server latency, there is no guarantee -that all (or any) of the requested data is returned. Likewise writes may not -proceed to completion. - -Hence asynchronous read and write methods need to iteratively perform the -nonblocking operation until the required data has been read or written. In -practice a timeout is likely to be required to cope with server outages. - -A further complication is that the ESP32 port had issues which required rather -unpleasant hacks for error-free operation. I have not tested whether this is -still the case. - -The file [sock_nonblock.py](./sock_nonblock.py) illustrates the sort of -techniques required. It is not a working demo, and solutions are likely to be -application dependent. - -### 7.6.1 WiFi issues - -The `uasyncio` stream mechanism is not good at detecting WiFi outages. I have -found it necessary to use nonblocking sockets to achieve resilient operation -and client reconnection in the presence of outages. - -[This doc](https://github.com/peterhinch/micropython-samples/blob/master/resilient/README.md) -describes issues I encountered in WiFi applications which keep sockets open for -long periods, and outlines a solution. - -[This repo](https://github.com/peterhinch/micropython-mqtt.git) offers a -resilent asynchronous MQTT client which ensures message integrity over WiFi -outages. [This repo](https://github.com/peterhinch/micropython-iot.git) -provides a simple asynchronous full-duplex serial channel between a wirelessly -connected client and a wired server with guaranteed message delivery. - -###### [Contents](./TUTORIAL.md#contents) - -## 7.7 Event loop constructor args - -A subtle bug can arise if you need to instantiate the event loop with non -default values. Instantiation should be performed before running any other -`asyncio` code. This is because the code may acquire the event loop. In -doing so it initialises it to the default values: - -```python -import uasyncio as asyncio -import some_module -bar = some_module.Bar() # Constructor calls get_event_loop() -# and renders these args inoperative -loop = asyncio.get_event_loop(runq_len=40, waitq_len=40) -``` - -Given that importing a module can run code the safest way is to instantiate -the event loop immediately after importing `uasyncio`. - -```python -import uasyncio as asyncio -loop = asyncio.get_event_loop(runq_len=40, waitq_len=40) -import some_module -bar = some_module.Bar() # The get_event_loop() call is now safe -``` - -My preferred approach to this is as follows. If writing modules for use by -other programs avoid running `uasyncio` code on import. Write functions and -methods to expect the event loop as an arg. Then ensure that only the top level -application calls `get_event_loop`: - -```python -import uasyncio as asyncio -import my_module # Does not run code on loading -loop = asyncio.get_event_loop(runq_len=40, waitq_len=40) -bar = my_module.Bar(loop) -``` - -Ref [this issue](https://github.com/micropython/micropython-lib/issues/295). - -###### [Contents](./TUTORIAL.md#contents) - -# 8 Notes for beginners - -These notes are intended for those new to asynchronous code. They start by -outlining the problems which schedulers seek to solve, and give an overview of -the `uasyncio` approach to a solution. - -[Section 8.5](./TUTORIAL.md#85-why-cooperative-rather-than-pre-emptive) -discusses the relative merits of `uasyncio` and the `_thread` module and why -you may prefer use cooperative (`uasyncio`) over pre-emptive (`_thread`) -scheduling. - -###### [Contents](./TUTORIAL.md#contents) - -## 8.1 Problem 1: event loops - -A typical firmware application runs continuously and is required to respond to -external events. These might include a voltage change on an ADC, the arrival of -a hard interrupt, a character arriving on a UART, or data being available on a -socket. These events occur asynchronously and the code must be able to respond -regardless of the order in which they occur. Further the application may be -required to perform time-dependent tasks such as flashing LED's. - -The obvious way to do this is with an event loop. The following is not -practical code but serves to illustrate the general form of an event loop. - -```python -def event_loop(): - led_1_time = 0 - led_2_time = 0 - switch_state = switch.state() # Current state of a switch - while True: - time_now = utime.time() - if time_now >= led_1_time: # Flash LED #1 - led1.toggle() - led_1_time = time_now + led_1_period - if time_now >= led_2_time: # Flash LED #2 - led2.toggle() - led_2_time = time_now + led_2_period - # Handle LEDs 3 upwards - - if switch.value() != switch_state: - switch_state = switch.value() - # do something - if uart.any(): - # handle UART input -``` - -This works for simple examples but event loops rapidly become unwieldy as the -number of events increases. They also violate the principles of object oriented -programming by lumping much of the program logic in one place rather than -associating code with the object being controlled. We want to design a class -for an LED capable of flashing which could be put in a module and imported. An -OOP approach to flashing an LED might look like this: - -```python -import pyb -class LED_flashable(): - def __init__(self, led_no): - self.led = pyb.LED(led_no) - - def flash(self, period): - while True: - self.led.toggle() - # somehow wait for period but allow other - # things to happen at the same time -``` - -A cooperative scheduler such as `uasyncio` enables classes such as this to be -created. - -###### [Contents](./TUTORIAL.md#contents) - -## 8.2 Problem 2: blocking methods - -Assume you need to read a number of bytes from a socket. If you call -`socket.read(n)` with a default blocking socket it will "block" (i.e. fail to -return) until `n` bytes have been received. During this period the application -will be unresponsive to other events. - -With `uasyncio` and a non-blocking socket you can write an asynchronous read -method. The task requiring the data will (necessarily) block until it is -received but during that period other tasks will be scheduled enabling the -application to remain responsive. - -## 8.3 The uasyncio approach - -The following class provides for an LED which can be turned on and off, and -which can also be made to flash at an arbitrary rate. A `LED_async` instance -has a `run` method which can be considered to run continuously. The LED's -behaviour can be controlled by methods `on()`, `off()` and `flash(secs)`. - -```python -import pyb -import uasyncio as asyncio - -class LED_async(): - def __init__(self, led_no): - self.led = pyb.LED(led_no) - self.rate = 0 - loop = asyncio.get_event_loop() - loop.create_task(self.run()) - - async def run(self): - while True: - if self.rate <= 0: - await asyncio.sleep_ms(200) - else: - self.led.toggle() - await asyncio.sleep_ms(int(500 / self.rate)) - - def flash(self, rate): - self.rate = rate - - def on(self): - self.led.on() - self.rate = 0 - - def off(self): - self.led.off() - self.rate = 0 -``` - -Note that `on()`, `off()` and `flash()` are conventional synchronous methods. -They change the behaviour of the LED but return immediately. The flashing -occurs "in the background". This is explained in detail in the next section. - -The class conforms with the OOP principle of keeping the logic associated with -the device within the class. Further, the way `uasyncio` works ensures that -while the LED is flashing the application can respond to other events. The -example below flashes the four Pyboard LED's at different rates while also -responding to the USR button which terminates the program. - -```python -import pyb -import uasyncio as asyncio -from led_async import LED_async # Class as listed above - -async def killer(): - sw = pyb.Switch() - while not sw.value(): - await asyncio.sleep_ms(100) - -leds = [LED_async(n) for n in range(1, 4)] -for n, led in enumerate(leds): - led.flash(0.7 + n/4) -loop = asyncio.get_event_loop() -loop.run_until_complete(killer()) -``` - -In contrast to the event loop example the logic associated with the switch is -in a function separate from the LED functionality. Note the code used to start -the scheduler: - -```python -loop = asyncio.get_event_loop() -loop.run_until_complete(killer()) # Execution passes to coroutines. - # It only continues here once killer() terminates, when the - # scheduler has stopped. -``` - -###### [Contents](./TUTORIAL.md#contents) - -## 8.4 Scheduling in uasyncio - -Python 3.5 and MicroPython support the notion of an asynchronous function, -also known as a coroutine (coro) or task. A coro must include at least one -`await` statement. - -```python -async def hello(): - for _ in range(10): - print('Hello world.') - await asyncio.sleep(1) -``` - -This function prints the message ten times at one second intervals. While the -function is paused pending the time delay asyncio will schedule other tasks, -providing an illusion of concurrency. - -When a coro issues `await asyncio.sleep_ms()` or `await asyncio.sleep()` the -current task pauses: it is placed on a queue which is ordered on time due, and -execution passes to the task at the top of the queue. The queue is designed so -that even if the specified sleep is zero other due tasks will run before the -current one is resumed. This is "fair round-robin" scheduling. It is common -practice to issue `await asyncio.sleep(0)` in loops to ensure a task doesn't -hog execution. The following shows a busy-wait loop which waits for another -task to set the global `flag`. Alas it monopolises the CPU preventing other -coros from running: - -```python -async def bad_code(): - global flag - while not flag: - pass - flag = False - # code omitted -``` - -The problem here is that while the `flag` is `False` the loop never yields to -the scheduler so no other task will get to run. The correct approach is: - -```python -async def good_code(): - global flag - while not flag: - await asyncio.sleep(0) - flag = False - # code omitted -``` - -For the same reason it's bad practice to issue delays like `utime.sleep(1)` -because that will lock out other tasks for 1s; use `await asyncio.sleep(1)`. -Note that the delays implied by `uasyncio` methods `sleep` and `sleep_ms` can -overrun the specified time. This is because while the delay is in progress -other tasks will run. When the delay period completes, execution will not -resume until the running task issues `await` or terminates. A well-behaved coro -will always issue `await` at regular intervals. Where a precise delay is -required, especially one below a few ms, it may be necessary to use -`utime.sleep_us(us)`. - -###### [Contents](./TUTORIAL.md#contents) - -## 8.5 Why cooperative rather than pre-emptive? - -The initial reaction of beginners to the idea of cooperative multi-tasking is -often one of disappointment. Surely pre-emptive is better? Why should I have to -explicitly yield control when the Python virtual machine can do it for me? - -When it comes to embedded systems the cooperative model has two advantages. -Firstly, it is lightweight. It is possible to have large numbers of coroutines -because unlike descheduled threads, paused coroutines contain little state. -Secondly it avoids some of the subtle problems associated with pre-emptive -scheduling. In practice cooperative multi-tasking is widely used, notably in -user interface applications. - -To make a case for the defence a pre-emptive model has one advantage: if -someone writes - -```python -for x in range(1000000): - # do something time consuming -``` - -it won't lock out other threads. Under cooperative schedulers the loop must -explicitly yield control every so many iterations e.g. by putting the code in -a coro and periodically issuing `await asyncio.sleep(0)`. - -Alas this benefit of pre-emption pales into insignificance compared to the -drawbacks. Some of these are covered in the documentation on writing -[interrupt handlers](http://docs.micropython.org/en/latest/reference/isr_rules.html). -In a pre-emptive model every thread can interrupt every other thread, changing -data which might be used in other threads. It is generally much easier to find -and fix a lockup resulting from a coro which fails to yield than locating the -sometimes deeply subtle and rarely occurring bugs which can occur in -pre-emptive code. - -To put this in simple terms, if you write a MicroPython coroutine, you can be -sure that variables won't suddenly be changed by another coro: your coro has -complete control until it issues `await asyncio.sleep(0)`. - -Bear in mind that interrupt handlers are pre-emptive. This applies to both hard -and soft interrupts, either of which can occur at any point in your code. - -An eloquent discussion of the evils of threading may be found -[in threads are bad](https://glyph.twistedmatrix.com/2014/02/unyielding.html). - -###### [Contents](./TUTORIAL.md#contents) - -## 8.6 Communication - -In non-trivial applications coroutines need to communicate. Conventional Python -techniques can be employed. These include the use of global variables or -declaring coros as object methods: these can then share instance variables. -Alternatively a mutable object may be passed as a coro argument. - -Pre-emptive systems mandate specialist classes to achieve "thread safe" -communications; in a cooperative system these are seldom required. - -###### [Contents](./TUTORIAL.md#contents) - -## 8.7 Polling - -Some hardware devices such as the Pyboard accelerometer don't support -interrupts, and therefore must be polled (i.e. checked periodically). Polling -can also be used in conjunction with interrupt handlers: the interrupt handler -services the hardware and sets a flag. A coro polls the flag: if it's set it -handles the data and clears the flag. A better approach is to use an `Event`. - -###### [Contents](./TUTORIAL.md#contents) diff --git a/UNDER_THE_HOOD.md b/UNDER_THE_HOOD.md deleted file mode 100644 index 64a3fff..0000000 --- a/UNDER_THE_HOOD.md +++ /dev/null @@ -1,377 +0,0 @@ -# uasyncio: Under the hood - -This document aims to explain the operation of `uasyncio` as I understand it. I -did not write the library so the information presented is a result of using it, -studying the code, experiment and inference. There may be errors, in which case -please raise an issue. None of this information is required to use the library: -it is intended to satisfy the curiosity of scheduler geeks or to help those -wishing to modify it. - -# 0. Contents - - 1. [Introduction](./UNDER_THE_HOOD.md#1-introduction) - 2. [Generators and coroutines](./UNDER_THE_HOOD.md#2-generators-and-coroutines) - 2.1 [pend_throw](./UNDER_THE_HOOD.md#21-pend_throw) - 3. [Coroutine yield types](./UNDER_THE_HOOD.md#3-coroutine-yield-types) - 3.1 [SysCall1 classes](./UNDER_THE_HOOD.md#31-syscall1-classes) - 4. [The EventLoop](./UNDER_THE_HOOD.md#4-the-eventloop) - 4.1 [Exceptions](./UNDER_THE_HOOD.md#41-exceptions) - 4.2 [Task Cancellation and Timeouts](./UNDER_THE_HOOD.md#42-task-cancellation-and-timeouts) - 5. [Stream I/O](./UNDER_THE_HOOD.md#5-stream-io) - 5.1 [StreamReader](./UNDER_THE_HOOD.md#51-streamreader) - 5.2 [StreamWriter](./UNDER_THE_HOOD.md#52-streamwriter) - 5.3 [PollEventLoop wait method](./UNDER_THE_HOOD.md#53-polleventloop-wait-method) - 6. [Modifying uasyncio](./UNDER_THE_HOOD.md#6-modifying-uasyncio) - 7. [Links](./UNDER_THE_HOOD.md#7-links) - -# 1. Introduction - -Where the versions differ, this explanation relates to the `fast_io` version. -Note that the code in `fast_io` contains additional comments to explain its -operation. The code the `fast_io` directory is also in -[my micropython-lib fork](https://github.com/peterhinch/micropython-lib.git), -`uasyncio-io-fast-and-rw` branch. - -This doc assumes a good appreciation of the use of `uasyncio`. An understanding -of Python generators is also essential, in particular the use of `yield from` -and an appreciation of the difference between a generator and a generator -function: - -```python -def gen_func(n): # gen_func is a generator function - while True: - yield n - n += 1 - -my_gen = gen_func(7) # my_gen is a generator -``` - -The code for the `fast_io` variant of `uasyncio` may be found in: - -``` -fast_io/__init__.py -fast_io/core.py -``` - -This has additional code comments to aid in its understanding. - -###### [Main README](./README.md) - -# 2. Generators and coroutines - -In MicroPython coroutines and generators are identical: this differs from -CPython. The knowledge that a coro is a generator is crucial to understanding -`uasyncio`'s operation. Consider this code fragment: - -```python -async def bar(): - await asyncio.sleep(1) - -async def foo(): - await bar() -``` - -In MicroPython the `async def` syntax allows a generator function to lack a -`yield` statement. Thus `bar` is a generator function, hence `bar()` returns a -generator. - -The `await bar()` syntax is equivalent to `yield from bar()`. So transferring -execution to the generator instantiated by `bar()` does not involve the -scheduler. `asyncio.sleep` is a generator function so `await asyncio.sleep(1)` -creates a generator and transfers execution to it via `yield from`. The -generator yields a value of 1000; this is passed to the scheduler to invoke the -delay by placing the coro onto a `timeq` (see below). - -## 2.1 pend_throw - -Generators in MicroPython have a nonstandard method `pend_throw`. The Python -`throw` method causes the generator immediately to run and to handle the passed -exception. `pend_throw` retains the exception until the generator (coroutine) -is next scheduled, when the exception is raised. In `fast_io` the task -cancellation and timeout mechanisms aim to ensure that the task is scheduled as -soon as possible to minimise latency. - -The `pend_throw` method serves a secondary purpose in `uasyncio`: to store -state in a coro which is paused pending execution. This works because the -object returned from `pend_throw` is that which was previously passed to it, or -`None` on the first call. - -```python -a = my_coro.pend_throw(42) -b = my_coro.pend_throw(None) # Coro can now safely be executed -``` -In the above instance `a` will be `None` if it was the first call to -`pend_throw` and `b` will be 42. This is used to determine if a paused task is -on a `timeq` or waiting on I/O. A task on a `timeq` will have an integer value, -being the `ID` of the task; one pending I/O will have `False`. - -If a coro is actually run, the only acceptable stored values are `None` or an -exception. The error "exception must be derived from base exception" indicates -an error in the scheduler whereby this constraint has not been satisfied. - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -# 3. Coroutine yield types - -Because coroutines are generators it is valid to issue `yield` in a coroutine, -behaviour which would cause a syntax error in CPython. While explicitly issuing -`yield` in a user application is best avoided for CPython compatibility, it is -used internally in `uasyncio`. Further, because `await` is equivalent to -`yield from`, the behaviour of the scheduler in response to `yield` is crucial -to understanding its operation. - -Where a coroutine (perhaps at the end of a `yield from` chain) executes - -```python -yield some_object -``` - -the scheduler regains execution. This is because the scheduler passed execution -to the user coroutine with - -```python -ret = next(cb) -``` - -so `ret` contains the object yielded. Subsequent scheduler behaviour depends on -the type of that object. The following object types are handled: - - * `None` The coro is rescheduled and will run in round-robin fashion. - Hence `yield` is functionally equivalent to `await asyncio.sleep(0)`. - * An integer `N`: equivalent to `await asyncio.sleep_ms(N)`. - * `False` The coro terminates and is not rescheduled. - * A coro/generator: the yielded coro is scheduled. The coro which issued the - `yield` is rescheduled. - * A `SysCall1` instance. See below. - -## 3.1 SysCall1 classes - -The `SysCall1` constructor takes a single argument stored in `self.arg`. It is -effectively an abstract base class: only subclasses are instantiated. When a -coro yields a `SysCall1` instance, the scheduler's behaviour is determined by -the type of the object and the contents of its `.arg`. - -The following subclasses exist: - - * `SleepMs` `.arg` holds the delay in ms. Effectively a singleton with the - instance in `sleep_ms`. Its `.__call__` enables `await asyncio.sleep_ms(n)`. - * `StopLoop` Stops the scheduler. `.arg` is returned to the caller. - * `IORead` Causes an interface to be polled for data ready. `.arg` is the - interface. - * `IOWrite` Causes an interface to be polled for ready to accept data. `.arg` - is the interface. - * `IOReadDone` These stop polling of an interface (in `.arg`). - * `IOWriteDone` - -The `IO*` classes are for the exclusive use of `StreamReader` and `StreamWriter` -objects. - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -# 4. The EventLoop - -The file `core.py` defines an `EventLoop` class which is subclassed by -`PollEventLoop` in `__init__.py`. The latter extends the base class to support -stream I/O. In particular `.wait()` is overridden in the subclass. - -The `fast_io` `EventLoop` maintains four queues, `.runq`, `.waitq`, `.lpq` and -`.ioq`. The latter two are only instantiated if specified to the -`get_event_loop` method. Official `uasyncio` does not have `.lpq` or `.ioq`. - -Tasks are appended to the bottom of the run queue and retrieved from the top; -in other words it is a First In First Out (FIFO) queue. The I/O queue is -similar. Tasks on `.waitq` and `.lpq` are sorted in order of the time when they -are to run, the task having the soonest time to run at the top. - -When a task issues `await asyncio.sleep(t)` or `await asyncio.sleep_ms(t)` and -t > 0 the task is placed on the wait queue. If t == 0 it is placed on the run -queue (by `.call_soon()`). Callbacks are placed on the queues in a similar way -to tasks. - -The following is a somewhat broad-brush explanation of an iteration of the -event loop's `run_forever()` method intended to aid in following the code. - -The method first checks the wait queue. Any tasks which have become due (or -overdue) are removed and placed on the run queue. - -The run queue is then processed. The number of tasks on it is determined: only -that number of tasks will be run. Because the run queue is FIFO this guarantees -that exactly those tasks which were on the queue at the start of processing -this queue will run (even when tasks are appended). - -The topmost task/callback is removed and run. If it is a callback the loop -iterates to the next entry. If it is a task, it runs then either yields or -raises an exception. If it yields, the return type is examined as described -above. If the task yields with a zero delay it will be appended to the run -queue, but as described above it will not be rescheduled in this pass through -the queue. If it yields a nonzero delay it will be added to `.waitq` (it has -already been removed from `.runq`). - -Once every task which was initially on the run queue has been scheduled, the -queue may or may not be empty depending on whether tasks yielded a zero delay. - -At the end of the outer loop a `delay` value is determined. This will be zero -if the run queue is not empty: tasks are ready for scheduling. If the run queue -is empty `delay` is determined from the time to run of the topmost (most -current) task on the wait queue. - -The `.wait()` method is called with this delay. If the delay is > 0 the -scheduler pauses for this period (polling I/O). On a zero delay I/O is checked -once: if nothing is pending it returns quickly. - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -## 4.1 Exceptions - -There are two "normal" cases where tasks raise an exception: when the task is -complete (`StopIteration`) and when it is cancelled (`CancelledError`). In both -these cases the exception is trapped and the loop proceeds to the next item on -the run queue - the task is simply not rescheduled. - -If an unhandled exception occurs in a task this will be propagated to the -caller of `run_forever()` or `run_until_complete` a explained in the tutorial. - -## 4.2 Task Cancellation and Timeouts - -The `cancel` function uses `pend_throw` to pass a `CancelledError` to the coro -to be cancelled. The generator's `.throw` and `.close` methods cause the coro -to execute code immediately. This is incorrect behaviour for a de-scheduled -coro. The `.pend_throw` method causes the exception to be processed the next -time the coro is scheduled. - -In the `fast_io` version the `cancel` function puts the task onto `.runq` or -`.ioq` for "immediate" excecution. In the case where the task is on `.waitq` or -`.lpq` the task ID is added to a `set` `.canned`. When the task reaches the top -of the timeq it is ignored and removed from `.canned`. This Python approach is -less efficient than that in the Paul Sokolovsky fork, but his approach uses a -special version of the C `utimeq` object and so requires his firmware. - -Timeouts use a similar mechanism. - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -# 5. Stream IO - -Stream I/O is an efficient way of polling stream devices using `select.poll`. -Device drivers for this mechanism must provide an `ioctl` method which reports -whether a read device has data ready, or whether a write device is capable of -accepting data. Stream I/O is handled via `StreamReader` and `StreamWriter` -instances (defined in `__init__.py`). - -## 5.1 StreamReader - -The class supports three read coros which work in a similar fashion. The coro -yields an `IORead` instance with the device to be polled as its arg. It is -rescheduled when `ioctl` has reported that some data is available. The coro -reads the device by calling the device driver's `read` or `readline` method. -If all available data has been read, the device's read methods must update the -status returned by its `ioctl` method. - -The `StreamReader` read coros iterate until the required data has been read, -when the coro yields `IOReadDone(object_to_poll)` before returning the data. If -during this process, `ioctl` reports that no data is available, the coro -yields `IORead(object_to_poll)`. This causes the coro to be descheduled until -data is again available. - -The mechanism which causes it to be rescheduled is discussed below (`.wait()`). - -When `IORead(object_to_poll)` is yielded the `EventLoop` calls `.add_reader()`. -This registers the device with `select.poll` as a reader, and saves the coro -for later rescheduling. - -The `PollEventLoop` maintains three dictionaries indexed by the `id` of the -object being polled. These are: - - * `rdobjmap` Value: the suspended read coro. - * `wrobjmap` Value: the suspended write coro (read and write coros may both be - in a suspended state). - * `flags` Value: bitmap of current poll flags. - -The `add_reader` method saves the coro in `.rdobjmap` and updates `.flags` and -the poll flags so that `ioctl` will respond to a `MP_STREAM_POLL_RD` query. - -When the `StreamReader` read method completes it yields -`IOReadDone(object_to_poll)`: this updates `.flags` and the poll flags so that -`ioctl` no longer responds to an `MP_STREAM_POLL_RD` query. - -## 5.2 StreamWriter - -This supports the `awrite` coro which works in a similar way to `StreamReader`, -yielding `IOWrite(object_to_poll)` until all data has been written, followed -by `IOWriteDone(object_to_poll)`. - -The mechanism is the same as for reading, except that when `ioctl` returns a -"ready" state for a writeable device it means the device is capable of writing -at least one character. - -## 5.3 PollEventLoop wait method - -When this is called the `Poll` instance is checked in a one-shot mode. In this -mode it will return either when `delay` has elapsed or when at least one device -is ready. - -The poller's `ipoll` method uses the iterator protocol to return successive -`(sock, ev)` tuples where `sock` is the device driver and `ev` is a bitmap of -read and write ready status for that device. The `.wait` method iterates -through each device requiring service. - -If the read bit is set (i.e. `ioctl` reported data available) the read coro is -retrieved from `.rdobjmap` and queued for scheduling. This is done via -`._call_io`: this puts the coro onto `.runq` or `.ioq` depending on whether an -I/O queue has been instantiated. - -Writing is handled similarly. - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -# 6. Modifying uasyncio - -The library is designed to be extensible. By following these guidelines a -module can be constructed which alters the functionality of asyncio without the -need to change the official library. Such a module may be used where `uasyncio` -is implemented as frozen bytecode as in official release binaries. - -Assume that the aim is to alter the event loop. The module should issue - -```python -from uasyncio import * -``` - -The event loop should be subclassed from `PollEventLoop` (defined in -`__init__.py`). - -The event loop is instantiated by the first call to `get_event_loop()`: this -creates a singleton instance. This is returned by every call to -`get_event_loop()`. On the assumption that the constructor arguments for the -new class differ from those of the base class, the module will need to redefine -`get_event_loop()` along the following lines: - -```python -_event_loop = None # The singleton instance -_event_loop_class = MyNewEventLoopClass # The class, not an instance -def get_event_loop(args): - global _event_loop - if _event_loop is None: - _event_loop = _event_loop_class(args) # Instantiate once only - return _event_loop -``` - -###### [Contents](./UNDER_THE_HOOD.md#0-contents) - -# 7. Links - -Initial discussion of priority I/O scheduling [here](https://github.com/micropython/micropython/issues/2664). - -MicroPython PR enabling stream device drivers to be written in Python -[PR #3836: io.IOBase](https://github.com/micropython/micropython/pull/3836). -Includes discussion of the read/write bug. - -My outstanding uasyncio PR's: fast I/O -[PR #287](https://github.com/micropython/micropython-lib/pull/287) improved -error reporting -[PR #292](https://github.com/micropython/micropython-lib/pull/292). - -This caught my attention for usefulness and compliance with CPython: -[PR #270](https://github.com/micropython/micropython-lib/pull/270). - -###### [Main README](./README.md) diff --git a/aledflash.py b/aledflash.py deleted file mode 100644 index 420a0d4..0000000 --- a/aledflash.py +++ /dev/null @@ -1,32 +0,0 @@ -# aledflash.py Demo/test program for MicroPython asyncio -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license -# Flashes the onboard LED's each at a different rate. Stops after ten seconds. -# Run on MicroPython board bare hardware - -import pyb -import uasyncio as asyncio - -async def killer(duration): - await asyncio.sleep(duration) - -async def toggle(objLED, time_ms): - while True: - await asyncio.sleep_ms(time_ms) - objLED.toggle() - -# TEST FUNCTION - -def test(duration): - loop = asyncio.get_event_loop() - duration = int(duration) - if duration > 0: - print("Flash LED's for {:3d} seconds".format(duration)) - leds = [pyb.LED(x) for x in range(1,5)] # Initialise all four on board LED's - for x, led in enumerate(leds): # Create a coroutine for each LED - t = int((0.2 + x/2) * 1000) - loop.create_task(toggle(leds[x], t)) - loop.run_until_complete(killer(duration)) - loop.close() - -test(10) diff --git a/apoll.py b/apoll.py deleted file mode 100644 index eeff59a..0000000 --- a/apoll.py +++ /dev/null @@ -1,64 +0,0 @@ -# Demonstration of a device driver using a coroutine to poll a dvice. -# Runs on Pyboard: displays results from the onboard accelerometer. -# Uses crude filtering to discard noisy data. - -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license - -import uasyncio as asyncio -import pyb -import utime as time - -class Accelerometer(object): - threshold_squared = 16 - def __init__(self, accelhw, timeout): - self.loop = asyncio.get_event_loop() - self.accelhw = accelhw - self.timeout = timeout - self.last_change = self.loop.time() - self.coords = [accelhw.x(), accelhw.y(), accelhw.z()] - - def dsquared(self, xyz): # Return the square of the distance between this and a passed - return sum(map(lambda p, q : (p-q)**2, self.coords, xyz)) # acceleration vector - - def poll(self): # Device is noisy. Only update if change exceeds a threshold - xyz = [self.accelhw.x(), self.accelhw.y(), self.accelhw.z()] - if self.dsquared(xyz) > Accelerometer.threshold_squared: - self.coords = xyz - self.last_change = self.loop.time() - return 0 - return time.ticks_diff(self.loop.time(), self.last_change) - - def vector(self): - return self.coords - - def timed_out(self): # Time since last change or last timeout report - if time.ticks_diff(self.loop.time(), self.last_change) > self.timeout: - self.last_change = self.loop.time() - return True - return False - -async def accel_coro(timeout = 2000): - loop = asyncio.get_event_loop() - accelhw = pyb.Accel() # Instantiate accelerometer hardware - await asyncio.sleep_ms(30) # Allow it to settle - accel = Accelerometer(accelhw, timeout) - while True: - result = accel.poll() - if result == 0: # Value has changed - x, y, z = accel.vector() - print("Value x:{:3d} y:{:3d} z:{:3d}".format(x, y, z)) - elif accel.timed_out(): # Report every 2 secs - print("Timeout waiting for accelerometer change") - await asyncio.sleep_ms(100) # Poll every 100ms - - -async def main(delay): - print('Testing accelerometer for {} secs. Move the Pyboard!'.format(delay)) - print('Test runs for 20s.') - await asyncio.sleep(delay) - print('Test complete!') - -loop = asyncio.get_event_loop() -loop.create_task(accel_coro()) -loop.run_until_complete(main(20)) diff --git a/aqtest.py b/aqtest.py deleted file mode 100644 index afb5ffb..0000000 --- a/aqtest.py +++ /dev/null @@ -1,35 +0,0 @@ -# aqtest.py Demo/test program for MicroPython library micropython-uasyncio.queues -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license - -import uasyncio as asyncio - -from uasyncio.queues import Queue - -q = Queue() - -async def slow_process(): - await asyncio.sleep(2) - return 42 - -async def bar(): - print('Waiting for slow process.') - result = await slow_process() - print('Putting result onto queue') - await q.put(result) # Put result on q - -async def foo(): - print("Running foo()") - result = await(q.get()) - print('Result was {}'.format(result)) - -async def main(delay): - await asyncio.sleep(delay) - print("I've seen starships burn off the shoulder of Orion...") - print("Time to die...") - -print('Test takes 3 secs') -loop = asyncio.get_event_loop() -loop.create_task(foo()) -loop.create_task(bar()) -loop.run_until_complete(main(3)) diff --git a/astests.py b/astests.py deleted file mode 100644 index 0120be5..0000000 --- a/astests.py +++ /dev/null @@ -1,132 +0,0 @@ -# Test/demo programs for the aswitch module. -# Tested on Pyboard but should run on other microcontroller platforms -# running MicroPython with uasyncio library. -# Author: Peter Hinch. -# Copyright Peter Hinch 2017-2018 Released under the MIT license. - -from machine import Pin -from pyb import LED -from aswitch import Switch, Pushbutton -import uasyncio as asyncio - -helptext = ''' -Test using switch or pushbutton between X1 and gnd. -Ground pin X2 to terminate test. -Soft reset (ctrl-D) after each test. - -''' -tests = ''' -Available tests: -test_sw Switch test -test_swcb Switch with callback -test_btn Pushutton launching coros -test_btncb Pushbutton launching callbacks -''' -print(tests) - -# Pulse an LED (coroutine) -async def pulse(led, ms): - led.on() - await asyncio.sleep_ms(ms) - led.off() - -# Toggle an LED (callback) -def toggle(led): - led.toggle() - -# Quit test by connecting X2 to ground -async def killer(): - pin = Pin('X2', Pin.IN, Pin.PULL_UP) - while pin.value(): - await asyncio.sleep_ms(50) - -# Test for the Switch class passing coros -def test_sw(): - s = ''' -close pulses green -open pulses red -''' - print('Test of switch scheduling coroutines.') - print(helptext) - print(s) - pin = Pin('X1', Pin.IN, Pin.PULL_UP) - red = LED(1) - green = LED(2) - sw = Switch(pin) - # Register coros to launch on contact close and open - sw.close_func(pulse, (green, 1000)) - sw.open_func(pulse, (red, 1000)) - loop = asyncio.get_event_loop() - loop.run_until_complete(killer()) - -# Test for the switch class with a callback -def test_swcb(): - s = ''' -close toggles red -open toggles green -''' - print('Test of switch executing callbacks.') - print(helptext) - print(s) - pin = Pin('X1', Pin.IN, Pin.PULL_UP) - red = LED(1) - green = LED(2) - sw = Switch(pin) - # Register a coro to launch on contact close - sw.close_func(toggle, (red,)) - sw.open_func(toggle, (green,)) - loop = asyncio.get_event_loop() - loop.run_until_complete(killer()) - -# Test for the Pushbutton class (coroutines) -# Pass True to test suppress -def test_btn(suppress=False, lf=True, df=True): - s = ''' -press pulses red -release pulses green -double click pulses yellow -long press pulses blue -''' - print('Test of pushbutton scheduling coroutines.') - print(helptext) - print(s) - pin = Pin('X1', Pin.IN, Pin.PULL_UP) - red = LED(1) - green = LED(2) - yellow = LED(3) - blue = LED(4) - pb = Pushbutton(pin, suppress) - pb.press_func(pulse, (red, 1000)) - pb.release_func(pulse, (green, 1000)) - if df: - print('Doubleclick enabled') - pb.double_func(pulse, (yellow, 1000)) - if lf: - print('Long press enabled') - pb.long_func(pulse, (blue, 1000)) - loop = asyncio.get_event_loop() - loop.run_until_complete(killer()) - -# Test for the Pushbutton class (callbacks) -def test_btncb(): - s = ''' -press toggles red -release toggles green -double click toggles yellow -long press toggles blue -''' - print('Test of pushbutton executing callbacks.') - print(helptext) - print(s) - pin = Pin('X1', Pin.IN, Pin.PULL_UP) - red = LED(1) - green = LED(2) - yellow = LED(3) - blue = LED(4) - pb = Pushbutton(pin) - pb.press_func(toggle, (red,)) - pb.release_func(toggle, (green,)) - pb.double_func(toggle, (yellow,)) - pb.long_func(toggle, (blue,)) - loop = asyncio.get_event_loop() - loop.run_until_complete(killer()) diff --git a/aswitch.py b/aswitch.py deleted file mode 100644 index 4269ce9..0000000 --- a/aswitch.py +++ /dev/null @@ -1,231 +0,0 @@ -# aswitch.py Switch and pushbutton classes for asyncio -# Delay_ms A retriggerable delay class. Can schedule a coro on timeout. -# Switch Simple debounced switch class for normally open grounded switch. -# Pushbutton extend the above to support logical state, long press and -# double-click events -# Tested on Pyboard but should run on other microcontroller platforms -# running MicroPython and uasyncio. - -# The MIT License (MIT) -# -# Copyright (c) 2017 Peter Hinch -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - -import uasyncio as asyncio -import utime as time -# Remove dependency on asyn to save RAM: -# launch: run a callback or initiate a coroutine depending on which is passed. -async def _g(): - pass -type_coro = type(_g()) - -# If a callback is passed, run it and return. -# If a coro is passed initiate it and return. -# coros are passed by name i.e. not using function call syntax. -def launch(func, tup_args): - res = func(*tup_args) - if isinstance(res, type_coro): - loop = asyncio.get_event_loop() - loop.create_task(res) - - -class Delay_ms: - verbose = False - def __init__(self, func=None, args=(), can_alloc=True, duration=1000): - self.func = func - self.args = args - self.can_alloc = can_alloc - self.duration = duration # Default duration - self._tstop = None # Killer not running - self._running = False # Timer not running - self.loop = asyncio.get_event_loop() - if not can_alloc: - self.loop.create_task(self._run()) - - async def _run(self): - while True: - if not self._running: # timer not running - await asyncio.sleep_ms(0) - else: - await self._killer() - - def stop(self): - self._running = False - # If uasyncio is ever fixed we should cancel .killer - - def trigger(self, duration=0): # Update end time - self._running = True - if duration <= 0: - duration = self.duration - tn = time.ticks_add(time.ticks_ms(), duration) # new end time - self.verbose and self._tstop is not None and self._tstop > tn \ - and print("Warning: can't reduce Delay_ms time.") - # Start killer if can allocate and killer is not running - sk = self.can_alloc and self._tstop is None - # The following indicates ._killer is running: it will be - # started either here or in ._run - self._tstop = tn - if sk: # ._killer stops the delay when its period has elapsed - self.loop.create_task(self._killer()) - - def running(self): - return self._running - - __call__ = running - - async def _killer(self): - twait = time.ticks_diff(self._tstop, time.ticks_ms()) - while twait > 0: # Must loop here: might be retriggered - await asyncio.sleep_ms(twait) - if self._tstop is None: - break # Return if stop() called during wait - twait = time.ticks_diff(self._tstop, time.ticks_ms()) - if self._running and self.func is not None: - launch(self.func, self.args) # Timed out: execute callback - self._tstop = None # killer not running - self._running = False # timer is stopped - -class Switch: - debounce_ms = 50 - def __init__(self, pin): - self.pin = pin # Should be initialised for input with pullup - self._open_func = False - self._close_func = False - self.switchstate = self.pin.value() # Get initial state - loop = asyncio.get_event_loop() - loop.create_task(self.switchcheck()) # Thread runs forever - - def open_func(self, func, args=()): - self._open_func = func - self._open_args = args - - def close_func(self, func, args=()): - self._close_func = func - self._close_args = args - - # Return current state of switch (0 = pressed) - def __call__(self): - return self.switchstate - - async def switchcheck(self): - while True: - state = self.pin.value() - if state != self.switchstate: - # State has changed: act on it now. - self.switchstate = state - if state == 0 and self._close_func: - launch(self._close_func, self._close_args) - elif state == 1 and self._open_func: - launch(self._open_func, self._open_args) - # Ignore further state changes until switch has settled - await asyncio.sleep_ms(Switch.debounce_ms) - -# An alternative Pushbutton solution with lower RAM use is available here -# https://github.com/kevinkk525/pysmartnode/blob/dev/pysmartnode/utils/abutton.py -class Pushbutton: - debounce_ms = 50 - long_press_ms = 1000 - double_click_ms = 400 - def __init__(self, pin, suppress=False): - self.pin = pin # Initialise for input - self._supp = suppress - self._dblpend = False # Doubleclick waiting for 2nd click - self._dblran = False # Doubleclick executed user function - self._tf = False - self._ff = False - self._df = False - self._lf = False - self._ld = False # Delay_ms instance for long press - self._dd = False # Ditto for doubleclick - self.sense = pin.value() # Convert from electrical to logical value - self.state = self.rawstate() # Initial state - loop = asyncio.get_event_loop() - loop.create_task(self.buttoncheck()) # Thread runs forever - - def press_func(self, func, args=()): - self._tf = func - self._ta = args - - def release_func(self, func, args=()): - self._ff = func - self._fa = args - - def double_func(self, func, args=()): - self._df = func - self._da = args - - def long_func(self, func, args=()): - self._lf = func - self._la = args - - # Current non-debounced logical button state: True == pressed - def rawstate(self): - return bool(self.pin.value() ^ self.sense) - - # Current debounced state of button (True == pressed) - def __call__(self): - return self.state - - def _ddto(self): # Doubleclick timeout: no doubleclick occurred - self._dblpend = False - if self._supp and not self.state: - if not self._ld or (self._ld and not self._ld()): - launch(self._ff, self._fa) - - async def buttoncheck(self): - if self._lf: # Instantiate timers if funcs exist - self._ld = Delay_ms(self._lf, self._la) - if self._df: - self._dd = Delay_ms(self._ddto) - while True: - state = self.rawstate() - # State has changed: act on it now. - if state != self.state: - self.state = state - if state: # Button pressed: launch pressed func - if self._tf: - launch(self._tf, self._ta) - if self._lf: # There's a long func: start long press delay - self._ld.trigger(Pushbutton.long_press_ms) - if self._df: - if self._dd(): # Second click: timer running - self._dd.stop() - self._dblpend = False - self._dblran = True # Prevent suppressed launch on release - launch(self._df, self._da) - else: - # First click: start doubleclick timer - self._dd.trigger(Pushbutton.double_click_ms) - self._dblpend = True # Prevent suppressed launch on release - else: # Button release. Is there a release func? - if self._ff: - if self._supp: - d = self._ld - # If long delay exists, is running and doubleclick status is OK - if not self._dblpend and not self._dblran: - if (d and d()) or not d: - launch(self._ff, self._fa) - else: - launch(self._ff, self._fa) - if self._ld: - self._ld.stop() # Avoid interpreting a second click as a long push - self._dblran = False - # Ignore state changes until switch has settled - await asyncio.sleep_ms(Pushbutton.debounce_ms) diff --git a/asyn.py b/asyn.py deleted file mode 100644 index c87c175..0000000 --- a/asyn.py +++ /dev/null @@ -1,470 +0,0 @@ -# asyn.py 'micro' synchronisation primitives for uasyncio -# Test/demo programs asyntest.py, barrier_test.py -# Provides Lock, Event, Barrier, Semaphore, BoundedSemaphore, Condition, -# NamedTask and Cancellable classes, also sleep coro. -# Updated 31 Dec 2017 for uasyncio.core V1.6 and to provide task cancellation. - -# The MIT License (MIT) -# -# Copyright (c) 2017 Peter Hinch -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - -# CPython 3.5 compatibility -# (ignore RuntimeWarning: coroutine '_g' was never awaited) - -try: - import uasyncio as asyncio -except ImportError: - import asyncio - - -async def _g(): - pass -type_coro = type(_g()) - -# If a callback is passed, run it and return. -# If a coro is passed initiate it and return. -# coros are passed by name i.e. not using function call syntax. -def launch(func, tup_args): - res = func(*tup_args) - if isinstance(res, type_coro): - loop = asyncio.get_event_loop() - loop.create_task(res) - - -# To access a lockable resource a coro should issue -# async with lock_instance: -# access the locked resource - -# Alternatively: -# await lock.acquire() -# try: -# do stuff with locked resource -# finally: -# lock.release -# Uses normal scheduling on assumption that locks are held briefly. -class Lock(): - def __init__(self, delay_ms=0): - self._locked = False - self.delay_ms = delay_ms - - def locked(self): - return self._locked - - async def __aenter__(self): - await self.acquire() - return self - - async def __aexit__(self, *args): - self.release() - await asyncio.sleep(0) - - async def acquire(self): - while True: - if self._locked: - await asyncio.sleep_ms(self.delay_ms) - else: - self._locked = True - break - - def release(self): - if not self._locked: - raise RuntimeError('Attempt to release a lock which has not been set') - self._locked = False - - -# A coro waiting on an event issues await event -# A coro rasing the event issues event.set() -# When all waiting coros have run -# event.clear() should be issued -class Event(): - def __init__(self, delay_ms=0): - self.delay_ms = delay_ms - self.clear() - - def clear(self): - self._flag = False - self._data = None - - async def wait(self): # CPython comptaibility - while not self._flag: - await asyncio.sleep_ms(self.delay_ms) - - def __await__(self): - while not self._flag: - await asyncio.sleep_ms(self.delay_ms) - - __iter__ = __await__ - - def is_set(self): - return self._flag - - def set(self, data=None): - self._flag = True - self._data = data - - def value(self): - return self._data - -# A Barrier synchronises N coros. Each issues await barrier. -# Execution pauses until all other participant coros are waiting on it. -# At that point the callback is executed. Then the barrier is 'opened' and -# execution of all participants resumes. - -# The nowait arg is to support task cancellation. It enables usage where one or -# more coros can register that they have reached the barrier without waiting -# for it. Any coros waiting normally on the barrier will pause until all -# non-waiting coros have passed the barrier and all waiting ones have reached -# it. The use of nowait promotes efficiency by enabling tasks which have been -# cancelled to leave the task queue as soon as possible. - -class Barrier(): - def __init__(self, participants, func=None, args=()): - self._participants = participants - self._func = func - self._args = args - self._reset(True) - - def __await__(self): - self._update() - if self._at_limit(): # All other threads are also at limit - if self._func is not None: - launch(self._func, self._args) - self._reset(not self._down) # Toggle direction to release others - return - - direction = self._down - while True: # Wait until last waiting thread changes the direction - if direction != self._down: - return - await asyncio.sleep_ms(0) - - __iter__ = __await__ - - def trigger(self): - self._update() - if self._at_limit(): # All other threads are also at limit - if self._func is not None: - launch(self._func, self._args) - self._reset(not self._down) # Toggle direction to release others - - def _reset(self, down): - self._down = down - self._count = self._participants if down else 0 - - def busy(self): - if self._down: - done = self._count == self._participants - else: - done = self._count == 0 - return not done - - def _at_limit(self): # Has count reached up or down limit? - limit = 0 if self._down else self._participants - return self._count == limit - - def _update(self): - self._count += -1 if self._down else 1 - if self._count < 0 or self._count > self._participants: - raise ValueError('Too many tasks accessing Barrier') - -# A Semaphore is typically used to limit the number of coros running a -# particular piece of code at once. The number is defined in the constructor. -class Semaphore(): - def __init__(self, value=1): - self._count = value - - async def __aenter__(self): - await self.acquire() - return self - - async def __aexit__(self, *args): - self.release() - await asyncio.sleep(0) - - async def acquire(self): - while self._count == 0: - await asyncio.sleep_ms(0) - self._count -= 1 - - def release(self): - self._count += 1 - -class BoundedSemaphore(Semaphore): - def __init__(self, value=1): - super().__init__(value) - self._initial_value = value - - def release(self): - if self._count < self._initial_value: - self._count += 1 - else: - raise ValueError('Semaphore released more than acquired') - -# Task Cancellation -try: - StopTask = asyncio.CancelledError # More descriptive name -except AttributeError: - raise OSError('asyn.py requires uasyncio V1.7.1 or above.') - -class TaskId(): - def __init__(self, taskid): - self.taskid = taskid - - def __call__(self): - return self.taskid - -# Sleep coro breaks up a sleep into shorter intervals to ensure a rapid -# response to StopTask exceptions. Only relevant to official uasyncio V2.0. -async def sleep(t, granularity=100): # 100ms default - if granularity <= 0: - raise ValueError('sleep granularity must be > 0') - t = int(t * 1000) # ms - if t <= granularity: - await asyncio.sleep_ms(t) - else: - n, rem = divmod(t, granularity) - for _ in range(n): - await asyncio.sleep_ms(granularity) - await asyncio.sleep_ms(rem) - -# Anonymous cancellable tasks. These are members of a group which is identified -# by a user supplied name/number (default 0). Class method cancel_all() cancels -# all tasks in a group and awaits confirmation. Confirmation of ending (whether -# normally or by cancellation) is signalled by a task calling the _stopped() -# class method. Handled by the @cancellable decorator. - - -class Cancellable(): - task_no = 0 # Generated task ID, index of tasks dict - tasks = {} # Value is [coro, group, barrier] indexed by integer task_no - - @classmethod - def _cancel(cls, task_no): - task = cls.tasks[task_no][0] - asyncio.cancel(task) - - @classmethod - async def cancel_all(cls, group=0, nowait=False): - tokill = cls._get_task_nos(group) - barrier = Barrier(len(tokill) + 1) # Include this task - for task_no in tokill: - cls.tasks[task_no][2] = barrier - cls._cancel(task_no) - if nowait: - barrier.trigger() - else: - await barrier - - @classmethod - def _is_running(cls, group=0): - tasks = cls._get_task_nos(group) - if tasks == []: - return False - for task_no in tasks: - barrier = cls.tasks[task_no][2] - if barrier is None: # Running, not yet cancelled - return True - if barrier.busy(): - return True - return False - - @classmethod - def _get_task_nos(cls, group): # Return task nos in a group - return [task_no for task_no in cls.tasks if cls.tasks[task_no][1] == group] - - @classmethod - def _get_group(cls, task_no): # Return group given a task_no - return cls.tasks[task_no][1] - - @classmethod - def _stopped(cls, task_no): - if task_no in cls.tasks: - barrier = cls.tasks[task_no][2] - if barrier is not None: # Cancellation in progress - barrier.trigger() - del cls.tasks[task_no] - - def __init__(self, gf, *args, group=0, **kwargs): - task = gf(TaskId(Cancellable.task_no), *args, **kwargs) - if task in self.tasks: - raise ValueError('Task already exists.') - self.tasks[Cancellable.task_no] = [task, group, None] - self.task_no = Cancellable.task_no # For subclass - Cancellable.task_no += 1 - self.task = task - - def __call__(self): - return self.task - - def __await__(self): # Return any value returned by task. - return (yield from self.task) - - __iter__ = __await__ - - -# @cancellable decorator - -def cancellable(f): - def new_gen(*args, **kwargs): - if isinstance(args[0], TaskId): # Not a bound method - task_id = args[0] - g = f(*args[1:], **kwargs) - else: # Task ID is args[1] if a bound method - task_id = args[1] - args = (args[0],) + args[2:] - g = f(*args, **kwargs) - try: - res = await g - return res - finally: - NamedTask._stopped(task_id) - return new_gen - -# The NamedTask class enables a coro to be identified by a user defined name. -# It constrains Cancellable to allow groups of one coro only. -# It maintains a dict of barriers indexed by name. -class NamedTask(Cancellable): - instances = {} - - @classmethod - async def cancel(cls, name, nowait=True): - if name in cls.instances: - await cls.cancel_all(group=name, nowait=nowait) - return True - return False - - @classmethod - def is_running(cls, name): - return cls._is_running(group=name) - - @classmethod - def _stopped(cls, task_id): # On completion remove it - name = cls._get_group(task_id()) # Convert task_id to task_no - if name in cls.instances: - instance = cls.instances[name] - barrier = instance.barrier - if barrier is not None: - barrier.trigger() - del cls.instances[name] - Cancellable._stopped(task_id()) - - def __init__(self, name, gf, *args, barrier=None, **kwargs): - if name in self.instances: - raise ValueError('Task name "{}" already exists.'.format(name)) - super().__init__(gf, *args, group=name, **kwargs) - self.barrier = barrier - self.instances[name] = self - - -# @namedtask -namedtask = cancellable # compatibility with old code - -# Condition class - -class Condition(): - def __init__(self, lock=None): - self.lock = Lock() if lock is None else lock - self.events = [] - - async def acquire(self): - await self.lock.acquire() - -# enable this syntax: -# with await condition [as cond]: - def __await__(self): - yield from self.lock.acquire() - return self - - __iter__ = __await__ - - def __enter__(self): - return self - - def __exit__(self, *_): - self.lock.release() - - def locked(self): - return self.lock.locked() - - def release(self): - self.lock.release() # Will raise RuntimeError if not locked - - def notify(self, n=1): # Caller controls lock - if not self.lock.locked(): - raise RuntimeError('Condition notify with lock not acquired.') - for _ in range(min(n, len(self.events))): - ev = self.events.pop() - ev.set() - - def notify_all(self): - self.notify(len(self.events)) - - async def wait(self): - if not self.lock.locked(): - raise RuntimeError('Condition wait with lock not acquired.') - ev = Event() - self.events.append(ev) - self.lock.release() - await ev - await self.lock.acquire() - assert ev not in self.events, 'condition wait assertion fail' - return True # CPython compatibility - - async def wait_for(self, predicate): - result = predicate() - while not result: - await self.wait() - result = predicate() - return result - -# Provide functionality similar to asyncio.gather() - -class Gather(): - def __init__(self, gatherables): - ncoros = len(gatherables) - self.barrier = Barrier(ncoros + 1) - self.results = [None] * ncoros - loop = asyncio.get_event_loop() - for n, gatherable in enumerate(gatherables): - loop.create_task(self.wrap(gatherable, n)()) - - def __iter__(self): - yield from self.barrier.__await__() - return self.results - - def wrap(self, gatherable, idx): - async def wrapped(): - coro, args, kwargs = gatherable() - try: - tim = kwargs.pop('timeout') - except KeyError: - self.results[idx] = await coro(*args, **kwargs) - else: - self.results[idx] = await asyncio.wait_for(coro(*args, **kwargs), tim) - self.barrier.trigger() - return wrapped - -class Gatherable(): - def __init__(self, coro, *args, **kwargs): - self.arguments = coro, args, kwargs - - def __call__(self): - return self.arguments diff --git a/asyn_demos.py b/asyn_demos.py deleted file mode 100644 index e4781b1..0000000 --- a/asyn_demos.py +++ /dev/null @@ -1,131 +0,0 @@ -# asyn_demos.py Simple demos of task cancellation -# Test/demo of official asyncio library and official Lock class - -# The MIT License (MIT) -# -# Copyright (c) 2017 Peter Hinch -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - -import uasyncio as asyncio -import asyn - -def print_tests(): - st = '''Minimal demo programs of uasyncio task cancellation. -Issue ctrl-D to soft reset the board between test runs. -Available demos: -cancel_test() Demo of Cancellable tasks. -named_test() Demo of NamedTask. -method_test() Cancellable and NamedTask coros as bound methods. -''' - print('\x1b[32m') - print(st) - print('\x1b[39m') - -print_tests() - -# Cancellable task minimal example -@asyn.cancellable -async def print_nums(num): - while True: - print(num) - num += 1 - await asyn.sleep(1) - -@asyn.cancellable -async def add_one(num): - num += 1 - await asyn.sleep(1) - return num - -async def run_cancel_test(loop): - res = await asyn.Cancellable(add_one, 41) - print('Result: ', res) - loop.create_task(asyn.Cancellable(print_nums, res)()) - await asyn.sleep(7.5) - # Cancel any cancellable tasks still running - await asyn.Cancellable.cancel_all() - print('Done') - -def cancel_test(): - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test(loop)) - -# NamedTask minimal example - -@asyn.cancellable -async def print_nums_named(num): - while True: - print(num) - num += 1 - await asyn.sleep(1) - -@asyn.cancellable -async def add_one_named(num): - num += 1 - await asyn.sleep(1) - return num - -async def run_named_test(loop): - res = await asyn.NamedTask('not cancelled', add_one_named, 99) - print('Result: ', res) - loop.create_task(asyn.NamedTask('print nums', print_nums_named, res)()) - await asyn.sleep(7.5) - asyn.NamedTask.cancel('not cancelled') # Nothing to do: task has finished - asyn.NamedTask.cancel('print nums') # Stop the continuously running task - print('Done') - -def named_test(): - loop = asyncio.get_event_loop() - loop.run_until_complete(run_named_test(loop)) - -# Tasks as bound methods - -class CanDemo(): - async def start(self, loop): - loop.create_task(asyn.Cancellable(self.foo, 1)()) # 3 instances in default group 0 - loop.create_task(asyn.Cancellable(self.foo, 2)()) - loop.create_task(asyn.Cancellable(self.foo, 3)()) - loop.create_task(asyn.NamedTask('my bar', self.bar, 4)()) - print('bar running status is', asyn.NamedTask.is_running('my bar')) - await asyncio.sleep(4.5) - await asyn.NamedTask.cancel('my bar') - print('bar instance scheduled for cancellation.') - await asyn.Cancellable.cancel_all() - print('foo instances have been cancelled.') - await asyncio.sleep(0.2) # Allow for 100ms latency in bar() - print('bar running status is', asyn.NamedTask.is_running('my bar')) - print('Done') - - @asyn.cancellable - async def foo(self, arg): - while True: - await asyn.sleep(1) - print('foo running, arg', arg) - - @asyn.cancellable - async def bar(self, arg): - while True: - await asyn.sleep(1) - print('bar running, arg', arg) - -def method_test(): - cantest = CanDemo() - loop = asyncio.get_event_loop() - loop.run_until_complete(cantest.start(loop)) diff --git a/asyntest.py b/asyntest.py deleted file mode 100644 index 26c874c..0000000 --- a/asyntest.py +++ /dev/null @@ -1,426 +0,0 @@ -# asyntest.py Test/demo of the 'micro' Event, Barrier and Semaphore classes -# Test/demo of official asyncio library and official Lock class - -# The MIT License (MIT) -# -# Copyright (c) 2017-2018 Peter Hinch -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - -# CPython 3.5 compatibility -# (ignore RuntimeWarning: coroutine '_g' was never awaited) - -try: - import uasyncio as asyncio -except ImportError: - import asyncio - -import asyn - -def print_tests(): - st = '''Available functions: -print_tests() Print this list. -ack_test() Test event acknowledge. -event_test() Test Event and Lock objects. -barrier_test() Test the Barrier class. -semaphore_test(bounded=False) Test Semaphore or BoundedSemaphore. -condition_test(new=False) Test the Condition class. Set arg True for new uasyncio. -gather_test() Test the Gather class - -Recommended to issue ctrl-D after running each test. -''' - print('\x1b[32m') - print(st) - print('\x1b[39m') - -print_tests() - -def printexp(exp, runtime=0): - print('Expected output:') - print('\x1b[32m') - print(exp) - print('\x1b[39m') - if runtime: - print('Running (runtime = {}s):'.format(runtime)) - else: - print('Running (runtime < 1s):') - -# ************ Test Event class ************ -# Demo use of acknowledge event - -async def event_wait(event, ack_event, n): - await event - print('Eventwait {} got event with value {}'.format(n, event.value())) - ack_event.set() - -async def run_ack(): - loop = asyncio.get_event_loop() - event = asyn.Event() - ack1 = asyn.Event() - ack2 = asyn.Event() - count = 0 - while True: - loop.create_task(event_wait(event, ack1, 1)) - loop.create_task(event_wait(event, ack2, 2)) - event.set(count) - count += 1 - print('event was set') - await ack1 - ack1.clear() - print('Cleared ack1') - await ack2 - ack2.clear() - print('Cleared ack2') - event.clear() - print('Cleared event') - await asyncio.sleep(1) - -async def ack_coro(delay): - await asyncio.sleep(delay) - print("I've seen attack ships burn on the shoulder of Orion...") - print("Time to die...") - -def ack_test(): - printexp('''event was set -Eventwait 1 got event with value 0 -Eventwait 2 got event with value 0 -Cleared ack1 -Cleared ack2 -Cleared event -event was set -Eventwait 1 got event with value 1 -Eventwait 2 got event with value 1 - -... text omitted ... - -Eventwait 1 got event with value 9 -Eventwait 2 got event with value 9 -Cleared ack1 -Cleared ack2 -Cleared event -I've seen attack ships burn on the shoulder of Orion... -Time to die... -''', 10) - loop = asyncio.get_event_loop() - loop.create_task(run_ack()) - loop.run_until_complete(ack_coro(10)) - -# ************ Test Lock and Event classes ************ - -async def run_lock(n, lock): - print('run_lock {} waiting for lock'.format(n)) - await lock.acquire() - print('run_lock {} acquired lock'.format(n)) - await asyncio.sleep(1) # Delay to demo other coros waiting for lock - lock.release() - print('run_lock {} released lock'.format(n)) - -async def eventset(event): - print('Waiting 5 secs before setting event') - await asyncio.sleep(5) - event.set() - print('event was set') - -async def eventwait(event): - print('waiting for event') - await event - print('got event') - event.clear() - -async def run_event_test(): - print('Test Lock class') - loop = asyncio.get_event_loop() - lock = asyn.Lock() - loop.create_task(run_lock(1, lock)) - loop.create_task(run_lock(2, lock)) - loop.create_task(run_lock(3, lock)) - print('Test Event class') - event = asyn.Event() - loop.create_task(eventset(event)) - await eventwait(event) # run_event_test runs fast until this point - print('Event status {}'.format('Incorrect' if event.is_set() else 'OK')) - print('Tasks complete') - -def event_test(): - printexp('''Test Lock class -Test Event class -waiting for event -run_lock 1 waiting for lock -run_lock 1 acquired lock -run_lock 2 waiting for lock -run_lock 3 waiting for lock -Waiting 5 secs before setting event -run_lock 1 released lock -run_lock 2 acquired lock -run_lock 2 released lock -run_lock 3 acquired lock -run_lock 3 released lock -event was set -got event -Event status OK -Tasks complete -''', 5) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_event_test()) - -# ************ Barrier test ************ - -async def killer(duration): - await asyncio.sleep(duration) - -def callback(text): - print(text) - -async def report(barrier): - for i in range(5): - print('{} '.format(i), end='') - await barrier - -def barrier_test(): - printexp('''0 0 0 Synch -1 1 1 Synch -2 2 2 Synch -3 3 3 Synch -4 4 4 Synch -''') - barrier = asyn.Barrier(3, callback, ('Synch',)) - loop = asyncio.get_event_loop() - for _ in range(3): - loop.create_task(report(barrier)) - loop.run_until_complete(killer(2)) - loop.close() - -# ************ Semaphore test ************ - -async def run_sema(n, sema, barrier): - print('run_sema {} trying to access semaphore'.format(n)) - async with sema: - print('run_sema {} acquired semaphore'.format(n)) - # Delay demonstrates other coros waiting for semaphore - await asyncio.sleep(1 + n/10) # n/10 ensures deterministic printout - print('run_sema {} has released semaphore'.format(n)) - barrier.trigger() - -async def run_sema_test(bounded): - num_coros = 5 - loop = asyncio.get_event_loop() - barrier = asyn.Barrier(num_coros + 1) - if bounded: - semaphore = asyn.BoundedSemaphore(3) - else: - semaphore = asyn.Semaphore(3) - for n in range(num_coros): - loop.create_task(run_sema(n, semaphore, barrier)) - await barrier # Quit when all coros complete - try: - semaphore.release() - except ValueError: - print('Bounded semaphore exception test OK') - -def semaphore_test(bounded=False): - if bounded: - exp = '''run_sema 0 trying to access semaphore -run_sema 0 acquired semaphore -run_sema 1 trying to access semaphore -run_sema 1 acquired semaphore -run_sema 2 trying to access semaphore -run_sema 2 acquired semaphore -run_sema 3 trying to access semaphore -run_sema 4 trying to access semaphore -run_sema 0 has released semaphore -run_sema 4 acquired semaphore -run_sema 1 has released semaphore -run_sema 3 acquired semaphore -run_sema 2 has released semaphore -run_sema 4 has released semaphore -run_sema 3 has released semaphore -Bounded semaphore exception test OK - -Exact sequence of acquisition may vary when 3 and 4 compete for semaphore.''' - else: - exp = '''run_sema 0 trying to access semaphore -run_sema 0 acquired semaphore -run_sema 1 trying to access semaphore -run_sema 1 acquired semaphore -run_sema 2 trying to access semaphore -run_sema 2 acquired semaphore -run_sema 3 trying to access semaphore -run_sema 4 trying to access semaphore -run_sema 0 has released semaphore -run_sema 3 acquired semaphore -run_sema 1 has released semaphore -run_sema 4 acquired semaphore -run_sema 2 has released semaphore -run_sema 3 has released semaphore -run_sema 4 has released semaphore - -Exact sequence of acquisition may vary when 3 and 4 compete for semaphore.''' - printexp(exp, 3) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_sema_test(bounded)) - -# ************ Condition test ************ - -cond = asyn.Condition() -tim = 0 - -@asyn.cancellable -async def cond01(): - while True: - await asyncio.sleep(2) - with await cond: - cond.notify(2) # Notify 2 tasks - -@asyn.cancellable -async def cond03(): # Maintain a count of seconds - global tim - await asyncio.sleep(0.5) - while True: - await asyncio.sleep(1) - tim += 1 - -async def cond01_new(): - while True: - await asyncio.sleep(2) - with await cond: - cond.notify(2) # Notify 2 tasks - -async def cond03_new(): # Maintain a count of seconds - global tim - await asyncio.sleep(0.5) - while True: - await asyncio.sleep(1) - tim += 1 - -async def cond02(n, barrier): - with await cond: - print('cond02', n, 'Awaiting notification.') - await cond.wait() - print('cond02', n, 'triggered. tim =', tim) - barrier.trigger() - -def predicate(): - return tim >= 8 # 12 - -async def cond04(n, barrier): - with await cond: - print('cond04', n, 'Awaiting notification and predicate.') - await cond.wait_for(predicate) - print('cond04', n, 'triggered. tim =', tim) - barrier.trigger() - -async def cond_go(loop, new): - ntasks = 7 - barrier = asyn.Barrier(ntasks + 1) - if new: - t1 = asyncio.create_task(cond01_new()) - t3 = asyncio.create_task(cond03_new()) - else: - loop.create_task(asyn.Cancellable(cond01)()) - loop.create_task(asyn.Cancellable(cond03)()) - for n in range(ntasks): - loop.create_task(cond02(n, barrier)) - await barrier # All instances of cond02 have completed - # Test wait_for - barrier = asyn.Barrier(2) - loop.create_task(cond04(99, barrier)) - await barrier - # cancel continuously running coros. - if new: - t1.cancel() - t3.cancel() - await asyncio.sleep_ms(0) - else: - await asyn.Cancellable.cancel_all() - print('Done.') - -def condition_test(new=False): - printexp('''cond02 0 Awaiting notification. -cond02 1 Awaiting notification. -cond02 2 Awaiting notification. -cond02 3 Awaiting notification. -cond02 4 Awaiting notification. -cond02 5 Awaiting notification. -cond02 6 Awaiting notification. -cond02 5 triggered. tim = 1 -cond02 6 triggered. tim = 1 -cond02 3 triggered. tim = 3 -cond02 4 triggered. tim = 3 -cond02 1 triggered. tim = 5 -cond02 2 triggered. tim = 5 -cond02 0 triggered. tim = 7 -cond04 99 Awaiting notification and predicate. -cond04 99 triggered. tim = 9 -Done. -''', 13) - loop = asyncio.get_event_loop() - loop.run_until_complete(cond_go(loop, new)) - -# ************ Gather test ************ - -# Task with one positional arg. Demonstrate that result order depends on -# original list order not termination order. -async def gath01(n): - print('gath01', n, 'started') - await asyncio.sleep(3 - n/10) - print('gath01', n, 'done') - return n - -# Takes kwarg. This is last to terminate. -async def gath02(x, y, rats): - print('gath02 started') - await asyncio.sleep(7) - print('gath02 done') - return x * y, rats - -# Only quits on timeout -async def gath03(n): - print('gath03 started') - try: - while True: - await asyncio.sleep(1) - n += 1 - except asyncio.TimeoutError: - print('gath03 timeout') - return n - -async def gath_go(): - gatherables = [asyn.Gatherable(gath01, n) for n in range(4)] - gatherables.append(asyn.Gatherable(gath02, 7, 8, rats=77)) - gatherables.append(asyn.Gatherable(gath03, 0, timeout=5)) - res = await asyn.Gather(gatherables) - print(res) - -def gather_test(): - printexp('''gath01 0 started -gath01 1 started -gath01 2 started -gath01 3 started -gath02 started -gath03 started -gath01 3 done -gath01 2 done -gath01 1 done -gath01 0 done -gath03 timeout -gath02 done -[0, 1, 2, 3, (56, 77), 4] -''', 7) - loop = asyncio.get_event_loop() - loop.run_until_complete(gath_go()) diff --git a/auart.py b/auart.py deleted file mode 100644 index 8600529..0000000 --- a/auart.py +++ /dev/null @@ -1,25 +0,0 @@ -# Test of uasyncio stream I/O using UART -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license -# Link X1 and X2 to test. - -import uasyncio as asyncio -from pyb import UART -uart = UART(4, 9600) - -async def sender(): - swriter = asyncio.StreamWriter(uart, {}) - while True: - await swriter.awrite('Hello uart\n') - await asyncio.sleep(2) - -async def receiver(): - sreader = asyncio.StreamReader(uart) - while True: - res = await sreader.readline() - print('Recieved', res) - -loop = asyncio.get_event_loop() -loop.create_task(sender()) -loop.create_task(receiver()) -loop.run_forever() diff --git a/awaitable.py b/awaitable.py deleted file mode 100644 index a9087f6..0000000 --- a/awaitable.py +++ /dev/null @@ -1,32 +0,0 @@ -# awaitable.py Demo of an awaitable class -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license -# runs in CPython and MicroPython -# Trivial fix for MicroPython issue #2678 - -try: - import uasyncio as asyncio -except ImportError: - import asyncio - -class Hardware(object): - def __init__(self, count): - self.count = count - - def __await__(self): # Typical use, loop until an interface becomes ready. - while self.count: - print(self.count) - yield - self.count -= 1 - - __iter__ = __await__ # issue #2678 - -loop = asyncio.get_event_loop() - -hardware = Hardware(10) - -async def run(): - await hardware - print('Done') - -loop.run_until_complete(run()) diff --git a/benchmarks/call_lp.py b/benchmarks/call_lp.py deleted file mode 100644 index 813787f..0000000 --- a/benchmarks/call_lp.py +++ /dev/null @@ -1,43 +0,0 @@ -# call_lp.py Demo of low priority callback. Author Peter Hinch July 2018. -# Requires fast_io version of core.py - -import pyb -import uasyncio as asyncio -try: - if not(isinstance(asyncio.version, tuple)): - raise AttributeError -except AttributeError: - raise OSError('This program requires uasyncio fast_io version V0.24 or above.') - -loop = asyncio.get_event_loop(lp_len=16) - -count = 0 -numbers = 0 - -async def report(): - await asyncio.after(2) - print('Callback executed {} times. Expected count 2000/20 = 100 times.'.format(count)) - print('Avg. of {} random numbers in range 0 to 1023 was {}'.format(count, numbers // count)) - -def callback(num): - global count, numbers - count += 1 - numbers += num // 2**20 # range 0 to 1023 - -def cb(arg): - print(arg) - -async def run_test(): - loop = asyncio.get_event_loop() - loop.call_after(1, cb, 'One second has elapsed.') # Test args - loop.call_after_ms(500, cb, '500ms has elapsed.') - print('Callbacks scheduled.') - while True: - loop.call_after(0, callback, pyb.rng()) # demo use of args - yield 20 # 20ms - -print('Test runs for 2 seconds') -loop = asyncio.get_event_loop() -loop.create_task(run_test()) -loop.run_until_complete(report()) - diff --git a/benchmarks/latency.py b/benchmarks/latency.py deleted file mode 100644 index 786cd22..0000000 --- a/benchmarks/latency.py +++ /dev/null @@ -1,123 +0,0 @@ -# latency.py Benchmark for uasyncio. Author Peter Hinch July 2018. - -# This measures the scheduling latency of a notional device driver running in the -# presence of other coros. This can test asyncio_priority.py which incorporates -# the priority mechanism. (In the home directory of this repo). - -# When running the test that uses the priority mechanism the latency is 300us which -# is determined by the time it takes uasyncio to schedule a coro (see rate.py). -# This is because, when the priority() coro issues await device it is the only coro -# on the normal queue and it therefore is immediately scheduled. - -# When running the test without the priority mechanism, the latency is D*Nms where N -# is the number of instances of the foo() coro and D is the processing period of -# foo() in ms (2). This is because priority() will only be rescheduled after every -# foo() instance has run. - -# For compute-intensive tasks a yield every 2ms is reasonably efficient. A shorter -# period implies a significant proportion of CPU cycles being taken up in scheduling. - -import uasyncio as asyncio -lp_version = True -try: - if not(isinstance(asyncio.version, tuple)): - raise AttributeError -except AttributeError: - lp_version = False - -import pyb -import utime as time -import gc - -num_coros = (5, 10, 100, 200) -duration = 2 # Time to run for each number of coros -done = False - -tmax = 0 -tmin = 1000000 -dtotal = 0 -count = 0 -lst_tmax = [tmax] * len(num_coros) # Max, min and avg error values -lst_tmin = [tmin] * len(num_coros) -lst_sd = [0] * len(num_coros) - -class DummyDeviceDriver(): - def __iter__(self): - yield - -async def report(): - # Don't compromise results by executing too soon. Time round loop is duration + 1 - await after(1 + len(num_coros) * (duration + 1)) - print('Awaiting result...') - while not done: - await after_ms(1000) - s = 'Coros {:4d} Latency = {:6.2f}ms min. {:6.2f}ms max. {:6.2f}ms avg.' - for x, n in enumerate(num_coros): - print(s.format(n, lst_tmin[x] / 1000, lst_tmax[x] /1000, lst_sd[x] / 1000)) - -async def lp_task(delay): - await after_ms(0) # If running low priority get on LP queue ASAP - while True: - time.sleep_ms(delay) # Simulate processing - await after_ms(0) - -async def priority(): - global tmax, tmin, dtotal, count - device = DummyDeviceDriver() - while True: - await after(0) # Ensure low priority coros get to run - tstart = time.ticks_us() - await device # Measure the latency - delta = time.ticks_diff(time.ticks_us(), tstart) - tmax = max(tmax, delta) - tmin = min(tmin, delta) - dtotal += delta - count += 1 - -async def run_test(delay): - global done, tmax, tmin, dtotal, count - loop.create_task(priority()) - old_n = 0 - for n, n_coros in enumerate(num_coros): - print('{:4d} coros. Test for {}s'.format(n_coros, duration)) - for _ in range(n_coros - old_n): - loop.create_task(lp_task(delay)) - await asyncio.sleep(1) # ensure tasks are all on LP queue before we measure - gc.collect() # ensure gc doesn't cloud the issue - old_n = n_coros - tmax = 0 - tmin = 1000000 - dtotal = 0 - count = 0 - await asyncio.sleep(duration) - lst_tmin[n] = tmin - lst_tmax[n] = tmax - lst_sd[n] = dtotal / count - done = True - -def test(use_priority=True): - global after, after_ms, loop, lp_version - processing_delay = 2 # Processing time in low priority task (ms) - if use_priority and not lp_version: - print('To test priority mechanism you must use fast_io version of uasyncio.') - else: - ntasks = max(num_coros) + 10 #4 - if use_priority: - loop = asyncio.get_event_loop(ntasks, ntasks, 0, ntasks) - after = asyncio.after - after_ms = asyncio.after_ms - else: - lp_version = False - after = asyncio.sleep - after_ms = asyncio.sleep_ms - loop = asyncio.get_event_loop(ntasks, ntasks) - s = 'Testing latency of priority task with coros blocking for {}ms.' - print(s.format(processing_delay)) - if lp_version: - print('Using priority mechanism.') - else: - print('Not using priority mechanism.') - loop.create_task(run_test(processing_delay)) - loop.run_until_complete(report()) - -print('Issue latency.test() to test priority mechanism, latency.test(False) to test standard algo.') diff --git a/benchmarks/overdue.py b/benchmarks/overdue.py deleted file mode 100644 index a777f5e..0000000 --- a/benchmarks/overdue.py +++ /dev/null @@ -1,40 +0,0 @@ -# overdue.py Test for "low priority" uasyncio. Author Peter Hinch April 2017. -import uasyncio as asyncio -try: - if not(isinstance(asyncio.version, tuple)): - raise AttributeError -except AttributeError: - raise OSError('This program requires uasyncio fast_io version V0.24 or above.') - -loop = asyncio.get_event_loop(lp_len=16) -ntimes = 0 - -async def lp_task(): - global ntimes - while True: - await asyncio.after_ms(100) - print('LP task runs.') - ntimes += 1 - -async def hp_task(): # Hog the scheduler - while True: - await asyncio.sleep_ms(0) - -async def report(): - global ntimes - loop.max_overdue_ms(1000) - loop.create_task(hp_task()) - loop.create_task(lp_task()) - print('First test runs for 10 secs. Max overdue time = 1s.') - await asyncio.sleep(10) - print('Low priority coro was scheduled {} times: (should be 9).'.format(ntimes)) - loop.max_overdue_ms(0) - ntimes = 0 - print('Second test runs for 10 secs. Default scheduling.') - print('Low priority coro should not be scheduled.') - await asyncio.sleep(10) - print('Low priority coro was scheduled {} times: (should be 0).'.format(ntimes)) - -loop = asyncio.get_event_loop() -loop.run_until_complete(report()) - diff --git a/benchmarks/priority_test.py b/benchmarks/priority_test.py deleted file mode 100644 index b6a4636..0000000 --- a/benchmarks/priority_test.py +++ /dev/null @@ -1,78 +0,0 @@ -# priority_test.py -# Test/demo of task cancellation of low priority tasks -# Author: Peter Hinch -# Copyright Peter Hinch 2018 Released under the MIT license - -# Check availability of 'priority' version -import uasyncio as asyncio -try: - if not(isinstance(asyncio.version, tuple)): - raise AttributeError -except AttributeError: - raise OSError('This program requires uasyncio fast_io version V0.24 or above.') - -loop = asyncio.get_event_loop(lp_len=16) -import asyn - -def printexp(exp, runtime=0): - print('Expected output:') - print('\x1b[32m') - print(exp) - print('\x1b[39m') - if runtime: - print('Running (runtime = {}s):'.format(runtime)) - else: - print('Running (runtime < 1s):') - -@asyn.cancellable -async def foo(num): - print('Starting foo', num) - try: - await asyncio.after(1) - print('foo', num, 'ran to completion.') - except asyn.StopTask: - print('foo', num, 'was cancelled.') - -async def kill(task_name): - if await asyn.NamedTask.cancel(task_name): - print(task_name, 'will be cancelled when next scheduled') - else: - print(task_name, 'was not cancellable.') - -# Example of a task which cancels another -async def bar(): - await asyncio.sleep(1) - await kill('foo 0') # Will fail because it has completed - await kill('foo 1') - await kill('foo 3') # Will fail because not yet scheduled - -async def run_cancel_test(): - loop = asyncio.get_event_loop() - await asyn.NamedTask('foo 0', foo, 0) - loop.create_task(asyn.NamedTask('foo 1', foo, 1)()) - loop.create_task(bar()) - await asyncio.sleep(5) - await asyn.NamedTask('foo 2', foo, 2) - await asyn.NamedTask('foo 4', foo, 4) - loop.create_task(asyn.NamedTask('foo 3', foo, 3)()) - await asyncio.sleep(5) - -def test(): - printexp('''Starting foo 0 -foo 0 ran to completion. -Starting foo 1 -foo 0 was not cancellable. -foo 1 will be cancelled when next scheduled -foo 3 was not cancellable. -foo 1 was cancelled. -Starting foo 2 -foo 2 ran to completion. -Starting foo 4 -foo 4 ran to completion. -Starting foo 3 -foo 3 ran to completion. -''', 14) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test()) - -test() diff --git a/benchmarks/rate.py b/benchmarks/rate.py deleted file mode 100644 index 8b7ceb8..0000000 --- a/benchmarks/rate.py +++ /dev/null @@ -1,48 +0,0 @@ -# rate.py Benchmark for uasyncio. Author Peter Hinch Feb 2018. -# Benchmark uasyncio round-robin scheduling performance -# This measures the rate at which uasyncio can schedule a minimal coro which -# mereley increments a global. - -# Outcome: 100 minimal coros are scheduled at an interval of ~156μs on official -# uasyncio V2. On fast_io version 0.1 (including low priority) at 162μs. -# fast_io overhead is < 4% - -import uasyncio as asyncio - -num_coros = (100, 200, 500, 1000) -iterations = [0, 0, 0, 0] -duration = 2 # Time to run for each number of coros -count = 0 -done = False - -async def report(): - while not done: - await asyncio.sleep(1) - for x, n in enumerate(num_coros): - print('Coros {:4d} Iterations/sec {:5d} Duration {:3d}us'.format( - n, int(iterations[x]/duration), int(duration*1000000/iterations[x]))) - -async def foo(): - global count - while True: - yield - count += 1 - -async def test(): - global count, done - old_n = 0 - for n, n_coros in enumerate(num_coros): - print('Testing {} coros for {}secs'.format(n_coros, duration)) - count = 0 - for _ in range(n_coros - old_n): - loop.create_task(foo()) - old_n = n_coros - await asyncio.sleep(duration) - iterations[n] = count - done = True - -ntasks = max(num_coros) + 2 -loop = asyncio.get_event_loop(ntasks, ntasks) -loop.create_task(test()) -loop.run_until_complete(report()) - diff --git a/benchmarks/rate_esp.py b/benchmarks/rate_esp.py deleted file mode 100644 index a2a54e4..0000000 --- a/benchmarks/rate_esp.py +++ /dev/null @@ -1,50 +0,0 @@ -# rate_esp.py Benchmark for uasyncio. Author Peter Hinch April 2017. -# Benchmark uasyncio round-robin scheduling performance -# This measures the rate at which uasyncio can schedule a minimal coro which -# mereley increments a global. - -# Test for ESP8266. Times below at 160/80MHz -# Outcome: minimal coros are scheduled at an interval of ~1.2/1.76ms with 'yield' -# 1.7/2.5ms with 'await asyncio.sleep_ms(0)' - -import uasyncio as asyncio -from machine import freq -freq(80000000) - -num_coros = (10,) -iterations = [0,] -duration = 10 # Time to run for each number of coros -count = 0 -done = False - -async def report(): - while not done: - await asyncio.sleep(1) - for x, n in enumerate(num_coros): - print('Coros {:4d} Iterations/sec {:5d} Duration {:3d}us'.format( - n, int(iterations[x]/duration), int(duration*1000000/iterations[x]))) - -async def foo(): - global count - while True: - yield - count += 1 - -async def test(): - global count, done - old_n = 0 - for n, n_coros in enumerate(num_coros): - print('Testing {} coros for {}secs'.format(n_coros, duration)) - count = 0 - for _ in range(n_coros - old_n): - loop.create_task(foo()) - old_n = n_coros - await asyncio.sleep(duration) - iterations[n] = count - done = True - -ntasks = max(num_coros) + 2 -loop = asyncio.get_event_loop(ntasks, ntasks) -loop.create_task(test()) -loop.run_until_complete(report()) - diff --git a/benchmarks/rate_fastio.py b/benchmarks/rate_fastio.py deleted file mode 100644 index d1ce969..0000000 --- a/benchmarks/rate_fastio.py +++ /dev/null @@ -1,48 +0,0 @@ -# rate_fastio.py Benchmark for uasyncio. Author Peter Hinch July 2018. -# This version tests the fast_io version when I/O is not pending. -# Benchmark uasyncio round-robin scheduling performance -# This measures the rate at which uasyncio can schedule a minimal coro which -# mereley increments a global. - -# This is identical to rate.py but instantiates io and lp queues -# Outcome: minimal coros are scheduled at an interval of ~206μs - -import uasyncio as asyncio - -num_coros = (100, 200, 500, 1000) -iterations = [0, 0, 0, 0] -duration = 2 # Time to run for each number of coros -count = 0 -done = False - -async def report(): - while not done: - await asyncio.sleep(1) - for x, n in enumerate(num_coros): - print('Coros {:4d} Iterations/sec {:5d} Duration {:3d}us'.format( - n, int(iterations[x]/duration), int(duration*1000000/iterations[x]))) - -async def foo(): - global count - while True: - yield - count += 1 - -async def test(): - global count, done - old_n = 0 - for n, n_coros in enumerate(num_coros): - print('Testing {} coros for {}secs'.format(n_coros, duration)) - count = 0 - for _ in range(n_coros - old_n): - loop.create_task(foo()) - old_n = n_coros - await asyncio.sleep(duration) - iterations[n] = count - done = True - -ntasks = max(num_coros) + 2 -loop = asyncio.get_event_loop(ntasks, ntasks, 6, 6) -loop.create_task(test()) -loop.run_until_complete(report()) - diff --git a/cantest.py b/cantest.py deleted file mode 100644 index e6cc43f..0000000 --- a/cantest.py +++ /dev/null @@ -1,422 +0,0 @@ -# cantest.py Tests of task cancellation - -# The MIT License (MIT) -# -# Copyright (c) 2017-2018 Peter Hinch -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -# THE SOFTWARE. - - -import uasyncio as asyncio -import asyn -import utime as time - -def print_tests(): - st = '''Available functions: -test1() Basic NamedTask cancellation. -test2() Use of Barrier to synchronise NamedTask cancellation. Demo of latency. -test3() Cancellation of a NamedTask which has run to completion. -test4() Test of Cancellable class. -test5() Cancellable and NamedTask instances as bound methods. -test6() Test of NamedTask.is_running() and awaiting NamedTask cancellation. -Recommended to issue ctrl-D after running each test. -''' - print('\x1b[32m') - print(st) - print('\x1b[39m') - -print_tests() - -def printexp(exp, runtime=0): - print('Expected output:') - print('\x1b[32m') - print(exp) - print('\x1b[39m') - if runtime: - print('Running (runtime = {}s):'.format(runtime)) - else: - print('Running (runtime < 1s):') - -# cancel_test1() - -@asyn.cancellable -async def foo(num): - try: - await asyncio.sleep(4) - except asyn.StopTask: - print('foo was cancelled.') - return -1 - else: - return num + 42 - -async def kill(task_name): - res = await asyn.NamedTask.cancel(task_name) - if res: - print(task_name, 'will be cancelled when next scheduled') - else: - print(task_name, 'was not cancellable.') - -# Example of a task which cancels another -async def bar(): - await asyncio.sleep(1) - await kill('foo') - await kill('not me') # Will fail because not yet scheduled - -async def run_cancel_test1(): - loop = asyncio.get_event_loop() - loop.create_task(bar()) - res = await asyn.NamedTask('foo', foo, 5) - print(res, asyn.NamedTask.is_running('foo')) - res = await asyn.NamedTask('not me', foo, 0) # Runs to completion - print(res, asyn.NamedTask.is_running('not me')) - -def test1(): - printexp('''foo will be cancelled when next scheduled -not me was not cancellable. -foo was cancelled. --1 False -42 False -''', 8) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test1()) - -# test2() -# This test uses a barrier so that cancelling task pauses until cancelled tasks -# have actually terminated. Also tests the propagation of the thrown exception -# to the awaiting coro. - -async def forever(n): - print('Started forever() instance', n) - while True: - await asyncio.sleep(7 + n) - print('Running instance', n) - -# Intercepting the StopTask exception. -@asyn.cancellable -async def rats(n): - try: - await forever(n) - except asyn.StopTask: - print('Instance', n, 'was cancelled') - -async def run_cancel_test2(): - barrier = asyn.Barrier(3) - loop = asyncio.get_event_loop() - loop.create_task(asyn.NamedTask('rats_1', rats, 1, barrier=barrier)()) - loop.create_task(asyn.NamedTask('rats_2', rats, 2, barrier=barrier)()) - print('Running two tasks') - await asyncio.sleep(10) - print('About to cancel tasks') - await asyn.NamedTask.cancel('rats_1') # These will stop when their wait is complete - await asyn.NamedTask.cancel('rats_2') - await barrier # So wait for that to occur. - print('tasks were cancelled') - -def test2(): - printexp('''Running two tasks -Started forever() instance 1 -Started forever() instance 2 -Running instance 1 -Running instance 2 -About to cancel tasks -Instance 1 was cancelled -Instance 2 was cancelled -tasks were cancelled -''', 20) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test2()) - -# test3() -# Test of cancelling a task which has already terminated - -# Intercepting the StopTask exception. -@asyn.cancellable -async def cant3(): - try: - await asyncio.sleep(1) - print('Task cant3 has ended.') - except asyn.StopTask: - print('Task cant3 was cancelled') - -async def run_cancel_test3(): - barrier = asyn.Barrier(2) - loop = asyncio.get_event_loop() - loop.create_task(asyn.NamedTask('cant3', cant3, barrier=barrier)()) - print('Task cant3 running status', asyn.NamedTask.is_running('cant3')) - await asyncio.sleep(3) - print('Task cant3 running status', asyn.NamedTask.is_running('cant3')) - print('About to cancel task') - await asyn.NamedTask.cancel('cant3') - print('Cancelled') - print('Task cant3 running status', asyn.NamedTask.is_running('cant3')) - await barrier - print('tasks were cancelled') - -def test3(): - printexp('''Task cant3 running status True -Task cant3 has ended. -Task cant3 running status False -About to cancel task -Cancelled -Task cant3 running status False -tasks were cancelled -''', 3) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test3()) - -# test4() -# Test of cancelling a task which has already terminated - -# Cancellable coros can trap the StopTask. They are passed the -# task_id automatically - -@asyn.cancellable -async def cant40(num): - while True: - try: - await asyn.sleep(1) - print('Task cant40 no. {} running.'.format(num)) - except asyn.StopTask: - print('Task cant40 no. {} was cancelled'.format(num)) - return - -@asyn.cancellable -async def cant41(num, arg=0): - try: - await asyn.sleep(1) - print('Task cant41 no. {} running, arg {}.'.format(num, arg)) - except asyn.StopTask: - print('Task cant41 no. {} was cancelled.'.format(num)) - return - else: - print('Task cant41 no. {} ended.'.format(num)) - -async def cant42(num): - while True: - print('Task cant42 no. {} running'.format(num)) - await asyn.sleep(1.2) - -# Test await syntax and throwing exception to subtask -@asyn.cancellable -async def chained(num, x, y, *, red, blue): - print('Args:', x, y, red, blue) # Test args and kwargs - try: - await cant42(num) - except asyn.StopTask: - print('Task chained no. {} was cancelled'.format(num)) - -async def run_cancel_test4(): - await asyn.Cancellable(cant41, 0, 5) - loop = asyncio.get_event_loop() - loop.create_task(asyn.Cancellable(cant40, 1)()) # 3 instances in default group 0 - loop.create_task(asyn.Cancellable(cant40, 2)()) - loop.create_task(asyn.Cancellable(cant40, 3)()) - loop.create_task(asyn.Cancellable(chained, 4, 1, 2, red=3, blue=4, group=1)()) - loop.create_task(asyn.Cancellable(cant41, 5)()) # Runs to completion - print('Running tasks') - await asyncio.sleep(3) - print('About to cancel group 0 tasks') - await asyn.Cancellable.cancel_all() # All in default group 0 - print('Group 0 tasks were cancelled') - await asyncio.sleep(1) # Demo chained still running - print('About to cancel group 1 tasks') - await asyn.Cancellable.cancel_all(1) # Group 1 - print('Group 1 tasks were cancelled') - await asyncio.sleep(1) - -def test4(): - printexp('''Task cant41 no. 0 running, arg 5. -Task cant41 no. 0 ended. -Running tasks -Args: 1 2 3 4 -Task cant42 no. 4 running -Task cant40 no. 1 running. -Task cant40 no. 2 running. -Task cant40 no. 3 running. -Task cant41 no. 5 running, arg 0. -Task cant41 no. 5 ended. -Task cant42 no. 4 running -Task cant40 no. 1 running. -Task cant40 no. 2 running. -Task cant40 no. 3 running. -Task cant42 no. 4 running -About to cancel group 0 tasks -Task cant40 no. 1 was cancelled -Task cant40 no. 2 was cancelled -Task cant40 no. 3 was cancelled -Group 0 tasks were cancelled -Task cant42 no. 4 running -About to cancel group 1 tasks -Task chained no. 4 was cancelled -Group 1 tasks were cancelled -''', 6) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test4()) - -# test5 -# Test of task cancellation where tasks are bound methods - -class CanTest(): - async def start(self, loop): - loop.create_task(asyn.Cancellable(self.foo, 1)()) # 3 instances in default group 0 - loop.create_task(asyn.Cancellable(self.foo, 2)()) - loop.create_task(asyn.Cancellable(self.foo, 3)()) - loop.create_task(asyn.NamedTask('my bar', self.bar, 4, y=42)()) - await asyncio.sleep(4.5) - await asyn.NamedTask.cancel('my bar') - await asyn.Cancellable.cancel_all() - await asyncio.sleep(1) - print('Done') - - @asyn.cancellable - async def foo(self, arg): - try: - while True: - await asyn.sleep(1) - print('foo running, arg', arg) - except asyn.StopTask: - print('foo was cancelled') - - @asyn.cancellable - async def bar(self, arg, *, x=1, y=2): - try: - while True: - await asyn.sleep(1) - print('bar running, arg', arg, x, y) - except asyn.StopTask: - print('bar was cancelled') - -def test5(): - printexp('''foo running, arg 1 -foo running, arg 2 -foo running, arg 3 -bar running, arg 4 1 42 -foo running, arg 1 -foo running, arg 2 -foo running, arg 3 -bar running, arg 4 1 42 -foo running, arg 1 -foo running, arg 2 -foo running, arg 3 -bar running, arg 4 1 42 -foo running, arg 1 -foo running, arg 2 -foo running, arg 3 -bar running, arg 4 1 42 -foo was cancelled -foo was cancelled -foo was cancelled -bar was cancelled -Done -''', 6) - cantest = CanTest() - loop = asyncio.get_event_loop() - loop.run_until_complete(cantest.start(loop)) - -# test 6: test NamedTask.is_running() -@asyn.cancellable -async def cant60(name): - print('Task cant60 name \"{}\" running.'.format(name)) - try: - for _ in range(5): - await asyncio.sleep(2) # 2 secs latency. - except asyn.StopTask: - print('Task cant60 name \"{}\" was cancelled.'.format(name)) - return - else: - print('Task cant60 name \"{}\" ended.'.format(name)) - -@asyn.cancellable -async def cant61(): - try: - while True: - for name in ('complete', 'cancel me'): - res = asyn.NamedTask.is_running(name) - print('Task \"{}\" running: {}'.format(name, res)) - await asyncio.sleep(1) - except asyn.StopTask: - print('Task cant61 cancelled.') - -async def run_cancel_test6(loop): - for name in ('complete', 'cancel me'): - loop.create_task(asyn.NamedTask(name, cant60, name)()) - loop.create_task(asyn.Cancellable(cant61)()) - await asyncio.sleep(4.5) - print('Cancelling task \"{}\". 1.5 secs latency.'.format(name)) - await asyn.NamedTask.cancel(name) - await asyncio.sleep(7) - name = 'cancel wait' - loop.create_task(asyn.NamedTask(name, cant60, name)()) - await asyncio.sleep(0.5) - print('Cancelling task \"{}\". 1.5 secs latency.'.format(name)) - t = time.ticks_ms() - await asyn.NamedTask.cancel('cancel wait', nowait=False) - print('Was cancelled in {} ms'.format(time.ticks_diff(time.ticks_ms(), t))) - print('Cancelling cant61') - await asyn.Cancellable.cancel_all() - print('Done') - - -def test6(): - printexp('''Task cant60 name "complete" running. -Task cant60 name "cancel me" running. -Task "complete" running: True -Task "cancel me" running: True -Task "complete" running: True -Task "cancel me" running: True -Task "complete" running: True -Task "cancel me" running: True -Task "complete" running: True -Task "cancel me" running: True -Task "complete" running: True -Task "cancel me" running: True -Cancelling task "cancel me". 1.5 secs latency. -Task "complete" running: True -Task "cancel me" running: True -Task cant60 name "cancel me" was cancelled. -Task "complete" running: True -Task "cancel me" running: False -Task "complete" running: True -Task "cancel me" running: False -Task "complete" running: True -Task "cancel me" running: False -Task "complete" running: True -Task "cancel me" running: False -Task cant60 name "complete" ended. -Task "complete" running: False -Task "cancel me" running: False -Task "complete" running: False -Task "cancel me" running: False -Task cant60 name "cancel wait" running. -Cancelling task "cancel wait". 1.5 secs latency. -Task "complete" running: False -Task "cancel me" running: False -Task "complete" running: False -Task "cancel me" running: False -Task cant60 name "cancel wait" was cancelled. -Was cancelled in 1503 ms -Cancelling cant61 -Task cant61 cancelled. -Done - - -[Duration of cancel wait may vary depending on platform 1500 <= range <= 1600ms] -''', 14) - loop = asyncio.get_event_loop() - loop.run_until_complete(run_cancel_test6(loop)) diff --git a/chain.py b/chain.py deleted file mode 100644 index 38e6f1a..0000000 --- a/chain.py +++ /dev/null @@ -1,20 +0,0 @@ -# chain.py Demo of chained coros under MicroPython uasyncio -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license -try: - import uasyncio as asyncio -except ImportError: - import asyncio - -async def compute(x, y): - print("Compute %s + %s ..." % (x, y)) - await asyncio.sleep(1.0) - return x + y - -async def print_sum(x, y): - result = await compute(x, y) - print("%s + %s = %s" % (x, y, result)) - -loop = asyncio.get_event_loop() -loop.run_until_complete(print_sum(1, 2)) -loop.close() diff --git a/check_async_code.py b/check_async_code.py deleted file mode 100755 index f7907a7..0000000 --- a/check_async_code.py +++ /dev/null @@ -1,206 +0,0 @@ -#! /usr/bin/python3 -# -*- coding: utf-8 -*- -# check_async_code.py -# A simple script to identify a common error which causes silent failures under -# MicroPython (issue #3241). -# This is where a task is declared with async def and then called as if it were -# a regular function. -# Copyright Peter Hinch 2017 -# Issued under the MIT licence - -import sys -import re - -tasks = set() -mismatch = False - -def pass1(part, lnum): - global mismatch - opart = part - sysnames = ('__aenter__', '__aexit__', '__aiter__', '__anext__') - # These are the commonest system functions declared with async def. - # Mimimise spurious duplicate function definition error messages. - good = True - if not part.startswith('#'): - mismatch = False - part = stripquotes(part, lnum) # Remove quoted strings (which might contain code) - good &= not mismatch - if part.startswith('async'): - pos = part.find('def') - if pos >= 0: - part = part[pos + 3:] - part = part.lstrip() - pos = part.find('(') - if pos >= 0: - fname = part[:pos].strip() - if fname in tasks and fname not in sysnames: - # Note this gives a false positive if a method of the same name - # exists in more than one class. - print('Duplicate function declaration "{}" in line {}'.format(fname, lnum)) - print(opart) - print() - good = False - else: - tasks.add(fname) - return good - -# Strip quoted strings (which may contain code) -def stripquotes(part, lnum=0): - global mismatch - for qchar in ('"', "'"): - pos = part.find(qchar) - if pos >= 0: - part = part[:pos] + part[pos + 1:] # strip 1st qchar - pos1 = part.find(qchar) - if pos > 0: - part = part[:pos] + part[pos1+1:] # Strip whole quoted string - part = stripquotes(part, lnum) - else: - print('Mismatched quotes in line', lnum) - mismatch = True - return part # for what it's worth - return part - -def pass2(part, lnum): - global mismatch - opart = part - good = True - if not part.startswith('#') and not part.startswith('async'): - mismatch = False - part = stripquotes(part, lnum) # Remove quoted strings (which might contain code) - good &= not mismatch - for task in tasks: - sstr = ''.join((task, r'\w*')) - match = re.search(sstr, part) - if match is None: # No match - continue - if match.group(0) != task: # No exact match - continue - # Accept await task, await task(args), a = await task(args) - sstr = ''.join((r'.*await[ \t]+', task)) - if re.search(sstr, part): - continue - # Accept await obj.task, await obj.task(args), a = await obj.task(args) - sstr = ''.join((r'.*await[ \t]+\w+\.', task)) - if re.search(sstr, part): - continue - # Accept assignments e.g. a = mytask or - # after = asyncio.after if p_version else asyncio.sleep - # or comparisons thistask == thattask - sstr = ''.join((r'=[ \t]*', task, r'[ \t]*[^(]')) - if re.search(sstr, part): - continue - # Not awaited but could be passed to function e.g. - # run_until_complete(mytask(args)) - sstr = ''.join((r'.*\w+[ \t]*\([ \t]*', task, r'[ \t]*\(')) - if re.search(sstr, part): - sstr = r'run_until_complete|run_forever|create_task|NamedTask' - if re.search(sstr, part): - continue - print('Please review line {}: async function "{}" is passed to a function.'.format(lnum, task)) - print(opart) - print() - good = False - continue - # func(mytask, more_args) may or may not be an error - sstr = ''.join((r'.*\w+[ \t]*\([ \t]*', task, r'[ \t]*[^\(]')) - if re.search(sstr, part): - print('Please review line {}: async function "{}" is passed to a function.'.format(lnum, task)) - print(opart) - print() - good = False - continue - - # Might be a method. Discard object. - sstr = ''.join((r'.*\w+[ \t]*\([ \t]*\w+\.', task)) - if re.search(sstr, part): - continue - print('Please review line {}: async function "{}" is not awaited.'.format(lnum, task)) - print(opart) - print() - good = False - return good - -txt = '''check_async_code.py -usage: check_async_code.py sourcefile.py - -This rather crude script is designed to locate a single type of coding error -which leads to silent runtime failure and hence can be hard to locate. - -It is intended to be used on otherwise correct source files and is not robust -in the face of syntax errors. Use pylint or other tools for general syntax -checking. - -It assumes code is written in the style advocated in the tutorial where coros -are declared with "async def". - -Under certain circumstances it can produce false positives. In some cases this -is by design. Given an asynchronous function foo the following is correct: -loop.run_until_complete(foo()) -The following line may or may not be an error depending on the design of bar() -bar(foo, args) -Likewise asynchronous functions can be put into objects such as dicts, lists or -sets. You may wish to review such lines to check that the intention was to put -the function rather than its result into the object. - -A false positive which is a consequence of the hacky nature of this script is -where a task has the same name as a synchronous bound method of some class. A -call to the bound method will produce an erroneous warning. This is because the -code does not parse class definitions. - -In practice the odd false positive is easily spotted in the code. -''' - -def usage(code=0): - print(txt) - sys.exit(code) - -# Process a line -in_triple_quote = False -def do_line(line, passn, lnum): - global in_triple_quote - ignore = False - good = True - # TODO The following isn't strictly correct. A line might be of the form - # erroneous Python ; ''' start of string - # It could therefore miss the error. - if re.search(r'[^"]*"""|[^\']*\'\'\'', line): - if in_triple_quote: - # Discard rest of line which terminates triple quote - ignore = True - in_triple_quote = not in_triple_quote - if not in_triple_quote and not ignore: - parts = line.split(';') - for part in parts: - # discard comments and whitespace at start and end - part = part.split('#')[0].strip() - if part: - good &= passn(part, lnum) - return good - -def main(fn): - global in_triple_quote - good = True - try: - with open(fn, 'r') as f: - for passn in (pass1, pass2): - in_triple_quote = False - lnum = 1 - for line in f: - good &= do_line(line, passn, lnum) - lnum += 1 - f.seek(0) - - except FileNotFoundError: - print('File {} does not exist.'.format(fn)) - return - if good: - print('No errors found!') - -if __name__ == "__main__": - if len(sys.argv) !=2: - usage(1) - arg = sys.argv[1].strip() - if arg == '--help' or arg == '-h': - usage() - main(arg) diff --git a/client_server/userver.py b/client_server/userver.py deleted file mode 100644 index ec7c07c..0000000 --- a/client_server/userver.py +++ /dev/null @@ -1,64 +0,0 @@ -# userver.py Demo of simple uasyncio-based echo server - -# Released under the MIT licence -# Copyright (c) Peter Hinch 2019 - -import usocket as socket -import uasyncio as asyncio -import uselect as select -import ujson -from heartbeat import heartbeat # Optional LED flash - -class Server: - async def run(self, loop, port=8123): - addr = socket.getaddrinfo('0.0.0.0', port, 0, socket.SOCK_STREAM)[0][-1] - s_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # server socket - s_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) - s_sock.bind(addr) - s_sock.listen(5) - self.socks = [s_sock] # List of current sockets for .close() - print('Awaiting connection on port', port) - poller = select.poll() - poller.register(s_sock, select.POLLIN) - client_id = 1 # For user feedback - while True: - res = poller.poll(1) # 1ms block - if res: # Only s_sock is polled - c_sock, _ = s_sock.accept() # get client socket - loop.create_task(self.run_client(c_sock, client_id)) - client_id += 1 - await asyncio.sleep_ms(200) - - async def run_client(self, sock, cid): - self.socks.append(sock) - sreader = asyncio.StreamReader(sock) - swriter = asyncio.StreamWriter(sock, {}) - print('Got connection from client', cid) - try: - while True: - res = await sreader.readline() - if res == b'': - raise OSError - print('Received {} from client {}'.format(ujson.loads(res.rstrip()), cid)) - await swriter.awrite(res) # Echo back - except OSError: - pass - print('Client {} disconnect.'.format(cid)) - sock.close() - self.socks.remove(sock) - - def close(self): - print('Closing {} sockets.'.format(len(self.socks))) - for sock in self.socks: - sock.close() - -loop = asyncio.get_event_loop() -# Optional fast heartbeat to confirm nonblocking operation -loop.create_task(heartbeat(100)) -server = Server() -try: - loop.run_until_complete(server.run(loop)) -except KeyboardInterrupt: - print('Interrupted') # This mechanism doesn't work on Unix build. -finally: - server.close() diff --git a/fast_io/__init__.py b/fast_io/__init__.py deleted file mode 100644 index 5a2239c..0000000 --- a/fast_io/__init__.py +++ /dev/null @@ -1,377 +0,0 @@ -# uasyncio.__init__ fast_io -# (c) 2014-2018 Paul Sokolovsky. MIT license. - -# This is a fork of official MicroPython uasynco. It is recommended to use -# the official version unless the specific features of this fork are required. - -# Changes copyright (c) Peter Hinch 2018 -# Code at https://github.com/peterhinch/micropython-async.git -# fork: peterhinch/micropython-lib branch: uasyncio-io-fast-and-rw - -import uerrno -import uselect as select -import usocket as _socket -import sys -from uasyncio.core import * - -DEBUG = 0 -log = None - -def set_debug(val): - global DEBUG, log - DEBUG = val - if val: - import logging - log = logging.getLogger("uasyncio") - - -class PollEventLoop(EventLoop): - - def __init__(self, runq_len=16, waitq_len=16, fast_io=0, lp_len=0): - EventLoop.__init__(self, runq_len, waitq_len, fast_io, lp_len) - self.poller = select.poll() - self.rdobjmap = {} - self.wrobjmap = {} - self.flags = {} - - # Remove registration of sock for reading or writing. - def _unregister(self, sock, objmap, flag): - # If StreamWriter.awrite() wrote entire buf on 1st pass sock will never - # have been registered. So test for presence in .flags. - if id(sock) in self.flags: - flags = self.flags[id(sock)] - if flags & flag: # flag is currently registered - flags &= ~flag # Clear current flag - if flags: # Another flag is present - self.flags[id(sock)] = flags - self.poller.register(sock, flags) - else: - del self.flags[id(sock)] # Clear all flags - self.poller.unregister(sock) - del objmap[id(sock)] # Remove coro from appropriate dict - - # Additively register sock for reading or writing - def _register(self, sock, flag): - if id(sock) in self.flags: - self.flags[id(sock)] |= flag - else: - self.flags[id(sock)] = flag - self.poller.register(sock, self.flags[id(sock)]) - - def add_reader(self, sock, cb, *args): - if DEBUG and __debug__: - log.debug("add_reader%s", (sock, cb, args)) - # HACK This should read - # self._register(sock, select.POLLIN) - # Temporary workround for https://github.com/micropython/micropython/issues/5172 - # The following is not compliant with POSIX or with the docs - self._register(sock, select.POLLIN | select.POLLHUP | select.POLLERR) # t35tB0t add HUP and ERR to force LWIP revents - if args: - self.rdobjmap[id(sock)] = (cb, args) - else: - self.rdobjmap[id(sock)] = cb - - def remove_reader(self, sock): - if DEBUG and __debug__: - log.debug("remove_reader(%s)", sock) - self._unregister(sock, self.rdobjmap, select.POLLIN) - - def add_writer(self, sock, cb, *args): - if DEBUG and __debug__: - log.debug("add_writer%s", (sock, cb, args)) - # HACK Should read - # self._register(sock, select.POLLOUT) - # Temporary workround for https://github.com/micropython/micropython/issues/5172 - # The following is not compliant with POSIX or with the docs - self._register(sock, select.POLLOUT | select.POLLHUP | select.POLLERR) # t35tB0t add HUP and ERR to force LWIP revents - if args: - self.wrobjmap[id(sock)] = (cb, args) - else: - self.wrobjmap[id(sock)] = cb - - def remove_writer(self, sock): - if DEBUG and __debug__: - log.debug("remove_writer(%s)", sock) - self._unregister(sock, self.wrobjmap, select.POLLOUT) - - def wait(self, delay): - if DEBUG and __debug__: - log.debug("poll.wait(%d)", delay) - # We need one-shot behavior (second arg of 1 to .poll()) - res = self.poller.ipoll(delay, 1) - #log.debug("poll result: %s", res) - for sock, ev in res: - if ev & select.POLLOUT: - cb = self.wrobjmap[id(sock)] - if cb is None: - continue # Not yet ready. - # Invalidate objmap: can get adverse timing in fast_io whereby add_writer - # is not called soon enough. Ignore poll events occurring before we are - # ready to handle them. - self.wrobjmap[id(sock)] = None - if ev & (select.POLLHUP | select.POLLERR): - # These events are returned even if not requested, and - # are sticky, i.e. will be returned again and again. - # If the caller doesn't do proper error handling and - # unregister this sock, we'll busy-loop on it, so we - # as well can unregister it now "just in case". - self.remove_writer(sock) - if DEBUG and __debug__: - log.debug("Calling IO callback: %r", cb) - if isinstance(cb, tuple): - cb[0](*cb[1]) - else: - prev = cb.pend_throw(None) # Enable task to run. - #if isinstance(prev, Exception): - #print('Put back exception') - #cb.pend_throw(prev) - self._call_io(cb) # Put coro onto runq (or ioq if one exists) - if ev & select.POLLIN: - cb = self.rdobjmap[id(sock)] - if cb is None: - continue - self.rdobjmap[id(sock)] = None - if ev & (select.POLLHUP | select.POLLERR): - # These events are returned even if not requested, and - # are sticky, i.e. will be returned again and again. - # If the caller doesn't do proper error handling and - # unregister this sock, we'll busy-loop on it, so we - # as well can unregister it now "just in case". - self.remove_reader(sock) - if DEBUG and __debug__: - log.debug("Calling IO callback: %r", cb) - if isinstance(cb, tuple): - cb[0](*cb[1]) - else: - prev = cb.pend_throw(None) # Enable task to run. - #if isinstance(prev, Exception): - #cb.pend_throw(prev) - #print('Put back exception') - self._call_io(cb) - - -class StreamReader: - - def __init__(self, polls, ios=None): - if ios is None: - ios = polls - self.polls = polls - self.ios = ios - - def read(self, n=-1): - while True: - yield IORead(self.polls) - res = self.ios.read(n) # Call the device's read method - if res is not None: - break - # This should not happen for real sockets, but can easily - # happen for stream wrappers (ssl, websockets, etc.) - #log.warn("Empty read") - yield IOReadDone(self.polls) # uasyncio.core calls remove_reader - # This de-registers device as a read device with poll via - # PollEventLoop._unregister - return res # Next iteration raises StopIteration and returns result - - def readinto(self, buf, n=0): # See comments in .read - while True: - yield IORead(self.polls) - if n: - res = self.ios.readinto(buf, n) # Call device's readinto method - else: - res = self.ios.readinto(buf) - if res is not None: - break - #log.warn("Empty read") - yield IOReadDone(self.polls) - return res - - def readexactly(self, n): - buf = b"" - while n: - yield IORead(self.polls) - # socket may become unreadable inbetween - # subsequent readline may return None - res = self.ios.read(n) - # returns none if socket not readable vs no data b'' - if res is None: - if DEBUG and __debug__: - log.debug('WARNING: socket write returned type(None)') - # socket may be in HUP or ERR state, so loop back ask poller - continue - else: - if not res: # All done - break - buf += res - n -= len(res) - yield IOReadDone(self.polls) - return buf - - def readline(self): - if DEBUG and __debug__: - log.debug("StreamReader.readline()") - buf = b"" - while True: - yield IORead(self.polls) - # socket may become unreadable inbetween - # subsequent readline may return None - res = self.ios.readline() - if res is None: - if DEBUG and __debug__: - log.debug('WARNING: socket read returned type(None)') - # socket may be in HUP or ERR state, so loop back and ask poller - continue - else: - if not res: - break - buf += res - if buf[-1] == 0x0a: - break - if DEBUG and __debug__: - log.debug("StreamReader.readline(): %s", buf) - yield IOReadDone(self.polls) - return buf - - def aclose(self): - yield IOReadDone(self.polls) - self.ios.close() - - def __repr__(self): - return "" % (self.polls, self.ios) - - -class StreamWriter: - - def __init__(self, s, extra): - self.s = s - self.extra = extra - - def awrite(self, buf, off=0, sz=-1): - # This method is called awrite (async write) to not proliferate - # incompatibility with original asyncio. Unlike original asyncio - # whose .write() method is both not a coroutine and guaranteed - # to return immediately (which means it has to buffer all the - # data), this method is a coroutine. - if sz == -1: - sz = len(buf) - off - if DEBUG and __debug__: - log.debug("StreamWriter.awrite(): spooling %d bytes", sz) - while True: - # Check socket write status first - yield IOWrite(self.s) - # socket may become unwritable inbetween - # subsequent writes may return None - res = self.s.write(buf, off, sz) - if res is None: - if DEBUG and __debug__: - log.debug('WARNING: socket write returned type(None)') - # socket may be in HUP or ERR state, so loop back and ask poller - continue - # If we spooled everything, return immediately - if res == sz: - if DEBUG and __debug__: - log.debug("StreamWriter.awrite(): completed spooling %d bytes", res) - break - if DEBUG and __debug__: - log.debug("StreamWriter.awrite(): spooled partial %d bytes", res) - assert res < sz - off += res - sz -= res - yield IOWrite(self.s) - if DEBUG and __debug__: - log.debug("StreamWriter.awrite(): can write more") - # remove_writer de-registers device as a writer - yield IOWriteDone(self.s) - - # Write piecewise content from iterable (usually, a generator) - def awriteiter(self, iterable): - for buf in iterable: - yield from self.awrite(buf) - - def aclose(self): - yield IOWriteDone(self.s) - self.s.close() - - def get_extra_info(self, name, default=None): - return self.extra.get(name, default) - - def __repr__(self): - return "" % self.s - - -def open_connection(host, port, ssl=False): - if DEBUG and __debug__: - log.debug("open_connection(%s, %s)", host, port) - ai = _socket.getaddrinfo(host, port, 0, _socket.SOCK_STREAM) - ai = ai[0] - s = _socket.socket(ai[0], ai[1], ai[2]) - s.setblocking(False) - try: - s.connect(ai[-1]) - except OSError as e: - if e.args[0] != uerrno.EINPROGRESS: - raise - if DEBUG and __debug__: - log.debug("open_connection: After connect") - yield IOWrite(s) -# if __debug__: -# assert s2.fileno() == s.fileno() - if DEBUG and __debug__: - log.debug("open_connection: After iowait: %s", s) - if ssl: - print("Warning: uasyncio SSL support is alpha") - import ussl - s.setblocking(True) - s2 = ussl.wrap_socket(s) - s.setblocking(False) - return StreamReader(s, s2), StreamWriter(s2, {}) - return StreamReader(s), StreamWriter(s, {}) - - -def start_server(client_coro, host, port, backlog=10): - if DEBUG and __debug__: - log.debug("start_server(%s, %s)", host, port) - ai = _socket.getaddrinfo(host, port, 0, _socket.SOCK_STREAM) - ai = ai[0] - s = _socket.socket(ai[0], ai[1], ai[2]) - s.setblocking(False) - - s.setsockopt(_socket.SOL_SOCKET, _socket.SO_REUSEADDR, 1) - s.bind(ai[-1]) - s.listen(backlog) - try: - while True: - try: - if DEBUG and __debug__: - log.debug("start_server: Before accept") - yield IORead(s) - if DEBUG and __debug__: - log.debug("start_server: After iowait") - s2, client_addr = s.accept() - s2.setblocking(False) - if DEBUG and __debug__: - log.debug("start_server: After accept: %s", s2) - extra = {"peername": client_addr} - # Detach the client_coro: put it on runq - yield client_coro(StreamReader(s2), StreamWriter(s2, extra)) - s2 = None - - except Exception as e: - if len(e.args)==0: - # This happens but shouldn't. Firmware bug? - # Handle exception as an unexpected unknown error: - # collect details here then close try to continue running - print('start_server:Unknown error: continuing') - sys.print_exception(e) - if not uerrno.errorcode.get(e.args[0], False): - # Handle exception as internal error: close and terminate - # handler (user must trap or crash) - print('start_server:Unexpected error: terminating') - raise - finally: - if s2: - s2.close() - s.close() - - -import uasyncio.core -uasyncio.core._event_loop_class = PollEventLoop diff --git a/fast_io/core.py b/fast_io/core.py deleted file mode 100644 index 7eadcfc..0000000 --- a/fast_io/core.py +++ /dev/null @@ -1,462 +0,0 @@ -# uasyncio.core fast_io -# (c) 2014-2018 Paul Sokolovsky. MIT license. - -# This is a fork of official MicroPython uasynco. It is recommended to use -# the official version unless the specific features of this fork are required. - -# Changes copyright (c) Peter Hinch 2018, 2019 -# Code at https://github.com/peterhinch/micropython-async.git -# fork: peterhinch/micropython-lib branch: uasyncio-io-fast-and-rw - -version = ('fast_io', '0.26') -try: - import rtc_time as time # Low power timebase using RTC -except ImportError: - import utime as time -import utimeq -import ucollections - - -type_gen = type((lambda: (yield))()) -type_genf = type((lambda: (yield))) # Type of a generator function upy iss #3241 - -DEBUG = 0 -log = None - -def set_debug(val): - global DEBUG, log - DEBUG = val - if val: - import logging - log = logging.getLogger("uasyncio.core") - - -class CancelledError(Exception): - pass - - -class TimeoutError(CancelledError): - pass - - -class EventLoop: - - def __init__(self, runq_len=16, waitq_len=16, ioq_len=0, lp_len=0): - self.runq = ucollections.deque((), runq_len, True) - self._max_od = 0 - self.lpq = utimeq.utimeq(lp_len) if lp_len else None - self.ioq_len = ioq_len - self.canned = set() - if ioq_len: - self.ioq = ucollections.deque((), ioq_len, True) - self._call_io = self._call_now - else: - self._call_io = self.call_soon - self.waitq = utimeq.utimeq(waitq_len) - # Current task being run. Task is a top-level coroutine scheduled - # in the event loop (sub-coroutines executed transparently by - # yield from/await, event loop "doesn't see" them). - self.cur_task = None - - def time(self): - return time.ticks_ms() - - def create_task(self, coro): - # CPython 3.4.2 - assert not isinstance(coro, type_genf), 'Coroutine arg expected.' # upy issue #3241 - # create_task with a callable would work, so above assert only traps the easily-made error - self.call_later_ms(0, coro) - # CPython asyncio incompatibility: we don't return Task object - - def _call_now(self, callback, *args): # For stream I/O only - if __debug__ and DEBUG: - log.debug("Scheduling in ioq: %s", (callback, args)) - self.ioq.append(callback) - if not isinstance(callback, type_gen): - self.ioq.append(args) - - def max_overdue_ms(self, t=None): - if t is not None: - self._max_od = int(t) - return self._max_od - - # Low priority versions of call_later() call_later_ms() and call_at_() - def call_after_ms(self, delay, callback, *args): - self.call_at_lp_(time.ticks_add(self.time(), delay), callback, *args) - - def call_after(self, delay, callback, *args): - self.call_at_lp_(time.ticks_add(self.time(), int(delay * 1000)), callback, *args) - - def call_at_lp_(self, time, callback, *args): - if self.lpq is not None: - self.lpq.push(time, callback, args) - if isinstance(callback, type_gen): - callback.pend_throw(id(callback)) - else: - raise OSError('No low priority queue exists.') - - def call_soon(self, callback, *args): - if __debug__ and DEBUG: - log.debug("Scheduling in runq: %s", (callback, args)) - self.runq.append(callback) - if not isinstance(callback, type_gen): - self.runq.append(args) - - def call_later(self, delay, callback, *args): - self.call_at_(time.ticks_add(self.time(), int(delay * 1000)), callback, args) - - def call_later_ms(self, delay, callback, *args): - if not delay: - return self.call_soon(callback, *args) - self.call_at_(time.ticks_add(self.time(), delay), callback, args) - - def call_at_(self, time, callback, args=()): - if __debug__ and DEBUG: - log.debug("Scheduling in waitq: %s", (time, callback, args)) - self.waitq.push(time, callback, args) - if isinstance(callback, type_gen): - callback.pend_throw(id(callback)) - - def wait(self, delay): - # Default wait implementation, to be overriden in subclasses - # with IO scheduling - if __debug__ and DEBUG: - log.debug("Sleeping for: %s", delay) - time.sleep_ms(delay) - - def run_forever(self): - cur_task = [0, 0, 0] - # Put a task on the runq unless it was cancelled - def runq_add(): - if isinstance(cur_task[1], type_gen): - tid = id(cur_task[1]) - if tid in self.canned: - self.canned.remove(tid) - else: - cur_task[1].pend_throw(None) - self.call_soon(cur_task[1], *cur_task[2]) - else: - self.call_soon(cur_task[1], *cur_task[2]) - - while True: - # Expire entries in waitq and move them to runq - tnow = self.time() - if self.lpq: - # Schedule a LP task if overdue or if no normal task is ready - to_run = False # Assume no LP task is to run - t = self.lpq.peektime() - tim = time.ticks_diff(t, tnow) - to_run = self._max_od > 0 and tim < -self._max_od - if not (to_run or self.runq): # No overdue LP task or task on runq - # zero delay tasks go straight to runq. So don't schedule LP if runq - to_run = tim <= 0 # True if LP task is due - if to_run and self.waitq: # Set False if normal tasks due. - t = self.waitq.peektime() - to_run = time.ticks_diff(t, tnow) > 0 # No normal task is ready - if to_run: - self.lpq.pop(cur_task) - runq_add() - - while self.waitq: - t = self.waitq.peektime() - delay = time.ticks_diff(t, tnow) - if delay > 0: - break - self.waitq.pop(cur_task) - if __debug__ and DEBUG: - log.debug("Moving from waitq to runq: %s", cur_task[1]) - runq_add() - - # Process runq. This can append tasks to the end of .runq so get initial - # length so we only process those items on the queue at the start. - l = len(self.runq) - if __debug__ and DEBUG: - log.debug("Entries in runq: %d", l) - cur_q = self.runq # Default: always get tasks from runq - dl = 1 # Subtract this from entry count l - while l or self.ioq_len: - if self.ioq_len: # Using fast_io - self.wait(0) # Schedule I/O. Can append to ioq. - if self.ioq: - cur_q = self.ioq - dl = 0 # No effect on l - elif l == 0: - break # Both queues are empty - else: - cur_q = self.runq - dl = 1 - l -= dl - cb = cur_q.popleft() # Remove most current task - args = () - if not isinstance(cb, type_gen): # It's a callback not a generator so get args - args = cur_q.popleft() - l -= dl - if __debug__ and DEBUG: - log.info("Next callback to run: %s", (cb, args)) - cb(*args) # Call it - continue # Proceed to next runq entry - - if __debug__ and DEBUG: - log.info("Next coroutine to run: %s", (cb, args)) - self.cur_task = cb # Stored in a bound variable for TimeoutObj - delay = 0 - low_priority = False # Assume normal priority - try: - if args is (): - ret = next(cb) # Schedule the coro, get result - else: - ret = cb.send(*args) - if __debug__ and DEBUG: - log.info("Coroutine %s yield result: %s", cb, ret) - if isinstance(ret, SysCall1): # Coro returned a SysCall1: an object with an arg spcified in its constructor - arg = ret.arg - if isinstance(ret, SleepMs): - delay = arg - if isinstance(ret, AfterMs): - low_priority = True - if isinstance(ret, After): - delay = int(delay*1000) - elif isinstance(ret, IORead): # coro was a StreamReader read method - cb.pend_throw(False) # Marks the task as waiting on I/O for cancellation/timeout - # If task is cancelled or times out, it is put on runq to process exception. - # Debug note: if task is scheduled other than by wait (which does pend_throw(None) - # an exception (exception doesn't inherit from Exception) is thrown - self.add_reader(arg, cb) # Set up select.poll for read and store the coro in object map - continue # Don't reschedule. Coro is scheduled by wait() when poll indicates h/w ready - elif isinstance(ret, IOWrite): # coro was StreamWriter.awrite. Above comments apply. - cb.pend_throw(False) - self.add_writer(arg, cb) - continue - elif isinstance(ret, IOReadDone): # update select.poll registration and if necessary remove coro from map - self.remove_reader(arg) - self._call_io(cb, args) # Next call produces StopIteration enabling result to be returned - continue - elif isinstance(ret, IOWriteDone): - self.remove_writer(arg) - self._call_io(cb, args) # Next call produces StopIteration: see StreamWriter.aclose - continue - elif isinstance(ret, StopLoop): # e.g. from run_until_complete. run_forever() terminates - return arg - else: - assert False, "Unknown syscall yielded: %r (of type %r)" % (ret, type(ret)) - elif isinstance(ret, type_gen): # coro has yielded a coro (or generator) - self.call_soon(ret) # append to .runq - elif isinstance(ret, int): # If coro issued yield N, delay = N ms - delay = ret - elif ret is None: - # coro issued yield. delay == 0 so code below will put the current task back on runq - pass - elif ret is False: - # yield False causes coro not to be rescheduled i.e. it stops. - continue - else: - assert False, "Unsupported coroutine yield value: %r (of type %r)" % (ret, type(ret)) - except StopIteration as e: - if __debug__ and DEBUG: - log.debug("Coroutine finished: %s", cb) - continue - except CancelledError as e: - if __debug__ and DEBUG: - log.debug("Coroutine cancelled: %s", cb) - continue - # Currently all syscalls don't return anything, so we don't - # need to feed anything to the next invocation of coroutine. - # If that changes, need to pass that value below. - if low_priority: - self.call_after_ms(delay, cb) # Put on lpq - elif delay: - self.call_later_ms(delay, cb) - else: - self.call_soon(cb) - - # Wait until next waitq task or I/O availability - delay = 0 - if not self.runq: - delay = -1 - if self.waitq: - tnow = self.time() - t = self.waitq.peektime() - delay = time.ticks_diff(t, tnow) - if delay < 0: - delay = 0 - if self.lpq: - t = self.lpq.peektime() - lpdelay = time.ticks_diff(t, tnow) - if lpdelay < 0: - lpdelay = 0 - if lpdelay < delay or delay < 0: - delay = lpdelay # waitq is empty or lp task is more current - self.wait(delay) - - def run_until_complete(self, coro): - assert not isinstance(coro, type_genf), 'Coroutine arg expected.' # upy issue #3241 - def _run_and_stop(): - ret = yield from coro # https://github.com/micropython/micropython-lib/pull/270 - yield StopLoop(ret) - self.call_soon(_run_and_stop()) - return self.run_forever() - - def stop(self): - self.call_soon((lambda: (yield StopLoop(0)))()) - - def close(self): - pass - - -class SysCall: - - def __init__(self, *args): - self.args = args - - def handle(self): - raise NotImplementedError - -# Optimized syscall with 1 arg -class SysCall1(SysCall): - - def __init__(self, arg): - self.arg = arg - -class StopLoop(SysCall1): - pass - -class IORead(SysCall1): - pass - -class IOWrite(SysCall1): - pass - -class IOReadDone(SysCall1): - pass - -class IOWriteDone(SysCall1): - pass - - -_event_loop = None -_event_loop_class = EventLoop -def get_event_loop(runq_len=16, waitq_len=16, ioq_len=0, lp_len=0): - global _event_loop - if _event_loop is None: - _event_loop = _event_loop_class(runq_len, waitq_len, ioq_len, lp_len) - return _event_loop - -# Allow user classes to determine prior event loop instantiation. -def get_running_loop(): - if _event_loop is None: - raise RuntimeError('Event loop not instantiated') - return _event_loop - -def got_event_loop(): # Kept to avoid breaking code - return _event_loop is not None - -def sleep(secs): - yield int(secs * 1000) - -# Implementation of sleep_ms awaitable with zero heap memory usage -class SleepMs(SysCall1): - - def __init__(self): - self.v = None - self.arg = None - - def __call__(self, arg): - self.v = arg - #print("__call__") - return self - - def __iter__(self): - #print("__iter__") - return self - - def __next__(self): - if self.v is not None: - #print("__next__ syscall enter") - self.arg = self.v - self.v = None - return self - #print("__next__ syscall exit") - _stop_iter.__traceback__ = None - raise _stop_iter - -_stop_iter = StopIteration() -sleep_ms = SleepMs() - - -def cancel(coro): - prev = coro.pend_throw(CancelledError()) - if prev is False: # Waiting on I/O. Not on q so put it there. - _event_loop._call_io(coro) - elif isinstance(prev, int): # On waitq or lpq - # task id - _event_loop.canned.add(prev) # Alas this allocates - _event_loop._call_io(coro) # Put on runq/ioq - else: - assert prev is None - - -class TimeoutObj: - def __init__(self, coro): - self.coro = coro - - -def wait_for_ms(coro, timeout): - - def waiter(coro, timeout_obj): - res = yield from coro - if __debug__ and DEBUG: - log.debug("waiter: cancelling %s", timeout_obj) - timeout_obj.coro = None - return res - - def timeout_func(timeout_obj): - if timeout_obj.coro: - if __debug__ and DEBUG: - log.debug("timeout_func: cancelling %s", timeout_obj.coro) - prev = timeout_obj.coro.pend_throw(TimeoutError()) - if prev is False: # Waiting on I/O - _event_loop._call_io(timeout_obj.coro) - elif isinstance(prev, int): # On waitq or lpq - # prev==task id - _event_loop.canned.add(prev) # Alas this allocates - _event_loop._call_io(timeout_obj.coro) # Put on runq/ioq - else: - assert prev is None - - timeout_obj = TimeoutObj(_event_loop.cur_task) - _event_loop.call_later_ms(timeout, timeout_func, timeout_obj) - return (yield from waiter(coro, timeout_obj)) - - -def wait_for(coro, timeout): - return wait_for_ms(coro, int(timeout * 1000)) - - -def coroutine(f): - return f - -# Low priority -class AfterMs(SleepMs): - pass - -class After(AfterMs): - pass - -after_ms = AfterMs() -after = After() - -# -# The functions below are deprecated in uasyncio, and provided only -# for compatibility with CPython asyncio -# - -def ensure_future(coro, loop=_event_loop): - _event_loop.call_soon(coro) - # CPython asyncio incompatibility: we don't return Task object - return coro - - -# CPython asyncio incompatibility: Task is a function, not a class (for efficiency) -def Task(coro, loop=_event_loop): - # Same as async() - _event_loop.call_soon(coro) diff --git a/fast_io/fast_can_test.py b/fast_io/fast_can_test.py deleted file mode 100644 index 1600080..0000000 --- a/fast_io/fast_can_test.py +++ /dev/null @@ -1,67 +0,0 @@ -# fast_can_test.py Test of cancellation of tasks which call sleep - -# Copyright (c) Peter Hinch 2019 -# Released under the MIT licence - -import uasyncio as asyncio -import sys -ermsg = 'This test requires the fast_io version of uasyncio V2.4 or later.' -try: - print('Uasyncio version', asyncio.version) - if not isinstance(asyncio.version, tuple): - print(ermsg) - sys.exit(0) -except AttributeError: - print(ermsg) - sys.exit(0) - -# If a task times out the TimeoutError can't be trapped: -# no exception is thrown to the task - -async def foo(t): - try: - print('foo started') - await asyncio.sleep(t) - print('foo ended', t) - except asyncio.CancelledError: - print('foo cancelled', t) - -async def lpfoo(t): - try: - print('lpfoo started') - await asyncio.after(t) - print('lpfoo ended', t) - except asyncio.CancelledError: - print('lpfoo cancelled', t) - -async def run(coro, t): - await asyncio.wait_for(coro, t) - -async def bar(loop): - foo1 = foo(1) - foo5 = foo(5) - lpfoo1 = lpfoo(1) - lpfoo5 = lpfoo(5) - loop.create_task(foo1) - loop.create_task(foo5) - loop.create_task(lpfoo1) - loop.create_task(lpfoo5) - await asyncio.sleep(2) - print('Cancelling tasks') - asyncio.cancel(foo1) - asyncio.cancel(foo5) - asyncio.cancel(lpfoo1) - asyncio.cancel(lpfoo5) - await asyncio.sleep(0) # Allow cancellation to occur - print('Pausing 7s to ensure no task still running.') - await asyncio.sleep(7) - print('Launching tasks with 2s timeout') - loop.create_task(run(foo(1), 2)) - loop.create_task(run(lpfoo(1), 2)) - loop.create_task(run(foo(20), 2)) - loop.create_task(run(lpfoo(20), 2)) - print('Pausing 7s to ensure no task still running.') - await asyncio.sleep(7) - -loop = asyncio.get_event_loop(ioq_len=16, lp_len=16) -loop.run_until_complete(bar(loop)) diff --git a/fast_io/iorw_can.py b/fast_io/iorw_can.py deleted file mode 100644 index 8ef7929..0000000 --- a/fast_io/iorw_can.py +++ /dev/null @@ -1,140 +0,0 @@ -# iorw_can.py Emulate a device which can read and write one character at a time -# and test cancellation. - -# Copyright (c) Peter Hinch 2019 -# Released under the MIT licence - -# This requires the modified version of uasyncio (fast_io directory). -# Slow hardware is emulated using timers. -# MyIO.write() ouputs a single character and sets the hardware not ready. -# MyIO.readline() returns a single character and sets the hardware not ready. -# Timers asynchronously set the hardware ready. - -import io, pyb -import uasyncio as asyncio -import micropython -import sys -try: - print('Uasyncio version', asyncio.version) - if not isinstance(asyncio.version, tuple): - print('Please use fast_io version 0.24 or later.') - sys.exit(0) -except AttributeError: - print('ERROR: This test requires the fast_io version. It will not run correctly') - print('under official uasyncio V2.0 owing to a bug which prevents concurrent') - print('input and output.') - sys.exit(0) - -print('Issue iorw_can.test(True) to test ioq, iorw_can.test() to test runq.') -print('Tasks time out after 15s.') -print('Issue ctrl-d after each run.') - -micropython.alloc_emergency_exception_buf(100) - -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL_WR = const(4) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -def printbuf(this_io): - print(bytes(this_io.wbuf[:this_io.wprint_len]).decode(), end='') - -class MyIO(io.IOBase): - def __init__(self, read=False, write=False): - self.ready_rd = False # Read and write not ready - self.rbuf = b'ready\n' # Read buffer - self.ridx = 0 - pyb.Timer(4, freq = 5, callback = self.do_input) - self.wch = b'' - self.wbuf = bytearray(100) # Write buffer - self.wprint_len = 0 - self.widx = 0 - pyb.Timer(5, freq = 10, callback = self.do_output) - - # Read callback: emulate asynchronous input from hardware. - # Typically would put bytes into a ring buffer and set .ready_rd. - def do_input(self, t): - self.ready_rd = True # Data is ready to read - - # Write timer callback. Emulate hardware: if there's data in the buffer - # write some or all of it - def do_output(self, t): - if self.wch: - self.wbuf[self.widx] = self.wch - self.widx += 1 - if self.wch == ord('\n'): - self.wprint_len = self.widx # Save for schedule - micropython.schedule(printbuf, self) - self.widx = 0 - self.wch = b'' - - - def ioctl(self, req, arg): # see ports/stm32/uart.c - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - if self.ready_rd: - ret |= MP_STREAM_POLL_RD - if arg & MP_STREAM_POLL_WR: - if not self.wch: - ret |= MP_STREAM_POLL_WR # Ready if no char pending - return ret - - # Test of device that produces one character at a time - def readline(self): - self.ready_rd = False # Cleared by timer cb do_input - ch = self.rbuf[self.ridx] - if ch == ord('\n'): - self.ridx = 0 - else: - self.ridx += 1 - return chr(ch) - - # Emulate unbuffered hardware which writes one character: uasyncio waits - # until hardware is ready for the next. Hardware ready is emulated by write - # timer callback. - def write(self, buf, off, sz): - self.wch = buf[off] # Hardware starts to write a char - return 1 # 1 byte written. uasyncio waits on ioctl write ready - -# Note that trapping the exception and returning is still mandatory. -async def receiver(myior): - sreader = asyncio.StreamReader(myior) - try: - while True: - res = await sreader.readline() - print('Received', res) - except asyncio.CancelledError: - print('Receiver cancelled') - -async def sender(myiow): - swriter = asyncio.StreamWriter(myiow, {}) - await asyncio.sleep(1) - count = 0 - try: # Trap in outermost scope to catch cancellation of .sleep - while True: - count += 1 - tosend = 'Wrote Hello MyIO {}\n'.format(count) - await swriter.awrite(tosend.encode('UTF8')) - await asyncio.sleep(2) - except asyncio.CancelledError: - print('Sender cancelled') - -async def cannem(coros, t): - await asyncio.sleep(t) - for coro in coros: - asyncio.cancel(coro) - await asyncio.sleep(1) - -def test(ioq=False): - myio = MyIO() - if ioq: - loop = asyncio.get_event_loop(ioq_len=16) - else: - loop = asyncio.get_event_loop() - rx = receiver(myio) - tx = sender(myio) - loop.create_task(rx) - loop.create_task(tx) - loop.run_until_complete(cannem((rx, tx), 15)) diff --git a/fast_io/iorw_to.py b/fast_io/iorw_to.py deleted file mode 100644 index 79e05fd..0000000 --- a/fast_io/iorw_to.py +++ /dev/null @@ -1,143 +0,0 @@ -# iorw_to.py Emulate a device which can read and write one character at a time -# and test timeouts. - -# Copyright (c) Peter Hinch 2019 -# Released under the MIT licence - -# This requires the modified version of uasyncio (fast_io directory). -# Slow hardware is emulated using timers. -# MyIO.write() ouputs a single character and sets the hardware not ready. -# MyIO.readline() returns a single character and sets the hardware not ready. -# Timers asynchronously set the hardware ready. - -import io, pyb -import uasyncio as asyncio -import micropython -import sys -try: - print('Uasyncio version', asyncio.version) - if not isinstance(asyncio.version, tuple): - print('Please use fast_io version 0.24 or later.') - sys.exit(0) -except AttributeError: - print('ERROR: This test requires the fast_io version. It will not run correctly') - print('under official uasyncio V2.0 owing to a bug which prevents concurrent') - print('input and output.') - sys.exit(0) - -print('Issue iorw_to.test(True) to test ioq, iorw_to.test() to test runq.') -print('Test runs until interrupted. Tasks time out after 15s.') -print('Issue ctrl-d after each run.') - -micropython.alloc_emergency_exception_buf(100) - -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL_WR = const(4) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -def printbuf(this_io): - print(bytes(this_io.wbuf[:this_io.wprint_len]).decode(), end='') - -class MyIO(io.IOBase): - def __init__(self, read=False, write=False): - self.ready_rd = False # Read and write not ready - self.rbuf = b'ready\n' # Read buffer - self.ridx = 0 - pyb.Timer(4, freq = 5, callback = self.do_input) - self.wch = b'' - self.wbuf = bytearray(100) # Write buffer - self.wprint_len = 0 - self.widx = 0 - pyb.Timer(5, freq = 10, callback = self.do_output) - - # Read callback: emulate asynchronous input from hardware. - # Typically would put bytes into a ring buffer and set .ready_rd. - def do_input(self, t): - self.ready_rd = True # Data is ready to read - - # Write timer callback. Emulate hardware: if there's data in the buffer - # write some or all of it - def do_output(self, t): - if self.wch: - self.wbuf[self.widx] = self.wch - self.widx += 1 - if self.wch == ord('\n'): - self.wprint_len = self.widx # Save for schedule - micropython.schedule(printbuf, self) - self.widx = 0 - self.wch = b'' - - - def ioctl(self, req, arg): # see ports/stm32/uart.c - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - if self.ready_rd: - ret |= MP_STREAM_POLL_RD - if arg & MP_STREAM_POLL_WR: - if not self.wch: - ret |= MP_STREAM_POLL_WR # Ready if no char pending - return ret - - # Test of device that produces one character at a time - def readline(self): - self.ready_rd = False # Cleared by timer cb do_input - ch = self.rbuf[self.ridx] - if ch == ord('\n'): - self.ridx = 0 - else: - self.ridx += 1 - return chr(ch) - - # Emulate unbuffered hardware which writes one character: uasyncio waits - # until hardware is ready for the next. Hardware ready is emulated by write - # timer callback. - def write(self, buf, off, sz): - self.wch = buf[off] # Hardware starts to write a char - return 1 # 1 byte written. uasyncio waits on ioctl write ready - -# Note that trapping the exception and returning is still mandatory. -async def receiver(myior): - sreader = asyncio.StreamReader(myior) - try: - while True: - res = await sreader.readline() - print('Received', res) - except asyncio.TimeoutError: - print('Receiver timeout') - -async def sender(myiow): - swriter = asyncio.StreamWriter(myiow, {}) - await asyncio.sleep(1) - count = 0 - try: # Trap in outermost scope to catch cancellation of .sleep - while True: - count += 1 - tosend = 'Wrote Hello MyIO {}\n'.format(count) - await swriter.awrite(tosend.encode('UTF8')) - await asyncio.sleep(2) - except asyncio.TimeoutError: - print('Sender timeout') - -async def run(coro, t): - await asyncio.wait_for_ms(coro, t) - -async def do_test(loop, t): - myio = MyIO() - while True: - tr = t * 1000 + (pyb.rng() >> 20) # Add ~1s uncertainty - tw = t * 1000 + (pyb.rng() >> 20) - print('Timeouts: {:7.3f}s read {:7.3f}s write'.format(tr/1000, tw/1000)) - loop.create_task(run(receiver(myio), tr)) - await run(sender(myio), tw) - await asyncio.sleep(2) # Wait out timing randomness - -def test(ioq=False): - if ioq: - loop = asyncio.get_event_loop(ioq_len=16) - else: - loop = asyncio.get_event_loop() - loop.create_task(do_test(loop, 15)) - loop.run_forever() diff --git a/fast_io/ms_timer.py b/fast_io/ms_timer.py deleted file mode 100644 index f539289..0000000 --- a/fast_io/ms_timer.py +++ /dev/null @@ -1,33 +0,0 @@ -# ms_timer.py A relatively high precision delay class for the fast_io version -# of uasyncio - -import uasyncio as asyncio -import utime -import io -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -class MillisecTimer(io.IOBase): - def __init__(self): - self.end = 0 - self.sreader = asyncio.StreamReader(self) - - def __iter__(self): - await self.sreader.readline() - - def __call__(self, ms): - self.end = utime.ticks_add(utime.ticks_ms(), ms) - return self - - def readline(self): - return b'\n' - - def ioctl(self, req, arg): - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - if utime.ticks_diff(utime.ticks_ms(), self.end) >= 0: - ret |= MP_STREAM_POLL_RD - return ret diff --git a/fast_io/ms_timer_test.py b/fast_io/ms_timer_test.py deleted file mode 100644 index 5870317..0000000 --- a/fast_io/ms_timer_test.py +++ /dev/null @@ -1,45 +0,0 @@ -# ms_timer_test.py Test/demo program for MillisecTimer - -import uasyncio as asyncio -import utime -import ms_timer - -async def timer_test(n): - timer = ms_timer.MillisecTimer() - while True: - t = utime.ticks_ms() - await timer(30) - print(n, utime.ticks_diff(utime.ticks_ms(), t)) - await asyncio.sleep(0.5 + n/5) - -async def foo(): - while True: - await asyncio.sleep(0) - utime.sleep_ms(10) # Emulate slow processing - -async def killer(): - await asyncio.sleep(10) - -def test(fast_io=True): - loop = asyncio.get_event_loop(ioq_len=6 if fast_io else 0) - for _ in range(10): - loop.create_task(foo()) - for n in range(3): - loop.create_task(timer_test(n)) - loop.run_until_complete(killer()) - -s = '''This test creates ten tasks each of which blocks for 10ms. -It also creates three tasks each of which runs a MillisecTimer for 30ms, -timing the period which elapses while it runs. Under the fast_io version -the elapsed time is ~30ms as expected. Under the normal version it is -about 300ms because of competetion from the blocking coros. - -This competetion is worse than might be expected because of inefficiency -in the way the official version handles I/O. - -Run test() to test fast I/O, test(False) to test normal I/O. - -Test prints the task number followed by the actual elapsed time in ms. -Test runs for 10s.''' - -print(s) diff --git a/fast_io/pin_cb.py b/fast_io/pin_cb.py deleted file mode 100644 index 91e69e2..0000000 --- a/fast_io/pin_cb.py +++ /dev/null @@ -1,47 +0,0 @@ -# pin_cb.py Demo of device driver using fast I/O to schedule a callback -# PinCall class allows a callback to be associated with a change in pin state. - -# This class is not suitable for switch I/O because of contact bounce: -# see Switch and Pushbutton classes in aswitch.py - -import uasyncio as asyncio -import io -MP_STREAM_POLL_RD = const(1) -MP_STREAM_POLL = const(3) -MP_STREAM_ERROR = const(-1) - -class PinCall(io.IOBase): - def __init__(self, pin, *, cb_rise=None, cbr_args=(), cb_fall=None, cbf_args=()): - self.pin = pin - self.cb_rise = cb_rise - self.cbr_args = cbr_args - self.cb_fall = cb_fall - self.cbf_args = cbf_args - self.pinval = pin.value() - self.sreader = asyncio.StreamReader(self) - loop = asyncio.get_event_loop() - loop.create_task(self.run()) - - async def run(self): - while True: - await self.sreader.read(1) - - def read(self, _): - v = self.pinval - if v and self.cb_rise is not None: - self.cb_rise(*self.cbr_args) - return b'\n' - if not v and self.cb_fall is not None: - self.cb_fall(*self.cbf_args) - return b'\n' - - def ioctl(self, req, arg): - ret = MP_STREAM_ERROR - if req == MP_STREAM_POLL: - ret = 0 - if arg & MP_STREAM_POLL_RD: - v = self.pin.value() - if v != self.pinval: - self.pinval = v - ret = MP_STREAM_POLL_RD - return ret diff --git a/fast_io/pin_cb_test.py b/fast_io/pin_cb_test.py deleted file mode 100644 index 60dab70..0000000 --- a/fast_io/pin_cb_test.py +++ /dev/null @@ -1,68 +0,0 @@ -# ********* TEST ********** - -# With fast_io false latency is up to 50.96ms -# With fast_io True we see ~450μs to 5.208ms. - -import utime -import pyb -import uasyncio as asyncio -from pin_cb import PinCall - -t = 0 # Time of last output transition -max_latency = 0 -pinout = pyb.Pin(pyb.Pin.board.X1, pyb.Pin.OUT) - -# Timer callback: generate asynchronous pin state changes -def toggle(_): - global t - pinout.value(not pinout.value()) - t = utime.ticks_us() - -# Callback for basic test -def cb(pin, ud): - print('Callback', pin.value(), ud) - -# Callback for latency test -def cbl(pinin): - global max_latency - dt = utime.ticks_diff(utime.ticks_us(), t) - max_latency = max(max_latency, dt) - print('Latency {:6d}μs {:6d}μs max'.format(dt, max_latency)) - -async def dummy(): - while True: - await asyncio.sleep(0) - utime.sleep_ms(5) # Emulate slow processing - -async def killer(): - await asyncio.sleep(20) - -def test(fast_io=True, latency=False): - loop = asyncio.get_event_loop(ioq_len=6 if fast_io else 0) - pinin = pyb.Pin(pyb.Pin.board.X2, pyb.Pin.IN) - pyb.Timer(4, freq = 2.1, callback = toggle) - for _ in range(5): - loop.create_task(dummy()) - if latency: - pin_cb = PinCall(pinin, cb_rise = cbl, cbr_args = (pinin,)) - else: - pincall = PinCall(pinin, cb_rise = cb, cbr_args = (pinin, 'rise'), cb_fall = cb, cbf_args = (pinin, 'fall')) - loop.run_until_complete(killer()) - -print('''Link Pyboard pins X1 and X2. - -This test uses a timer to toggle pin X1, recording the time of each state change. - -The basic test with latency False just demonstrates the callbacks. -The latency test measures the time between the leading edge of X1 output and the -driver detecting the state change. This is in the presence of five competing coros -each of which blocks for 5ms. Latency is on the order of 5ms max under fast_io, -50ms max under official V2.0. - -Issue ctrl-D between runs. - -test(fast_io=True, latency=False) -args: -fast_io test fast I/O mechanism. -latency test latency (delay between X1 and X2 leading edge). -Tests run for 20s.''') diff --git a/gps/LICENSE b/gps/LICENSE deleted file mode 100644 index 798b35f..0000000 --- a/gps/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2017 Calvin McCoy - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/gps/ast_pbrw.py b/gps/ast_pbrw.py deleted file mode 100644 index ec5a760..0000000 --- a/gps/ast_pbrw.py +++ /dev/null @@ -1,174 +0,0 @@ -# ast_pb.py -# Basic test/demo of AS_GPS class (asynchronous GPS device driver) -# Runs on a Pyboard with GPS data on pin X2. -# Copyright (c) Peter Hinch 2018 -# Released under the MIT License (MIT) - see LICENSE file -# Test asynchronous GPS device driver as_rwGPS - -# LED's: -# Green indicates data is being received. -# Red toggles on RMC message received. -# Yellow and blue: coroutines have 4s loop delay. -# Yellow toggles on position reading. -# Blue toggles on date valid. - -import pyb -import uasyncio as asyncio -import aswitch -import as_GPS -import as_rwGPS - -# Avoid multiple baudrates. Tests use 9600 or 19200 only. -BAUDRATE = 19200 -red, green, yellow, blue = pyb.LED(1), pyb.LED(2), pyb.LED(3), pyb.LED(4) -ntimeouts = 0 - -def callback(gps, _, timer): - red.toggle() - green.on() - timer.trigger(10000) # Outage is declared after 10s - -def cb_timeout(): - global ntimeouts - green.off() - ntimeouts += 1 - -def message_cb(gps, segs): - print('Message received:', segs) - -# Print satellite data every 10s -async def sat_test(gps): - while True: - d = await gps.get_satellite_data() - print('***** SATELLITE DATA *****') - print('Data is Valid:', hex(gps._valid)) - for i in d: - print(i, d[i]) - print() - await asyncio.sleep(10) - -# Print statistics every 30s -async def stats(gps): - while True: - await gps.data_received(position=True) # Wait for a valid fix - await asyncio.sleep(30) - print('***** STATISTICS *****') - print('Outages:', ntimeouts) - print('Sentences Found:', gps.clean_sentences) - print('Sentences Parsed:', gps.parsed_sentences) - print('CRC_Fails:', gps.crc_fails) - print('Antenna status:', gps.antenna) - print('Firmware vesrion:', gps.version) - print('Enabled sentences:', gps.enabled) - print() - -# Print navigation data every 4s -async def navigation(gps): - while True: - await asyncio.sleep(4) - await gps.data_received(position=True) - yellow.toggle() - print('***** NAVIGATION DATA *****') - print('Data is Valid:', hex(gps._valid)) - print('Longitude:', gps.longitude(as_GPS.DD)) - print('Latitude', gps.latitude(as_GPS.DD)) - print() - -async def course(gps): - while True: - await asyncio.sleep(4) - await gps.data_received(course=True) - print('***** COURSE DATA *****') - print('Data is Valid:', hex(gps._valid)) - print('Speed:', gps.speed_string(as_GPS.MPH)) - print('Course', gps.course) - print('Compass Direction:', gps.compass_direction()) - print() - -async def date(gps): - while True: - await asyncio.sleep(4) - await gps.data_received(date=True) - blue.toggle() - print('***** DATE AND TIME *****') - print('Data is Valid:', hex(gps._valid)) - print('UTC Time:', gps.utc) - print('Local time:', gps.local_time) - print('Date:', gps.date_string(as_GPS.LONG)) - print() - -async def change_status(gps, uart): - await asyncio.sleep(10) - print('***** Changing status. *****') - await gps.baudrate(BAUDRATE) - uart.init(BAUDRATE) - print('***** baudrate 19200 *****') - await asyncio.sleep(5) # Ensure baudrate is sorted - print('***** Query VERSION *****') - await gps.command(as_rwGPS.VERSION) - await asyncio.sleep(10) - print('***** Query ENABLE *****') - await gps.command(as_rwGPS.ENABLE) - await asyncio.sleep(10) # Allow time for 1st report - await gps.update_interval(2000) - print('***** Update interval 2s *****') - await asyncio.sleep(10) - await gps.enable(gsv = False, chan = False) - print('***** Disable satellite in view and channel messages *****') - await asyncio.sleep(10) - print('***** Query ENABLE *****') - await gps.command(as_rwGPS.ENABLE) - -# See README.md re antenna commands -# await asyncio.sleep(10) -# await gps.command(as_rwGPS.ANTENNA) -# print('***** Antenna reports requested *****') -# await asyncio.sleep(60) -# await gps.command(as_rwGPS.NO_ANTENNA) -# print('***** Antenna reports turned off *****') -# await asyncio.sleep(10) - -async def gps_test(): - global gps, uart # For shutdown - print('Initialising') - # Adapt UART instantiation for other MicroPython hardware - uart = pyb.UART(4, 9600, read_buf_len=200) - # read_buf_len is precautionary: code runs reliably without it. - sreader = asyncio.StreamReader(uart) - swriter = asyncio.StreamWriter(uart, {}) - timer = aswitch.Delay_ms(cb_timeout) - sentence_count = 0 - gps = as_rwGPS.GPS(sreader, swriter, local_offset=1, fix_cb=callback, - fix_cb_args=(timer,), msg_cb = message_cb) - await asyncio.sleep(2) - await gps.command(as_rwGPS.DEFAULT_SENTENCES) - print('Set sentence frequencies to default') - #await gps.command(as_rwGPS.FULL_COLD_START) - #print('Performed FULL_COLD_START') - print('awaiting first fix') - loop = asyncio.get_event_loop() - loop.create_task(sat_test(gps)) - loop.create_task(stats(gps)) - loop.create_task(navigation(gps)) - loop.create_task(course(gps)) - loop.create_task(date(gps)) - await gps.data_received(True, True, True, True) # all messages - loop.create_task(change_status(gps, uart)) - -async def shutdown(): - # Normally UART is already at BAUDRATE. But if last session didn't restore - # factory baudrate we can restore connectivity in the subsequent stuck - # session with ctrl-c. - uart.init(BAUDRATE) - await asyncio.sleep(1) - await gps.command(as_rwGPS.FULL_COLD_START) - print('Factory reset') - #print('Restoring default baudrate.') - #await gps.baudrate(9600) - -loop = asyncio.get_event_loop() -loop.create_task(gps_test()) -try: - loop.run_forever() -finally: - loop.run_until_complete(shutdown()) diff --git a/gps/astests.py b/gps/astests.py deleted file mode 100755 index 6bfbebd..0000000 --- a/gps/astests.py +++ /dev/null @@ -1,178 +0,0 @@ -#!/usr/bin/env python3.5 -# -*- coding: utf-8 -*- - -# astests.py -# Tests for AS_GPS module (asynchronous GPS device driver) -# Based on tests for MicropyGPS by Michael Calvin McCoy -# https://github.com/inmcm/micropyGPS - -# Copyright (c) 2018 Peter Hinch -# Released under the MIT License (MIT) - see LICENSE file -# Run under CPython 3.5+ or MicroPython - -import as_GPS -try: - import uasyncio as asyncio -except ImportError: - import asyncio - -async def run(): - sentence_count = 0 - - test_RMC = ['$GPRMC,081836,A,3751.65,S,14507.36,E,000.0,360.0,130998,011.3,E*62\n', - '$GPRMC,123519,A,4807.038,N,01131.000,E,022.4,084.4,230394,003.1,W*6A\n', - '$GPRMC,225446,A,4916.45,N,12311.12,W,000.5,054.7,191194,020.3,E*68\n', - '$GPRMC,180041.896,A,3749.1851,N,08338.7891,W,001.9,154.9,240911,,,A*7A\n', - '$GPRMC,180049.896,A,3749.1808,N,08338.7869,W,001.8,156.3,240911,,,A*70\n', - '$GPRMC,092751.000,A,5321.6802,N,00630.3371,W,0.06,31.66,280511,,,A*45\n'] - - test_VTG = ['$GPVTG,232.9,T,,M,002.3,N,004.3,K,A*01\n'] - test_GGA = ['$GPGGA,180050.896,3749.1802,N,08338.7865,W,1,07,1.1,397.4,M,-32.5,M,,0000*6C\n'] - test_GSA = ['$GPGSA,A,3,07,11,28,24,26,08,17,,,,,,2.0,1.1,1.7*37\n', - '$GPGSA,A,3,07,02,26,27,09,04,15,,,,,,1.8,1.0,1.5*33\n'] - test_GSV = ['$GPGSV,3,1,12,28,72,355,39,01,52,063,33,17,51,272,44,08,46,184,38*74\n', - '$GPGSV,3,2,12,24,42,058,33,11,34,053,33,07,20,171,40,20,15,116,*71\n', - '$GPGSV,3,3,12,04,12,204,34,27,11,324,35,32,11,089,,26,10,264,40*7B\n', - '$GPGSV,3,1,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00*74\n', - '$GPGSV,3,2,11,14,25,170,00,16,57,208,39,18,67,296,40,19,40,246,00*74\n', - '$GPGSV,3,3,11,22,42,067,42,24,14,311,43,27,05,244,00,,,,*4D\n', - '$GPGSV,4,1,14,22,81,349,25,14,64,296,22,18,54,114,21,51,40,212,*7D\n', - '$GPGSV,4,2,14,24,30,047,22,04,22,312,26,31,22,204,,12,19,088,23*72\n', - '$GPGSV,4,3,14,25,17,127,18,21,16,175,,11,09,315,16,19,05,273,*72\n', - '$GPGSV,4,4,14,32,05,303,,15,02,073,*7A\n'] - test_GLL = ['$GPGLL,3711.0942,N,08671.4472,W,000812.000,A,A*46\n', - '$GPGLL,4916.45,N,12311.12,W,225444,A,*1D\n', - '$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n', - '$GPGLL,0000.0000,N,00000.0000,E,235947.000,V*2D\n'] - - my_gps = as_GPS.AS_GPS(None) - sentence = '' - for sentence in test_RMC: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('RMC sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('Longitude:', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Timestamp:', my_gps.utc) - print('Speed:', my_gps.speed()) - print('Date Stamp:', my_gps.date) - print('Course', my_gps.course) - print('Data is Valid:', bool(my_gps._valid & 1)) - print('Compass Direction:', my_gps.compass_direction()) - print('') - - for sentence in test_GLL: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('GLL sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('Longitude:', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Timestamp:', my_gps.utc) - print('Data is Valid:', bool(my_gps._valid & 2)) - print('') - - for sentence in test_VTG: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('VTG sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('Speed:', my_gps.speed()) - print('Course', my_gps.course) - print('Compass Direction:', my_gps.compass_direction()) - print('Data is Valid:', bool(my_gps._valid & 4)) - print('') - - for sentence in test_GGA: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('GGA sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('Longitude', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Timestamp:', my_gps.utc) - print('Altitude:', my_gps.altitude) - print('Height Above Geoid:', my_gps.geoid_height) - print('Horizontal Dilution of Precision:', my_gps.hdop) - print('Satellites in Use by Receiver:', my_gps.satellites_in_use) - print('Data is Valid:', bool(my_gps._valid & 8)) - print('') - - for sentence in test_GSA: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('GSA sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('Satellites Used', my_gps.satellites_used) - print('Horizontal Dilution of Precision:', my_gps.hdop) - print('Vertical Dilution of Precision:', my_gps.vdop) - print('Position Dilution of Precision:', my_gps.pdop) - print('Data is Valid:', bool(my_gps._valid & 16)) - print('') - - for sentence in test_GSV: - my_gps._valid = 0 - sentence_count += 1 - sentence = await my_gps._update(sentence) - if sentence is None: - print('GSV sentence is invalid.') - else: - print('Parsed a', sentence, 'Sentence') - print('SV Sentences Parsed', my_gps._last_sv_sentence) - print('SV Sentences in Total', my_gps._total_sv_sentences) - print('# of Satellites in View:', my_gps.satellites_in_view) - print('Data is Valid:', bool(my_gps._valid & 32)) - data_valid = my_gps._total_sv_sentences > 0 and my_gps._total_sv_sentences == my_gps._last_sv_sentence - print('Is Satellite Data Valid?:', data_valid) - if data_valid: - print('Satellite Data:', my_gps._satellite_data) - print('Satellites Visible:', list(my_gps._satellite_data.keys())) - print('') - - print("Pretty Print Examples:") - print('Latitude (degs):', my_gps.latitude_string(as_GPS.DD)) - print('Longitude (degs):', my_gps.longitude_string(as_GPS.DD)) - print('Latitude (dms):', my_gps.latitude_string(as_GPS.DMS)) - print('Longitude (dms):', my_gps.longitude_string(as_GPS.DMS)) - print('Latitude (kml):', my_gps.latitude_string(as_GPS.KML)) - print('Longitude (kml):', my_gps.longitude_string(as_GPS.KML)) - print('Latitude (degs, mins):', my_gps.latitude_string()) - print('Longitude (degs, mins):', my_gps.longitude_string()) - print('Speed:', my_gps.speed_string(as_GPS.KPH), 'or', - my_gps.speed_string(as_GPS.MPH), 'or', - my_gps.speed_string(as_GPS.KNOT)) - print('Date (Long Format):', my_gps.date_string(as_GPS.LONG)) - print('Date (Short D/M/Y Format):', my_gps.date_string(as_GPS.DMY)) - print('Date (Short M/D/Y Format):', my_gps.date_string(as_GPS.MDY)) - print('Time:', my_gps.time_string()) - print() - - print('### Final Results ###') - print('Sentences Attempted:', sentence_count) - print('Sentences Found:', my_gps.clean_sentences) - print('Sentences Parsed:', my_gps.parsed_sentences) - print('Unsupported sentences:', my_gps.unsupported_sentences) - print('CRC_Fails:', my_gps.crc_fails) - -def run_tests(): - loop = asyncio.get_event_loop() - loop.run_until_complete(run()) - -if __name__ == "__main__": - run_tests() diff --git a/gps/astests_pyb.py b/gps/astests_pyb.py deleted file mode 100755 index 5846fe9..0000000 --- a/gps/astests_pyb.py +++ /dev/null @@ -1,150 +0,0 @@ -# astests_pyb.py - -# Tests for AS_GPS module. Emulates a GPS unit using a UART loopback. -# Run on a Pyboard with X1 and X2 linked -# Tests for AS_GPS module (asynchronous GPS device driver) -# Based on tests for MicropyGPS by Michael Calvin McCoy -# https://github.com/inmcm/micropyGPS - -# Copyright (c) 2018 Peter Hinch -# Released under the MIT License (MIT) - see LICENSE file - -import as_GPS -from machine import UART -import uasyncio as asyncio - -def callback(gps, _, arg): - print('Fix callback. Time:', gps.utc, arg) - -async def run_tests(): - uart = UART(4, 9600, read_buf_len=200) - swriter = asyncio.StreamWriter(uart, {}) - sreader = asyncio.StreamReader(uart) - sentence_count = 0 - - test_RMC = ['$GPRMC,180041.896,A,3749.1851,N,08338.7891,W,001.9,154.9,240911,,,A*7A\n', - '$GPRMC,180049.896,A,3749.1808,N,08338.7869,W,001.8,156.3,240911,,,A*70\n', - '$GPRMC,092751.000,A,5321.6802,N,00630.3371,W,0.06,31.66,280511,,,A*45\n'] - - test_VTG = ['$GPVTG,232.9,T,,M,002.3,N,004.3,K,A*01\n'] - test_GGA = ['$GPGGA,180050.896,3749.1802,N,08338.7865,W,1,07,1.1,397.4,M,-32.5,M,,0000*6C\n'] - test_GSA = ['$GPGSA,A,3,07,11,28,24,26,08,17,,,,,,2.0,1.1,1.7*37\n', - '$GPGSA,A,3,07,02,26,27,09,04,15,,,,,,1.8,1.0,1.5*33\n'] - test_GSV = ['$GPGSV,3,1,12,28,72,355,39,01,52,063,33,17,51,272,44,08,46,184,38*74\n', - '$GPGSV,3,2,12,24,42,058,33,11,34,053,33,07,20,171,40,20,15,116,*71\n', - '$GPGSV,3,3,12,04,12,204,34,27,11,324,35,32,11,089,,26,10,264,40*7B\n', - '$GPGSV,3,1,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00*74\n', - '$GPGSV,3,2,11,14,25,170,00,16,57,208,39,18,67,296,40,19,40,246,00*74\n', - '$GPGSV,3,3,11,22,42,067,42,24,14,311,43,27,05,244,00,,,,*4D\n', - '$GPGSV,4,1,14,22,81,349,25,14,64,296,22,18,54,114,21,51,40,212,*7D\n', - '$GPGSV,4,2,14,24,30,047,22,04,22,312,26,31,22,204,,12,19,088,23*72\n', - '$GPGSV,4,3,14,25,17,127,18,21,16,175,,11,09,315,16,19,05,273,*72\n', - '$GPGSV,4,4,14,32,05,303,,15,02,073,*7A\n'] - test_GLL = ['$GPGLL,3711.0942,N,08671.4472,W,000812.000,A,A*46\n', - '$GPGLL,4916.45,N,12311.12,W,225444,A,*1D\n', - '$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n', - '$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n',] - -# '$GPGLL,0000.0000,N,00000.0000,E,235947.000,V*2D\n', # Will ignore this one - - my_gps = as_GPS.AS_GPS(sreader, fix_cb=callback, fix_cb_args=(42,)) - sentence = '' - for sentence in test_RMC: - sentence_count += 1 - await swriter.awrite(sentence) - await my_gps.data_received(date=True) - print('Longitude:', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Time:', my_gps.utc) - print('Speed:', my_gps.speed()) - print('Date Stamp:', my_gps.date) - print('Course', my_gps.course) - print('Data is Valid:', my_gps._valid) - print('Compass Direction:', my_gps.compass_direction()) - print('') - - for sentence in test_GLL: - sentence_count += 1 - await swriter.awrite(sentence) - await my_gps.data_received(position=True) - print('Longitude:', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Time:', my_gps.utc) - print('Data is Valid:', my_gps._valid) - print('') - - for sentence in test_VTG: - print('Test VTG', sentence) - sentence_count += 1 - await swriter.awrite(sentence) - await asyncio.sleep_ms(200) # Can't wait for course because of position check - print('Speed:', my_gps.speed()) - print('Course', my_gps.course) - print('Compass Direction:', my_gps.compass_direction()) - print('') - - for sentence in test_GGA: - sentence_count += 1 - await swriter.awrite(sentence) - await my_gps.data_received(position=True) - print('Longitude', my_gps.longitude()) - print('Latitude', my_gps.latitude()) - print('UTC Time:', my_gps.utc) -# print('Fix Status:', my_gps.fix_stat) - print('Altitude:', my_gps.altitude) - print('Height Above Geoid:', my_gps.geoid_height) - print('Horizontal Dilution of Precision:', my_gps.hdop) - print('Satellites in Use by Receiver:', my_gps.satellites_in_use) - print('') - - for sentence in test_GSA: - sentence_count += 1 - await swriter.awrite(sentence) - await asyncio.sleep_ms(200) - print('Satellites Used', my_gps.satellites_used) - print('Horizontal Dilution of Precision:', my_gps.hdop) - print('Vertical Dilution of Precision:', my_gps.vdop) - print('Position Dilution of Precision:', my_gps.pdop) - print('') - - for sentence in test_GSV: - sentence_count += 1 - await swriter.awrite(sentence) - await asyncio.sleep_ms(200) - print('SV Sentences Parsed', my_gps._last_sv_sentence) - print('SV Sentences in Total', my_gps._total_sv_sentences) - print('# of Satellites in View:', my_gps.satellites_in_view) - data_valid = my_gps._total_sv_sentences > 0 and my_gps._total_sv_sentences == my_gps._last_sv_sentence - print('Is Satellite Data Valid?:', data_valid) - if data_valid: - print('Satellite Data:', my_gps._satellite_data) - print('Satellites Visible:', list(my_gps._satellite_data.keys())) - print('') - - print("Pretty Print Examples:") - print('Latitude (degs):', my_gps.latitude_string(as_GPS.DD)) - print('Longitude (degs):', my_gps.longitude_string(as_GPS.DD)) - print('Latitude (dms):', my_gps.latitude_string(as_GPS.DMS)) - print('Longitude (dms):', my_gps.longitude_string(as_GPS.DMS)) - print('Latitude (kml):', my_gps.latitude_string(as_GPS.KML)) - print('Longitude (kml):', my_gps.longitude_string(as_GPS.KML)) - print('Latitude (degs, mins):', my_gps.latitude_string()) - print('Longitude (degs, mins):', my_gps.longitude_string()) - print('Speed:', my_gps.speed_string(as_GPS.KPH), 'or', - my_gps.speed_string(as_GPS.MPH), 'or', - my_gps.speed_string(as_GPS.KNOT)) - print('Date (Long Format):', my_gps.date_string(as_GPS.LONG)) - print('Date (Short D/M/Y Format):', my_gps.date_string(as_GPS.DMY)) - print('Date (Short M/D/Y Format):', my_gps.date_string(as_GPS.MDY)) - print('Time:', my_gps.time_string()) - print() - - print('### Final Results ###') - print('Sentences Attempted:', sentence_count) - print('Sentences Found:', my_gps.clean_sentences) - print('Sentences Parsed:', my_gps.parsed_sentences) - print('Unsupported sentences:', my_gps.unsupported_sentences) - print('CRC_Fails:', my_gps.crc_fails) - -loop = asyncio.get_event_loop() -loop.run_until_complete(run_tests()) diff --git a/htu21d/README.md b/htu21d/README.md deleted file mode 100644 index 947a679..0000000 --- a/htu21d/README.md +++ /dev/null @@ -1,50 +0,0 @@ -# The HTU21D temperature/humidity sensor. - -A breakout board is available from -[Sparkfun](https://www.sparkfun.com/products/12064). - -This driver was derived from the synchronous Pyboard-specific driver -[here](https://github.com/manitou48/pyboard/blob/master/htu21d.py). It is -designed to be multi-platform and uses `uasyncio` to achieve asynchronous (non- -blocking) operation. The driver maintains `temperature` and `humidity` bound -variables as a non-blocking background task. Consequently reading the values is -effectively instantaneous. - -###### [Main README](../README.md) - -# Files - - 1. `htu21d_mc.py` The asynchronous driver. - 2. `htu_test.py` Test/demo program. - -# The driver - -This provides a single class `HTU21D`. - -Constructor. -This takes two args, `i2c` (mandatory) and an optional `read_delay=10`. The -former must be an initialised I2C bus instance. The `read_delay` (secs) -determines how frequently the data values are updated. - -Public bound values - 1. `temperature` Latest value in Celcius. - 2. `humidity` Latest value of relative humidity (%). - -Initial readings will not be complete until about 120ms after the class is -instantiated. Prior to this the values will be `None`. To avoid such invalid -readings the class is awaitable and may be used as follows. - -```python -async def show_values(): - htu = htu21d_mc.HTU21D(i2c) - await htu # Will pause ~120ms - # Data is now valid - while True: - fstr = 'Temp {:5.1f} Humidity {:5.1f}' - print(fstr.format(htu.temperature, htu.humidity)) - await asyncio.sleep(5) -``` - -Thermal inertia of the chip packaging means that there is a lag between the -occurrence of a temperature change and the availability of accurate readings. -There is therefore little practical benefit in reducing the `read_delay`. diff --git a/io.py b/io.py deleted file mode 100644 index 348f1a3..0000000 --- a/io.py +++ /dev/null @@ -1,39 +0,0 @@ -# io.py Failed attempt to use uasyncio IORead mechanism in a custom class. -# It turns out that the necessary support has not been implemented, and -# it is unlikely that this will occur. -import uasyncio as asyncio - -MP_STREAM_POLL_RD = 1 -MP_STREAM_POLL = 3 - -import uasyncio as asyncio -class Device(): - def __init__(self): - self.ready = False - - def fileno(self): - return 999 - - def ioctl(self, cmd, flags): - res = 0 - print('Got here') - if cmd == MP_STREAM_POLL and (flags & MP_STREAM_POLL_RD): - if self.ready: - res = MP_STREAM_POLL_RD - return res - - def read(self): - return - def write(self): - return - - async def readloop(self): - while True: - print('About to yield') - yield asyncio.IORead(self) - print('Should never happen') - -loop = asyncio.get_event_loop() -device = Device() -loop.create_task(device.readloop()) -loop.run_forever() diff --git a/lowpower/README.md b/lowpower/README.md deleted file mode 100644 index acaedc8..0000000 --- a/lowpower/README.md +++ /dev/null @@ -1,496 +0,0 @@ -# A low power usayncio adaptation - -Release 0.13 17th Oct 2019 - -API changes: low power applications must now import `rtc_time_cfg` and set its -`enabled` flag. -`Latency` is now a functor rather than a class. - -This module is specific to Pyboards including the D series. - - 1. [Introduction](./README.md#1-introduction) - 2. [Installation](./README.md#2-installation) - 2.1 [Files](./README.md#21-files) - 3. [Low power uasyncio operation](./README.md#3-low-power-uasyncio-operation) - 3.1 [The official uasyncio package](./README.md#31-the-official-uasyncio-package) - 3.2 [The low power adaptation](./README.md#32-the-low-power-adaptation) - 3.2.1 [Consequences of stop mode](./README.md#321-consequences-of-stop-mode) - 3.2.1.1 [Timing Accuracy and rollover](./README.md#3211-timing-accuracy-and-rollover) - 3.2.1.2 [USB](./README.md#3212-usb) - 3.2.2 [Measured results Pyboard 1](./README.md#322-measured-results-pyboard-1) - 3.2.3 [Current waveforms Pyboard 1](./README.md#323-current-waveforms-pyboard-1) - 3.2.4 [Pyboard D measurements](./README.md#324-pyboard-d-measurements) - 4. [The rtc_time module](./README.md#4-the-rtc_time-module) - 4.1 [rtc_time_cfg](./README.md#41-rtc_time_cfg) - 5. [Application design](./README.md#5-application-design) - 5.1 [Hardware](./README.md#51-hardware) - 5.2 [Application Code](./README.md#52-application-code) - 6. [Note on the design](./README.md#6-note-on-the-design) - -###### [Main README](../README.md) - -# 1. Introduction - -This adaptation is specific to the Pyboard and compatible platforms, namely -those capable of running the `pyb` module; this supports two low power modes -`standby` and `stop` -[see docs](http://docs.micropython.org/en/latest/pyboard/library/pyb.html). - -Use of `standby` is simple in concept: the application runs and issues -`standby`. The board goes into a very low power mode until it is woken by one -of a limited set of external events, when it behaves similarly to after a hard -reset. In that respect a `uasyncio` application is no different from any other. -If the application can cope with the fact that execution state is lost during -the delay, it will correctly resume. - -This adaptation modifies `uasyncio` such that it can enter `stop` mode for much -of the time, minimising power consumption while retaining state. The two -approaches can be combined, with a device waking from `shutdown` to run a low -power `uasyncio` application before again entering `shutdown`. - -The adaptation trades a reduction in scheduling performance for a substantial -reduction in power consumption. This tradeoff can be dynamically altered at -runtime. An application can wait with low power consumption for a trigger such -as a button push. Or it could periodically self-trigger by issuing -`await ayncio.sleep(long_time)`. For the duration of running the scheduler -latency can be reduced to improve performance at the cost of temporarily -higher power consumption, with the code reverting to low power mode while -waiting for a new trigger. - -Some general notes on low power Pyboard applications may be found -[here](https://github.com/peterhinch/micropython-micropower). - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -# 2. Installation - -Ensure that the version of `uasyncio` in this repository is installed and -tested. Copy the files `rtc_time.py` and `rtc_time_cfg.py` to the device so -that they are on `sys.path`. - -## 2.1 Files - - * `rtc_time.py` Low power library. - * `rtc_time_cfg` Configuration file to enable `uasyncio` to use above. - * `lpdemo.py` A basic application which waits for a pushbutton to be pressed - before running. A second button press terminates it. While "off" and waiting - very low power is consumed. A normally open pushbutton should be connected - between `X1` and `Gnd`. This program is intended as a basic template for - similar applications. - * `howlow.py` A lower power version of the above. Polls the switch every 200ms - rather than running debouncing code. - * `lp_uart.py` Send and receive messages on UART4, echoing received messages - to UART1 at a different baudrate. This consumes about 1.4mA and serves to - demonstrate that interrupt-driven devices operate correctly. Requires a link - between pins X1 and X2 to enable UART 4 to receive data via a loopback. - * `mqtt_log.py` A publish-only MQTT application for Pyboard D. See below. - -`mqtt_log.py` requires the `umqtt.simple` library. This may be installed with -upip. See [Installing library modules](https://github.com/peterhinch/micropython-samples/tree/master/micropip). -``` ->>> import upip ->>> upip.install('micropython-umqtt.simple') -``` -Owing to [this issue](https://github.com/micropython/micropython/issues/5152) -this test is currently broken and I suspect that any usage of WiFi in low power -mode will fail. - -This test is "experimental". Pyboard D support for low power WiFi is currently -incomplete. I have seen anomolous results where power was low initially before -jumping to ~30mA after a few hours. The application continued to run, but the -evidence suggested that the WiFi chip was consuming power. See Damien's comment -in [this issue](https://github.com/micropython/micropython/issues/4686). -An option would be to shut down the WiFi chip after each connection. The need -periodically to reconnect would consume power, but in applications which log at -low rates this should avoid the above issue. Or wait for the firmware to mature. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -# 3 Low power uasyncio operation - -## 3.1 The official uasyncio package - -The official `uasyncio` library is unsuited to low power operation for two -reasons. Firstly because of its method of I/O polling. In periods when no task -is ready for execution, it determines the time when the most current task will -be ready to run. It then calls `select.poll`'s `ipoll` method with a timeout -calculated on that basis. This consumes power. - -The second issue is that it uses `utime`'s millisecond timing utilities for -timing. This ensures portability across MicroPython platforms. Unfortunately on -the Pyboard the clock responsible for `utime` stops for the duration of -`pyb.stop()`. If an application were to use `pyb.stop` to conserve power it -would cause `uasyncio` timing to become highly inaccurate. - -## 3.2 The low power adaptation - -If running on a Pyboard the version of `uasyncio` in this repo attempts to -import the file `rtc_time.py`. If this succeeds and there is no USB connection -to the board it derives its millisecond timing from the RTC; this continues to -run through `stop`. So code such as the following will behave as expected: -```python -async def foo(): - await asyncio.sleep(10) - bar() - await asyncio.sleep_ms(100) -``` -Libraries and applications using `uasyncio` will run unmodified. Code adapted -to invoke power saving (as described below) may exhibit reduced performance: -there is a tradeoff beween power consumption and speed. - -To avoid the power drain caused by `select.poll` the user code must issue the -following: - -```python -import rtc_time_cfg -rtc_time_cfg.enabled = True # Must be done before importing uasyncio - -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') -from rtc_time import Latency - # Instantiate event loop with any args before running code that uses it -loop = asyncio.get_event_loop() -Latency(100) # Define latency in ms -``` - -`Latency` is a functor: its only interface is with function call syntax, which -takes a single argument being the `lightsleep` duration in ms. If the lowpower -mode is in operation the first call instantiates a coroutine with a -continuously running loop that executes `pyb.stop` before yielding with a zero -delay. The duration of the `lightsleep` condition can be dynamically varied by -further `Latency(time_in_ms)` calls. If the arg is zero the scheduler will run -at full speed. The `yield` allows each pending task to run once before the -scheduler is again paused (if the current latency value is > 0). - -The line -```python -rtc_time_cfg.enabled = True -``` -must be issued before importing `uasyncio` and before importing any modules -which use it, otherwise low-power mode will not be engaged. It is wise to do -this at the start of application code. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -### 3.2.1 Consequences of stop mode - -#### 3.2.1.1 Timing Accuracy and rollover - -A minor limitation is that the Pyboard 1.x RTC cannot resolve times of less -than 4ms so there is a theoretical reduction in the accuracy of delays. This -does not apply to the Pyboard D. This is somewhat academic. As explained in the -[tutorial](../TUTORIAL.md), issuing - -```python -await asyncio.sleep_ms(t) -``` - -specifies a minimum delay: the maximum may be substantially higher depending on -the behaviour of other tasks. Also the `latency` value will be added to `t`. - -RTC time rollover is at 7 days. The maximum supported `asyncio.sleep()` value -is 302399999 seconds (3.5 days - 1s). - -#### 3.2.1.2 USB - -Programs using `pyb.stop` disable the USB connection to the PC. This is -inconvenient for debugging so `rtc_time.py` detects an active USB connection -and disables power saving. This enables an application to be developed normally -via a USB connected PC. The board can then be disconnected from the PC and run -from a separate power source for power measurements, the application being -started by `main.py`. - -An active USB connection is one where a PC application is accessing the port: -an unused port can power the Pyboard and the library will assume low-power -mode. If the Pyboard is booted in safe mode to bypass `main.py` and the -application is started at the REPL, USB detection will disable low power mode -to keep the connection open. - -Applications can detect which timebase is in use by issuing: - -```python -import rtc_time_cfg -rtc_time_cfg.enabled = True # Must be done before importing uasyncio - -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') -import rtc_time -if rtc_time.use_utime: - # Timebase is utime: either a USB connection exists or not a Pyboard -else: - # Running on RTC timebase with no USB connection -``` - -Debugging at low power is facilitated by using `pyb.repl_uart` with an FTDI -adaptor. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -### 3.2.2 Measured results Pyboard 1 - -The `lpdemo.py` script consumes a mean current of 980μA with 100ms latency, and -730μA with 200ms latency, while awaiting a button press. - -The following script consumes about 380μA between wakeups (usb is disabled in -`boot.py`): - -```python -import pyb -for pin in [p for p in dir(pyb.Pin.board) if p[0] in 'XY']: - pin_x = pyb.Pin(pin, pyb.Pin.IN, pyb.Pin.PULL_UP) -rtc = pyb.RTC() -rtc.wakeup(10000) -while True: - pyb.stop() -``` - -This accords with the 500μA maximum specification for `stop`. So current -consumption can be estimated by -`i = ib + n/latency` -`ib` is the stopped current (in my case 380μA). -`n` is a factor dependent on the amount of code which runs when the latency -period expires. - -A data logging application might tolerate latencies of many seconds while -waiting for a long delay to expire: getting close to `ib` may be practicable -for such applications during their waiting period. - -### 3.2.3 Current waveforms Pyboard 1 - -Running `lpdemo.py` while it waits for a button press with latency = 200ms. -It consumes 380μA except for brief peaks while polling the switch. -Vertical 20mA/div -Horizontal 50ms/div -![Image](./current.png) - -The following shows that peak on a faster timebase. This type of waveform is -typical that experienced when Python code is running. -Vertical 20mA/div -Horizontal 500μs/div -![Image](./current1.png) - -### 3.2.4 Pyboard D measurements - -As of this release the `lpdemo.py` script consumes around 1.1mA. I believe this -can be reduced because some unused pins are floating. When I discover which -pins can be set to input with pullups as per the Pyboard 1.x implementation I -hope to see figures comparable to Pyboard 1.x. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -# 4. The rtc_time module - -This provides the following. - -Variables (treat as read-only): - * `use_utime` `True` if the `uasyncio` timebase is `utime`, `False` if it is - the RTC. - * `d_series` `True` if running on Pyboard D, `False` if on Pyboard 1.x. - -Functions: -If the timebase is `utime` these are references to the corresponding `utime` -functions. Otherwise they are direct replacements but using the RTC as their -timebase. See the `utime` -[official documentation](http://docs.micropython.org/en/latest/pyboard/library/utime.html) -for these. - * `ticks_ms` - * `ticks_add` - * `ticks_diff` - -It also exposes `sleep_ms`. This is always a reference to `utime.sleep_ms`. The -reason is explained in the code comments. It is recommended to use the `utime` -method explicitly if needed. - -Latency Class: - * Constructor: Positional args `loop` - the event loop, `t_ms=100` - period - for which the scheduler enters `stop` i.e. initial latency period. - * Method: `value` Arg `val=None`. Controls period for which scheduler - stops. It returns the period in ms prior to any change in value. If the - default `None` is passed the value is unchanged. If 0 is passed the scheduler - runs at full speed. A value > 0 sets the stop period in ms. - -The higher the value, the greater the latency experienced by other tasks and -by I/O. Smaller values will result in higher power consumption with other tasks -being scheduled more frequently. - -The class is a singleton consequently there is no need to pass an instance -around or to make it global. Once instantiated, latency may be changed by - -```python -Latency(t) -``` - -## 4.1 rtc_time_cfg - -This consists of the following: -```python -enabled = False -disable_3v3 = False -disable_leds = False -disable_pins = False -``` -These variables may selectively be set `True` by the application prior to -importing `uasyncio`. Setting `enabled` is mandatory if low power mode is to be -engaged. The other variables control the 3.3V regulator, the LED drivers and -GPIO pins: the latter may be set to inputs with pulldown resistors to minimise -current draw. Unfortunately at the time of writing this feature seems to have -a fatal effect. I am investigating. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -# 5. Application design - -Attention to detail is required to minimise power consumption, both in terms of -hardware and code. The only *required* change to application code is to add - -```python -import rtc_time_cfg -rtc_time_cfg.enabled = True # Must be done before importing uasyncio - -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') - # Do this import before configuring any pins or I/O: -from rtc_time import Latency - # Instantiate event loop with any args before running code that uses it: -loop = asyncio.get_event_loop() -Latency(100) # Define latency in ms - # Run application code -``` - -However optimising the power draw/performance tradeoff benefits from further -optimisations. - -## 5.1 Hardware - -Hardware issues are covered [here](https://github.com/peterhinch/micropython-micropower). -To summarise an SD card consumes on the order of 150μA. For lowest power -consumption use the onboard flash memory. Peripherals usually consume power -even when not in use: consider switching their power source under program -control. - -Floating Pyboard I/O pins can consume power. Further there are 4.7KΩ pullups on -the I2C pins. The `rtc_time` module sets all pins as inputs with internal -pullups. The application should import `rtc_time` before configuring any pins -or instantiating any drivers which use pins. If I2C is to be used there are -implications regarding the onboard pullups: see the above reference. - -## 5.2 Application Code - -The Pyboard has only one RTC and the `Latency` class needs sole use of -`pyb.stop` and `rtc.wakeup`; these functions should not be used in application -code. Setting the RTC at runtime is likely to be problematic: the effect on -scheduling can be assumed to be malign. If required, the RTC should be set -prior to instantiating the event loop. - -For short delays use `utime.sleep_ms` or `utime.sleep_us`. Such delays use -power and hog execution preventing other tasks from running. - -A task only consumes power when it runs: power may be reduced by using larger -values of `t` in - -```python -await asyncio.sleep(t) -``` - -The implications of the time value of the `Latency` instance should be -considered. During periods when the Pyboard is in a `stop` state, other tasks -will not be scheduled. I/O from interrupt driven devices such as UARTs will be -buffered for processing when stream I/O is next scheduled. The size of buffers -needs to be determined in conjunction with data rates and the latency period. - -Long values of latency affect the minimum time delays which can be expected of -`await asyncio.sleep_ms`. Such values will affect the aggregate amount of CPU -time any task will acquire in any period. If latency is 200ms the task - -```python -async def foo(): - while True: - # Do some processing - await asyncio.sleep(0) -``` - -will execute (at best) at a rate of 5Hz; possibly less, depending on the -behaviour of competing tasks. Likewise with 200ms latency - -```python -async def bar(): - while True: - # Do some processing - await asyncio.sleep_ms(10) -``` - -the 10ms sleep will be >=200ms dependent on other application tasks. - -Latency may be changed dynamically by issuing `Latency(time_in_ms)`. A typical -application (as in `howlow.py`) might wait on a "Start" button with a high -latency value, before running the application code with a lower (or zero) -latency. On completion it could revert to waiting for "Start" with high latency -to conserve battery. Logging applications might pause for a duration or wait on -a specific RTC time with a high latency value. - -Pyboard D users should note that firmware support for low power WiFi is -incomplete. Consider turning off the WiFi chip when not in use: -``` -sta_if = network.WLAN() -while True: - # Wait for trigger - sta_if.active(True) # Enable WiFi - sta_if.connect(SSID, PW) - # Use the network - sta_if.deinit() # Turns off WiFi chip -``` -[ref](https://github.com/micropython/micropython/issues/4681) - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) - -# 6. Note on the design - -This module uses the old `pyb` in preference to `machine`. This is because the -Pyboard 1.x `machine` module does not have an `RTC` class. - -The `rtc_time` module represents a compromise designed to minimise changes to -`uasyncio`. The aim is to have zero effect on the performance of applications -not using `rtc_time` or ones running on non-Pyboard hardware. - -An alternative approach is to modify the `PollEventLoop` `wait` method to -invoke `stop` conditions when required. It would have the advantage of removing -the impact of latency on `sleep_ms` times. It proved rather involved and was -abandoned on the grounds of its impact on performance of normal applications. -Despite its name, `.wait` is time-critical in the common case of a zero delay; -increased code is best avoided. - -The approach used ensures that there is always at least one task waiting on a -zero delay. This guarantees that `PollEventLoop` `wait` is always called with a -zero delay: consequently `self.poller.ipoll(delay, 1)` will always return -immediately minimising power consumption. Consequently there is no change to -the design of the scheduler beyond the use of a different timebase. It does, -however, rely on the fact that the scheduler algorithm behaves as described -above. - -By default `uasyncio` uses the `utime` module for timing. For the timing to be -derived from the RTC the following conditions must be met: - * Hardware must be a Pyboard 1.x, Pyboard D or compatible (i.e. able to use - the `pyb` module). - * The application must import `rtc_time_cfg` and set its `enabled` flag `True` - before importing `uasyncio`. - * `uasyncio` must be the `fast_io` version 2.4 or later. - * The `rtc_time` module must be on the MicroPython search path. - * There must be no active USB connection. - -These constraints ensure there is no performance penalty unless an application -specifically requires micropower operation. They also enable a USB connection -to work if required for debugging. - -###### [Contents](./README.md#a-low-power-usayncio-adaptation) diff --git a/lowpower/current.png b/lowpower/current.png deleted file mode 100644 index df45b56..0000000 Binary files a/lowpower/current.png and /dev/null differ diff --git a/lowpower/current1.png b/lowpower/current1.png deleted file mode 100644 index feec091..0000000 Binary files a/lowpower/current1.png and /dev/null differ diff --git a/lowpower/lp_uart.py b/lowpower/lp_uart.py deleted file mode 100644 index 42cc586..0000000 --- a/lowpower/lp_uart.py +++ /dev/null @@ -1,53 +0,0 @@ -# lp_uart.py Demo of using uasyncio to reduce Pyboard power consumption -# Author: Peter Hinch -# Copyright Peter Hinch 2018 Released under the MIT license - -# The files rtc_time.py and rtc_time_cfg.py must be on the path. -# Requires a link between X1 and X2. -# Periodically sends a line on UART4 at 9600 baud. -# This is received on UART4 and re-sent on UART1 (pin Y1) at 115200 baud. - -import pyb -import rtc_time_cfg -rtc_time_cfg.enabled = True # Must be done before importing uasyncio - -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') -from rtc_time import Latency, use_utime - -# Stop the test after a period -async def killer(duration): - await asyncio.sleep(duration) - -# Periodically send text through UART -async def sender(uart): - swriter = asyncio.StreamWriter(uart, {}) - while True: - await swriter.awrite('Hello uart\n') - await asyncio.sleep(1.3) - -# Each time a message is received echo it on uart 4 -async def receiver(uart_in, uart_out): - sreader = asyncio.StreamReader(uart_in) - swriter = asyncio.StreamWriter(uart_out, {}) - while True: - res = await sreader.readline() - await swriter.awrite(res) - -def test(duration): - if use_utime: # Not running in low power mode - pyb.LED(3).on() - uart2 = pyb.UART(1, 115200) - uart4 = pyb.UART(4, 9600) - # Instantiate event loop before using it in Latency class - loop = asyncio.get_event_loop() - lp = Latency(50) # ms - loop.create_task(sender(uart4)) - loop.create_task(receiver(uart4, uart2)) - loop.run_until_complete(killer(duration)) - -test(60) diff --git a/lowpower/lpdemo.py b/lowpower/lpdemo.py deleted file mode 100644 index 1f68631..0000000 --- a/lowpower/lpdemo.py +++ /dev/null @@ -1,75 +0,0 @@ -# lpdemo.py Demo/test program for MicroPython asyncio low power operation -# Author: Peter Hinch -# Copyright Peter Hinch 2018-2019 Released under the MIT license - -import rtc_time_cfg -rtc_time_cfg.enabled = True - -from pyb import LED, Pin -import aswitch -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') -from rtc_time import Latency - -class Button(aswitch.Switch): - def __init__(self, pin): - super().__init__(pin) - self.close_func(self._sw_close) - self._flag = False - - def pressed(self): - f = self._flag - self._flag = False - return f - - def _sw_close(self): - self._flag = True - -running = False -def start(loop, leds, tims): - global running - running = True - coros = [] - # Demo: assume app requires higher speed (not true in this instance) - Latency(50) - # Here you might apply power to external hardware - for x, led in enumerate(leds): # Create a coroutine for each LED - coros.append(toggle(led, tims[x])) - loop.create_task(coros[-1]) - return coros - -def stop(leds, coros): - global running - running = False - while coros: - asyncio.cancel(coros.pop()) - # Remove power from external hardware - for led in leds: - led.off() - Latency(200) # Slow down scheduler to conserve power - -async def monitor(loop, button): - leds = [LED(x) for x in (1, 2, 3)] # Create list of LED's and times - tims = [200, 700, 1200] - coros = start(loop, leds, tims) - while True: - if button.pressed(): - if running: - stop(leds, coros) - else: - coros = start(loop, leds, tims) - await asyncio.sleep_ms(0) - -async def toggle(objLED, time_ms): - while True: - await asyncio.sleep_ms(time_ms) - objLED.toggle() - -loop = asyncio.get_event_loop() -button = Button(Pin('X1', Pin.IN, Pin.PULL_UP)) -loop.create_task(monitor(loop, button)) -loop.run_forever() diff --git a/lowpower/mqtt_log.py b/lowpower/mqtt_log.py deleted file mode 100644 index 21e50d9..0000000 --- a/lowpower/mqtt_log.py +++ /dev/null @@ -1,63 +0,0 @@ -# mqtt_log.py Demo/test program for MicroPython asyncio low power operation -# Author: Peter Hinch -# Copyright Peter Hinch 2019 Released under the MIT license - -# MQTT Demo publishes an incremental count and the RTC time periodically. -# On my SF_2W board consumption while paused was 170μA. - -# Test reception e.g. with: -# mosquitto_sub -h 192.168.0.10 -t result - -import rtc_time_cfg -rtc_time_cfg.enabled = True - -from pyb import LED, RTC -from umqtt.simple import MQTTClient -import network -import ujson -from local import SERVER, SSID, PW # Local configuration: change this file - -import uasyncio as asyncio -try: - if asyncio.version[0] != 'fast_io': - raise AttributeError -except AttributeError: - raise OSError('This requires fast_io fork of uasyncio.') -from rtc_time import Latency - -def publish(s): - c = MQTTClient('umqtt_client', SERVER) - c.connect() - c.publish(b'result', s.encode('UTF8')) - c.disconnect() - -async def main(loop): - rtc = RTC() - red = LED(1) - red.on() - grn = LED(2) - sta_if = network.WLAN() - sta_if.active(True) - sta_if.connect(SSID, PW) - while sta_if.status() in (1, 2): # https://github.com/micropython/micropython/issues/4682 - await asyncio.sleep(1) - grn.toggle() - if sta_if.isconnected(): - red.off() - grn.on() - await asyncio.sleep(1) # 1s of green == success. - grn.off() # Conserve power - Latency(2000) - count = 0 - while True: - print('Publish') - publish(ujson.dumps([count, rtc.datetime()])) - count += 1 - print('Wait 2 mins') - await asyncio.sleep(120) # 2 mins - else: # Fail to connect - red.on() - grn.off() - -loop = asyncio.get_event_loop() -loop.run_until_complete(main(loop)) diff --git a/lowpower/rtc_time.py b/lowpower/rtc_time.py deleted file mode 100644 index b6b7137..0000000 --- a/lowpower/rtc_time.py +++ /dev/null @@ -1,166 +0,0 @@ -# rtc_time.py Pyboard-only RTC based timing for low power uasyncio -# Author: Peter Hinch -# Copyright Peter Hinch 2018-2019 Released under the MIT license - -# Code based on extmod/utime_mphal.c -# millisecs roll over on 7 days rather than 12.42757 days - -# If not running on a Pyboard the normal utime timebase is used. This is also -# used on a USB connected Pyboard to keep the USB connection open. -# On an externally powered Pyboard an RTC timebase is substituted. - -import sys -import utime -from os import uname -from rtc_time_cfg import enabled, disable_3v3, disable_leds, disable_pins - -if not enabled: # uasyncio traps this and uses utime - raise ImportError('rtc_time is not enabled.') - -# sleep_ms is defined to stop things breaking if someone imports uasyncio.core -# Power won't be saved if this is done. -sleep_ms = utime.sleep_ms - -d_series = uname().machine[:4] == 'PYBD' -use_utime = True # Assume the normal utime timebase - -if sys.platform == 'pyboard': - import pyb - mode = pyb.usb_mode() - if mode is None: # USB is disabled - use_utime = False # use RTC timebase - elif 'VCP' in mode: # User has enabled VCP in boot.py - # Detect an active connection (not just a power source) - if pyb.USB_VCP().isconnected(): # USB will work normally - print('USB connection: rtc_time disabled.') - else: - pyb.usb_mode(None) # Save power - use_utime = False # use RTC timebase -else: - raise OSError('rtc_time.py is Pyboard-specific.') - -# For lowest power consumption set unused pins as inputs with pullups. -# Note the 4K7 I2C pullups on X9 X10 Y9 Y10 (Pyboard 1.x). - -# Pulling Pyboard D pins should be disabled if using WiFi as it now seems to -# interfere with it. Although until issue #5152 is fixed it's broken anyway. -if d_series: - print('Running on Pyboard D') - if not use_utime: - def low_power_pins(): - pins = [ - # user IO pins - 'A0', 'A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'A10', 'A11', 'A12', 'A13', 'A14', 'A15', - 'B0', 'B1', 'B3', 'B4', 'B5', 'B7', 'B8', 'B9', 'B10', 'B11', 'B12', 'B13', - 'C0', 'C1', 'C2', 'C3', 'C4', 'C5', 'C6', - 'D0', 'D3', 'D8', 'D9', - 'E0', 'E1', 'E12', 'E14', 'E15', - 'F1', 'F6', 'F7', 'F8', 'F9', 'F10', 'F11', 'F13', 'F14', 'F15', - 'H2', 'H3', 'H5', 'H6', 'H7', 'H8', - 'I0', 'I1', - - # internal pins - 'D1', 'D14', 'D15', - 'F0', 'F12', - 'G0', 'G1', 'G2', 'G3', 'G4', 'G5', #'G6', - 'H4', 'H9', 'H10', 'H11', 'H12', 'H13', 'H14', 'H15', - 'I2', 'I3', - ] - pins_led = ['F3', 'F4', 'F5',] - pins_sdmmc = ['D6', 'D7', 'G9', 'G10', 'G11', 'G12'] - pins_wlan = ['D2', 'D4', 'I7', 'I8', 'I9', 'I11'] - pins_bt = ['D5', 'D10', 'E3', 'E4', 'E5', 'E6', 'G8', 'G13', 'G14', 'G15', 'I4', 'I5', 'I6', 'I10'] - pins_qspi1 = ['B2', 'B6', 'D11', 'D12', 'D13', 'E2'] - pins_qspi2 = ['E7', 'E8', 'E9', 'E10', 'E11', 'E13'] - if disable_pins: - for p in pins: - pyb.Pin(p, pyb.Pin.IN, pyb.Pin.PULL_DOWN) - if disable_3v3: - pyb.Pin('EN_3V3', pyb.Pin.IN, None) - if disable_leds: - for p in pins_led: - pyb.Pin(p, pyb.Pin.IN, pyb.Pin.PULL_UP) - low_power_pins() -else: - print('Running on Pyboard 1.x') - for pin in [p for p in dir(pyb.Pin.board) if p[0] in 'XY']: - pin_x = pyb.Pin(pin, pyb.Pin.IN, pyb.Pin.PULL_UP) -# User code redefines any pins in use - -if use_utime: - ticks_ms = utime.ticks_ms - ticks_add = utime.ticks_add - ticks_diff = utime.ticks_diff -else: # All conditions met for low power operation - _PERIOD = const(604800000) # ms in 7 days - _PERIOD_2 = const(302400000) # half period - _SS_TO_MS = 1000/256 # Subsecs to ms - rtc = pyb.RTC() - # dt: (year, month, day, weekday, hours, minutes, seconds, subseconds) - # weekday is 1-7 for Monday through Sunday. - if d_series: - # Subseconds are μs - def ticks_ms(): - dt = rtc.datetime() - return ((dt[3] - 1)*86400000 + dt[4]*3600000 + dt[5]*60000 + dt[6]*1000 + - int(dt[7] / 1000)) - else: - # subseconds counts down from 255 to 0 - def ticks_ms(): - dt = rtc.datetime() - return ((dt[3] - 1)*86400000 + dt[4]*3600000 + dt[5]*60000 + dt[6]*1000 + - int(_SS_TO_MS * (255 - dt[7]))) - - def ticks_add(a, b): - return (a + b) % _PERIOD - - def ticks_diff(end, start): - return ((end - start + _PERIOD_2) % _PERIOD) - _PERIOD_2 - -import uasyncio as asyncio - -def functor(cls): - instance = None - def getinstance(*args, **kwargs): - nonlocal instance - if instance is None: - instance = cls(*args, **kwargs) - return instance - return instance(*args, **kwargs) - return getinstance - -@functor -class Latency: - def __init__(self, t_ms=100): - if use_utime: # Not in low power mode: t_ms stays zero - self._t_ms = 0 - else: - if asyncio.got_event_loop(): - self._t_ms = max(t_ms, 0) - loop = asyncio.get_event_loop() - loop.create_task(self._run()) - else: - raise OSError('Event loop not instantiated.') - - def _run(self): - print('Low power mode is ON.') - rtc = pyb.RTC() - rtc.wakeup(self._t_ms) - t_ms = self._t_ms - while True: - if t_ms > 0: - pyb.stop() - # Pending tasks run once, may change self._t_ms - yield - if t_ms != self._t_ms: # Has changed: update wakeup - t_ms = self._t_ms - if t_ms > 0: - rtc.wakeup(t_ms) - else: - rtc.wakeup(None) - - def __call__(self, t_ms=None): - v = self._t_ms - if t_ms is not None: - self._t_ms = max(t_ms, 0) - return v diff --git a/lowpower/rtc_time_cfg.py b/lowpower/rtc_time_cfg.py deleted file mode 100644 index c7c7d5e..0000000 --- a/lowpower/rtc_time_cfg.py +++ /dev/null @@ -1,5 +0,0 @@ -# rtc_time_cfg.py -enabled = False -disable_3v3 = False -disable_leds = False -disable_pins = False diff --git a/nec_ir/art.py b/nec_ir/art.py deleted file mode 100644 index c861a50..0000000 --- a/nec_ir/art.py +++ /dev/null @@ -1,47 +0,0 @@ -# art.py Test program for IR remote control decoder aremote.py -# Supports Pyboard and ESP8266 - -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license - -# Run this to characterise a remote. - -from sys import platform -import uasyncio as asyncio -ESP32 = platform == 'esp32' or platform == 'esp32_LoBo' - -if platform == 'pyboard': - from pyb import Pin -elif platform == 'esp8266' or ESP32: - from machine import Pin, freq -else: - print('Unsupported platform', platform) - -from aremote import * - -errors = {BADSTART : 'Invalid start pulse', BADBLOCK : 'Error: bad block', - BADREP : 'Error: repeat', OVERRUN : 'Error: overrun', - BADDATA : 'Error: invalid data', BADADDR : 'Error: invalid address'} - -def cb(data, addr): - if data == REPEAT: - print('Repeat') - elif data >= 0: - print(hex(data), hex(addr)) - else: - print('{} Address: {}'.format(errors[data], hex(addr))) - -def test(): - print('Test for IR receiver. Assumes NEC protocol.') - if platform == 'pyboard': - p = Pin('X3', Pin.IN) - elif platform == 'esp8266': - freq(160000000) - p = Pin(13, Pin.IN) - elif ESP32: - p = Pin(23, Pin.IN) - ir = NEC_IR(p, cb, True) # Assume r/c uses extended addressing - loop = asyncio.get_event_loop() - loop.run_forever() - -test() diff --git a/roundrobin.py b/roundrobin.py deleted file mode 100644 index 5aefae1..0000000 --- a/roundrobin.py +++ /dev/null @@ -1,37 +0,0 @@ -# roundrobin.py Test/demo of round-robin scheduling -# Author: Peter Hinch -# Copyright Peter Hinch 2017 Released under the MIT license - -# Result on Pyboard with print('Foo', n) commented out -# executions/second: -# Using yield: 4249 -# Using sleep_ms(0) 2750 -# Note using yield in a coro is "unofficial" and may not -# work in future uasyncio revisions. - -import uasyncio as asyncio - -count = 0 -period = 5 - - -async def foo(n): - global count - while True: -# yield - await asyncio.sleep_ms(0) - count += 1 - print('Foo', n) - - -async def main(delay): - print('Testing for {} seconds'.format(period)) - await asyncio.sleep(delay) - - -loop = asyncio.get_event_loop() -loop.create_task(foo(1)) -loop.create_task(foo(2)) -loop.create_task(foo(3)) -loop.run_until_complete(main(period)) -print('Coro executions per sec =', count/period) diff --git a/sock_nonblock.py b/sock_nonblock.py deleted file mode 100644 index 2f44464..0000000 --- a/sock_nonblock.py +++ /dev/null @@ -1,110 +0,0 @@ -# sock_nonblock.py Illustration of the type of code required to use nonblocking -# sockets. It is not a working demo and probably has silly errors. -# It is intended as an outline of requirements and also to illustrate some of the -# nasty hacks required on current builds of ESP32 firmware. Platform detection is -# done at runtime. -# If running on ESP8266 these hacks can be eliminated. -# Working implementations may be found in the asynchronous MQTT library. -# https://github.com/peterhinch/micropython-mqtt - -# Author: Peter Hinch -# Copyright Peter Hinch 2018 Released under the MIT license - -import usocket as socket -import network -import machine -import sys -from micropython import const -from uerrno import EINPROGRESS, ETIMEDOUT -from utime import ticks_ms, ticks_diff, sleep_ms - -ESP32 = sys.platform == 'esp32' - -BUSY_ERRORS = [EINPROGRESS, ETIMEDOUT] - -# ESP32. It is not enough to regularly yield to RTOS with machine.idle(). There are -# two cases where an explicit sleep() is required. Where data has been written to the -# socket and a response is awaited, a timeout may occur without a >= 20ms sleep. -# Secondly during WiFi connection sleeps are required to prevent hangs. -if ESP32: - # https://forum.micropython.org/viewtopic.php?f=16&t=3608&p=20942#p20942 - BUSY_ERRORS += [118, 119] # Add in weird ESP32 errors - # 20ms seems about the minimum before we miss data read from a socket. - def esp32_pause(): # https://github.com/micropython/micropython-esp32/issues/167 - sleep_ms(20) # This is horrible. -else: - esp32_pause = lambda *_ : None # Do nothing on sane platforms - -# How long to delay between polls. Too long affects throughput, too short can -# starve other coroutines. -_SOCKET_POLL_DELAY = const(5) # ms -_RESPONSE_TIME = const(30000) # ms. max server latency before timeout - -class FOO: - def __init__(self, server, port): - # On ESP32 need to submit WiFi credentials - self._sta_if = network.WLAN(network.STA_IF) - self._sta_if.active(True) - # Note that the following blocks, potentially for seconds, owing to DNS lookup - self._addr = socket.getaddrinfo(server, port)[0][-1] - self._sock = socket.socket() - self._sock.setblocking(False) - try: - self._sock.connect(addr) - except OSError as e: - if e.args[0] not in BUSY_ERRORS: - raise - if ESP32: # Revolting kludge :-( - loop = asyncio.get_event_loop() - loop.create_task(self._idle_task()) - - def _timeout(self, t): - return ticks_diff(ticks_ms(), t) > _RESPONSE_TIME - - # Read and return n bytes. Raise OSError on timeout ( caught by superclass). - async def _as_read(self, n): - sock = self._sock - data = b'' - t = ticks_ms() - while len(data) < n: - esp32_pause() # Necessary on ESP32 or we can time out. - if self._timeout(t) or not self._sta_if.isconnected(): - raise OSError(-1) - try: - msg = sock.read(n - len(data)) - except OSError as e: # ESP32 issues weird 119 errors here - msg = None - if e.args[0] not in BUSY_ERRORS: - raise - if msg == b'': # Connection closed by host (?) - raise OSError(-1) - if msg is not None: # data received - data = b''.join((data, msg)) - t = ticks_ms() # reset timeout - await asyncio.sleep_ms(_SOCKET_POLL_DELAY) - return data - - # Write a buffer - async def _as_write(self, bytes_wr): - sock = self._sock - t = ticks_ms() - while bytes_wr: - if self._timeout(t) or not self._sta_if.isconnected(): - raise OSError(-1) - try: - n = sock.write(bytes_wr) - except OSError as e: # ESP32 issues weird 119 errors here - n = 0 - if e.args[0] not in BUSY_ERRORS: - raise - if n: # Bytes still to write - t = ticks_ms() # Something was written: reset t/o - bytes_wr = bytes_wr[n:] - esp32_pause() # Precaution. How to prove whether it's necessary? - await asyncio.sleep_ms(_SOCKET_POLL_DELAY) - - # ESP32 kludge :-( - async def _idle_task(self): - while True: - await asyncio.sleep_ms(10) - machine.idle() # Yield to underlying RTOS diff --git a/v3/README.md b/v3/README.md new file mode 100644 index 0000000..95d7344 --- /dev/null +++ b/v3/README.md @@ -0,0 +1,155 @@ +# 1. Guide to asyncio + +MicroPython's `asyncio` is pre-installed on all platforms except severely +constrained ones such as the 1MB ESP8266. It supports CPython 3.8 syntax and +aims to be a compatible subset of `asyncio`. The current version is 3.0.0. + +## 1.1 Documents + +[asyncio official docs](http://docs.micropython.org/en/latest/library/asyncio.html) + +[Tutorial](./docs/TUTORIAL.md) Intended for users with all levels of experience +of asynchronous programming, including beginners. + +[Drivers](https://github.com/peterhinch/micropython-async/blob/master/v3/docs/DRIVERS.md) +describes device drivers for switches, pushbuttons, ESP32 touch buttons, ADC's +and incremental encoders. + +[Interrupts](https://github.com/peterhinch/micropython-async/blob/master/v3/docs/INTERRUPTS.md) +is a guide to interfacing interrupts to `asyncio`. + +[Event-based programming](./docs/EVENTS.md) is a guide to a way of writing +applications and device drivers which largely does away with callbacks. The doc +assumes some knowledge of `asyncio`. + +[Threading](./docs/THREADING.md) is a guide to the use of multi-threaded and +multi-core programming. Code is offered to enable a `asyncio` application to +deal with blocking functions. + +## 1.2 Debugging tools + +[aiorepl](https://github.com/micropython/micropython-lib/tree/master/micropython/aiorepl) +This official tool enables an application to launch a REPL which is active +while the application is running. From this you can modify and query the +application and run `asyncio` scripts concurrently with the running +application. Author Jim Mussared @jimmo. + +[aioprof](https://gitlab.com/alelec/aioprof/-/tree/main) A profiler for +`asyncio` applications: show the number of calls and the total time used by +each task. Author Andrew Leech @andrewleech. + +[monitor](https://github.com/peterhinch/micropython-monitor) enables a running +`asyncio` application to be monitored using a Pi Pico, ideally with a scope or +logic analyser. Normally requires only one GPIO pin on the target. + +![Image](https://github.com/peterhinch/micropython-monitor/raw/master/images/monitor.jpg) + +## 1.3 Resources in this repo + +### 1.3.1 Test/demo scripts + +Documented in the [tutorial](./docs/TUTORIAL.md). + +### 1.3.2 Synchronisation primitives + +Documented in the [tutorial](./docs/TUTORIAL.md). Comprises: + * Implementations of unsupported CPython primitives including `barrier`, + `queue` and others. + * A software retriggerable monostable timer class `Delay_ms`, similar to a + watchdog. + * Two primitives enabling waiting on groups of `Event` instances. + +### 1.3.3 Threadsafe primitives + +[This doc](https://github.com/peterhinch/micropython-async/blob/master/v3/docs/THREADING.md) +describes issues linking `asyncio` code with code running on other cores or in +other threads. The `threadsafe` directory provides: + + * A threadsafe primitive `Message`. + * `ThreadSafeQueue` + * `ThreadSafeEvent` Extends `ThreadsafeFlag`. + +The doc also provides code to enable `asyncio` to handle blocking functions +using threading. + +### 1.3.4 Asynchronous device drivers + +These are documented +[here](https://github.com/peterhinch/micropython-async/blob/master/v3/docs/DRIVERS.md): + * Classes for interfacing switches, pushbuttons and ESP32 touch buttons. + * Drivers for ADC's + * Drivers for incremental encoders. + +### 1.3.5 A scheduler + +This [lightweight scheduler](./docs/SCHEDULE.md) enables tasks to be scheduled +at future times. These can be assigned in a flexible way: a task might run at +4.10am on Monday and Friday if there's no "r" in the month. + +### 1.3.6 Asynchronous interfaces + +These device drivers are intended as examples of asynchronous code which are +useful in their own right: + + * [GPS driver](./docs/GPS.md) Includes various GPS utilities. + * [HTU21D](./docs/HTU21D.md) Temperature and humidity sensor. + * [I2C](./docs/I2C.md) Use Pyboard I2C slave mode to implement a UART-like + asynchronous stream interface. Uses: communication with ESP8266, or (with + coding) to interface a Pyboard to I2C masters. + * [NEC IR](./docs/NEC_IR.md) A receiver for signals from IR remote controls + using the popular NEC protocol. + * [HD44780](./docs/hd44780.md) Driver for common character based LCD displays + based on the Hitachi HD44780 controller. + +# 2. V3 Overview + +These notes are intended for users familiar with `asyncio` under CPython. + +The MicroPython language is based on CPython 3.4. The `asyncio` library now +supports a subset of the CPython 3.8 `asyncio` library. There are non-standard +extensions to optimise services such as millisecond level timing. Its design +focus is on high performance. Scheduling runs without RAM allocation. + +The `asyncio` library supports the following features: + + * `async def` and `await` syntax. + * Awaitable classes (using `__iter__` rather than `__await__`). + * Asynchronous context managers. + * Asynchronous iterators. + * `asyncio.sleep(seconds)`. + * Timeouts (`asyncio.wait_for`). + * Task cancellation (`Task.cancel`). + * Gather. + +It supports millisecond level timing with the following: + * `asyncio.sleep_ms(time)` + +It includes the following CPython compatible synchronisation primitives: + * `Event`. + * `Lock`. + * `gather`. + +This repo includes code for the CPython primitives which are not yet officially +supported. + +The `Future` class is not supported, nor are the `event_loop` methods +`call_soon`, `call_later`, `call_at`. + +## 2.1 Outstanding issues with V3 + +V3 is still a work in progress. The following is a list of issues which I hope +will be addressed in due course. + +### 2.1.1 Fast I/O scheduling + +There is currently no support for this: I/O is scheduled in round robin fashion +with other tasks. There are situations where this is too slow and the scheduler +should be able to poll I/O whenever it gains control. + +### 2.1.2 Synchronisation primitives + +These CPython primitives are outstanding: + * `Semaphore`. + * `BoundedSemaphore`. + * `Condition`. + * `Queue`. diff --git a/v3/__init__.py b/v3/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/v3/as_demos/__init__.py b/v3/as_demos/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/v3/as_demos/aledflash.py b/v3/as_demos/aledflash.py new file mode 100644 index 0000000..6e8fb12 --- /dev/null +++ b/v3/as_demos/aledflash.py @@ -0,0 +1,39 @@ +# aledflash.py Demo/test program for MicroPython asyncio +# Author: Peter Hinch +# Copyright Peter Hinch 2020 Released under the MIT license +# Flashes the onboard LED's each at a different rate. Stops after ten seconds. +# Run on MicroPython board bare hardware + +import pyb +import asyncio + + +async def toggle(objLED, time_ms): + while True: + await asyncio.sleep_ms(time_ms) + objLED.toggle() + + +# TEST FUNCTION + + +async def main(duration): + print("Flash LED's for {} seconds".format(duration)) + leds = [pyb.LED(x) for x in range(1, 4)] # Initialise three on board LED's + for x, led in enumerate(leds): # Create a task for each LED + t = int((0.2 + x / 2) * 1000) + asyncio.create_task(toggle(leds[x], t)) + await asyncio.sleep(duration) + + +def test(duration=10): + try: + asyncio.run(main(duration)) + except KeyboardInterrupt: + print("Interrupted") + finally: + asyncio.new_event_loop() + print("as_demos.aledflash.test() to run again.") + + +test() diff --git a/v3/as_demos/apoll.py b/v3/as_demos/apoll.py new file mode 100644 index 0000000..abb609d --- /dev/null +++ b/v3/as_demos/apoll.py @@ -0,0 +1,65 @@ +# Demonstration of a device driver using a coroutine to poll a device. +# Runs on Pyboard: displays results from the onboard accelerometer. +# Uses crude filtering to discard noisy data. + +# Author: Peter Hinch +# Copyright Peter Hinch 2017 Released under the MIT license + +import asyncio +import pyb +import utime as time + + +class Accelerometer(object): + threshold_squared = 16 + + def __init__(self, accelhw, timeout): + self.accelhw = accelhw + self.timeout = timeout + self.last_change = time.ticks_ms() + self.coords = [accelhw.x(), accelhw.y(), accelhw.z()] + + def dsquared(self, xyz): # Return the square of the distance between this and a passed + return sum(map(lambda p, q: (p - q) ** 2, self.coords, xyz)) # acceleration vector + + def poll(self): # Device is noisy. Only update if change exceeds a threshold + xyz = [self.accelhw.x(), self.accelhw.y(), self.accelhw.z()] + if self.dsquared(xyz) > Accelerometer.threshold_squared: + self.coords = xyz + self.last_change = time.ticks_ms() + return 0 + return time.ticks_diff(time.ticks_ms(), self.last_change) + + def vector(self): + return self.coords + + def timed_out(self): # Time since last change or last timeout report + if time.ticks_diff(time.ticks_ms(), self.last_change) > self.timeout: + self.last_change = time.ticks_ms() + return True + return False + + +async def accel_coro(timeout=2000): + accelhw = pyb.Accel() # Instantiate accelerometer hardware + await asyncio.sleep_ms(30) # Allow it to settle + accel = Accelerometer(accelhw, timeout) + while True: + result = accel.poll() + if result == 0: # Value has changed + x, y, z = accel.vector() + print("Value x:{:3d} y:{:3d} z:{:3d}".format(x, y, z)) + elif accel.timed_out(): # Report every 2 secs + print("Timeout waiting for accelerometer change") + await asyncio.sleep_ms(100) # Poll every 100ms + + +async def main(delay): + print("Testing accelerometer for {} secs. Move the Pyboard!".format(delay)) + print("Test runs for {}s.".format(delay)) + asyncio.create_task(accel_coro()) + await asyncio.sleep(delay) + print("Test complete!") + + +asyncio.run(main(20)) diff --git a/v3/as_demos/auart.py b/v3/as_demos/auart.py new file mode 100644 index 0000000..1a312b0 --- /dev/null +++ b/v3/as_demos/auart.py @@ -0,0 +1,45 @@ +# Test of uasyncio stream I/O using UART +# Author: Peter Hinch +# Copyright Peter Hinch 2017-2022 Released under the MIT license +# Link X1 and X2 to test. + +# We run with no UART timeout: UART read never blocks. +import asyncio +from machine import UART + +uart = UART(4, 9600, timeout=0) + + +async def sender(): + swriter = asyncio.StreamWriter(uart, {}) + while True: + swriter.write("Hello uart\n") + await swriter.drain() + await asyncio.sleep(2) + + +async def receiver(): + sreader = asyncio.StreamReader(uart) + while True: + res = await sreader.readline() + print("Received", res) + + +async def main(): + asyncio.create_task(sender()) + asyncio.create_task(receiver()) + while True: + await asyncio.sleep(1) + + +def test(): + try: + asyncio.run(main()) + except KeyboardInterrupt: + print("Interrupted") + finally: + asyncio.new_event_loop() + print("as_demos.auart.test() to run again.") + + +test() diff --git a/auart_hd.py b/v3/as_demos/auart_hd.py similarity index 65% rename from auart_hd.py rename to v3/as_demos/auart_hd.py index da2b33e..5a6783f 100644 --- a/auart_hd.py +++ b/v3/as_demos/auart_hd.py @@ -1,6 +1,6 @@ # auart_hd.py # Author: Peter Hinch -# Copyright Peter Hinch 2018 Released under the MIT license +# Copyright Peter Hinch 2018-2020 Released under the MIT license # Demo of running a half-duplex protocol to a device. The device never sends # unsolicited messages. An example is a communications device which responds @@ -9,26 +9,24 @@ # lines of data. The master assumes that the device has sent all its data when # a timeout has elapsed. -# In this test a physical device is emulated by the DEVICE class +# In this test a physical device is emulated by the Device class # To test link X1-X4 and X2-X3 from pyb import UART -import uasyncio as asyncio -import aswitch +import asyncio +from primitives.delay_ms import Delay_ms # Dummy device waits for any incoming line and responds with 4 lines at 1 second # intervals. -class DEVICE(): - def __init__(self, uart_no = 4): +class Device: + def __init__(self, uart_no=4): self.uart = UART(uart_no, 9600) - self.loop = asyncio.get_event_loop() self.swriter = asyncio.StreamWriter(self.uart, {}) self.sreader = asyncio.StreamReader(self.uart) - loop = asyncio.get_event_loop() - loop.create_task(self._run()) + asyncio.create_task(self._run()) async def _run(self): - responses = ['Line 1', 'Line 2', 'Line 3', 'Goodbye'] + responses = ["Line 1", "Line 2", "Line 3", "Goodbye"] while True: res = await self.sreader.readline() for response in responses: @@ -36,23 +34,22 @@ async def _run(self): # Demo the fact that the master tolerates slow response. await asyncio.sleep_ms(300) + # The master's send_command() method sends a command and waits for a number of # lines from the device. The end of the process is signified by a timeout, when # a list of lines is returned. This allows line-by-line processing. # A special test mode demonstrates the behaviour with a non-responding device. If # None is passed, no commend is sent. The master waits for a response which never # arrives and returns an empty list. -class MASTER(): - def __init__(self, uart_no = 2, timeout=4000): +class Master: + def __init__(self, uart_no=2, timeout=4000): self.uart = UART(uart_no, 9600) self.timeout = timeout - self.loop = asyncio.get_event_loop() self.swriter = asyncio.StreamWriter(self.uart, {}) self.sreader = asyncio.StreamReader(self.uart) - self.delay = aswitch.Delay_ms() + self.delay = Delay_ms() self.response = [] - loop = asyncio.get_event_loop() - loop.create_task(self._recv()) + asyncio.create_task(self._recv()) async def _recv(self): while True: @@ -63,44 +60,60 @@ async def _recv(self): async def send_command(self, command): self.response = [] # Discard any pending messages if command is None: - print('Timeout test.') + print("Timeout test.") else: await self.swriter.awrite("{}\r\n".format(command)) - print('Command sent:', command) + print("Command sent:", command) self.delay.trigger(self.timeout) # Re-initialise timer while self.delay.running(): await asyncio.sleep(1) # Wait for 4s after last msg received return self.response -async def test(): - print('This test takes 10s to complete.') - for cmd in ['Run', None]: + +async def main(): + print("This test takes 10s to complete.") + master = Master() + device = Device() + for cmd in ["Run", None]: print() res = await master.send_command(cmd) # can use b''.join(res) if a single string is required. if res: - print('Result is:') + print("Result is:") for line in res: - print(line.decode('UTF8'), end='') + print(line.decode("UTF8"), end="") else: - print('Timed out waiting for result.') - -loop = asyncio.get_event_loop() -master = MASTER() -device = DEVICE() -loop.run_until_complete(test()) - -# Expected output -# >>> import auart_hd -# This test takes 10s to complete. -# -# Command sent: Run -# Result is: -# Line 1 -# Line 2 -# Line 3 -# Goodbye -# -# Timeout test. -# Timed out waiting for result. -# >>> + print("Timed out waiting for result.") + + +def printexp(): + st = """Expected output: +This test takes 10s to complete. + +Command sent: Run +Result is: +Line 1 +Line 2 +Line 3 +Goodbye + +Timeout test. +Timed out waiting for result. +""" + print("\x1b[32m") + print(st) + print("\x1b[39m") + + +def test(): + printexp() + try: + asyncio.run(main()) + except KeyboardInterrupt: + print("Interrupted") + finally: + asyncio.new_event_loop() + print("as_demos.auart_hd.test() to run again.") + + +test() diff --git a/v3/as_demos/gather.py b/v3/as_demos/gather.py new file mode 100644 index 0000000..45205f9 --- /dev/null +++ b/v3/as_demos/gather.py @@ -0,0 +1,108 @@ +# gather.py Demo of Gatherable coroutines. Includes 3 cases: +# 1. A normal coro +# 2. A coro with a timeout +# 3. A cancellable coro + +import asyncio + + +async def barking(n): + print("Start normal coro barking()") + for _ in range(6): + await asyncio.sleep(1) + print("Done barking.") + return 2 * n + + +async def foo(n): + print("Start timeout coro foo()") + try: + while True: + await asyncio.sleep(1) + n += 1 + except asyncio.CancelledError: + print("Trapped foo timeout.") + raise + return n + + +async def bar(n): + print("Start cancellable bar()") + try: + while True: + await asyncio.sleep(1) + n += 1 + except asyncio.CancelledError: # Demo of trapping + print("Trapped bar cancellation.") + raise + return n + + +async def do_cancel(task): + await asyncio.sleep(5) + print("About to cancel bar") + task.cancel() + + +async def main(rex): + bar_task = asyncio.create_task(bar(70)) # Note args here + tasks = [] + tasks.append(barking(21)) + tasks.append(asyncio.wait_for(foo(10), 7)) + asyncio.create_task(do_cancel(bar_task)) + try: + res = await asyncio.gather(*tasks, return_exceptions=rex) + except asyncio.TimeoutError: + print("foo timed out.") + res = "No result" + print("Result: ", res) + + +exp_false = """Test runs for 10s. Expected output: + +Start cancellable bar() +Start normal coro barking() +Start timeout coro foo() +About to cancel bar +Trapped bar cancellation. +Done barking. +Trapped foo timeout. +foo timed out. +Result: No result + +""" +exp_true = """Test runs for 10s. Expected output: + +Start cancellable bar() +Start normal coro barking() +Start timeout coro foo() +About to cancel bar +Trapped bar cancellation. +Done barking. +Trapped foo timeout. +Result: [42, TimeoutError()] + +""" + + +def printexp(st): + print("\x1b[32m") + print(st) + print("\x1b[39m") + + +def test(rex): + st = exp_true if rex else exp_false + printexp(st) + try: + asyncio.run(main(rex)) + except KeyboardInterrupt: + print("Interrupted") + finally: + asyncio.new_event_loop() + print() + print("as_demos.gather.test() to run again.") + print("as_demos.gather.test(True) to see effect of return_exceptions.") + + +test(rex=False) diff --git a/iorw.py b/v3/as_demos/iorw.py similarity index 72% rename from iorw.py rename to v3/as_demos/iorw.py index f3a8502..8d91f5d 100644 --- a/iorw.py +++ b/v3/as_demos/iorw.py @@ -1,14 +1,14 @@ # iorw.py Emulate a device which can read and write one character at a time. -# This requires the modified version of uasyncio (fast_io directory). # Slow hardware is emulated using timers. # MyIO.write() ouputs a single character and sets the hardware not ready. # MyIO.readline() returns a single character and sets the hardware not ready. # Timers asynchronously set the hardware ready. import io, pyb -import uasyncio as asyncio +import asyncio import micropython + micropython.alloc_emergency_exception_buf(100) MP_STREAM_POLL_RD = const(1) @@ -16,20 +16,22 @@ MP_STREAM_POLL = const(3) MP_STREAM_ERROR = const(-1) + def printbuf(this_io): - print(bytes(this_io.wbuf[:this_io.wprint_len]).decode(), end='') + print(bytes(this_io.wbuf[: this_io.wprint_len]).decode(), end="") + class MyIO(io.IOBase): def __init__(self, read=False, write=False): self.ready_rd = False # Read and write not ready - self.rbuf = b'ready\n' # Read buffer + self.rbuf = b"ready\n" # Read buffer self.ridx = 0 - pyb.Timer(4, freq = 5, callback = self.do_input) - self.wch = b'' + pyb.Timer(4, freq=5, callback=self.do_input) + self.wch = b"" self.wbuf = bytearray(100) # Write buffer self.wprint_len = 0 self.widx = 0 - pyb.Timer(5, freq = 10, callback = self.do_output) + pyb.Timer(5, freq=10, callback=self.do_output) # Read callback: emulate asynchronous input from hardware. # Typically would put bytes into a ring buffer and set .ready_rd. @@ -42,12 +44,11 @@ def do_output(self, t): if self.wch: self.wbuf[self.widx] = self.wch self.widx += 1 - if self.wch == ord('\n'): + if self.wch == ord("\n"): self.wprint_len = self.widx # Save for schedule micropython.schedule(printbuf, self) self.widx = 0 - self.wch = b'' - + self.wch = b"" def ioctl(self, req, arg): # see ports/stm32/uart.c ret = MP_STREAM_ERROR @@ -63,9 +64,9 @@ def ioctl(self, req, arg): # see ports/stm32/uart.c # Test of device that produces one character at a time def readline(self): - self.ready_rd = False # Cleared by timer cb do_input + self.ready_rd = False # Set by timer cb do_input ch = self.rbuf[self.ridx] - if ch == ord('\n'): + if ch == ord("\n"): self.ridx = 0 else: self.ridx += 1 @@ -74,15 +75,17 @@ def readline(self): # Emulate unbuffered hardware which writes one character: uasyncio waits # until hardware is ready for the next. Hardware ready is emulated by write # timer callback. - def write(self, buf, off, sz): + def write(self, buf, off=0, sz=0): self.wch = buf[off] # Hardware starts to write a char return 1 # 1 byte written. uasyncio waits on ioctl write ready + async def receiver(myior): sreader = asyncio.StreamReader(myior) while True: res = await sreader.readline() - print('Received', res) + print("Received", res) + async def sender(myiow): swriter = asyncio.StreamWriter(myiow, {}) @@ -90,12 +93,32 @@ async def sender(myiow): count = 0 while True: count += 1 - tosend = 'Wrote Hello MyIO {}\n'.format(count) - await swriter.awrite(tosend.encode('UTF8')) + tosend = "Wrote Hello MyIO {}\n".format(count) + await swriter.awrite(tosend.encode("UTF8")) await asyncio.sleep(2) + +def printexp(): + st = """Received b'ready\\n' +Received b'ready\\n' +Received b'ready\\n' +Received b'ready\\n' +Received b'ready\\n' +Wrote Hello MyIO 1 +Received b'ready\\n' +Received b'ready\\n' +Received b'ready\\n' +Wrote Hello MyIO 2 +Received b'ready\\n' +... +Runs until interrupted (ctrl-c). +""" + print("\x1b[32m") + print(st) + print("\x1b[39m") + + +printexp() myio = MyIO() -loop = asyncio.get_event_loop() -loop.create_task(receiver(myio)) -loop.create_task(sender(myio)) -loop.run_forever() +asyncio.create_task(receiver(myio)) +asyncio.run(sender(myio)) diff --git a/v3/as_demos/monitor/README.md b/v3/as_demos/monitor/README.md new file mode 100644 index 0000000..9595dc7 --- /dev/null +++ b/v3/as_demos/monitor/README.md @@ -0,0 +1,3 @@ +# This repo has moved + +[new location](https://github.com/peterhinch/micropython-monitor) diff --git a/v3/as_demos/rate.py b/v3/as_demos/rate.py new file mode 100644 index 0000000..46cb5b2 --- /dev/null +++ b/v3/as_demos/rate.py @@ -0,0 +1,62 @@ +# rate.py Benchmark for uasyncio. Author Peter Hinch Feb 2018-Apr 2020. +# Benchmark uasyncio round-robin scheduling performance +# This measures the rate at which uasyncio can schedule a minimal coro which +# mereley increments a global. + +# Outcome on a Pyboard 1.1 +# 100 minimal coros are scheduled at an interval of 195μs on uasyncio V3 +# Compares with ~156μs on official uasyncio V2. + +# Results for 100 coros on other platforms at standard clock rate: +# Pyboard D SF2W 124μs +# Pico 481μs +# ESP32 322μs +# ESP8266 1495μs (could not run 500 or 1000 coros) + +# Note that ESP32 benchmarks are notoriously fickle. Above figure was for +# the reference board running MP V1.18. Results may vary with firmware +# depending on the layout of code in RAM/IRAM + +import asyncio + +num_coros = (100, 200, 500, 1000) +iterations = [0, 0, 0, 0] +duration = 2 # Time to run for each number of coros +count = 0 +done = False + + +async def foo(): + global count + while True: + await asyncio.sleep_ms(0) + count += 1 + + +async def test(): + global count, done + old_n = 0 + for n, n_coros in enumerate(num_coros): + print("Testing {} coros for {}secs".format(n_coros, duration)) + count = 0 + for _ in range(n_coros - old_n): + asyncio.create_task(foo()) + old_n = n_coros + await asyncio.sleep(duration) + iterations[n] = count + done = True + + +async def report(): + asyncio.create_task(test()) + while not done: + await asyncio.sleep(1) + for x, n in enumerate(num_coros): + print( + "Coros {:4d} Iterations/sec {:5d} Duration {:3d}us".format( + n, int(iterations[x] / duration), int(duration * 1000000 / iterations[x]) + ) + ) + + +asyncio.run(report()) diff --git a/v3/as_demos/roundrobin.py b/v3/as_demos/roundrobin.py new file mode 100644 index 0000000..79bc60d --- /dev/null +++ b/v3/as_demos/roundrobin.py @@ -0,0 +1,34 @@ +# roundrobin.py Test/demo of round-robin scheduling +# Author: Peter Hinch +# Copyright Peter Hinch 2017-2020 Released under the MIT license + +# Result on Pyboard 1.1 with print('Foo', n) commented out +# executions/second 5575.6 on uasyncio V3 + +# uasyncio V2 produced the following results +# 4249 - with a hack where sleep_ms(0) was replaced with yield +# Using sleep_ms(0) 2750 + +import asyncio + +count = 0 +period = 5 + + +async def foo(n): + global count + while True: + await asyncio.sleep_ms(0) + count += 1 + print("Foo", n) + + +async def main(delay): + for n in range(1, 4): + asyncio.create_task(foo(n)) + print("Testing for {:d} seconds".format(delay)) + await asyncio.sleep(delay) + + +asyncio.run(main(period)) +print("Coro executions per sec =", count / period) diff --git a/v3/as_demos/stream_to.py b/v3/as_demos/stream_to.py new file mode 100644 index 0000000..a0d7267 --- /dev/null +++ b/v3/as_demos/stream_to.py @@ -0,0 +1,74 @@ +# stream_to.py Demo of StreamReader with timeout. +# Hardware: Pico or Pico W with pin GPIO0 linked to GPIO1 +# Copyright Peter Hinch 2024 Released under the MIT license + +import asyncio +from primitives import Delay_ms +from machine import UART + +_uart = UART(0, 115200, tx=0, rx=1, timeout=0) # Adapt for other hardware + +# Class extends StreamReader to enable read with timeout +class StreamReaderTo(asyncio.StreamReader): + def __init__(self, source): + super().__init__(source) + self._delay_ms = Delay_ms() # Allocate once only + + # Task cancels itself if timeout elapses without a byte being received + async def readintotim(self, buf: bytearray, toms: int) -> int: # toms: timeout in ms + mvb = memoryview(buf) + timer = self._delay_ms + timer.callback(asyncio.current_task().cancel) + timer.trigger(toms) # Start cancellation timer + n = 0 + nbytes = len(buf) + try: + while n < nbytes: + n += await super().readinto(mvb[n:]) + timer.trigger(toms) # Retrigger when bytes received + except asyncio.CancelledError: + pass + timer.stop() + return n + + +# Simple demo +EOT = b"QUIT" # End of transmission + + +async def sender(writer): + s = "The quick brown fox jumps over the lazy dog!" + for _ in range(2): + writer.write(s) + writer.drain() + await asyncio.sleep(1) # < reader timeout + writer.write(s) + writer.drain() + await asyncio.sleep(4) # > reader timeout + writer.write(EOT) + writer.drain() + + +async def receiver(reader): + buf = bytearray(16) # Read in blocks of 16 cbytes + print("Receiving. Demo runs for ~15s...") + while not buf.startswith(EOT): + n = await reader.readintotim(buf, 3000) + if n < len(buf): + print("Timeout: ", end="") + print(bytes(buf[:n])) + if n < len(buf): + print("") + print("Demo complete.") + + +async def main(): + reader = StreamReaderTo(_uart) + writer = asyncio.StreamWriter(_uart, {}) + await asyncio.gather(sender(writer), receiver(reader)) + + +try: + asyncio.run(main()) +finally: + _ = asyncio.new_event_loop() diff --git a/v3/as_drivers/as_GPS/__init__.py b/v3/as_drivers/as_GPS/__init__.py new file mode 100644 index 0000000..e7979ed --- /dev/null +++ b/v3/as_drivers/as_GPS/__init__.py @@ -0,0 +1 @@ +from .as_GPS import * diff --git a/gps/as_GPS.py b/v3/as_drivers/as_GPS/as_GPS.py similarity index 85% rename from gps/as_GPS.py rename to v3/as_drivers/as_GPS/as_GPS.py index 1a912d5..f1f553c 100644 --- a/gps/as_GPS.py +++ b/v3/as_drivers/as_GPS/as_GPS.py @@ -11,15 +11,14 @@ # astests.py runs under CPython but not MicroPython because mktime is missing # from Unix build of utime -try: - import uasyncio as asyncio -except ImportError: - import asyncio +# Ported to uasyncio V3 OK. + +import asyncio try: from micropython import const except ImportError: - const = lambda x : x + const = lambda x: x from math import modf @@ -64,10 +63,12 @@ class AS_GPS(object): # https://stackoverflow.com/questions/9847213/how-do-i-get-the-day-of-week-given-a-date-in-python?noredirect=1&lq=1 # Adapted for Python 3 and Pyboard RTC format. @staticmethod - def _week_day(year, month, day, offset = [0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334]): + def _week_day( + year, month, day, offset=[0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334] + ): aux = year - 1700 - (1 if month <= 2 else 0) # day_of_week for 1700/1/1 = 5, Friday - day_of_week = 5 + day_of_week = 5 # partial sum of days betweem current date and 1700/1/1 day_of_week += (aux + (1 if month <= 2 else 0)) * 365 # leap year correction @@ -87,12 +88,14 @@ def _crc_check(res, ascii_crc): return False x = 1 crc_xor = 0 - while res[x] != '*': + while res[x] != "*": crc_xor ^= ord(res[x]) x += 1 return crc_xor == crc - def __init__(self, sreader, local_offset=0, fix_cb=lambda *_ : None, cb_mask=RMC, fix_cb_args=()): + def __init__( + self, sreader, local_offset=0, fix_cb=lambda *_: None, cb_mask=RMC, fix_cb_args=() + ): self._sreader = sreader # If None testing: update is called with simulated data self._fix_cb = fix_cb self.cb_mask = cb_mask @@ -102,6 +105,7 @@ def __init__(self, sreader, local_offset=0, fix_cb=lambda *_ : None, cb_mask=RMC # CPython compatibility. Import utime or time for fix time handling. try: import utime + self._get_time = utime.ticks_ms self._time_diff = utime.ticks_diff self._localtime = utime.localtime @@ -110,19 +114,21 @@ def __init__(self, sreader, local_offset=0, fix_cb=lambda *_ : None, cb_mask=RMC # Otherwise default to time module for non-embedded implementations # Should still support millisecond resolution. import time + self._get_time = time.time self._time_diff = lambda start, end: 1000 * (start - end) self._localtime = time.localtime self._mktime = time.mktime # Key: currently supported NMEA sentences. Value: parse method. - self.supported_sentences = {'RMC': self._gprmc, - 'GGA': self._gpgga, - 'VTG': self._gpvtg, - 'GSA': self._gpgsa, - 'GSV': self._gpgsv, - 'GLL': self._gpgll, - } + self.supported_sentences = { + "RMC": self._gprmc, + "GGA": self._gpgga, + "VTG": self._gpvtg, + "GSA": self._gpgsa, + "GSV": self._gpgsv, + "GLL": self._gpgll, + } ##################### # Object Status Flags @@ -147,8 +153,8 @@ def __init__(self, sreader, local_offset=0, fix_cb=lambda *_ : None, cb_mask=RMC self.msecs = 0 # Position/Motion - self._latitude = [0, 0.0, 'N'] # (°, mins, N/S) - self._longitude = [0, 0.0, 'W'] # (°, mins, E/W) + self._latitude = [0, 0.0, "N"] # (°, mins, N/S) + self._longitude = [0, 0.0, "W"] # (°, mins, E/W) self._speed = 0.0 # Knot self.course = 0.0 # ° clockwise from N self.altitude = 0.0 # Metres @@ -172,35 +178,34 @@ def __init__(self, sreader, local_offset=0, fix_cb=lambda *_ : None, cb_mask=RMC # Received status self._valid = 0 # Bitfield of received sentences if sreader is not None: # Running with UART data - loop = asyncio.get_event_loop() - loop.create_task(self._run(loop)) + asyncio.create_task(self._run()) ########################################## # Data Stream Handler Functions ########################################## - async def _run(self, loop): + async def _run(self): while True: res = await self._sreader.readline() try: - res = res.decode('utf8') + res = res.decode("utf8") except UnicodeError: # Garbage: can happen e.g. on baudrate change continue - loop.create_task(self._update(res)) + asyncio.create_task(self._update(res)) await asyncio.sleep(0) # Ensure task runs and res is copied # Update takes a line of text async def _update(self, line): line = line.rstrip() # Copy line # Basic integrity check: may have received partial line e.g on power up - if not line.startswith('$') or not '*' in line or len(line) > self._SENTENCE_LIMIT: + if not line.startswith("$") or not "*" in line or len(line) > self._SENTENCE_LIMIT: return # 2.4ms on Pyboard: if self.FULL_CHECK and not all(10 <= ord(c) <= 126 for c in line): return # Bad character received - a = line.split(',') - segs = a[:-1] + a[-1].split('*') + a = line.split(",") + segs = a[:-1] + a[-1].split("*") await asyncio.sleep(0) if self.FULL_CHECK: # 6ms on Pyboard @@ -214,7 +219,7 @@ async def _update(self, line): segs = segs[:-1] # and checksum seg0 = segs[0] # e.g. GPGLL segx = seg0[2:] # e.g. GLL - if seg0.startswith('G') and segx in self.supported_sentences: + if seg0.startswith("G") and segx in self.supported_sentences: try: s_type = self.supported_sentences[segx](segs) # Parse except ValueError: @@ -257,7 +262,7 @@ def _fix(self, gps_segments, idx_lat, idx_long): lon_mins = float(l_string[3:]) lon_hemi = gps_segments[idx_long + 1] - if lat_hemi not in 'NS'or lon_hemi not in 'EW': + if lat_hemi not in "NS" or lon_hemi not in "EW": raise ValueError self._latitude[0] = lat_degs # In-place to avoid allocation self._latitude[1] = lat_mins @@ -296,12 +301,12 @@ def _set_date_time(self, utc_string, date_string): # Sentence Parsers ######################################## -# For all parsers: -# Initially the ._valid bit for the sentence type is cleared. -# On error a ValueError is raised: trapped by the caller. -# On successful parsing the ._valid bit is set. -# The ._valid mechanism enables the data_received coro to determine what -# sentence types have been received. + # For all parsers: + # Initially the ._valid bit for the sentence type is cleared. + # On error a ValueError is raised: trapped by the caller. + # On successful parsing the ._valid bit is set. + # The ._valid mechanism enables the data_received coro to determine what + # sentence types have been received. # Chip sends rubbish RMC messages before first PPS pulse, but these have # data valid set to 'V' (void) @@ -309,13 +314,13 @@ def _gprmc(self, gps_segments): # Parse RMC sentence self._valid &= ~RMC # Check Receiver Data Valid Flag ('A' active) if not self.battery: - if gps_segments[2] != 'A': + if gps_segments[2] != "A": raise ValueError # UTC Timestamp and date. Can raise ValueError. self._set_date_time(gps_segments[1], gps_segments[9]) # Check Receiver Data Valid Flag ('A' active) - if gps_segments[2] != 'A': + if gps_segments[2] != "A": raise ValueError # Data from Receiver is Valid/Has Fix. Longitude / Latitude @@ -323,14 +328,16 @@ def _gprmc(self, gps_segments): # Parse RMC sentence self._fix(gps_segments, 3, 5) # Speed spd_knt = float(gps_segments[7]) - # Course - course = float(gps_segments[8]) + # Course: adapt for Ublox ZED-F9P + course = float(gps_segments[8]) if gps_segments[8] else 0.0 # Add Magnetic Variation if firmware supplies it if gps_segments[10]: - mv = float(gps_segments[10]) # Float conversions can throw ValueError, caught by caller. - if gps_segments[11] not in ('EW'): + mv = float( + gps_segments[10] + ) # Float conversions can throw ValueError, caught by caller. + if gps_segments[11] not in ("EW"): raise ValueError - self.magvar = mv if gps_segments[11] == 'E' else -mv + self.magvar = mv if gps_segments[11] == "E" else -mv # Update Object Data self._speed = spd_knt self.course = course @@ -340,7 +347,7 @@ def _gprmc(self, gps_segments): # Parse RMC sentence def _gpgll(self, gps_segments): # Parse GLL sentence self._valid &= ~GLL # Check Receiver Data Valid Flag - if gps_segments[6] != 'A': # Invalid. Don't update data + if gps_segments[6] != "A": # Invalid. Don't update data raise ValueError # Data from Receiver is Valid/Has Fix. Longitude / Latitude @@ -428,9 +435,13 @@ def _gpgsv(self, gps_segments): # Calculate Number of Satelites to pull data for and thus how many segment positions to read if num_sv_sentences == current_sv_sentence: - sat_segment_limit = ((sats_in_view % 4) * 4) + 4 # Last sentence may have 1-4 satellites + sat_segment_limit = ( + (sats_in_view % 4) * 4 + ) + 4 # Last sentence may have 1-4 satellites else: - sat_segment_limit = 20 # Non-last sentences have 4 satellites and thus read up to position 20 + sat_segment_limit = ( + 20 # Non-last sentences have 4 satellites and thus read up to position 20 + ) # Try to recover data for up to 4 satellites in sentence for sats in range(4, sat_segment_limit, 4): @@ -443,18 +454,18 @@ def _gpgsv(self, gps_segments): raise ValueError # Abandon try: # elevation can be null (no value) when not tracking - elevation = int(gps_segments[sats+1]) - except (ValueError,IndexError): + elevation = int(gps_segments[sats + 1]) + except (ValueError, IndexError): elevation = None try: # azimuth can be null (no value) when not tracking - azimuth = int(gps_segments[sats+2]) - except (ValueError,IndexError): + azimuth = int(gps_segments[sats + 2]) + except (ValueError, IndexError): azimuth = None try: # SNR can be null (no value) when not tracking - snr = int(gps_segments[sats+3]) - except (ValueError,IndexError): + snr = int(gps_segments[sats + 3]) + except (ValueError, IndexError): snr = None # If no PRN is found, then the sentence has no more satellites to read else: @@ -483,8 +494,7 @@ def _gpgsv(self, gps_segments): ######################################### # Data Validity. On startup data may be invalid. During an outage it will be absent. - async def data_received(self, position=False, course=False, date=False, - altitude=False): + async def data_received(self, position=False, course=False, date=False, altitude=False): self._valid = 0 # Assume no messages at start result = False while not result: @@ -516,7 +526,7 @@ def latitude(self, coord_format=DD): return [self._latitude[0], mins, seconds, self._latitude[2]] elif coord_format == DM: return self._latitude - raise ValueError('Unknown latitude format.') + raise ValueError("Unknown latitude format.") def longitude(self, coord_format=DD): # Format Longitude Data Correctly @@ -529,7 +539,7 @@ def longitude(self, coord_format=DD): return [self._longitude[0], mins, seconds, self._longitude[2]] elif coord_format == DM: return self._longitude - raise ValueError('Unknown longitude format.') + raise ValueError("Unknown longitude format.") def speed(self, units=KNOT): if units == KNOT: @@ -538,7 +548,7 @@ def speed(self, units=KNOT): return self._speed * 1.852 if units == MPH: return self._speed * 1.151 - raise ValueError('Unknown speed units.') + raise ValueError("Unknown speed units.") async def get_satellite_data(self): self._total_sv_sentences = 0 @@ -554,37 +564,38 @@ def time_since_fix(self): # ms since last valid fix return self._time_diff(self._get_time(), self._fix_time) def compass_direction(self): # Return cardinal point as string. - from as_GPS_utils import compass_direction + from .as_GPS_utils import compass_direction + return compass_direction(self) def latitude_string(self, coord_format=DM): if coord_format == DD: - return '{:3.6f}° {:s}'.format(*self.latitude(DD)) + return "{:3.6f}° {:s}".format(*self.latitude(DD)) if coord_format == DMS: return """{:3d}° {:2d}' {:2d}" {:s}""".format(*self.latitude(DMS)) if coord_format == KML: form_lat = self.latitude(DD) - return '{:4.6f}'.format(form_lat[0] if form_lat[1] == 'N' else -form_lat[0]) + return "{:4.6f}".format(form_lat[0] if form_lat[1] == "N" else -form_lat[0]) return "{:3d}° {:3.4f}' {:s}".format(*self.latitude(coord_format)) def longitude_string(self, coord_format=DM): if coord_format == DD: - return '{:3.6f}° {:s}'.format(*self.longitude(DD)) + return "{:3.6f}° {:s}".format(*self.longitude(DD)) if coord_format == DMS: return """{:3d}° {:2d}' {:2d}" {:s}""".format(*self.longitude(DMS)) if coord_format == KML: form_long = self.longitude(DD) - return '{:4.6f}'.format(form_long[0] if form_long[1] == 'E' else -form_long[0]) + return "{:4.6f}".format(form_long[0] if form_long[1] == "E" else -form_long[0]) return "{:3d}° {:3.4f}' {:s}".format(*self.longitude(coord_format)) def speed_string(self, unit=KPH): - sform = '{:3.2f} {:s}' + sform = "{:3.2f} {:s}" speed = self.speed(unit) if unit == MPH: - return sform.format(speed, 'mph') + return sform.format(speed, "mph") elif unit == KNOT: - return sform.format(speed, 'knots') - return sform.format(speed, 'km/h') + return sform.format(speed, "knots") + return sform.format(speed, "km/h") # Return local time (hrs: int, mins: int, secs:float) @property @@ -607,8 +618,9 @@ def utc(self): def time_string(self, local=True): hrs, mins, secs = self.local_time if local else self.utc - return '{:02d}:{:02d}:{:02d}'.format(hrs, mins, secs) + return "{:02d}:{:02d}:{:02d}".format(hrs, mins, secs) def date_string(self, formatting=MDY): - from as_GPS_utils import date_string + from .as_GPS_utils import date_string + return date_string(self, formatting) diff --git a/gps/as_GPS_time.py b/v3/as_drivers/as_GPS/as_GPS_time.py similarity index 57% rename from gps/as_GPS_time.py rename to v3/as_drivers/as_GPS/as_GPS_time.py index 02028d1..5742c33 100644 --- a/gps/as_GPS_time.py +++ b/v3/as_drivers/as_GPS/as_GPS_time.py @@ -2,55 +2,61 @@ # Using GPS for precision timing and for calibrating Pyboard RTC # This is STM-specific: requires pyb module. -# Requires asyn.py from this repo. -# Copyright (c) 2018 Peter Hinch +# Copyright (c) 2018-2020 Peter Hinch # Released under the MIT License (MIT) - see LICENSE file -import uasyncio as asyncio +import asyncio import pyb import utime import math -import asyn -import as_tGPS +from .as_tGPS import GPS_Timer +from threadsafe.message import Message # Hardware assumptions. Change as required. PPS_PIN = pyb.Pin.board.X3 UART_ID = 4 -print('Available tests:') -print('calibrate(minutes=5) Set and calibrate the RTC.') -print('drift(minutes=5) Repeatedly print the difference between RTC and GPS time.') -print('time(minutes=1) Print get_ms() and get_t_split values.') -print('usec(minutes=1) Measure accuracy of usec timer.') -print('Press ctrl-d to reboot after each test.') +print("Available tests:") +print("calibrate(minutes=5) Set and calibrate the RTC.") +print("drift(minutes=5) Repeatedly print the difference between RTC and GPS time.") +print("time(minutes=1) Print get_ms() and get_t_split values.") +print("usec(minutes=1) Measure accuracy of usec timer.") +print("Press ctrl-d to reboot after each test.") -# Setup for tests. Red LED toggles on fix, blue on PPS interrupt. +# Setup for tests. Red LED toggles on fix, green on PPS interrupt. async def setup(): red = pyb.LED(1) - blue = pyb.LED(4) + green = pyb.LED(2) uart = pyb.UART(UART_ID, 9600, read_buf_len=200) sreader = asyncio.StreamReader(uart) pps_pin = pyb.Pin(PPS_PIN, pyb.Pin.IN) - return as_tGPS.GPS_Timer(sreader, pps_pin, local_offset=1, - fix_cb=lambda *_: red.toggle(), - pps_cb=lambda *_: blue.toggle()) + return GPS_Timer( + sreader, + pps_pin, + local_offset=1, + fix_cb=lambda *_: red.toggle(), + pps_cb=lambda *_: green.toggle(), + ) + # Test terminator: task sets the passed event after the passed time. async def killer(end_event, minutes): - print('Will run for {} minutes.'.format(minutes)) + print("Will run for {} minutes.".format(minutes)) await asyncio.sleep(minutes * 60) end_event.set() + # ******** Calibrate and set the Pyboard RTC ******** async def do_cal(minutes): gps = await setup() await gps.calibrate(minutes) gps.close() + def calibrate(minutes=5): - loop = asyncio.get_event_loop() - loop.run_until_complete(do_cal(minutes)) + asyncio.run(do_cal(minutes)) + # ******** Drift test ******** # Every 10s print the difference between GPS time and RTC time @@ -58,44 +64,44 @@ async def drift_test(terminate, gps): dstart = await gps.delta() while not terminate.is_set(): dt = await gps.delta() - print('{} Delta {}μs'.format(gps.time_string(), dt)) + print("{} Delta {}μs".format(gps.time_string(), dt)) await asyncio.sleep(10) return dt - dstart + async def do_drift(minutes): - print('Setting up GPS.') + print("Setting up GPS.") gps = await setup() - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) - print('Setting RTC.') + terminate = asyncio.Event() + asyncio.create_task(killer(terminate, minutes)) + print("Setting RTC.") await gps.set_rtc() - print('Measuring drift.') + print("Measuring drift.") change = await drift_test(terminate, gps) - ush = int(60 * change/minutes) + ush = int(60 * change / minutes) spa = int(ush * 365 * 24 / 1000000) - print('Rate of change {}μs/hr {}secs/year'.format(ush, spa)) + print("Rate of change {}μs/hr {}secs/year".format(ush, spa)) gps.close() + def drift(minutes=5): - loop = asyncio.get_event_loop() - loop.run_until_complete(do_drift(minutes)) + asyncio.run(do_drift(minutes)) + # ******** Time printing demo ******** # Every 10s print the difference between GPS time and RTC time async def do_time(minutes): - fstr = '{}ms Time: {:02d}:{:02d}:{:02d}:{:06d}' - print('Setting up GPS.') + fstr = "{}ms Time: {:02d}:{:02d}:{:02d}:{:06d}" + print("Setting up GPS.") gps = await setup() - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() - print('Setting RTC.') + print("Setting RTC.") await gps.set_rtc() - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) + terminate = asyncio.Event() + asyncio.create_task(killer(terminate, minutes)) while not terminate.is_set(): await asyncio.sleep(1) # In a precision app, get the time list without allocation: @@ -103,9 +109,10 @@ async def do_time(minutes): print(fstr.format(gps.get_ms(), t[0], t[1], t[2], t[3])) gps.close() + def time(minutes=1): - loop = asyncio.get_event_loop() - loop.run_until_complete(do_time(minutes)) + asyncio.run(do_time(minutes)) + # ******** Measure accracy of μs clock ******** # At 9600 baud see occasional lag of up to 3ms followed by similar lead. @@ -117,46 +124,54 @@ def time(minutes=1): # Callback occurs in interrupt context us_acquired = None + + def us_cb(my_gps, tick, led): global us_acquired # Time of previous PPS edge in ticks_us() if us_acquired is not None: - # Trigger event. Pass time between PPS measured by utime.ticks_us() + # Trigger Message. Pass time between PPS measured by utime.ticks_us() tick.set(utime.ticks_diff(my_gps.acquired, us_acquired)) us_acquired = my_gps.acquired led.toggle() + # Setup initialises with above callback async def us_setup(tick): red = pyb.LED(1) - blue = pyb.LED(4) + yellow = pyb.LED(3) uart = pyb.UART(UART_ID, 9600, read_buf_len=200) sreader = asyncio.StreamReader(uart) pps_pin = pyb.Pin(PPS_PIN, pyb.Pin.IN) - return as_tGPS.GPS_Timer(sreader, pps_pin, local_offset=1, - fix_cb=lambda *_: red.toggle(), - pps_cb=us_cb, pps_cb_args=(tick, blue)) + return GPS_Timer( + sreader, + pps_pin, + local_offset=1, + fix_cb=lambda *_: red.toggle(), + pps_cb=us_cb, + pps_cb_args=(tick, yellow), + ) + async def do_usec(minutes): - tick = asyn.Event() - print('Setting up GPS.') + tick = Message() + print("Setting up GPS.") gps = await us_setup(tick) - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() max_us = 0 min_us = 0 sd = 0 nsamples = 0 count = 0 - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) + terminate = asyncio.Event() + asyncio.create_task(killer(terminate, minutes)) while not terminate.is_set(): - await tick + await tick.wait() usecs = tick.value() tick.clear() err = 1000000 - usecs count += 1 - print('Timing discrepancy is {:4d}μs {}'.format(err, '(skipped)' if count < 3 else '')) + print("Timing discrepancy is {:4d}μs {}".format(err, "(skipped)" if count < 3 else "")) if count < 3: # Discard 1st two samples from statistics continue # as these can be unrepresentative max_us = max(max_us, err) @@ -164,10 +179,14 @@ async def do_usec(minutes): sd += err * err nsamples += 1 # SD: apply Bessel's correction for infinite population - sd = int(math.sqrt(sd/(nsamples - 1))) - print('Timing discrepancy is: {:5d}μs max {:5d}μs min. Standard deviation {:4d}μs'.format(max_us, min_us, sd)) + sd = int(math.sqrt(sd / (nsamples - 1))) + print( + "Timing discrepancy is: {:5d}μs max {:5d}μs min. Standard deviation {:4d}μs".format( + max_us, min_us, sd + ) + ) gps.close() + def usec(minutes=1): - loop = asyncio.get_event_loop() - loop.run_until_complete(do_usec(minutes)) + asyncio.run(do_usec(minutes)) diff --git a/gps/as_GPS_utils.py b/v3/as_drivers/as_GPS/as_GPS_utils.py similarity index 97% rename from gps/as_GPS_utils.py rename to v3/as_drivers/as_GPS/as_GPS_utils.py index 7deb5d6..6993baf 100644 --- a/gps/as_GPS_utils.py +++ b/v3/as_drivers/as_GPS/as_GPS_utils.py @@ -4,7 +4,7 @@ # Copyright (c) 2018 Peter Hinch # Released under the MIT License (MIT) - see LICENSE file -from as_GPS import MDY, DMY, LONG +from .as_GPS import MDY, DMY, LONG _DIRECTIONS = ('N', 'NNE', 'NE', 'ENE', 'E', 'ESE', 'SE', 'SSE', 'S', 'SSW', 'SW', 'WSW', 'W', 'WNW', 'NW', 'NNW') diff --git a/gps/as_rwGPS.py b/v3/as_drivers/as_GPS/as_rwGPS.py similarity index 96% rename from gps/as_rwGPS.py rename to v3/as_drivers/as_GPS/as_rwGPS.py index 2cb5540..d93ec71 100644 --- a/gps/as_rwGPS.py +++ b/v3/as_drivers/as_GPS/as_rwGPS.py @@ -7,7 +7,7 @@ # Copyright (c) 2018 Peter Hinch # Released under the MIT License (MIT) - see LICENSE file -import as_GPS +import as_drivers.as_GPS as as_GPS try: from micropython import const except ImportError: @@ -69,19 +69,19 @@ async def baudrate(self, value=9600): if value not in (4800,9600,14400,19200,38400,57600,115200): raise ValueError('Invalid baudrate {:d}.'.format(value)) - sentence = bytearray('$PMTK251,{:d}*00\r\n'.format(value)) + sentence = bytearray('$PMTK251,{:d}*00\r\n'.format(value).encode()) await self._send(sentence) async def update_interval(self, ms=1000): if ms < 100 or ms > 10000: raise ValueError('Invalid update interval {:d}ms.'.format(ms)) - sentence = bytearray('$PMTK220,{:d}*00\r\n'.format(ms)) + sentence = bytearray('$PMTK220,{:d}*00\r\n'.format(ms).encode()) await self._send(sentence) self._update_ms = ms # Save for timing driver async def enable(self, *, gll=0, rmc=1, vtg=1, gga=1, gsa=1, gsv=5, chan=0): fstr = '$PMTK314,{:d},{:d},{:d},{:d},{:d},{:d},0,0,0,0,0,0,0,0,0,0,0,0,{:d}*00\r\n' - sentence = bytearray(fstr.format(gll, rmc, vtg, gga, gsa, gsv, chan)) + sentence = bytearray(fstr.format(gll, rmc, vtg, gga, gsa, gsv, chan).encode()) await self._send(sentence) async def command(self, cmd): diff --git a/gps/as_rwGPS_time.py b/v3/as_drivers/as_GPS/as_rwGPS_time.py similarity index 60% rename from gps/as_rwGPS_time.py rename to v3/as_drivers/as_GPS/as_rwGPS_time.py index 09c7f13..5cf94cd 100644 --- a/gps/as_rwGPS_time.py +++ b/v3/as_drivers/as_GPS/as_rwGPS_time.py @@ -1,9 +1,8 @@ # as_rwGPS_time.py Test scripts for as_tGPS read-write driver. # Using GPS for precision timing and for calibrating Pyboard RTC # This is STM-specific: requires pyb module. -# Requires asyn.py from this repo. -# Copyright (c) 2018 Peter Hinch +# Copyright (c) 2018-2020 Peter Hinch # Released under the MIT License (MIT) - see LICENSE file # See README.md notes re setting baudrates. In particular 9600 does not work. @@ -13,13 +12,14 @@ # Data has (for 38400): $PMTK251,38400*27 # Sending: $PMTK251,38400*27\r\n' -import uasyncio as asyncio +import asyncio +from uasyncio import Event +from threadsafe.message import Message import pyb import utime import math -import asyn -import as_tGPS -import as_rwGPS +from .as_tGPS import GPS_RWTimer +from .as_rwGPS import FULL_COLD_START # Hardware assumptions. Change as required. PPS_PIN = pyb.Pin.board.X3 @@ -29,16 +29,17 @@ UPDATE_INTERVAL = 100 READ_BUF_LEN = 200 -print('Available tests:') -print('calibrate(minutes=5) Set and calibrate the RTC.') -print('drift(minutes=5) Repeatedly print the difference between RTC and GPS time.') -print('time(minutes=1) Print get_ms() and get_t_split values.') -print('usec(minutes=1) Measure accuracy of usec timer.') -print('Press ctrl-d to reboot after each test.') +print("Available tests:") +print("calibrate(minutes=5) Set and calibrate the RTC.") +print("drift(minutes=5) Repeatedly print the difference between RTC and GPS time.") +print("time(minutes=1) Print get_ms() and get_t_split values.") +print("usec(minutes=1) Measure accuracy of usec timer.") +print("Press ctrl-d to reboot after each test.") # Initially use factory baudrate uart = pyb.UART(UART_ID, 9600, read_buf_len=READ_BUF_LEN) + async def shutdown(): global gps # Normally UART is already at BAUDRATE. But if last session didn't restore @@ -46,18 +47,19 @@ async def shutdown(): # session with ctrl-c. uart.init(BAUDRATE) await asyncio.sleep(0.5) - await gps.command(as_rwGPS.FULL_COLD_START) - print('Factory reset') + await gps.command(FULL_COLD_START) + print("Factory reset") gps.close() # Stop ISR - #print('Restoring default baudrate (9600).') - #await gps.baudrate(9600) - #uart.init(9600) - #gps.close() # Stop ISR - #print('Restoring default 1s update rate.') - #await asyncio.sleep(0.5) - #await gps.update_interval(1000) # 1s update rate - #print('Restoring satellite data.') - #await gps.command(as_rwGPS.DEFAULT_SENTENCES) # Restore satellite data + # print('Restoring default baudrate (9600).') + # await gps.baudrate(9600) + # uart.init(9600) + # gps.close() # Stop ISR + # print('Restoring default 1s update rate.') + # await asyncio.sleep(0.5) + # await gps.update_interval(1000) # 1s update rate + # print('Restoring satellite data.') + # await gps.command(as_rwGPS.DEFAULT_SENTENCES) # Restore satellite data + # Setup for tests. Red LED toggles on fix, blue on PPS interrupt. async def setup(): @@ -67,9 +69,14 @@ async def setup(): sreader = asyncio.StreamReader(uart) swriter = asyncio.StreamWriter(uart, {}) pps_pin = pyb.Pin(PPS_PIN, pyb.Pin.IN) - gps = as_tGPS.GPS_RWTimer(sreader, swriter, pps_pin, local_offset=1, - fix_cb=lambda *_: red.toggle(), - pps_cb=lambda *_: blue.toggle()) + gps = GPS_RWTimer( + sreader, + swriter, + pps_pin, + local_offset=1, + fix_cb=lambda *_: red.toggle(), + pps_cb=lambda *_: blue.toggle(), + ) gps.FULL_CHECK = False await asyncio.sleep(2) await gps.baudrate(BAUDRATE) @@ -77,27 +84,30 @@ async def setup(): await asyncio.sleep(1) await gps.enable(gsa=0, gsv=0) # Disable satellite data await gps.update_interval(UPDATE_INTERVAL) - pstr = 'Baudrate {} update interval {}ms satellite messages disabled.' + pstr = "Baudrate {} update interval {}ms satellite messages disabled." print(pstr.format(BAUDRATE, UPDATE_INTERVAL)) return gps + # Test terminator: task sets the passed event after the passed time. async def killer(end_event, minutes): - print('Will run for {} minutes.'.format(minutes)) + print("Will run for {} minutes.".format(minutes)) await asyncio.sleep(minutes * 60) end_event.set() + # ******** Calibrate and set the Pyboard RTC ******** async def do_cal(minutes): gps = await setup() await gps.calibrate(minutes) + def calibrate(minutes=5): - loop = asyncio.get_event_loop() try: - loop.run_until_complete(do_cal(minutes)) + asyncio.run(do_cal(minutes)) finally: - loop.run_until_complete(shutdown()) + asyncio.run(shutdown()) + # ******** Drift test ******** # Every 10s print the difference between GPS time and RTC time @@ -105,61 +115,62 @@ async def drift_test(terminate, gps): dstart = await gps.delta() while not terminate.is_set(): dt = await gps.delta() - print('{} Delta {}μs'.format(gps.time_string(), dt)) + print("{} Delta {}μs".format(gps.time_string(), dt)) await asyncio.sleep(10) return dt - dstart + async def do_drift(minutes): global gps - print('Setting up GPS.') + print("Setting up GPS.") gps = await setup() - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() - print('Setting RTC.') + print("Setting RTC.") await gps.set_rtc() - print('Measuring drift.') - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) + print("Measuring drift.") + terminate = Event() + asyncio.create_task(killer(terminate, minutes)) change = await drift_test(terminate, gps) - ush = int(60 * change/minutes) + ush = int(60 * change / minutes) spa = int(ush * 365 * 24 / 1000000) - print('Rate of change {}μs/hr {}secs/year'.format(ush, spa)) + print("Rate of change {}μs/hr {}secs/year".format(ush, spa)) + def drift(minutes=5): - loop = asyncio.get_event_loop() try: - loop.run_until_complete(do_drift(minutes)) + asyncio.run(do_drift(minutes)) finally: - loop.run_until_complete(shutdown()) + asyncio.run(shutdown()) + # ******** Time printing demo ******** # Every 10s print the difference between GPS time and RTC time async def do_time(minutes): global gps - fstr = '{}ms Time: {:02d}:{:02d}:{:02d}:{:06d}' - print('Setting up GPS.') + fstr = "{}ms Time: {:02d}:{:02d}:{:02d}:{:06d}" + print("Setting up GPS.") gps = await setup() - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() - print('Setting RTC.') + print("Setting RTC.") await gps.set_rtc() - print('RTC is set.') - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) + print("RTC is set.") + terminate = Event() + asyncio.create_task(killer(terminate, minutes)) while not terminate.is_set(): await asyncio.sleep(1) # In a precision app, get the time list without allocation: t = gps.get_t_split() print(fstr.format(gps.get_ms(), t[0], t[1], t[2], t[3])) + def time(minutes=1): - loop = asyncio.get_event_loop() try: - loop.run_until_complete(do_time(minutes)) + asyncio.run(do_time(minutes)) finally: - loop.run_until_complete(shutdown()) + asyncio.run(shutdown()) + # ******** Measure accracy of μs clock ******** # Test produces better numbers at 57600 baud (SD 112μs) @@ -168,6 +179,8 @@ def time(minutes=1): # Callback occurs in interrupt context us_acquired = None # Time of previous PPS edge in ticks_us() + + def us_cb(my_gps, tick, led): global us_acquired if us_acquired is not None: @@ -176,6 +189,7 @@ def us_cb(my_gps, tick, led): us_acquired = my_gps.acquired led.toggle() + # Setup initialises with above callback async def us_setup(tick): global uart, gps # For shutdown @@ -184,9 +198,15 @@ async def us_setup(tick): sreader = asyncio.StreamReader(uart) swriter = asyncio.StreamWriter(uart, {}) pps_pin = pyb.Pin(PPS_PIN, pyb.Pin.IN) - gps = as_tGPS.GPS_RWTimer(sreader, swriter, pps_pin, local_offset=1, - fix_cb=lambda *_: red.toggle(), - pps_cb=us_cb, pps_cb_args=(tick, blue)) + gps = GPS_RWTimer( + sreader, + swriter, + pps_pin, + local_offset=1, + fix_cb=lambda *_: red.toggle(), + pps_cb=us_cb, + pps_cb_args=(tick, blue), + ) gps.FULL_CHECK = False await asyncio.sleep(2) await gps.baudrate(BAUDRATE) @@ -194,31 +214,31 @@ async def us_setup(tick): await asyncio.sleep(1) await gps.enable(gsa=0, gsv=0) # Disable satellite data await gps.update_interval(UPDATE_INTERVAL) - pstr = 'Baudrate {} update interval {}ms satellite messages disabled.' + pstr = "Baudrate {} update interval {}ms satellite messages disabled." print(pstr.format(BAUDRATE, UPDATE_INTERVAL)) + async def do_usec(minutes): global gps - tick = asyn.Event() - print('Setting up GPS.') + tick = Message() + print("Setting up GPS.") await us_setup(tick) - print('Waiting for time data.') + print("Waiting for time data.") await gps.ready() max_us = 0 min_us = 0 sd = 0 nsamples = 0 count = 0 - terminate = asyn.Event() - loop = asyncio.get_event_loop() - loop.create_task(killer(terminate, minutes)) + terminate = Event() + asyncio.create_task(killer(terminate, minutes)) while not terminate.is_set(): await tick usecs = tick.value() tick.clear() err = 1000000 - usecs count += 1 - print('Timing discrepancy is {:4d}μs {}'.format(err, '(skipped)' if count < 3 else '')) + print("Timing discrepancy is {:4d}μs {}".format(err, "(skipped)" if count < 3 else "")) if count < 3: # Discard 1st two samples from statistics continue # as these can be unrepresentative max_us = max(max_us, err) @@ -226,12 +246,16 @@ async def do_usec(minutes): sd += err * err nsamples += 1 # SD: apply Bessel's correction for infinite population - sd = int(math.sqrt(sd/(nsamples - 1))) - print('Timing discrepancy is: {:5d}μs max {:5d}μs min. Standard deviation {:4d}μs'.format(max_us, min_us, sd)) + sd = int(math.sqrt(sd / (nsamples - 1))) + print( + "Timing discrepancy is: {:5d}μs max {:5d}μs min. Standard deviation {:4d}μs".format( + max_us, min_us, sd + ) + ) + def usec(minutes=1): - loop = asyncio.get_event_loop() try: - loop.run_until_complete(do_usec(minutes)) + asyncio.run(do_usec(minutes)) finally: - loop.run_until_complete(shutdown()) + asyncio.run(shutdown()) diff --git a/gps/as_tGPS.py b/v3/as_drivers/as_GPS/as_tGPS.py similarity index 77% rename from gps/as_tGPS.py rename to v3/as_drivers/as_GPS/as_tGPS.py index df7c2aa..78bebf3 100644 --- a/gps/as_tGPS.py +++ b/v3/as_drivers/as_GPS/as_tGPS.py @@ -1,15 +1,15 @@ # as_tGPS.py Using GPS for precision timing and for calibrating Pyboard RTC -# This is STM-specific: requires pyb module. -# Hence not as RAM-critical as as_GPS -# Copyright (c) 2018 Peter Hinch +# Copyright (c) 2018-2020 Peter Hinch # Released under the MIT License (MIT) - see LICENSE file # TODO Test machine version. Replace LED with callback. Update tests and doc. -import uasyncio as asyncio +import asyncio import machine + try: import pyb + on_pyboard = True rtc = pyb.RTC() except ImportError: @@ -17,35 +17,57 @@ import utime import micropython import gc -import as_GPS -import as_rwGPS +from .as_GPS import RMC, AS_GPS +from .as_rwGPS import GPS micropython.alloc_emergency_exception_buf(100) # Convenience function. Return RTC seconds since midnight as float def rtc_secs(): if not on_pyboard: - raise OSError('Only available on STM targets.') + raise OSError("Only available on STM targets.") dt = rtc.datetime() - return 3600*dt[4] + 60*dt[5] + dt[6] + (255 - dt[7])/256 + return 3600 * dt[4] + 60 * dt[5] + dt[6] + (255 - dt[7]) / 256 + # Constructor for GPS_Timer class -def gps_ro_t_init(self, sreader, pps_pin, local_offset=0, - fix_cb=lambda *_ : None, cb_mask=as_GPS.RMC, fix_cb_args=(), - pps_cb=lambda *_ : None, pps_cb_args=()): - as_GPS.AS_GPS.__init__(self, sreader, local_offset, fix_cb, cb_mask, fix_cb_args) +def gps_ro_t_init( + self, + sreader, + pps_pin, + local_offset=0, + fix_cb=lambda *_: None, + cb_mask=RMC, + fix_cb_args=(), + pps_cb=lambda *_: None, + pps_cb_args=(), +): + AS_GPS.__init__(self, sreader, local_offset, fix_cb, cb_mask, fix_cb_args) self.setup(pps_pin, pps_cb, pps_cb_args) + # Constructor for GPS_RWTimer class -def gps_rw_t_init(self, sreader, swriter, pps_pin, local_offset=0, - fix_cb=lambda *_ : None, cb_mask=as_GPS.RMC, fix_cb_args=(), - msg_cb=lambda *_ : None, msg_cb_args=(), - pps_cb=lambda *_ : None, pps_cb_args=()): - as_rwGPS.GPS.__init__(self, sreader, swriter, local_offset, fix_cb, cb_mask, fix_cb_args, - msg_cb, msg_cb_args) +def gps_rw_t_init( + self, + sreader, + swriter, + pps_pin, + local_offset=0, + fix_cb=lambda *_: None, + cb_mask=RMC, + fix_cb_args=(), + msg_cb=lambda *_: None, + msg_cb_args=(), + pps_cb=lambda *_: None, + pps_cb_args=(), +): + GPS.__init__( + self, sreader, swriter, local_offset, fix_cb, cb_mask, fix_cb_args, msg_cb, msg_cb_args + ) self.setup(pps_pin, pps_cb, pps_cb_args) -class GPS_Tbase(): + +class GPS_Tbase: def setup(self, pps_pin, pps_cb, pps_cb_args): self._pps_pin = pps_pin self._pps_cb = pps_cb @@ -54,14 +76,13 @@ def setup(self, pps_pin, pps_cb, pps_cb_args): self.t_ms = 0 # ms since midnight self.acquired = None # Value of ticks_us at edge of PPS self._rtc_set = False # Set RTC flag - self._rtcbuf = [0]*8 # Buffer for RTC setting - self._time = [0]*4 # get_t_split() time buffer. - loop = asyncio.get_event_loop() - loop.create_task(self._start()) + self._rtcbuf = [0] * 8 # Buffer for RTC setting + self._time = [0] * 4 # get_t_split() time buffer. + asyncio.create_task(self._start()) async def _start(self): await self.data_received(date=True) - self._pps_pin.irq(self._isr, trigger = machine.Pin.IRQ_RISING) + self._pps_pin.irq(self._isr, trigger=machine.Pin.IRQ_RISING) def close(self): self._pps_pin.irq(None) @@ -72,7 +93,7 @@ def _isr(self, _): acquired = utime.ticks_us() # Save time of PPS # Time in last NMEA sentence was time of last PPS. # Reduce to integer secs since midnight local time. - isecs = (self.epoch_time + int(3600*self.local_offset)) % 86400 + isecs = (self.epoch_time + int(3600 * self.local_offset)) % 86400 # ms since midnight (28 bits). Add in any ms in RMC data msecs = isecs * 1000 + self.msecs # This PPS is presumed to be one update later @@ -111,7 +132,7 @@ def _dtset(self, wday): # Set flag and let ISR set the RTC. Pause until done. async def set_rtc(self): if not on_pyboard: - raise OSError('Only available on STM targets.') + raise OSError("Only available on STM targets.") self._rtc_set = True while self._rtc_set: await asyncio.sleep_ms(250) @@ -128,7 +149,7 @@ def _get_rtc_usecs(self): # PPS leading edge. async def delta(self): if not on_pyboard: - raise OSError('Only available on STM targets.') + raise OSError("Only available on STM targets.") rtc_time, gps_time = await self._await_pps() # μs since Y2K at time of latest PPS return rtc_time - gps_time @@ -144,25 +165,25 @@ async def _await_pps(self): while rtc.datetime()[7] == st: # Wait for RTC to change (4ms max) pass dt = utime.ticks_diff(utime.ticks_us(), self.acquired) - trtc = self._get_rtc_usecs() - dt # Read RTC now and adjust for PPS edge - tgps = 1000000 * (self.epoch_time + 3600*self.local_offset + 1) + trtc = self._get_rtc_usecs() - dt # Read RTC now and adjust for PPS edge + tgps = 1000000 * (self.epoch_time + 3600 * self.local_offset + 1) return trtc, tgps # Non-realtime calculation of calibration factor. times are in μs def _calculate(self, gps_start, gps_end, rtc_start, rtc_end): # Duration (μs) between PPS edges - pps_delta = (gps_end - gps_start) + pps_delta = gps_end - gps_start # Duration (μs) between PPS edges as measured by RTC and corrected - rtc_delta = (rtc_end - rtc_start) + rtc_delta = rtc_end - rtc_start ppm = (1000000 * (rtc_delta - pps_delta)) / pps_delta # parts per million - return int(-ppm/0.954) + return int(-ppm / 0.954) # Measure difference between RTC and GPS rate and return calibration factor # If 3 successive identical results are within 1 digit the outcome is considered # valid and the coro quits. async def _getcal(self, minutes=5): if minutes < 1: - raise ValueError('minutes must be >= 1') + raise ValueError("minutes must be >= 1") results = [0, 0, 0] # Last 3 cal results idx = 0 # Index into above circular buffer nresults = 0 # Count of results @@ -178,13 +199,13 @@ async def _getcal(self, minutes=5): # Get RTC time at instant of PPS rtc_end, gps_end = await self._await_pps() cal = self._calculate(gps_start, gps_end, rtc_start, rtc_end) - print('Mins {:d} cal factor {:d}'.format(n + 1, cal)) + print("Mins {:d} cal factor {:d}".format(n + 1, cal)) results[idx] = cal idx += 1 idx %= len(results) nresults += 1 if nresults >= 4 and (abs(max(results) - min(results)) <= 1): - return round(sum(results)/len(results)) + return round(sum(results) / len(results)) return cal # Pause until time/date message received and 1st PPS interrupt has occurred. @@ -194,16 +215,16 @@ async def ready(self): async def calibrate(self, minutes=5): if not on_pyboard: - raise OSError('Only available on STM targets.') - print('Waiting for GPS startup.') + raise OSError("Only available on STM targets.") + print("Waiting for GPS startup.") await self.ready() - print('Waiting up to {} minutes to acquire calibration factor...'.format(minutes)) + print("Waiting up to {} minutes to acquire calibration factor...".format(minutes)) cal = await self._getcal(minutes) if cal <= 512 and cal >= -511: rtc.calibration(cal) - print('Pyboard RTC is calibrated. Factor is {:d}.'.format(cal)) + print("Pyboard RTC is calibrated. Factor is {:d}.".format(cal)) else: - print('Calibration factor {:d} is out of range.'.format(cal)) + print("Calibration factor {:d} is out of range.".format(cal)) # User interface functions: accurate GPS time. # Return GPS time in ms since midnight (small int on 32 bit h/w). @@ -234,8 +255,9 @@ def get_t_split(self): self._time[0] = hrs self._time[1] = mins self._time[2] = secs + ds - self._time[3] = us + ims*1000 + self._time[3] = us + ims * 1000 return self._time -GPS_Timer = type('GPS_Timer', (GPS_Tbase, as_GPS.AS_GPS), {'__init__': gps_ro_t_init}) -GPS_RWTimer = type('GPS_RWTimer', (GPS_Tbase, as_rwGPS.GPS), {'__init__': gps_rw_t_init}) + +GPS_Timer = type("GPS_Timer", (GPS_Tbase, AS_GPS), {"__init__": gps_ro_t_init}) +GPS_RWTimer = type("GPS_RWTimer", (GPS_Tbase, GPS), {"__init__": gps_rw_t_init}) diff --git a/gps/ast_pb.py b/v3/as_drivers/as_GPS/ast_pb.py similarity index 50% rename from gps/ast_pb.py rename to v3/as_drivers/as_GPS/ast_pb.py index b9498bf..de93fed 100644 --- a/gps/ast_pb.py +++ b/v3/as_drivers/as_GPS/ast_pb.py @@ -6,96 +6,101 @@ # Test asynchronous GPS device driver as_pyGPS import pyb -import uasyncio as asyncio -import aswitch -import as_GPS +import asyncio +from primitives.delay_ms import Delay_ms +from .as_GPS import DD, MPH, LONG, AS_GPS red = pyb.LED(1) green = pyb.LED(2) ntimeouts = 0 + def callback(gps, _, timer): red.toggle() green.on() timer.trigger(10000) + def timeout(): global ntimeouts green.off() ntimeouts += 1 + # Print satellite data every 10s async def sat_test(gps): while True: d = await gps.get_satellite_data() - print('***** SATELLITE DATA *****') + print("***** SATELLITE DATA *****") for i in d: print(i, d[i]) print() await asyncio.sleep(10) + # Print statistics every 30s async def stats(gps): while True: await asyncio.sleep(30) - print('***** STATISTICS *****') - print('Outages:', ntimeouts) - print('Sentences Found:', gps.clean_sentences) - print('Sentences Parsed:', gps.parsed_sentences) - print('CRC_Fails:', gps.crc_fails) + print("***** STATISTICS *****") + print("Outages:", ntimeouts) + print("Sentences Found:", gps.clean_sentences) + print("Sentences Parsed:", gps.parsed_sentences) + print("CRC_Fails:", gps.crc_fails) print() + # Print navigation data every 4s async def navigation(gps): while True: await asyncio.sleep(4) await gps.data_received(position=True) - print('***** NAVIGATION DATA *****') - print('Data is Valid:', gps._valid) - print('Longitude:', gps.longitude(as_GPS.DD)) - print('Latitude', gps.latitude(as_GPS.DD)) + print("***** NAVIGATION DATA *****") + print("Data is Valid:", gps._valid) + print("Longitude:", gps.longitude(DD)) + print("Latitude", gps.latitude(DD)) print() + async def course(gps): while True: await asyncio.sleep(4) await gps.data_received(course=True) - print('***** COURSE DATA *****') - print('Data is Valid:', gps._valid) - print('Speed:', gps.speed_string(as_GPS.MPH)) - print('Course', gps.course) - print('Compass Direction:', gps.compass_direction()) + print("***** COURSE DATA *****") + print("Data is Valid:", gps._valid) + print("Speed:", gps.speed_string(MPH)) + print("Course", gps.course) + print("Compass Direction:", gps.compass_direction()) print() + async def date(gps): while True: await asyncio.sleep(4) await gps.data_received(date=True) - print('***** DATE AND TIME *****') - print('Data is Valid:', gps._valid) - print('UTC time:', gps.utc) - print('Local time:', gps.local_time) - print('Date:', gps.date_string(as_GPS.LONG)) + print("***** DATE AND TIME *****") + print("Data is Valid:", gps._valid) + print("UTC time:", gps.utc) + print("Local time:", gps.local_time) + print("Date:", gps.date_string(LONG)) print() + async def gps_test(): - print('Initialising') + print("Initialising") # Adapt for other MicroPython hardware uart = pyb.UART(4, 9600, read_buf_len=200) # read_buf_len is precautionary: code runs reliably without it.) sreader = asyncio.StreamReader(uart) - timer = aswitch.Delay_ms(timeout) + timer = Delay_ms(timeout) sentence_count = 0 - gps = as_GPS.AS_GPS(sreader, local_offset=1, fix_cb=callback, fix_cb_args=(timer,)) - loop = asyncio.get_event_loop() - print('awaiting first fix') - loop.create_task(sat_test(gps)) - loop.create_task(stats(gps)) - loop.create_task(navigation(gps)) - loop.create_task(course(gps)) - loop.create_task(date(gps)) - - -loop = asyncio.get_event_loop() -loop.create_task(gps_test()) -loop.run_forever() + gps = AS_GPS(sreader, local_offset=1, fix_cb=callback, fix_cb_args=(timer,)) + print("awaiting first fix") + asyncio.create_task(sat_test(gps)) + asyncio.create_task(stats(gps)) + asyncio.create_task(navigation(gps)) + asyncio.create_task(course(gps)) + await date(gps) + + +asyncio.run(gps_test()) diff --git a/v3/as_drivers/as_GPS/ast_pbrw.py b/v3/as_drivers/as_GPS/ast_pbrw.py new file mode 100644 index 0000000..3e177ff --- /dev/null +++ b/v3/as_drivers/as_GPS/ast_pbrw.py @@ -0,0 +1,185 @@ +# ast_pbrw.py +# Basic test/demo of AS_GPS class (asynchronous GPS device driver) +# Runs on a Pyboard with GPS data on pin X2. +# Copyright (c) Peter Hinch 2018-2020 +# Released under the MIT License (MIT) - see LICENSE file +# Test asynchronous GPS device driver as_rwGPS + +# LED's: +# Green indicates data is being received. +# Red toggles on RMC message received. +# Yellow: coroutine has 4s loop delay. +# Yellow toggles on position reading. + +import pyb +import asyncio +from primitives.delay_ms import Delay_ms +from .as_GPS import DD, LONG, MPH +from .as_rwGPS import * + +# Avoid multiple baudrates. Tests use 9600 or 19200 only. +BAUDRATE = 19200 +red, green, yellow = pyb.LED(1), pyb.LED(2), pyb.LED(3) +ntimeouts = 0 + + +def callback(gps, _, timer): + red.toggle() + green.on() + timer.trigger(10000) # Outage is declared after 10s + + +def cb_timeout(): + global ntimeouts + green.off() + ntimeouts += 1 + + +def message_cb(gps, segs): + print("Message received:", segs) + + +# Print satellite data every 10s +async def sat_test(gps): + while True: + d = await gps.get_satellite_data() + print("***** SATELLITE DATA *****") + print("Data is Valid:", hex(gps._valid)) + for i in d: + print(i, d[i]) + print() + await asyncio.sleep(10) + + +# Print statistics every 30s +async def stats(gps): + while True: + await gps.data_received(position=True) # Wait for a valid fix + await asyncio.sleep(30) + print("***** STATISTICS *****") + print("Outages:", ntimeouts) + print("Sentences Found:", gps.clean_sentences) + print("Sentences Parsed:", gps.parsed_sentences) + print("CRC_Fails:", gps.crc_fails) + print("Antenna status:", gps.antenna) + print("Firmware version:", gps.version) + print("Enabled sentences:", gps.enabled) + print() + + +# Print navigation data every 4s +async def navigation(gps): + while True: + await asyncio.sleep(4) + await gps.data_received(position=True) + yellow.toggle() + print("***** NAVIGATION DATA *****") + print("Data is Valid:", hex(gps._valid)) + print("Longitude:", gps.longitude(DD)) + print("Latitude", gps.latitude(DD)) + print() + + +async def course(gps): + while True: + await asyncio.sleep(4) + await gps.data_received(course=True) + print("***** COURSE DATA *****") + print("Data is Valid:", hex(gps._valid)) + print("Speed:", gps.speed_string(MPH)) + print("Course", gps.course) + print("Compass Direction:", gps.compass_direction()) + print() + + +async def date(gps): + while True: + await asyncio.sleep(4) + await gps.data_received(date=True) + print("***** DATE AND TIME *****") + print("Data is Valid:", hex(gps._valid)) + print("UTC Time:", gps.utc) + print("Local time:", gps.local_time) + print("Date:", gps.date_string(LONG)) + print() + + +async def change_status(gps, uart): + await asyncio.sleep(10) + print("***** Changing status. *****") + await gps.baudrate(BAUDRATE) + uart.init(BAUDRATE) + print("***** baudrate 19200 *****") + await asyncio.sleep(5) # Ensure baudrate is sorted + print("***** Query VERSION *****") + await gps.command(VERSION) + await asyncio.sleep(10) + print("***** Query ENABLE *****") + await gps.command(ENABLE) + await asyncio.sleep(10) # Allow time for 1st report + await gps.update_interval(2000) + print("***** Update interval 2s *****") + await asyncio.sleep(10) + await gps.enable(gsv=False, chan=False) + print("***** Disable satellite in view and channel messages *****") + await asyncio.sleep(10) + print("***** Query ENABLE *****") + await gps.command(ENABLE) + + +# See README.md re antenna commands +# await asyncio.sleep(10) +# await gps.command(ANTENNA) +# print('***** Antenna reports requested *****') +# await asyncio.sleep(60) +# await gps.command(NO_ANTENNA) +# print('***** Antenna reports turned off *****') +# await asyncio.sleep(10) + + +async def gps_test(): + global gps, uart # For shutdown + print("Initialising") + # Adapt UART instantiation for other MicroPython hardware + uart = pyb.UART(4, 9600, read_buf_len=200) + # read_buf_len is precautionary: code runs reliably without it. + sreader = asyncio.StreamReader(uart) + swriter = asyncio.StreamWriter(uart, {}) + timer = Delay_ms(cb_timeout) + sentence_count = 0 + gps = GPS( + sreader, swriter, local_offset=1, fix_cb=callback, fix_cb_args=(timer,), msg_cb=message_cb + ) + await asyncio.sleep(2) + await gps.command(DEFAULT_SENTENCES) + print("Set sentence frequencies to default") + # await gps.command(FULL_COLD_START) + # print('Performed FULL_COLD_START') + print("awaiting first fix") + asyncio.create_task(sat_test(gps)) + asyncio.create_task(stats(gps)) + asyncio.create_task(navigation(gps)) + asyncio.create_task(course(gps)) + asyncio.create_task(date(gps)) + await gps.data_received(True, True, True, True) # all messages + await change_status(gps, uart) + + +async def shutdown(): + # Normally UART is already at BAUDRATE. But if last session didn't restore + # factory baudrate we can restore connectivity in the subsequent stuck + # session with ctrl-c. + uart.init(BAUDRATE) + await asyncio.sleep(1) + await gps.command(FULL_COLD_START) + print("Factory reset") + # print('Restoring default baudrate.') + # await gps.baudrate(9600) + + +try: + asyncio.run(gps_test()) +except KeyboardInterrupt: + print("Interrupted") +finally: + asyncio.run(shutdown()) diff --git a/v3/as_drivers/as_GPS/astests.py b/v3/as_drivers/as_GPS/astests.py new file mode 100755 index 0000000..c71d70a --- /dev/null +++ b/v3/as_drivers/as_GPS/astests.py @@ -0,0 +1,193 @@ +#!/usr/bin/env python3.8 +# -*- coding: utf-8 -*- + +# astests.py +# Tests for AS_GPS module (asynchronous GPS device driver) +# Based on tests for MicropyGPS by Michael Calvin McCoy +# https://github.com/inmcm/micropyGPS + +# Copyright (c) 2018 Peter Hinch +# Released under the MIT License (MIT) - see LICENSE file +# Run under CPython 3.5+ or MicroPython + +from .as_GPS import * +import asyncio + + +async def run(): + sentence_count = 0 + + test_RMC = [ + "$GPRMC,081836,A,3751.65,S,14507.36,E,000.0,360.0,130998,011.3,E*62\n", + "$GPRMC,123519,A,4807.038,N,01131.000,E,022.4,084.4,230394,003.1,W*6A\n", + "$GPRMC,225446,A,4916.45,N,12311.12,W,000.5,054.7,191194,020.3,E*68\n", + "$GPRMC,180041.896,A,3749.1851,N,08338.7891,W,001.9,154.9,240911,,,A*7A\n", + "$GPRMC,180049.896,A,3749.1808,N,08338.7869,W,001.8,156.3,240911,,,A*70\n", + "$GPRMC,092751.000,A,5321.6802,N,00630.3371,W,0.06,31.66,280511,,,A*45\n", + ] + + test_VTG = ["$GPVTG,232.9,T,,M,002.3,N,004.3,K,A*01\n"] + test_GGA = ["$GPGGA,180050.896,3749.1802,N,08338.7865,W,1,07,1.1,397.4,M,-32.5,M,,0000*6C\n"] + test_GSA = [ + "$GPGSA,A,3,07,11,28,24,26,08,17,,,,,,2.0,1.1,1.7*37\n", + "$GPGSA,A,3,07,02,26,27,09,04,15,,,,,,1.8,1.0,1.5*33\n", + ] + test_GSV = [ + "$GPGSV,3,1,12,28,72,355,39,01,52,063,33,17,51,272,44,08,46,184,38*74\n", + "$GPGSV,3,2,12,24,42,058,33,11,34,053,33,07,20,171,40,20,15,116,*71\n", + "$GPGSV,3,3,12,04,12,204,34,27,11,324,35,32,11,089,,26,10,264,40*7B\n", + "$GPGSV,3,1,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00*74\n", + "$GPGSV,3,2,11,14,25,170,00,16,57,208,39,18,67,296,40,19,40,246,00*74\n", + "$GPGSV,3,3,11,22,42,067,42,24,14,311,43,27,05,244,00,,,,*4D\n", + "$GPGSV,4,1,14,22,81,349,25,14,64,296,22,18,54,114,21,51,40,212,*7D\n", + "$GPGSV,4,2,14,24,30,047,22,04,22,312,26,31,22,204,,12,19,088,23*72\n", + "$GPGSV,4,3,14,25,17,127,18,21,16,175,,11,09,315,16,19,05,273,*72\n", + "$GPGSV,4,4,14,32,05,303,,15,02,073,*7A\n", + ] + test_GLL = [ + "$GPGLL,3711.0942,N,08671.4472,W,000812.000,A,A*46\n", + "$GPGLL,4916.45,N,12311.12,W,225444,A,*1D\n", + "$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n", + "$GPGLL,0000.0000,N,00000.0000,E,235947.000,V*2D\n", + ] + + my_gps = AS_GPS(None) + sentence = "" + for sentence in test_RMC: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("RMC sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("Longitude:", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Timestamp:", my_gps.utc) + print("Speed:", my_gps.speed()) + print("Date Stamp:", my_gps.date) + print("Course", my_gps.course) + print("Data is Valid:", bool(my_gps._valid & 1)) + print("Compass Direction:", my_gps.compass_direction()) + print("") + + for sentence in test_GLL: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("GLL sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("Longitude:", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Timestamp:", my_gps.utc) + print("Data is Valid:", bool(my_gps._valid & 2)) + print("") + + for sentence in test_VTG: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("VTG sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("Speed:", my_gps.speed()) + print("Course", my_gps.course) + print("Compass Direction:", my_gps.compass_direction()) + print("Data is Valid:", bool(my_gps._valid & 4)) + print("") + + for sentence in test_GGA: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("GGA sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("Longitude", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Timestamp:", my_gps.utc) + print("Altitude:", my_gps.altitude) + print("Height Above Geoid:", my_gps.geoid_height) + print("Horizontal Dilution of Precision:", my_gps.hdop) + print("Satellites in Use by Receiver:", my_gps.satellites_in_use) + print("Data is Valid:", bool(my_gps._valid & 8)) + print("") + + for sentence in test_GSA: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("GSA sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("Satellites Used", my_gps.satellites_used) + print("Horizontal Dilution of Precision:", my_gps.hdop) + print("Vertical Dilution of Precision:", my_gps.vdop) + print("Position Dilution of Precision:", my_gps.pdop) + print("Data is Valid:", bool(my_gps._valid & 16)) + print("") + + for sentence in test_GSV: + my_gps._valid = 0 + sentence_count += 1 + sentence = await my_gps._update(sentence) + if sentence is None: + print("GSV sentence is invalid.") + else: + print("Parsed a", sentence, "Sentence") + print("SV Sentences Parsed", my_gps._last_sv_sentence) + print("SV Sentences in Total", my_gps._total_sv_sentences) + print("# of Satellites in View:", my_gps.satellites_in_view) + print("Data is Valid:", bool(my_gps._valid & 32)) + data_valid = ( + my_gps._total_sv_sentences > 0 + and my_gps._total_sv_sentences == my_gps._last_sv_sentence + ) + print("Is Satellite Data Valid?:", data_valid) + if data_valid: + print("Satellite Data:", my_gps._satellite_data) + print("Satellites Visible:", list(my_gps._satellite_data.keys())) + print("") + + print("Pretty Print Examples:") + print("Latitude (degs):", my_gps.latitude_string(DD)) + print("Longitude (degs):", my_gps.longitude_string(DD)) + print("Latitude (dms):", my_gps.latitude_string(DMS)) + print("Longitude (dms):", my_gps.longitude_string(DMS)) + print("Latitude (kml):", my_gps.latitude_string(KML)) + print("Longitude (kml):", my_gps.longitude_string(KML)) + print("Latitude (degs, mins):", my_gps.latitude_string()) + print("Longitude (degs, mins):", my_gps.longitude_string()) + print( + "Speed:", + my_gps.speed_string(KPH), + "or", + my_gps.speed_string(MPH), + "or", + my_gps.speed_string(KNOT), + ) + print("Date (Long Format):", my_gps.date_string(LONG)) + print("Date (Short D/M/Y Format):", my_gps.date_string(DMY)) + print("Date (Short M/D/Y Format):", my_gps.date_string(MDY)) + print("Time:", my_gps.time_string()) + print() + + print("### Final Results ###") + print("Sentences Attempted:", sentence_count) + print("Sentences Found:", my_gps.clean_sentences) + print("Sentences Parsed:", my_gps.parsed_sentences) + print("Unsupported sentences:", my_gps.unsupported_sentences) + print("CRC_Fails:", my_gps.crc_fails) + + +def run_tests(): + asyncio.run(run()) + + +if __name__ == "__main__": + run_tests() diff --git a/v3/as_drivers/as_GPS/astests_pyb.py b/v3/as_drivers/as_GPS/astests_pyb.py new file mode 100755 index 0000000..171714f --- /dev/null +++ b/v3/as_drivers/as_GPS/astests_pyb.py @@ -0,0 +1,170 @@ +# astests_pyb.py + +# Tests for AS_GPS module. Emulates a GPS unit using a UART loopback. +# Run on a Pyboard with X1 and X2 linked +# Tests for AS_GPS module (asynchronous GPS device driver) +# Based on tests for MicropyGPS by Michael Calvin McCoy +# https://github.com/inmcm/micropyGPS + +# Copyright (c) 2018-2020 Peter Hinch +# Released under the MIT License (MIT) - see LICENSE file + +# Ported to uasyncio V3 OK. + +from .as_GPS import * +from machine import UART +import asyncio + + +def callback(gps, _, arg): + print("Fix callback. Time:", gps.utc, arg) + + +async def run_tests(): + uart = UART(4, 9600, read_buf_len=200) + swriter = asyncio.StreamWriter(uart, {}) + sreader = asyncio.StreamReader(uart) + sentence_count = 0 + + test_RMC = [ + "$GPRMC,180041.896,A,3749.1851,N,08338.7891,W,001.9,154.9,240911,,,A*7A\n", + "$GPRMC,180049.896,A,3749.1808,N,08338.7869,W,001.8,156.3,240911,,,A*70\n", + "$GPRMC,092751.000,A,5321.6802,N,00630.3371,W,0.06,31.66,280511,,,A*45\n", + ] + + test_VTG = ["$GPVTG,232.9,T,,M,002.3,N,004.3,K,A*01\n"] + test_GGA = ["$GPGGA,180050.896,3749.1802,N,08338.7865,W,1,07,1.1,397.4,M,-32.5,M,,0000*6C\n"] + test_GSA = [ + "$GPGSA,A,3,07,11,28,24,26,08,17,,,,,,2.0,1.1,1.7*37\n", + "$GPGSA,A,3,07,02,26,27,09,04,15,,,,,,1.8,1.0,1.5*33\n", + ] + test_GSV = [ + "$GPGSV,3,1,12,28,72,355,39,01,52,063,33,17,51,272,44,08,46,184,38*74\n", + "$GPGSV,3,2,12,24,42,058,33,11,34,053,33,07,20,171,40,20,15,116,*71\n", + "$GPGSV,3,3,12,04,12,204,34,27,11,324,35,32,11,089,,26,10,264,40*7B\n", + "$GPGSV,3,1,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00*74\n", + "$GPGSV,3,2,11,14,25,170,00,16,57,208,39,18,67,296,40,19,40,246,00*74\n", + "$GPGSV,3,3,11,22,42,067,42,24,14,311,43,27,05,244,00,,,,*4D\n", + "$GPGSV,4,1,14,22,81,349,25,14,64,296,22,18,54,114,21,51,40,212,*7D\n", + "$GPGSV,4,2,14,24,30,047,22,04,22,312,26,31,22,204,,12,19,088,23*72\n", + "$GPGSV,4,3,14,25,17,127,18,21,16,175,,11,09,315,16,19,05,273,*72\n", + "$GPGSV,4,4,14,32,05,303,,15,02,073,*7A\n", + ] + test_GLL = [ + "$GPGLL,3711.0942,N,08671.4472,W,000812.000,A,A*46\n", + "$GPGLL,4916.45,N,12311.12,W,225444,A,*1D\n", + "$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n", + "$GPGLL,4250.5589,S,14718.5084,E,092204.999,A*2D\n", + ] + + # '$GPGLL,0000.0000,N,00000.0000,E,235947.000,V*2D\n', # Will ignore this one + + my_gps = AS_GPS(sreader, fix_cb=callback, fix_cb_args=(42,)) + sentence = "" + for sentence in test_RMC: + sentence_count += 1 + await swriter.awrite(sentence) + await my_gps.data_received(date=True) + print("Longitude:", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Time:", my_gps.utc) + print("Speed:", my_gps.speed()) + print("Date Stamp:", my_gps.date) + print("Course", my_gps.course) + print("Data is Valid:", my_gps._valid) + print("Compass Direction:", my_gps.compass_direction()) + print("") + + for sentence in test_GLL: + sentence_count += 1 + await swriter.awrite(sentence) + await my_gps.data_received(position=True) + print("Longitude:", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Time:", my_gps.utc) + print("Data is Valid:", my_gps._valid) + print("") + + for sentence in test_VTG: + print("Test VTG", sentence) + sentence_count += 1 + await swriter.awrite(sentence) + await asyncio.sleep_ms(200) # Can't wait for course because of position check + print("Speed:", my_gps.speed()) + print("Course", my_gps.course) + print("Compass Direction:", my_gps.compass_direction()) + print("") + + for sentence in test_GGA: + sentence_count += 1 + await swriter.awrite(sentence) + await my_gps.data_received(position=True) + print("Longitude", my_gps.longitude()) + print("Latitude", my_gps.latitude()) + print("UTC Time:", my_gps.utc) + # print('Fix Status:', my_gps.fix_stat) + print("Altitude:", my_gps.altitude) + print("Height Above Geoid:", my_gps.geoid_height) + print("Horizontal Dilution of Precision:", my_gps.hdop) + print("Satellites in Use by Receiver:", my_gps.satellites_in_use) + print("") + + for sentence in test_GSA: + sentence_count += 1 + await swriter.awrite(sentence) + await asyncio.sleep_ms(200) + print("Satellites Used", my_gps.satellites_used) + print("Horizontal Dilution of Precision:", my_gps.hdop) + print("Vertical Dilution of Precision:", my_gps.vdop) + print("Position Dilution of Precision:", my_gps.pdop) + print("") + + for sentence in test_GSV: + sentence_count += 1 + await swriter.awrite(sentence) + await asyncio.sleep_ms(200) + print("SV Sentences Parsed", my_gps._last_sv_sentence) + print("SV Sentences in Total", my_gps._total_sv_sentences) + print("# of Satellites in View:", my_gps.satellites_in_view) + data_valid = ( + my_gps._total_sv_sentences > 0 + and my_gps._total_sv_sentences == my_gps._last_sv_sentence + ) + print("Is Satellite Data Valid?:", data_valid) + if data_valid: + print("Satellite Data:", my_gps._satellite_data) + print("Satellites Visible:", list(my_gps._satellite_data.keys())) + print("") + + print("Pretty Print Examples:") + print("Latitude (degs):", my_gps.latitude_string(DD)) + print("Longitude (degs):", my_gps.longitude_string(DD)) + print("Latitude (dms):", my_gps.latitude_string(DMS)) + print("Longitude (dms):", my_gps.longitude_string(DMS)) + print("Latitude (kml):", my_gps.latitude_string(KML)) + print("Longitude (kml):", my_gps.longitude_string(KML)) + print("Latitude (degs, mins):", my_gps.latitude_string()) + print("Longitude (degs, mins):", my_gps.longitude_string()) + print( + "Speed:", + my_gps.speed_string(KPH), + "or", + my_gps.speed_string(MPH), + "or", + my_gps.speed_string(KNOT), + ) + print("Date (Long Format):", my_gps.date_string(LONG)) + print("Date (Short D/M/Y Format):", my_gps.date_string(DMY)) + print("Date (Short M/D/Y Format):", my_gps.date_string(MDY)) + print("Time:", my_gps.time_string()) + print() + + print("### Final Results ###") + print("Sentences Attempted:", sentence_count) + print("Sentences Found:", my_gps.clean_sentences) + print("Sentences Parsed:", my_gps.parsed_sentences) + print("Unsupported sentences:", my_gps.unsupported_sentences) + print("CRC_Fails:", my_gps.crc_fails) + + +asyncio.run(run_tests()) diff --git a/v3/as_drivers/as_GPS/baud.py b/v3/as_drivers/as_GPS/baud.py new file mode 100644 index 0000000..29852c3 --- /dev/null +++ b/v3/as_drivers/as_GPS/baud.py @@ -0,0 +1,60 @@ +# baud.py Test uasyncio at high baudrate +import pyb +import asyncio +import utime +import as_drivers.as_rwGPS as as_rwGPS + +# Outcome +# Sleep Buffer +# 0 None OK, length limit 74 +# 10 None Bad: length 111 also short weird RMC sentences +# 10 1000 OK, length 74, 37 +# 10 200 Bad: 100, 37 overruns +# 10 400 OK, 74,24 Short GSV sentence looked OK +# 4 200 OK, 74,35 Emulate parse time + +# as_GPS.py +# As written update blocks for 23.5ms parse for 3.8ms max +# with CRC check removed update blocks 17.3ms max +# CRC, bad char and line length removed update blocks 8.1ms max + +# At 10Hz update rate I doubt there's enough time to process the data +BAUDRATE = 115200 +red, green, yellow, blue = pyb.LED(1), pyb.LED(2), pyb.LED(3), pyb.LED(4) + + +async def setup(): + print("Initialising") + uart = pyb.UART(4, 9600) + sreader = asyncio.StreamReader(uart) + swriter = asyncio.StreamWriter(uart, {}) + gps = as_rwGPS.GPS(sreader, swriter, local_offset=1) + await asyncio.sleep(2) + await gps.baudrate(BAUDRATE) + uart.init(BAUDRATE) + + +def setbaud(): + asyncio.run(setup()) + print("Baudrate set to 115200.") + + +async def gps_test(): + print("Initialising") + uart = pyb.UART(4, BAUDRATE, read_buf_len=400) + sreader = asyncio.StreamReader(uart) + swriter = asyncio.StreamWriter(uart, {}) + maxlen = 0 + minlen = 100 + while True: + res = await sreader.readline() + l = len(res) + maxlen = max(maxlen, l) + minlen = min(minlen, l) + print(l, maxlen, minlen, res) + red.toggle() + utime.sleep_ms(10) + + +def test(): + asyncio.run(gps_test()) diff --git a/gps/log.kml b/v3/as_drivers/as_GPS/log.kml similarity index 100% rename from gps/log.kml rename to v3/as_drivers/as_GPS/log.kml diff --git a/gps/log_kml.py b/v3/as_drivers/as_GPS/log_kml.py similarity index 69% rename from gps/log_kml.py rename to v3/as_drivers/as_GPS/log_kml.py index 8fed4c6..e16b1b8 100644 --- a/gps/log_kml.py +++ b/v3/as_drivers/as_GPS/log_kml.py @@ -8,11 +8,11 @@ # Logging stops and the file is closed when the user switch is pressed. -import as_GPS -import uasyncio as asyncio +from .as_GPS import KML, AS_GPS +import asyncio import pyb -str_start = ''' +str_start = """