diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 0000000..68677b8 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,57 @@ +# Contributing + +When contributing to this repository, please first discuss the change you wish to make via an issue, email, or any other method with the owners of this repository before making a change. + +*Please note we have a Code of Conduct, please follow it in all your interactions with the project.* + +## Pull Request Process + +1. Update the `README.md` with details of changes to the interface, this includes new environment variables, useful file locations and container parameters. +2. Increase the version numbers in any examples files and the `README.md` to the new version that this Pull Request would represent. The versioning scheme we use is [SemVer](https://semver.org). + +## Code of Conduct + +### Our Pledge + +In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. + +### Our Standards + +Examples of behaviour that contributes to creating a positive environment include: + +* Using welcoming and inclusive language +* Being respectful of differing viewpoints and experiences +* Gracefully accepting constructive criticism +* Focusing on what is best for the community +* Showing empathy towards other community members + +Examples of unacceptable behaviour by participants include: + +* The use of sexualised language or imagery and unwelcome sexual attention or advances +* Trolling, insulting/derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or electronic address, without explicit permission +* Other conduct which could reasonably be considered inappropriate in a professional setting + +### Our Responsibilities + +Project maintainers are responsible for clarifying the standards of acceptable behaviour and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behaviour. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviours that they deem inappropriate, threatening, offensive, or harmful. + +### Scope + +This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. + +### Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behaviour may be reported by contacting the project team at `support[at]pycom.io` and adding the `[CoC]` tag to the subject line. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. + +Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. + +### Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] + +[homepage]: http://contributor-covenant.org +[version]: http://contributor-covenant.org/version/1/4 diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..3dd78c8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,29 @@ +--- +name: Bug report +about: Create a report to help us improve + +--- + +(Hi! πŸ‘‹ Thanks for reporting an issue! Please make sure you click the link above to view the issue guidelines, then fill out the blanks below.) + +## What are the steps to reproduce this issue? +1. +2. +3. + +## What happens? + + +## What were you expecting to happen? + + +## Any logs, error output, etc? +*(If it’s long, please paste to https://gist.github.com and insert the link here)* + + +## Any other comments? + + +## What versions of software are you using? +- **Board type and hardware version:** +- **`os.uname()` output:** diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..3b6e7a1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,22 @@ +--- +name: Feature request +about: Suggest an idea for this project + +--- + +# Feature Request πŸš€ + +## Is your feature request related to a problem? Please describe. +*A clear and concise description of what the problem is. E.g. I have an issue when...* + + +## Describe the solution you'd like +*A clear and concise description of what you want to happen. Add any considered drawbacks.* + + +## Describe alternatives you've considered +*A clear and concise description of any alternative solutions or features you've considered.* + + +## Teachability, Documentation, Adoption, Migration Strategy +*If you can, explain how users will be able to use this and possibly write out a version the docs. Maybe a screenshot or design?* diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..8b6fbab --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,18 @@ +(Hi! πŸ‘‹ Thanks for sending a pull request! Please make sure you click the link above to view the contribution guidelines, then fill out the blanks below.) + +## What does this implement/fix? Explain your changes. + + +## Does this close any currently open issues? + + +## Any relevant logs, error output, etc? +*(If it’s long, please paste to https://gist.github.com and insert the link here)* + + +## Any other comments? + + +## Where has this been tested? +- **Board type and hardware version:** +- **`os.uname()` output:** diff --git a/.gitignore b/.gitignore index beced4e..e943765 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,15 @@ *.conf .DS_Store +pyscan/* +pysense/* +pysense2/* +pytrack/* +pytrack2/* + +pyscan.zip +pysense.zip +pysense2.zip +pytrack.zip +pytrack2.zip + diff --git a/GoogleIOT/.gitignore b/GoogleIOT/.gitignore new file mode 100644 index 0000000..802aba7 --- /dev/null +++ b/GoogleIOT/.gitignore @@ -0,0 +1,3 @@ +db/* +*.pem +flash/config.py diff --git a/GoogleIOT/README.md b/GoogleIOT/README.md new file mode 100644 index 0000000..8cc8d43 --- /dev/null +++ b/GoogleIOT/README.md @@ -0,0 +1,22 @@ +

+ +# Google Cloud Iot Core MQTT connection library + +### requirement + +Pycom Firmware >= 1.20.0.rc11 + +You will need to setup a Google IoT core registry as described here: https://cloud.google.com/iot/docs/quickstart#create_a_device_registry + +During the activation please collect the following informations: 'project_id', + 'cloud_region' and 'registry_id'. + +### usage + +- create a device registry: +https://cloud.google.com/iot/docs/quickstart#create_a_device_registry +- generate a key using the provided tool genkey.sh and add it to the platform +- add the public key to Google IoT Core : +https://cloud.google.com/iot/docs/quickstart#add_a_device_to_the_registry +- copy config.example.py to config.py and edit the variable +- upload the project using pymakr diff --git a/lib/ADS1115/main.py b/GoogleIOT/flash/cert/.gitkeep similarity index 100% rename from lib/ADS1115/main.py rename to GoogleIOT/flash/cert/.gitkeep diff --git a/GoogleIOT/flash/config.example.py b/GoogleIOT/flash/config.example.py new file mode 100644 index 0000000..2578470 --- /dev/null +++ b/GoogleIOT/flash/config.example.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +''' Set here you setup config in this example +''' +CONFIG = { + 'wifi_ssid': "somewifi", + 'wifi_password': 'iforgot', + 'ntp_server': 'time.google.com', + 'project_id': 'pybytes-101', # replace with your Google project_id + 'cloud_region': 'us-central1', # replace + 'registry_id': 'goinvent', # replace with your Google registry_id + 'topic': '/devices/pysense2/events', # replace so match your device + 'device_id': 'pysense2' # +} diff --git a/GoogleIOT/flash/google_iot_core.py b/GoogleIOT/flash/google_iot_core.py new file mode 100644 index 0000000..18330b2 --- /dev/null +++ b/GoogleIOT/flash/google_iot_core.py @@ -0,0 +1,161 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +''' A MQTT wrapper for Google Cloud IoT MQTT bridge + Extended from umqtt.robust by Paul Sokolovsky with wrap of Google credentials + + https://github.com/micropython/micropython-lib/tree/master/umqtt.robust + https://github.com/micropython/micropython-lib/tree/master/umqtt.simple + + Quick API Reference: + connect(...) - Connect to a server. Returns True if this connection uses + persistent session stored on a server (this will be always + False if clean_session=True argument is used (default)). + disconnect() - Disconnect from a server, release resources. + ping() - Ping server (response is processed automatically by wait_msg()). + publish() - Publish a message. + subscribe() - Subscribe to a topic. + set_callback() - Set callback for received subscription messages. + wait_msg() - Wait for a server message. A subscription message will be + delivered to a callback set with set_callback(), any other + messages will be processed internally. + check_msg() - Check if there's pending message from server. If yes, process + the same way as wait_msg(), if not, return immediately. +''' + +import json +from binascii import b2a_base64 +from binascii import a2b_base64 +import ucrypto +import utime +import umqtt + +def _create_unsigned_jwt(project_id, expires=60 * 60 * 24): + header = { + 'alg': "RS256", + 'typ': 'JWT' + } + token = { + 'iat': utime.time(), + 'exp': utime.time() + expires, + 'aud': project_id + } + return b2a_base64(json.dumps(header)) + "." + \ + b2a_base64(json.dumps(token)) + +def _get_google_client_id( + project_id, + cloud_region, + registry_id, + device_id): + return "projects/%s/locations/%s/registries/%s/devices/%s" % ( + project_id, cloud_region, registry_id, device_id) + +def _create_google_jwt(project_id, private_key): + to_sign = _create_unsigned_jwt(project_id) + signed = ucrypto.generate_rsa_signature(to_sign, private_key) + return to_sign + b'.' + b2a_base64(signed) + + +class GoogleMQTTClient(umqtt.MQTTClient): + ''' Instanciate a mqtt client + Args: + var_int (int): An integer. + var_str (str): A string. + project_id (str): your google's project_id + private_key (bytes): private key bytes in pk8s format + cloud_region (str): your google's region + registry_id (str): the name you had given to your registry + device_id: (str): the human friendly device name + ''' + + DELAY = 2 + DEBUG = True + GOOGLE_CA = '/flash/cert/google_roots.pem' + GOOGLE_MQTT = 'mqtt.googleapis.com' + + def __init__( + self, + project_id, + private_key, + cloud_region, + registry_id, + device_id): + self.private_key = private_key + self.project_id = project_id + self.jwt = _create_google_jwt(self.project_id, self.private_key) + google_client_id = _get_google_client_id( + project_id, cloud_region, registry_id, device_id) + google_args = self._get_google_mqtt_args(self.jwt) + super().__init__(google_client_id, self.GOOGLE_MQTT, **google_args) + + def delay(self, i): + utime.sleep(self.DELAY + i) + + def log(self, in_reconnect, err): + if self.DEBUG: + if in_reconnect: + print("mqtt reconnect: %r" % err) + else: + print("mqtt: %r" % err) + + def reconnect(self): + i = 0 + while True: + if not self.is_jwt_valid(): + self.pswd = self.jwt = _create_google_jwt( + self.project_id, self.private_key) + try: + return super().connect(False) + except OSError as exception: + self.log(True, exception) + i += 1 + self.delay(i) + + def publish(self, topic, msg, retain=False, qos=0): + if qos == 2: + raise Exception("qos=2 not supported by mqtt bridge") + + while True: + try: + return super().publish(topic, msg, retain, qos) + except OSError as exception: + self.log(False, exception) + self.reconnect() + + def wait_msg(self): + while True: + try: + return super().wait_msg() + except OSError as exception: + self.log(False, exception) + self.reconnect() + + def _get_google_mqtt_args(self, jwt): + arguments = { + 'user': '', + 'password': jwt, + 'port': 8883, + 'ssl': True, + 'ssl_params': { + 'ca_certs': self.GOOGLE_CA + } + } + return arguments + + def is_jwt_valid(self): + try: + token = json.loads(a2b_base64(self.jwt.decode().split('.')[1])) + except Exception: + return False + return utime.time() - token.get('iat') < 60 * 60 * 24 + + def set_last_will(self, topic, msg, retain=False, qos=0): + raise Exception("set_last_will not supported by mqtt bridge") diff --git a/GoogleIOT/flash/main.py b/GoogleIOT/flash/main.py new file mode 100644 index 0000000..2f95055 --- /dev/null +++ b/GoogleIOT/flash/main.py @@ -0,0 +1,74 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +''' Example Google IoT Core connection +''' +import utime +import machine +import _thread +from network import WLAN +from google_iot_core import GoogleMQTTClient +from config import CONFIG + +# Connect to Wifi +WLAN_I = WLAN(mode=WLAN.STA, max_tx_pwr=78) +print('Connecting to WiFi %s' % CONFIG.get('wifi_ssid')) +WLAN_I.connect(CONFIG.get('wifi_ssid'), (WLAN.WPA2, CONFIG.get('wifi_password')), timeout=60000) +i = 0 +while not WLAN_I.isconnected(): + i = i + 1 + # print(".", end="") + utime.sleep(1) + if i > 60: + print("\nWifi not available") + break + +# Syncing time +RTCI = machine.RTC() +print('Syncing time with %s' % CONFIG.get('ntp_server'), end='') +RTCI.ntp_sync(CONFIG.get('ntp_server')) +while not RTCI.synced(): + print('.', end='') + utime.sleep(1) +print('') + +# read the private key +FILE_HANDLE = open("cert/%s-pk8.key" % CONFIG.get('device_id')) +PRIVATE_KEY = FILE_HANDLE.read() +FILE_HANDLE.close() + +# make a mqtt client, connect and publish an empty message +MQTT_CLIENT = GoogleMQTTClient(CONFIG.get('project_id'), + PRIVATE_KEY, + CONFIG.get('cloud_region'), + CONFIG.get('registry_id'), + CONFIG.get('device_id')) + + +MQTT_CLIENT.connect() +MQTT_CLIENT.publish(CONFIG.get('topic'), b'test') + +# make a demo callback +def _sub_cb(topic, msg): + ''' handle your message received here ... + ''' + print('received:', topic, msg) + +# register callback +MQTT_CLIENT.set_callback(_sub_cb) + +# example subscription +MQTT_CLIENT.subscribe('/devices/%s/config' % CONFIG.get('device_id'), qos=1) +while True: + # Non-blocking wait for message + MQTT_CLIENT.check_msg() + # Then need to sleep to avoid 100% CPU usage (in a real + # app other useful actions would be performed instead) + utime.sleep_ms(100) diff --git a/GoogleIOT/flash/umqtt.py b/GoogleIOT/flash/umqtt.py new file mode 100644 index 0000000..0d75984 --- /dev/null +++ b/GoogleIOT/flash/umqtt.py @@ -0,0 +1,259 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +'''umqtt is a simple MQTT client for MicroPython. + Original code: https://github.com/micropython/micropython-lib/tree/master/umqtt.simple''' +import time +import os +import usocket as socket +import ustruct as struct +from ubinascii import hexlify + + +class MQTTException(Exception): + pass + + +class MQTTClient: + + def __init__(self, client_id, server, port=0, user=None, password=None, keepalive=0, + ssl=False, ssl_params={}): + if port == 0: + port = 8883 if ssl else 1883 + self.client_id = client_id + self.sock = None + self.server = server + self.addr = socket.getaddrinfo(server, port)[0][-1] + self.ssl = ssl + self.ssl_params = ssl_params + self.pid = 0 + self.cb = None + self.user = user + self.pswd = password + self.keepalive = keepalive + self.lastWill_topic = None + self.lastWill_msg = None + self.lastWill_qos = 0 + self.lastWill_retain = False + + def _send_str(self, s): + self.sock.write(struct.pack("!H", len(s))) + self.sock.write(s) + + def _recv_len(self): + n = 0 + sh = 0 + while 1: + b = self.sock.read(1)[0] + n |= (b & 0x7f) << sh + if not b & 0x80: + return n + sh += 7 + + def set_callback(self, f): + self.cb = f + + def set_last_will(self, topic, msg, retain=False, qos=0): + assert 0 <= qos <= 2 + assert topic + self.lastWill_topic = topic + self.lastWill_msg = msg + self.lastWill_qos = qos + self.lastWill_retain = retain + + def connect(self, clean_session=True): + if (self.sock): + # https://pycomiot.atlassian.net/browse/PB-358 + try: + self.sock.send('') # can send == closeable + self.sock.close() + except Exception as e: + self.sock = None # socket is not closable + import gc + gc.collect() + + self.sock = socket.socket() + self.sock.connect(self.addr) + if self.ssl == True: + import ussl + pssl ={} + if self.ssl_params is not None and self.ssl_params.get('ca_certs') is not None: + try: + os.stat(self.ssl_params.get('ca_certs')) + pssl["cert_reqs"] = ussl.CERT_REQUIRED + pssl["ca_certs"] = self.ssl_params.get('ca_certs') + if self.ssl_params.get('keyfile') is not None: + pssl["keyfile"] = self.ssl_params.get('keyfile') + if self.ssl_params.get('certfile') is not None: + pssl["certfile"] = self.ssl_params.get('certfile') + except Exception as e: + print(e) + print("WARNING:TLS certificate validation for MQTT will be disabled",self.ssl_params.get('ca_file'), " is missing") + + else: + print("WARNING: consider enabling TSL certificate validation for MQTT") + # todo check the file if params and print error + # os.stat(self.ssl_params.get('keyfile')) + # os.stat(self.ssl_params.get('certfile')) + self.sock = ussl.wrap_socket(self.sock, **pssl) + else: + print("WARNING: consider enabling TLS for MQTT") + premsg = bytearray(b"\x10\0\0\0\0\0") + msg = bytearray(b"\x04MQTT\x04\x02\0\0") + + size = 10 + 2 + len(self.client_id) + msg[6] = clean_session << 1 + if self.user is not None: + size += 2 + len(self.user) + 2 + len(self.pswd) + msg[6] |= 0xC0 + if self.keepalive: + assert self.keepalive < 65536 + msg[7] |= self.keepalive >> 8 + msg[8] |= self.keepalive & 0x00FF + if self.lastWill_topic: + size += 2 + len(self.lastWill_topic) + 2 + len(self.lastWill_msg) + msg[6] |= 0x4 | (self.lastWill_qos & 0x1) << 3 | (self.lastWill_qos & 0x2) << 3 + msg[6] |= self.lastWill_retain << 5 + + i = 1 + while size > 0x7f: + premsg[i] = (size & 0x7f) | 0x80 + size >>= 7 + i += 1 + premsg[i] = size + + self.sock.write(premsg, i + 2) + self.sock.write(msg) + self._send_str(self.client_id) + + if self.lastWill_topic: + self._send_str(self.lastWill_topic) + self._send_str(self.lastWill_msg) + + if self.user is not None: + self._send_str(self.user) + self._send_str(self.pswd) + + resp = self.sock.read(4) + if len(resp) == 0: + print("[MQTT] ERR: server closed connection.") + return + assert resp[0] == 0x20 and resp[1] == 0x02 + if resp[3] != 0: + raise MQTTException(resp[3]) + print("[MQTT] %s OK!" % self.server) + return resp[2] & 1 + + def disconnect(self): + self.sock.write(b"\xe0\0") + self.sock.close() + + def ping(self): + self.sock.write(b"\xc0\0") + + def publish(self, topic, msg, retain=False, qos=0): + pkt = bytearray(b"\x30\0\0\0") + pkt[0] |= qos << 1 | retain + size = 2 + len(topic) + len(msg) + if qos > 0: + size += 2 + assert size < 2097152 + i = 1 + while size > 0x7f: + pkt[i] = (size & 0x7f) | 0x80 + size >>= 7 + i += 1 + pkt[i] = size + # print('publish', hex(len(pkt)), hexlify(pkt).decode('ascii')) + self.sock.write(pkt, i + 1) + self._send_str(topic) + if qos > 0: + self.pid += 1 + pid = self.pid + struct.pack_into("!H", pkt, 0, pid) + self.sock.write(pkt, 2) + self.sock.write(msg) + + if qos == 1: + while 1: + op = self.wait_msg() + if op == 0x40: + size = self.sock.read(1) + assert size == b"\x02" + rcv_pid = self.sock.read(2) + rcv_pid = rcv_pid[0] << 8 | rcv_pid[1] + if pid == rcv_pid: + return + elif qos == 2: + assert 0 + + def subscribe(self, topic, qos=0): + assert self.cb is not None, "Subscribe callback is not set" + pkt = bytearray(b"\x82\0\0\0") + self.pid += 1 + struct.pack_into("!BH", pkt, 1, 2 + 2 + len(topic) + 1, self.pid) + # print('subscribe', hex(len(pkt)), hexlify(pkt).decode('ascii')) + self.sock.write(pkt) + self._send_str(topic) + self.sock.write(qos.to_bytes(1, "little")) + while 1: + op = self.wait_msg() + if op == 0x90: + resp = self.sock.read(4) + # print('Response subscribe', hexlify(resp).decode('ascii')) + assert resp[1] == pkt[2] and resp[2] == pkt[3] + if resp[3] == 0x80: + raise MQTTException(resp[3]) + return + + # Wait for a single incoming MQTT message and process it. + # Subscribed messages are delivered to a callback previously + # set by .set_callback() method. Other (internal) MQTT + # messages processed internally. + def wait_msg(self): + res = self.sock.read(1) + self.sock.setblocking(True) + if res is None: + return None + if res == b"": + # other tls empty response happens ... + # raise OSError(-1) + return + if res == b"\xd0": # PING RESPONCE + size = self.sock.read(1)[0] + assert size == 0 + return None + op = res[0] + if op & 0xf0 != 0x30: + return op + size = self._recv_len() + topic_len = self.sock.read(2) + topic_len = (topic_len[0] << 8) | topic_len[1] + topic = self.sock.read(topic_len) + size -= topic_len + 2 + if op & 6: + pid = self.sock.read(2) + pid = pid[0] << 8 | pid[1] + size -= 2 + msg = self.sock.read(size) + self.cb(topic, msg) + if op & 6 == 2: + pkt = bytearray(b"\x40\x02\0\0") + struct.pack_into("!H", pkt, 2, pid) + self.sock.write(pkt) + elif op & 6 == 4: + assert 0 + + # Checks whether a pending message from server is available. + # If not, returns immediately with None. Otherwise, does + # the same processing as wait_msg. + def check_msg(self): + self.sock.setblocking(False) + return self.wait_msg() diff --git a/GoogleIOT/genkey.sh b/GoogleIOT/genkey.sh new file mode 100755 index 0000000..082bf79 --- /dev/null +++ b/GoogleIOT/genkey.sh @@ -0,0 +1,27 @@ +# Google Cloud IoT Core +if [[ $# -eq 0 ]] ; then + echo "Usage: $0 device_id" + exit 0 +fi + +DB_DIR='db' +DEVICE_ID=$1 + +if [ ! -f $DB_DIR/$DEVICE_ID-priv.pem ]; then + openssl genrsa -out $DB_DIR/$DEVICE_ID-priv.pem 2048 +fi + +if [ ! -f $DB_DIR/$DEVICE_ID-pub.pem ]; then + openssl rsa -in $DB_DIR/$DEVICE_ID-priv.pem -pubout -out $DB_DIR/$DEVICE_ID-pub.pem +fi + +if [ ! -f flash/cert/$DEVICE_ID-pk8.key ]; then + openssl pkcs8 -topk8 -nocrypt -in $DB_DIR/$DEVICE_ID-priv.pem -out flash/cert/$DEVICE_ID-pk8.key +fi + +if [ ! -f flash/cert/google_roots.pem ]; then + wget "https://pki.google.com/roots.pem" -O flash/cert/google_roots.pem +fi + +echo "Please add this public key to Google Cloud IoT Core Registry" +cat $DB_DIR/$DEVICE_ID-pub.pem diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..17034ef --- /dev/null +++ b/Makefile @@ -0,0 +1,102 @@ +release: pyscan pysense pysense2 pytrack pytrack2 + + +pyscan: + rm -rf pyscan + rm -f pyscan.zip + @echo "Making Pyscan" + mkdir pyscan + mkdir pyscan/lib + #sensors + cp shields/lib/LIS2HH12.py pyscan/lib/ + cp shields/lib/MFRC630.py pyscan/lib/ + cp shields/lib/SI7006A20.py pyscan/lib/ + cp shields/lib/LTR329ALS01.py pyscan/lib/ + #pycoproc + cp shields/lib/pycoproc_1.py pyscan/lib/ + #example + cp shields/pyscan_1.py pyscan/main.py + + zip -r pyscan.zip pyscan + +pysense: + rm -rf pysense + rm -f pysense.zip + @echo "Making Pysense" + mkdir pysense + mkdir pysense/lib + # sensors + cp shields/lib/LIS2HH12.py pysense/lib/ + cp shields/lib/LTR329ALS01.py pysense/lib/ + cp shields/lib/MPL3115A2.py pysense/lib/ + cp shields/lib/SI7006A20.py pysense/lib/ + # pycoproc + cp shields/lib/pycoproc_1.py pysense/lib/ + # example + cp shields/pysense_1.py pysense/main.py + + zip -r pysense.zip pysense + +pysense2: + rm -rf pysense2 + rm -f pysense2.zip + @echo "Making Pysense 2" + mkdir pysense2 + mkdir pysense2/lib + # sensors + cp shields/lib/LIS2HH12.py pysense2/lib/ + cp shields/lib/LTR329ALS01.py pysense2/lib/ + cp shields/lib/MPL3115A2.py pysense2/lib/ + cp shields/lib/SI7006A20.py pysense2/lib/ + # pycoproc + cp shields/lib/pycoproc_2.py pysense2/lib/ + # example + cp shields/pysense_2.py pysense2/main.py + + zip -r pysense2.zip pysense2 + +pytrack: + rm -rf pytrack + rm -f pytrack.zip + @echo "Making Pytrack" + mkdir pytrack + mkdir pytrack/lib + #sensors + cp shields/lib/L76GNSS.py pytrack/lib/ + cp shields/lib/LIS2HH12.py pytrack/lib/ + #pycoproc + cp shields/lib/pycoproc_1.py pytrack/lib/ + #example + cp shields/pytrack_1.py pytrack/main.py + + zip -r pytrack.zip pytrack + +pytrack2: + rm -rf pytrack2 + rm -f pytrack2.zip + @echo "Making Pytrack2" + mkdir pytrack2 + mkdir pytrack2/lib + #sensors + cp shields/lib/L76GNSS.py pytrack2/lib/ + cp shields/lib/LIS2HH12.py pytrack2/lib/ + #pycoproc + cp shields/lib/pycoproc_2.py pytrack2/lib/ + #example + cp shields/pytrack_2.py pytrack2/main.py + + zip -r pytrack2.zip pytrack2 + +clean: + @echo "Cleaning up files" + rm -rf pyscan + rm -rf pysense + rm -rf pysense2 + rm -rf pytrack + rm -rf pytrack2 + + rm -f pyscan.zip + rm -f pysense.zip + rm -f pysense2.zip + rm -f pytrack.zip + rm -f pytrack2.zip \ No newline at end of file diff --git a/README.md b/README.md index 3391f41..ce4e46c 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,16 @@

-# Introduction -This repository contains out of the box examples for Pycom devices, including Pysense and Pytrack. +# Pycom Libraries and Examples -# Table of Contents +## Introduction +This repository contains libraries and out of the box examples for Pycom devices, including the Shields: Pysense, Pytrack, and Pyscan. + +## Table of Contents * [Examples](/examples) * [Libraries](/lib) -* [Pysense](/pysense) -* [Pytrack](/pytrack) - -# Pysense & Pytrack - -To install the required libraries for Pysense & Pytrack, please download this repo as a ZIP file. You will then be able to extract and upload the library files to your Pycom device (via FTP or Pymakr Plugin). Please see https://docs.pycom.io for more information. +* [Shields](/shields) -# Links +## Links * [Pycom](https://pycom.io) * [Forum](https://forum.pycom.io) * [Docs](https://docs.pycom.io) diff --git a/deepsleep/deepsleep.py b/deepsleep/deepsleep.py index 7841ec7..80b4754 100644 --- a/deepsleep/deepsleep.py +++ b/deepsleep/deepsleep.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART from machine import Pin import pycom @@ -29,7 +39,7 @@ class DeepSleep: EXP_RTC_PERIOD = const(7000) def __init__(self): - self.uart = UART(1, baudrate=10000, pins=(COMM_PIN, ), timeout_chars=3) + self.uart = UART(1, baudrate=10000, pins=(COMM_PIN, ), timeout_chars=5) self.clk_cal_factor = 1 self.uart.read() # enable the weak pull-ups control @@ -103,20 +113,20 @@ def calibrate(self): # setbits, but limit the number of received bytes to avoid confusion with pattern self._magic(CTRL_0_ADDR, 0xFF, 1 << 2, 0, 0) - self._pulses = pycom.pulses_get(COMM_PIN, 50) - self.uart.init(baudrate=10000, pins=(COMM_PIN, ), timeout_chars=3) + self.uart.deinit() + self._pulses = pycom.pulses_get(COMM_PIN, 150) + self.uart.init(baudrate=10000, pins=(COMM_PIN, ), timeout_chars=5) + idx = 0 + for i in range(len(self._pulses)): + if self._pulses[i][1] > EXP_RTC_PERIOD: + idx = i + break try: - if len(self._pulses) > 6: - self.clk_cal_factor = (self._pulses[6][1] - self._pulses[4][1]) / EXP_RTC_PERIOD - else: - self.clk_cal_factor = (self._pulses[5][1] - self._pulses[3][1]) / EXP_RTC_PERIOD + self.clk_cal_factor = (self._pulses[idx][1] - self._pulses[(idx - 1)][1]) / EXP_RTC_PERIOD except: - pass + self.clk_cal_factor = 1 if self.clk_cal_factor > 1.25 or self.clk_cal_factor < 0.75: self.clk_cal_factor = 1 - # flush the buffer - self.uart.read() - self.get_wake_status() def enable_auto_poweroff(self): self.setbits(CTRL_0_ADDR, 1 << 1) diff --git a/examples/DS18X20/boot.py b/examples/DS18X20/boot.py index c9ace5d..f767f86 100644 --- a/examples/DS18X20/boot.py +++ b/examples/DS18X20/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/DS18X20/main.py b/examples/DS18X20/main.py index c5e5b60..3caecc7 100644 --- a/examples/DS18X20/main.py +++ b/examples/DS18X20/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time from machine import Pin from onewire import DS18X20 diff --git a/examples/DS18X20/onewire.py b/examples/DS18X20/onewire.py index ddf14df..8bf2331 100644 --- a/examples/DS18X20/onewire.py +++ b/examples/DS18X20/onewire.py @@ -1,4 +1,12 @@ -#!/usr/bin/env python3 +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# """ OneWire library for MicroPython diff --git a/examples/OTA-lorawan/LoraServer.py b/examples/OTA-lorawan/LoraServer.py new file mode 100644 index 0000000..8382ee1 --- /dev/null +++ b/examples/OTA-lorawan/LoraServer.py @@ -0,0 +1,262 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import urllib.request +import binascii +import base64 +import os +import json +import config + +login_payload = { + "password": "string", + "email": "string" +} + +mcGroup_payload = { + "multicastGroup": { + "dr": 0, + "fCnt": 0, + "frequency": 0, + "groupType": "CLASS_C", + "id": "string", + "mcAddr": "string", + "mcAppSKey": "string", + "mcNwkSKey": "string", + "name": "string", + "pingSlotPeriod": 0, + "serviceProfileID": "string" + } +} + +mcQueue_payload = { + "multicastQueueItem": { + "data": "string", + "fCnt": 0, + "fPort": 1, + "multicastGroupID": "string" + } +} + +mcAddDevice_payload = { + "devEUI": "string", + "multicastGroupID": "string" +} + +class LoraServerClient: + + def __init__(self): + self.server = config.LORASERVER_URL + self.port = config.LORASERVER_API_PORT + self.email = config.LORASERVER_EMAIL + self.passwd = config.LORASERVER_PASS + + def login(self): + url = self.server + ':' + str(self.port) + '/api/internal/login' + + login_payload["password"] = self.passwd + login_payload["email"] = self.email + + payload = bytes(json.dumps(login_payload),'utf-8') + + try: + r = urllib.request.Request(url, data= payload, method= 'POST') + r.add_header("Content-Type", "application/json") + r.add_header("Accept", "application/json") + + with urllib.request.urlopen(r) as f: + return json.loads(f.read().decode('utf-8'))['jwt'] + + except Exception as ex: + print("Error getting the jwt: {}".format(ex)) + + return None + + def parse_service_profile_list(self, response, profile_name): + + try: + json_obj = json.loads(response) + for sp_obj in json_obj["result"]: + if sp_obj["name"] == profile_name: + return sp_obj["id"] + except Exception as ex: + print("Error parsing service profile list: {}".format(ex)) + + return None + + def request_service_profile_id(self, profile_name, jwt): + url = self.server + ':' + str(self.port) + '/api/service-profiles?limit=100' + + try: + r = urllib.request.Request(url, method= 'GET') + r.add_header("Accept", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + return self.parse_service_profile_list(f.read().decode('utf-8'), profile_name) + + except Exception as ex: + print("Error getting service profile id: {}".format(ex)) + + return None + + def create_multicast_group(self, dr, freq, group_name, serviceProfileID, jwt): + + group_id = self.generate_random_id() + mcAddr = self.generate_randon_addr() + mcAppSKey = self.generate_random_key() + mcNwkSKey = self.generate_random_key() + + url = self.server + ':' + str(self.port) + '/api/multicast-groups' + + mcGroup_payload["multicastGroup"]["dr"] = dr + mcGroup_payload["multicastGroup"]["frequency"] = freq + mcGroup_payload["multicastGroup"]["id"] = group_id.decode("utf-8") + mcGroup_payload["multicastGroup"]["mcAddr"] = mcAddr.decode("utf-8") + mcGroup_payload["multicastGroup"]["mcAppSKey"] = mcAppSKey.decode("utf-8") + mcGroup_payload["multicastGroup"]["mcNwkSKey"] = mcNwkSKey.decode("utf-8") + mcGroup_payload["multicastGroup"]["name"] = group_name + mcGroup_payload["multicastGroup"]["serviceProfileID"] = serviceProfileID + + payload = bytes(json.dumps(mcGroup_payload),'utf-8') + + + try: + r = urllib.request.Request(url, data= payload, method= 'POST') + r.add_header("Content-Type", "application/json") + r.add_header("Accept", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + resp = f.read().decode('utf-8') + if '"id":' in resp: + multicast_id = json.loads(resp)["id"] + return (multicast_id, mcAddr, mcNwkSKey, mcAppSKey) + else: + return None + except Exception as ex: + print("Error creating multicast data: {}".format(ex)) + + return None + + def delete_multicast_group(self, group_id): + + url = self.server + ':' + str(self.port) + '/api/multicast-groups/' + group_id + + try: + r = urllib.request.Request(url, method = 'DELETE') + + with urllib.request.urlopen(r) as f: + return f.getcode() == 200 + + except Exception as ex: + print("Error deleting multicast group: {}".format(ex)) + + return False + + def add_device_multicast_group(self, devEUI, group_id, jwt): + + url = self.server + ':' + str(self.port) + '/api/multicast-groups/' + group_id + '/devices' + + mcAddDevice_payload["devEUI"] = devEUI + mcAddDevice_payload["multicastGroupID"] = group_id + + payload = json.dumps(mcAddDevice_payload).encode('utf-8') + + try: + r = urllib.request.Request(url, data= payload, method= 'POST') + r.add_header("Content-Type", "application/json") + r.add_header("Accept", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + return f.getcode() == 200 + + except Exception as ex: + print("Error adding device to multicast group: {}".format(ex)) + + return False + + def request_multicast_keys(self, group_id, jwt): + + url = self.server + ':' + str(self.port) + '/api/multicast-groups/' + group_id + + try: + r = urllib.request.Request(url, method= 'GET') + r.add_header("Accept", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + resp = f.read().decode('utf-8') + if "mcNwkSKey" in resp: + json_resp = json.loads(resp)["multicastGroup"] + return (json_resp["mcAddr"], json_resp["mcNwkSKey"], json_resp["mcAppSKey"]) + else: + return None + except Exception as ex: + print("Error getting multicast keys: {}".format(ex)) + + return None + + def generate_randon_addr(self): + return binascii.hexlify(os.urandom(4)) + + def generate_random_key(self): + return binascii.hexlify(os.urandom(16)) + + def generate_random_id(self): + return binascii.hexlify(os.urandom(4)) + b'-' + binascii.hexlify(os.urandom(2)) + b'-' + binascii.hexlify(os.urandom(2)) \ + + b'-' + binascii.hexlify(os.urandom(2)) + b'-' + binascii.hexlify(os.urandom(6)) + + def multicast_queue_length(self, jwt, multicast_group): + url = self.server + ':' + str(self.port) + '/api/multicast-groups/' + multicast_group + '/queue' + + try: + r = urllib.request.Request(url, method= 'GET') + r.add_header("Content-Type", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + resp = f.read().decode('utf-8') + if "multicastQueueItems" in resp: + print(resp) + json_resp = json.loads(resp)["multicastQueueItems"] + print("Len: {}".format(len(json_resp))) + return len(json_resp) + else: + return -1 + + except Exception as ex: + print("Error getting multicast queue length: {}".format(ex)) + + return -1 + + + def send(self, jwt, multicast_group, data): + url = self.server + ':' + str(self.port) + '/api/multicast-groups/' + multicast_group + '/queue' + + mcQueue_payload["multicastQueueItem"]["data"] = base64.b64encode(data).decode("utf-8") + mcQueue_payload["multicastQueueItem"]["multicastGroupID"] = multicast_group + + payload = bytes(json.dumps(mcQueue_payload),'utf-8') + + try: + r = urllib.request.Request(url, data= payload, method= 'POST') + r.add_header("Content-Type", "application/json") + r.add_header("Accept", "application/json") + r.add_header("Grpc-Metadata-Authorization", "Bearer " + jwt) + + with urllib.request.urlopen(r) as f: + return f.getcode() == 200 + + except Exception as ex: + print("Error sending multicast data: {}".format(ex)) + + return False diff --git a/examples/OTA-lorawan/README.md b/examples/OTA-lorawan/README.md new file mode 100644 index 0000000..218d4e5 --- /dev/null +++ b/examples/OTA-lorawan/README.md @@ -0,0 +1,2 @@ +Coming soon + diff --git a/examples/OTA-lorawan/config.py b/examples/OTA-lorawan/config.py new file mode 100644 index 0000000..b4e1239 --- /dev/null +++ b/examples/OTA-lorawan/config.py @@ -0,0 +1,25 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +#LORASERVER configuration +LORASERVER_IP = "127.0.0.1" +LORASERVER_URL = 'http://localhost/' +LORASERVER_MQTT_PORT = 1883 +LORASERVER_API_PORT = 8080 +LORASERVER_EMAIL = 'admin' +LORASERVER_PASS = 'admin' + +LORASERVER_SERVICE_PROFILE = 'ota_sp' +LORASERVER_DOWNLINK_DR = 5 +LORASERVER_DOWNLINK_FREQ = 869525000 +LORASERVER_APP_ID = 1 # Read from Web Interface / Applications + +#update configuration +UPDATE_DELAY = 300 diff --git a/examples/OTA-lorawan/diff_match_patch.py b/examples/OTA-lorawan/diff_match_patch.py new file mode 100644 index 0000000..7404534 --- /dev/null +++ b/examples/OTA-lorawan/diff_match_patch.py @@ -0,0 +1,1915 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +"""Diff Match and Patch +Copyright 2018 The diff-match-patch Authors. +https://github.com/google/diff-match-patch + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +"""Functions for diff, match and patch. + +Computes the difference between two texts to create a patch. +Applies the patch onto another text, allowing for errors. +""" + +__author__ = 'fraser@google.com (Neil Fraser)' + +import re +import sys +import time +import urllib.parse + + +class diff_match_patch: + """Class containing the diff, match and patch methods. + + Also contains the behaviour settings. + """ + + def __init__(self): + """Inits a diff_match_patch object with default settings. + Redefine these in your program to override the defaults. + """ + + # Number of seconds to map a diff before giving up (0 for infinity). + self.Diff_Timeout = 1.0 + # Cost of an empty edit operation in terms of edit characters. + self.Diff_EditCost = 4 + # At what point is no match declared (0.0 = perfection, 1.0 = very loose). + self.Match_Threshold = 0.5 + # How far to search for a match (0 = exact location, 1000+ = broad match). + # A match this many characters away from the expected location will add + # 1.0 to the score (0.0 is a perfect match). + self.Match_Distance = 1000 + # When deleting a large block of text (over ~64 characters), how close do + # the contents have to be to match the expected contents. (0.0 = perfection, + # 1.0 = very loose). Note that Match_Threshold controls how closely the + # end points of a delete need to match. + self.Patch_DeleteThreshold = 0.5 + # Chunk size for context length. + self.Patch_Margin = 4 + + # The number of bits in an int. + # Python has no maximum, thus to disable patch splitting set to 0. + # However to avoid long patches in certain pathological cases, use 32. + # Multiple short patches (using native ints) are much faster than long ones. + self.Match_MaxBits = 32 + + # DIFF FUNCTIONS + + # The data structure representing a diff is an array of tuples: + # [(DIFF_DELETE, "Hello"), (DIFF_INSERT, "Goodbye"), (DIFF_EQUAL, " world.")] + # which means: delete "Hello", add "Goodbye" and keep " world." + DIFF_DELETE = -1 + DIFF_INSERT = 1 + DIFF_EQUAL = 0 + + def diff_main(self, text1, text2, checklines=True, deadline=None): + """Find the differences between two texts. Simplifies the problem by + stripping any common prefix or suffix off the texts before diffing. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Optional speedup flag. If present and false, then don't run + a line-level diff first to identify the changed areas. + Defaults to true, which does a faster, slightly less optimal diff. + deadline: Optional time when the diff should be complete by. Used + internally for recursive calls. Users should set DiffTimeout instead. + + Returns: + Array of changes. + """ + # Set a deadline by which time the diff must be complete. + if deadline == None: + # Unlike in most languages, Python counts time in seconds. + if self.Diff_Timeout <= 0: + deadline = sys.maxsize + else: + deadline = time.time() + self.Diff_Timeout + + # Check for null inputs. + if text1 == None or text2 == None: + raise ValueError("Null inputs. (diff_main)") + + # Check for equality (speedup). + if text1 == text2: + if text1: + return [(self.DIFF_EQUAL, text1)] + return [] + + # Trim off common prefix (speedup). + commonlength = self.diff_commonPrefix(text1, text2) + commonprefix = text1[:commonlength] + text1 = text1[commonlength:] + text2 = text2[commonlength:] + + # Trim off common suffix (speedup). + commonlength = self.diff_commonSuffix(text1, text2) + if commonlength == 0: + commonsuffix = '' + else: + commonsuffix = text1[-commonlength:] + text1 = text1[:-commonlength] + text2 = text2[:-commonlength] + + # Compute the diff on the middle block. + diffs = self.diff_compute(text1, text2, checklines, deadline) + + # Restore the prefix and suffix. + if commonprefix: + diffs[:0] = [(self.DIFF_EQUAL, commonprefix)] + if commonsuffix: + diffs.append((self.DIFF_EQUAL, commonsuffix)) + self.diff_cleanupMerge(diffs) + return diffs + + def diff_compute(self, text1, text2, checklines, deadline): + """Find the differences between two texts. Assumes that the texts do not + have any common prefix or suffix. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Speedup flag. If false, then don't run a line-level diff + first to identify the changed areas. + If true, then run a faster, slightly less optimal diff. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + if not text1: + # Just add some text (speedup). + return [(self.DIFF_INSERT, text2)] + + if not text2: + # Just delete some text (speedup). + return [(self.DIFF_DELETE, text1)] + + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + i = longtext.find(shorttext) + if i != -1: + # Shorter text is inside the longer text (speedup). + diffs = [(self.DIFF_INSERT, longtext[:i]), (self.DIFF_EQUAL, shorttext), + (self.DIFF_INSERT, longtext[i + len(shorttext):])] + # Swap insertions for deletions if diff is reversed. + if len(text1) > len(text2): + diffs[0] = (self.DIFF_DELETE, diffs[0][1]) + diffs[2] = (self.DIFF_DELETE, diffs[2][1]) + return diffs + + if len(shorttext) == 1: + # Single character string. + # After the previous speedup, the character can't be an equality. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + # Check to see if the problem can be split in two. + hm = self.diff_halfMatch(text1, text2) + if hm: + # A half-match was found, sort out the return data. + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + # Send both pairs off for separate processing. + diffs_a = self.diff_main(text1_a, text2_a, checklines, deadline) + diffs_b = self.diff_main(text1_b, text2_b, checklines, deadline) + # Merge the results. + return diffs_a + [(self.DIFF_EQUAL, mid_common)] + diffs_b + + if checklines and len(text1) > 100 and len(text2) > 100: + return self.diff_lineMode(text1, text2, deadline) + + return self.diff_bisect(text1, text2, deadline) + + def diff_lineMode(self, text1, text2, deadline): + """Do a quick line-level diff on both strings, then rediff the parts for + greater accuracy. + This speedup can produce non-minimal diffs. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + + # Scan the text on a line-by-line basis first. + (text1, text2, linearray) = self.diff_linesToChars(text1, text2) + + diffs = self.diff_main(text1, text2, False, deadline) + + # Convert the diff back to original text. + self.diff_charsToLines(diffs, linearray) + # Eliminate freak matches (e.g. blank lines) + self.diff_cleanupSemantic(diffs) + + # Rediff any replacement blocks, this time character-by-character. + # Add a dummy entry at the end. + diffs.append((self.DIFF_EQUAL, '')) + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete >= 1 and count_insert >= 1: + # Delete the offending records and add the merged ones. + subDiff = self.diff_main(text_delete, text_insert, False, deadline) + diffs[pointer - count_delete - count_insert : pointer] = subDiff + pointer = pointer - count_delete - count_insert + len(subDiff) + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + pointer += 1 + + diffs.pop() # Remove the dummy entry at the end. + + return diffs + + def diff_bisect(self, text1, text2, deadline): + """Find the 'middle snake' of a diff, split the problem in two + and return the recursively constructed diff. + See Myers 1986 paper: An O(ND) Difference Algorithm and Its Variations. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + max_d = (text1_length + text2_length + 1) // 2 + v_offset = max_d + v_length = 2 * max_d + v1 = [-1] * v_length + v1[v_offset + 1] = 0 + v2 = v1[:] + delta = text1_length - text2_length + # If the total number of characters is odd, then the front path will + # collide with the reverse path. + front = (delta % 2 != 0) + # Offsets for start and end of k loop. + # Prevents mapping of space beyond the grid. + k1start = 0 + k1end = 0 + k2start = 0 + k2end = 0 + for d in range(max_d): + # Bail out if deadline is reached. + if time.time() > deadline: + break + + # Walk the front path one step. + for k1 in range(-d + k1start, d + 1 - k1end, 2): + k1_offset = v_offset + k1 + if k1 == -d or (k1 != d and + v1[k1_offset - 1] < v1[k1_offset + 1]): + x1 = v1[k1_offset + 1] + else: + x1 = v1[k1_offset - 1] + 1 + y1 = x1 - k1 + while (x1 < text1_length and y1 < text2_length and + text1[x1] == text2[y1]): + x1 += 1 + y1 += 1 + v1[k1_offset] = x1 + if x1 > text1_length: + # Ran off the right of the graph. + k1end += 2 + elif y1 > text2_length: + # Ran off the bottom of the graph. + k1start += 2 + elif front: + k2_offset = v_offset + delta - k1 + if k2_offset >= 0 and k2_offset < v_length and v2[k2_offset] != -1: + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - v2[k2_offset] + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Walk the reverse path one step. + for k2 in range(-d + k2start, d + 1 - k2end, 2): + k2_offset = v_offset + k2 + if k2 == -d or (k2 != d and + v2[k2_offset - 1] < v2[k2_offset + 1]): + x2 = v2[k2_offset + 1] + else: + x2 = v2[k2_offset - 1] + 1 + y2 = x2 - k2 + while (x2 < text1_length and y2 < text2_length and + text1[-x2 - 1] == text2[-y2 - 1]): + x2 += 1 + y2 += 1 + v2[k2_offset] = x2 + if x2 > text1_length: + # Ran off the left of the graph. + k2end += 2 + elif y2 > text2_length: + # Ran off the top of the graph. + k2start += 2 + elif not front: + k1_offset = v_offset + delta - k2 + if k1_offset >= 0 and k1_offset < v_length and v1[k1_offset] != -1: + x1 = v1[k1_offset] + y1 = v_offset + x1 - k1_offset + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - x2 + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Diff took too long and hit the deadline or + # number of diffs equals number of characters, no commonality at all. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + def diff_bisectSplit(self, text1, text2, x, y, deadline): + """Given the location of the 'middle snake', split the diff in two parts + and recurse. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + x: Index of split point in text1. + y: Index of split point in text2. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + text1a = text1[:x] + text2a = text2[:y] + text1b = text1[x:] + text2b = text2[y:] + + # Compute both diffs serially. + diffs = self.diff_main(text1a, text2a, False, deadline) + diffsb = self.diff_main(text1b, text2b, False, deadline) + + return diffs + diffsb + + def diff_linesToChars(self, text1, text2): + """Split two texts into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + + Args: + text1: First string. + text2: Second string. + + Returns: + Three element tuple, containing the encoded text1, the encoded text2 and + the array of unique strings. The zeroth element of the array of unique + strings is intentionally blank. + """ + lineArray = [] # e.g. lineArray[4] == "Hello\n" + lineHash = {} # e.g. lineHash["Hello\n"] == 4 + + # "\x00" is a valid character, but various debuggers don't like it. + # So we'll insert a junk entry to avoid generating a null character. + lineArray.append('') + + def diff_linesToCharsMunge(text): + """Split a text into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + Modifies linearray and linehash through being a closure. + + Args: + text: String to encode. + + Returns: + Encoded string. + """ + chars = [] + # Walk the text, pulling out a substring for each line. + # text.split('\n') would would temporarily double our memory footprint. + # Modifying text would create many large strings to garbage collect. + lineStart = 0 + lineEnd = -1 + while lineEnd < len(text) - 1: + lineEnd = text.find('\n', lineStart) + if lineEnd == -1: + lineEnd = len(text) - 1 + line = text[lineStart:lineEnd + 1] + + if line in lineHash: + chars.append(chr(lineHash[line])) + else: + if len(lineArray) == maxLines: + # Bail out at 1114111 because chr(1114112) throws. + line = text[lineStart:] + lineEnd = len(text) + lineArray.append(line) + lineHash[line] = len(lineArray) - 1 + chars.append(chr(len(lineArray) - 1)) + lineStart = lineEnd + 1 + return "".join(chars) + + # Allocate 2/3rds of the space for text1, the rest for text2. + maxLines = 666666 + chars1 = diff_linesToCharsMunge(text1) + maxLines = 1114111 + chars2 = diff_linesToCharsMunge(text2) + return (chars1, chars2, lineArray) + + def diff_charsToLines(self, diffs, lineArray): + """Rehydrate the text in a diff from a string of line hashes to real lines + of text. + + Args: + diffs: Array of diff tuples. + lineArray: Array of unique strings. + """ + for i in range(len(diffs)): + text = [] + for char in diffs[i][1]: + text.append(lineArray[ord(char)]) + diffs[i] = (diffs[i][0], "".join(text)) + + def diff_commonPrefix(self, text1, text2): + """Determine the common prefix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the start of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[0] != text2[0]: + return 0 + # Binary search. + # Performance analysis: https://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerstart = 0 + while pointermin < pointermid: + if text1[pointerstart:pointermid] == text2[pointerstart:pointermid]: + pointermin = pointermid + pointerstart = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonSuffix(self, text1, text2): + """Determine the common suffix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the end of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[-1] != text2[-1]: + return 0 + # Binary search. + # Performance analysis: https://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerend = 0 + while pointermin < pointermid: + if (text1[-pointermid:len(text1) - pointerend] == + text2[-pointermid:len(text2) - pointerend]): + pointermin = pointermid + pointerend = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonOverlap(self, text1, text2): + """Determine if the suffix of one string is the prefix of another. + + Args: + text1 First string. + text2 Second string. + + Returns: + The number of characters common to the end of the first + string and the start of the second string. + """ + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + # Eliminate the null case. + if text1_length == 0 or text2_length == 0: + return 0 + # Truncate the longer string. + if text1_length > text2_length: + text1 = text1[-text2_length:] + elif text1_length < text2_length: + text2 = text2[:text1_length] + text_length = min(text1_length, text2_length) + # Quick check for the worst case. + if text1 == text2: + return text_length + + # Start by looking for a single character match + # and increase length until no match is found. + # Performance analysis: https://neil.fraser.name/news/2010/11/04/ + best = 0 + length = 1 + while True: + pattern = text1[-length:] + found = text2.find(pattern) + if found == -1: + return best + length += found + if found == 0 or text1[-length:] == text2[:length]: + best = length + length += 1 + + def diff_halfMatch(self, text1, text2): + """Do the two texts share a substring which is at least half the length of + the longer text? + This speedup can produce non-minimal diffs. + + Args: + text1: First string. + text2: Second string. + + Returns: + Five element Array, containing the prefix of text1, the suffix of text1, + the prefix of text2, the suffix of text2 and the common middle. Or None + if there was no match. + """ + if self.Diff_Timeout <= 0: + # Don't risk returning a non-optimal diff if we have unlimited time. + return None + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + if len(longtext) < 4 or len(shorttext) * 2 < len(longtext): + return None # Pointless. + + def diff_halfMatchI(longtext, shorttext, i): + """Does a substring of shorttext exist within longtext such that the + substring is at least half the length of longtext? + Closure, but does not reference any external variables. + + Args: + longtext: Longer string. + shorttext: Shorter string. + i: Start index of quarter length substring within longtext. + + Returns: + Five element Array, containing the prefix of longtext, the suffix of + longtext, the prefix of shorttext, the suffix of shorttext and the + common middle. Or None if there was no match. + """ + seed = longtext[i:i + len(longtext) // 4] + best_common = '' + j = shorttext.find(seed) + while j != -1: + prefixLength = self.diff_commonPrefix(longtext[i:], shorttext[j:]) + suffixLength = self.diff_commonSuffix(longtext[:i], shorttext[:j]) + if len(best_common) < suffixLength + prefixLength: + best_common = (shorttext[j - suffixLength:j] + + shorttext[j:j + prefixLength]) + best_longtext_a = longtext[:i - suffixLength] + best_longtext_b = longtext[i + prefixLength:] + best_shorttext_a = shorttext[:j - suffixLength] + best_shorttext_b = shorttext[j + prefixLength:] + j = shorttext.find(seed, j + 1) + + if len(best_common) * 2 >= len(longtext): + return (best_longtext_a, best_longtext_b, + best_shorttext_a, best_shorttext_b, best_common) + else: + return None + + # First check if the second quarter is the seed for a half-match. + hm1 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 3) // 4) + # Check again based on the third quarter. + hm2 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 1) // 2) + if not hm1 and not hm2: + return None + elif not hm2: + hm = hm1 + elif not hm1: + hm = hm2 + else: + # Both matched. Select the longest. + if len(hm1[4]) > len(hm2[4]): + hm = hm1 + else: + hm = hm2 + + # A half-match was found, sort out the return data. + if len(text1) > len(text2): + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + else: + (text2_a, text2_b, text1_a, text1_b, mid_common) = hm + return (text1_a, text1_b, text2_a, text2_b, mid_common) + + def diff_cleanupSemantic(self, diffs): + """Reduce the number of edits by eliminating semantically trivial + equalities. + + Args: + diffs: Array of diff tuples. + """ + changes = False + equalities = [] # Stack of indices where equalities are found. + lastEquality = None # Always equal to diffs[equalities[-1]][1] + pointer = 0 # Index of current position. + # Number of chars that changed prior to the equality. + length_insertions1, length_deletions1 = 0, 0 + # Number of chars that changed after the equality. + length_insertions2, length_deletions2 = 0, 0 + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_EQUAL: # Equality found. + equalities.append(pointer) + length_insertions1, length_insertions2 = length_insertions2, 0 + length_deletions1, length_deletions2 = length_deletions2, 0 + lastEquality = diffs[pointer][1] + else: # An insertion or deletion. + if diffs[pointer][0] == self.DIFF_INSERT: + length_insertions2 += len(diffs[pointer][1]) + else: + length_deletions2 += len(diffs[pointer][1]) + # Eliminate an equality that is smaller or equal to the edits on both + # sides of it. + if (lastEquality and (len(lastEquality) <= + max(length_insertions1, length_deletions1)) and + (len(lastEquality) <= max(length_insertions2, length_deletions2))): + # Duplicate record. + diffs.insert(equalities[-1], (self.DIFF_DELETE, lastEquality)) + # Change second copy to insert. + diffs[equalities[-1] + 1] = (self.DIFF_INSERT, + diffs[equalities[-1] + 1][1]) + # Throw away the equality we just deleted. + equalities.pop() + # Throw away the previous equality (it needs to be reevaluated). + if len(equalities): + equalities.pop() + if len(equalities): + pointer = equalities[-1] + else: + pointer = -1 + # Reset the counters. + length_insertions1, length_deletions1 = 0, 0 + length_insertions2, length_deletions2 = 0, 0 + lastEquality = None + changes = True + pointer += 1 + + # Normalize the diff. + if changes: + self.diff_cleanupMerge(diffs) + self.diff_cleanupSemanticLossless(diffs) + + # Find any overlaps between deletions and insertions. + # e.g: abcxxxxxxdef + # -> abcxxxdef + # e.g: xxxabcdefxxx + # -> defxxxabc + # Only extract an overlap if it is as big as the edit ahead or behind it. + pointer = 1 + while pointer < len(diffs): + if (diffs[pointer - 1][0] == self.DIFF_DELETE and + diffs[pointer][0] == self.DIFF_INSERT): + deletion = diffs[pointer - 1][1] + insertion = diffs[pointer][1] + overlap_length1 = self.diff_commonOverlap(deletion, insertion) + overlap_length2 = self.diff_commonOverlap(insertion, deletion) + if overlap_length1 >= overlap_length2: + if (overlap_length1 >= len(deletion) / 2.0 or + overlap_length1 >= len(insertion) / 2.0): + # Overlap found. Insert an equality and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, + insertion[:overlap_length1])) + diffs[pointer - 1] = (self.DIFF_DELETE, + deletion[:len(deletion) - overlap_length1]) + diffs[pointer + 1] = (self.DIFF_INSERT, + insertion[overlap_length1:]) + pointer += 1 + else: + if (overlap_length2 >= len(deletion) / 2.0 or + overlap_length2 >= len(insertion) / 2.0): + # Reverse overlap found. + # Insert an equality and swap and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, deletion[:overlap_length2])) + diffs[pointer - 1] = (self.DIFF_INSERT, + insertion[:len(insertion) - overlap_length2]) + diffs[pointer + 1] = (self.DIFF_DELETE, deletion[overlap_length2:]) + pointer += 1 + pointer += 1 + pointer += 1 + + def diff_cleanupSemanticLossless(self, diffs): + """Look for single edits surrounded on both sides by equalities + which can be shifted sideways to align the edit to a word boundary. + e.g: The cat came. -> The cat came. + + Args: + diffs: Array of diff tuples. + """ + + def diff_cleanupSemanticScore(one, two): + """Given two strings, compute a score representing whether the + internal boundary falls on logical boundaries. + Scores range from 6 (best) to 0 (worst). + Closure, but does not reference any external variables. + + Args: + one: First string. + two: Second string. + + Returns: + The score. + """ + if not one or not two: + # Edges are the best. + return 6 + + # Each port of this function behaves slightly differently due to + # subtle differences in each language's definition of things like + # 'whitespace'. Since this function's purpose is largely cosmetic, + # the choice has been made to use each language's native features + # rather than force total conformity. + char1 = one[-1] + char2 = two[0] + nonAlphaNumeric1 = not char1.isalnum() + nonAlphaNumeric2 = not char2.isalnum() + whitespace1 = nonAlphaNumeric1 and char1.isspace() + whitespace2 = nonAlphaNumeric2 and char2.isspace() + lineBreak1 = whitespace1 and (char1 == "\r" or char1 == "\n") + lineBreak2 = whitespace2 and (char2 == "\r" or char2 == "\n") + blankLine1 = lineBreak1 and self.BLANKLINEEND.search(one) + blankLine2 = lineBreak2 and self.BLANKLINESTART.match(two) + + if blankLine1 or blankLine2: + # Five points for blank lines. + return 5 + elif lineBreak1 or lineBreak2: + # Four points for line breaks. + return 4 + elif nonAlphaNumeric1 and not whitespace1 and whitespace2: + # Three points for end of sentences. + return 3 + elif whitespace1 or whitespace2: + # Two points for whitespace. + return 2 + elif nonAlphaNumeric1 or nonAlphaNumeric2: + # One point for non-alphanumeric. + return 1 + return 0 + + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + equality1 = diffs[pointer - 1][1] + edit = diffs[pointer][1] + equality2 = diffs[pointer + 1][1] + + # First, shift the edit as far left as possible. + commonOffset = self.diff_commonSuffix(equality1, edit) + if commonOffset: + commonString = edit[-commonOffset:] + equality1 = equality1[:-commonOffset] + edit = commonString + edit[:-commonOffset] + equality2 = commonString + equality2 + + # Second, step character by character right, looking for the best fit. + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + bestScore = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + while edit and equality2 and edit[0] == equality2[0]: + equality1 += edit[0] + edit = edit[1:] + equality2[0] + equality2 = equality2[1:] + score = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + # The >= encourages trailing rather than leading whitespace on edits. + if score >= bestScore: + bestScore = score + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + + if diffs[pointer - 1][1] != bestEquality1: + # We have an improvement, save it back to the diff. + if bestEquality1: + diffs[pointer - 1] = (diffs[pointer - 1][0], bestEquality1) + else: + del diffs[pointer - 1] + pointer -= 1 + diffs[pointer] = (diffs[pointer][0], bestEdit) + if bestEquality2: + diffs[pointer + 1] = (diffs[pointer + 1][0], bestEquality2) + else: + del diffs[pointer + 1] + pointer -= 1 + pointer += 1 + + # Define some regex patterns for matching boundaries. + BLANKLINEEND = re.compile(r"\n\r?\n$") + BLANKLINESTART = re.compile(r"^\r?\n\r?\n") + + def diff_cleanupEfficiency(self, diffs): + """Reduce the number of edits by eliminating operationally trivial + equalities. + + Args: + diffs: Array of diff tuples. + """ + changes = False + equalities = [] # Stack of indices where equalities are found. + lastEquality = None # Always equal to diffs[equalities[-1]][1] + pointer = 0 # Index of current position. + pre_ins = False # Is there an insertion operation before the last equality. + pre_del = False # Is there a deletion operation before the last equality. + post_ins = False # Is there an insertion operation after the last equality. + post_del = False # Is there a deletion operation after the last equality. + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_EQUAL: # Equality found. + if (len(diffs[pointer][1]) < self.Diff_EditCost and + (post_ins or post_del)): + # Candidate found. + equalities.append(pointer) + pre_ins = post_ins + pre_del = post_del + lastEquality = diffs[pointer][1] + else: + # Not a candidate, and can never become one. + equalities = [] + lastEquality = None + + post_ins = post_del = False + else: # An insertion or deletion. + if diffs[pointer][0] == self.DIFF_DELETE: + post_del = True + else: + post_ins = True + + # Five types to be split: + # ABXYCD + # AXCD + # ABXC + # AXCD + # ABXC + + if lastEquality and ((pre_ins and pre_del and post_ins and post_del) or + ((len(lastEquality) < self.Diff_EditCost / 2) and + (pre_ins + pre_del + post_ins + post_del) == 3)): + # Duplicate record. + diffs.insert(equalities[-1], (self.DIFF_DELETE, lastEquality)) + # Change second copy to insert. + diffs[equalities[-1] + 1] = (self.DIFF_INSERT, + diffs[equalities[-1] + 1][1]) + equalities.pop() # Throw away the equality we just deleted. + lastEquality = None + if pre_ins and pre_del: + # No changes made which could affect previous entry, keep going. + post_ins = post_del = True + equalities = [] + else: + if len(equalities): + equalities.pop() # Throw away the previous equality. + if len(equalities): + pointer = equalities[-1] + else: + pointer = -1 + post_ins = post_del = False + changes = True + pointer += 1 + + if changes: + self.diff_cleanupMerge(diffs) + + def diff_cleanupMerge(self, diffs): + """Reorder and merge like edit sections. Merge equalities. + Any edit section can move as long as it doesn't cross an equality. + + Args: + diffs: Array of diff tuples. + """ + diffs.append((self.DIFF_EQUAL, '')) # Add a dummy entry at the end. + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete + count_insert > 1: + if count_delete != 0 and count_insert != 0: + # Factor out any common prefixies. + commonlength = self.diff_commonPrefix(text_insert, text_delete) + if commonlength != 0: + x = pointer - count_delete - count_insert - 1 + if x >= 0 and diffs[x][0] == self.DIFF_EQUAL: + diffs[x] = (diffs[x][0], diffs[x][1] + + text_insert[:commonlength]) + else: + diffs.insert(0, (self.DIFF_EQUAL, text_insert[:commonlength])) + pointer += 1 + text_insert = text_insert[commonlength:] + text_delete = text_delete[commonlength:] + # Factor out any common suffixies. + commonlength = self.diff_commonSuffix(text_insert, text_delete) + if commonlength != 0: + diffs[pointer] = (diffs[pointer][0], text_insert[-commonlength:] + + diffs[pointer][1]) + text_insert = text_insert[:-commonlength] + text_delete = text_delete[:-commonlength] + # Delete the offending records and add the merged ones. + new_ops = [] + if len(text_delete) != 0: + new_ops.append((self.DIFF_DELETE, text_delete)) + if len(text_insert) != 0: + new_ops.append((self.DIFF_INSERT, text_insert)) + pointer -= count_delete + count_insert + diffs[pointer : pointer + count_delete + count_insert] = new_ops + pointer += len(new_ops) + 1 + elif pointer != 0 and diffs[pointer - 1][0] == self.DIFF_EQUAL: + # Merge this equality with the previous one. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer][1]) + del diffs[pointer] + else: + pointer += 1 + + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + if diffs[-1][1] == '': + diffs.pop() # Remove the dummy entry at the end. + + # Second pass: look for single edits surrounded on both sides by equalities + # which can be shifted sideways to eliminate an equality. + # e.g: ABAC -> ABAC + changes = False + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + if diffs[pointer][1].endswith(diffs[pointer - 1][1]): + # Shift the edit over the previous equality. + if diffs[pointer - 1][1] != "": + diffs[pointer] = (diffs[pointer][0], + diffs[pointer - 1][1] + + diffs[pointer][1][:-len(diffs[pointer - 1][1])]) + diffs[pointer + 1] = (diffs[pointer + 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + del diffs[pointer - 1] + changes = True + elif diffs[pointer][1].startswith(diffs[pointer + 1][1]): + # Shift the edit over the next equality. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + diffs[pointer] = (diffs[pointer][0], + diffs[pointer][1][len(diffs[pointer + 1][1]):] + + diffs[pointer + 1][1]) + del diffs[pointer + 1] + changes = True + pointer += 1 + + # If shifts were made, the diff needs reordering and another shift sweep. + if changes: + self.diff_cleanupMerge(diffs) + + def diff_xIndex(self, diffs, loc): + """loc is a location in text1, compute and return the equivalent location + in text2. e.g. "The cat" vs "The big cat", 1->1, 5->8 + + Args: + diffs: Array of diff tuples. + loc: Location within text1. + + Returns: + Location within text2. + """ + chars1 = 0 + chars2 = 0 + last_chars1 = 0 + last_chars2 = 0 + for x in range(len(diffs)): + (op, text) = diffs[x] + if op != self.DIFF_INSERT: # Equality or deletion. + chars1 += len(text) + if op != self.DIFF_DELETE: # Equality or insertion. + chars2 += len(text) + if chars1 > loc: # Overshot the location. + break + last_chars1 = chars1 + last_chars2 = chars2 + + if len(diffs) != x and diffs[x][0] == self.DIFF_DELETE: + # The location was deleted. + return last_chars2 + # Add the remaining len(character). + return last_chars2 + (loc - last_chars1) + + def diff_prettyHtml(self, diffs): + """Convert a diff array into a pretty HTML report. + + Args: + diffs: Array of diff tuples. + + Returns: + HTML representation. + """ + html = [] + for (op, data) in diffs: + text = (data.replace("&", "&").replace("<", "<") + .replace(">", ">").replace("\n", "¶
")) + if op == self.DIFF_INSERT: + html.append("%s" % text) + elif op == self.DIFF_DELETE: + html.append("%s" % text) + elif op == self.DIFF_EQUAL: + html.append("%s" % text) + return "".join(html) + + def diff_text1(self, diffs): + """Compute and return the source text (all equalities and deletions). + + Args: + diffs: Array of diff tuples. + + Returns: + Source text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_INSERT: + text.append(data) + return "".join(text) + + def diff_text2(self, diffs): + """Compute and return the destination text (all equalities and insertions). + + Args: + diffs: Array of diff tuples. + + Returns: + Destination text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_DELETE: + text.append(data) + return "".join(text) + + def diff_levenshtein(self, diffs): + """Compute the Levenshtein distance; the number of inserted, deleted or + substituted characters. + + Args: + diffs: Array of diff tuples. + + Returns: + Number of changes. + """ + levenshtein = 0 + insertions = 0 + deletions = 0 + for (op, data) in diffs: + if op == self.DIFF_INSERT: + insertions += len(data) + elif op == self.DIFF_DELETE: + deletions += len(data) + elif op == self.DIFF_EQUAL: + # A deletion and an insertion is one substitution. + levenshtein += max(insertions, deletions) + insertions = 0 + deletions = 0 + levenshtein += max(insertions, deletions) + return levenshtein + + def diff_toDelta(self, diffs): + """Crush the diff into an encoded string which describes the operations + required to transform text1 into text2. + E.g. =3\t-2\t+ing -> Keep 3 chars, delete 2 chars, insert 'ing'. + Operations are tab-separated. Inserted text is escaped using %xx notation. + + Args: + diffs: Array of diff tuples. + + Returns: + Delta text. + """ + text = [] + for (op, data) in diffs: + if op == self.DIFF_INSERT: + # High ascii will raise UnicodeDecodeError. Use Unicode instead. + data = data.encode("utf-8") + text.append("+" + urllib.parse.quote(data, "!~*'();/?:@&=+$,# ")) + elif op == self.DIFF_DELETE: + text.append("-%d" % len(data)) + elif op == self.DIFF_EQUAL: + text.append("=%d" % len(data)) + return "\t".join(text) + + def diff_fromDelta(self, text1, delta): + """Given the original text1, and an encoded string which describes the + operations required to transform text1 into text2, compute the full diff. + + Args: + text1: Source string for the diff. + delta: Delta text. + + Returns: + Array of diff tuples. + + Raises: + ValueError: If invalid input. + """ + diffs = [] + pointer = 0 # Cursor in text1 + tokens = delta.split("\t") + for token in tokens: + if token == "": + # Blank tokens are ok (from a trailing \t). + continue + # Each token begins with a one character parameter which specifies the + # operation of this token (delete, insert, equality). + param = token[1:] + if token[0] == "+": + param = urllib.parse.unquote(param) + diffs.append((self.DIFF_INSERT, param)) + elif token[0] == "-" or token[0] == "=": + try: + n = int(param) + except ValueError: + raise ValueError("Invalid number in diff_fromDelta: " + param) + if n < 0: + raise ValueError("Negative number in diff_fromDelta: " + param) + text = text1[pointer : pointer + n] + pointer += n + if token[0] == "=": + diffs.append((self.DIFF_EQUAL, text)) + else: + diffs.append((self.DIFF_DELETE, text)) + else: + # Anything else is an error. + raise ValueError("Invalid diff operation in diff_fromDelta: " + + token[0]) + if pointer != len(text1): + raise ValueError( + "Delta length (%d) does not equal source text length (%d)." % + (pointer, len(text1))) + return diffs + + # MATCH FUNCTIONS + + def match_main(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc'. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Check for null inputs. + if text == None or pattern == None: + raise ValueError("Null inputs. (match_main)") + + loc = max(0, min(loc, len(text))) + if text == pattern: + # Shortcut (potentially not guaranteed by the algorithm) + return 0 + elif not text: + # Nothing to match. + return -1 + elif text[loc:loc + len(pattern)] == pattern: + # Perfect match at the perfect spot! (Includes case of null pattern) + return loc + else: + # Do a fuzzy compare. + match = self.match_bitap(text, pattern, loc) + return match + + def match_bitap(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc' using the + Bitap algorithm. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Python doesn't have a maxint limit, so ignore this check. + #if self.Match_MaxBits != 0 and len(pattern) > self.Match_MaxBits: + # raise ValueError("Pattern too long for this application.") + + # Initialise the alphabet. + s = self.match_alphabet(pattern) + + def match_bitapScore(e, x): + """Compute and return the score for a match with e errors and x location. + Accesses loc and pattern through being a closure. + + Args: + e: Number of errors in match. + x: Location of match. + + Returns: + Overall score for match (0.0 = good, 1.0 = bad). + """ + accuracy = float(e) / len(pattern) + proximity = abs(loc - x) + if not self.Match_Distance: + # Dodge divide by zero error. + return proximity and 1.0 or accuracy + return accuracy + (proximity / float(self.Match_Distance)) + + # Highest score beyond which we give up. + score_threshold = self.Match_Threshold + # Is there a nearby exact match? (speedup) + best_loc = text.find(pattern, loc) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + # What about in the other direction? (speedup) + best_loc = text.rfind(pattern, loc + len(pattern)) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + + # Initialise the bit arrays. + matchmask = 1 << (len(pattern) - 1) + best_loc = -1 + + bin_max = len(pattern) + len(text) + # Empty initialization added to appease pychecker. + last_rd = None + for d in range(len(pattern)): + # Scan for the best match each iteration allows for one more error. + # Run a binary search to determine how far from 'loc' we can stray at + # this error level. + bin_min = 0 + bin_mid = bin_max + while bin_min < bin_mid: + if match_bitapScore(d, loc + bin_mid) <= score_threshold: + bin_min = bin_mid + else: + bin_max = bin_mid + bin_mid = (bin_max - bin_min) // 2 + bin_min + + # Use the result from this iteration as the maximum for the next. + bin_max = bin_mid + start = max(1, loc - bin_mid + 1) + finish = min(loc + bin_mid, len(text)) + len(pattern) + + rd = [0] * (finish + 2) + rd[finish + 1] = (1 << d) - 1 + for j in range(finish, start - 1, -1): + if len(text) <= j - 1: + # Out of range. + charMatch = 0 + else: + charMatch = s.get(text[j - 1], 0) + if d == 0: # First pass: exact match. + rd[j] = ((rd[j + 1] << 1) | 1) & charMatch + else: # Subsequent passes: fuzzy match. + rd[j] = (((rd[j + 1] << 1) | 1) & charMatch) | ( + ((last_rd[j + 1] | last_rd[j]) << 1) | 1) | last_rd[j + 1] + if rd[j] & matchmask: + score = match_bitapScore(d, j - 1) + # This match will almost certainly be better than any existing match. + # But check anyway. + if score <= score_threshold: + # Told you so. + score_threshold = score + best_loc = j - 1 + if best_loc > loc: + # When passing loc, don't exceed our current distance from loc. + start = max(1, 2 * loc - best_loc) + else: + # Already passed loc, downhill from here on in. + break + # No hope for a (better) match at greater error levels. + if match_bitapScore(d + 1, loc) > score_threshold: + break + last_rd = rd + return best_loc + + def match_alphabet(self, pattern): + """Initialise the alphabet for the Bitap algorithm. + + Args: + pattern: The text to encode. + + Returns: + Hash of character locations. + """ + s = {} + for char in pattern: + s[char] = 0 + for i in range(len(pattern)): + s[pattern[i]] |= 1 << (len(pattern) - i - 1) + return s + + # PATCH FUNCTIONS + + def patch_addContext(self, patch, text): + """Increase the context until it is unique, + but don't let the pattern expand beyond Match_MaxBits. + + Args: + patch: The patch to grow. + text: Source text. + """ + if len(text) == 0: + return + pattern = text[patch.start2 : patch.start2 + patch.length1] + padding = 0 + + # Look for the first and last matches of pattern in text. If two different + # matches are found, increase the pattern length. + while (text.find(pattern) != text.rfind(pattern) and (self.Match_MaxBits == + 0 or len(pattern) < self.Match_MaxBits - self.Patch_Margin - + self.Patch_Margin)): + padding += self.Patch_Margin + pattern = text[max(0, patch.start2 - padding) : + patch.start2 + patch.length1 + padding] + # Add one chunk for good luck. + padding += self.Patch_Margin + + # Add the prefix. + prefix = text[max(0, patch.start2 - padding) : patch.start2] + if prefix: + patch.diffs[:0] = [(self.DIFF_EQUAL, prefix)] + # Add the suffix. + suffix = text[patch.start2 + patch.length1 : + patch.start2 + patch.length1 + padding] + if suffix: + patch.diffs.append((self.DIFF_EQUAL, suffix)) + + # Roll back the start points. + patch.start1 -= len(prefix) + patch.start2 -= len(prefix) + # Extend lengths. + patch.length1 += len(prefix) + len(suffix) + patch.length2 += len(prefix) + len(suffix) + + def patch_make(self, a, b=None, c=None): + """Compute a list of patches to turn text1 into text2. + Use diffs if provided, otherwise compute it ourselves. + There are four ways to call this function, depending on what data is + available to the caller: + Method 1: + a = text1, b = text2 + Method 2: + a = diffs + Method 3 (optimal): + a = text1, b = diffs + Method 4 (deprecated, use method 3): + a = text1, b = text2, c = diffs + + Args: + a: text1 (methods 1,3,4) or Array of diff tuples for text1 to + text2 (method 2). + b: text2 (methods 1,4) or Array of diff tuples for text1 to + text2 (method 3) or undefined (method 2). + c: Array of diff tuples for text1 to text2 (method 4) or + undefined (methods 1,2,3). + + Returns: + Array of Patch objects. + """ + text1 = None + diffs = None + if isinstance(a, str) and isinstance(b, str) and c is None: + # Method 1: text1, text2 + # Compute diffs from text1 and text2. + text1 = a + diffs = self.diff_main(text1, b, True) + if len(diffs) > 2: + self.diff_cleanupSemantic(diffs) + self.diff_cleanupEfficiency(diffs) + elif isinstance(a, list) and b is None and c is None: + # Method 2: diffs + # Compute text1 from diffs. + diffs = a + text1 = self.diff_text1(diffs) + elif isinstance(a, str) and isinstance(b, list) and c is None: + # Method 3: text1, diffs + text1 = a + diffs = b + elif (isinstance(a, str) and isinstance(b, str) and + isinstance(c, list)): + # Method 4: text1, text2, diffs + # text2 is not used. + text1 = a + diffs = c + else: + raise ValueError("Unknown call format to patch_make.") + + if not diffs: + return [] # Get rid of the None case. + patches = [] + patch = patch_obj() + char_count1 = 0 # Number of characters into the text1 string. + char_count2 = 0 # Number of characters into the text2 string. + prepatch_text = text1 # Recreate the patches to determine context info. + postpatch_text = text1 + for x in range(len(diffs)): + (diff_type, diff_text) = diffs[x] + if len(patch.diffs) == 0 and diff_type != self.DIFF_EQUAL: + # A new patch starts here. + patch.start1 = char_count1 + patch.start2 = char_count2 + if diff_type == self.DIFF_INSERT: + # Insertion + patch.diffs.append(diffs[x]) + patch.length2 += len(diff_text) + postpatch_text = (postpatch_text[:char_count2] + diff_text + + postpatch_text[char_count2:]) + elif diff_type == self.DIFF_DELETE: + # Deletion. + patch.length1 += len(diff_text) + patch.diffs.append(diffs[x]) + postpatch_text = (postpatch_text[:char_count2] + + postpatch_text[char_count2 + len(diff_text):]) + elif (diff_type == self.DIFF_EQUAL and + len(diff_text) <= 2 * self.Patch_Margin and + len(patch.diffs) != 0 and len(diffs) != x + 1): + # Small equality inside a patch. + patch.diffs.append(diffs[x]) + patch.length1 += len(diff_text) + patch.length2 += len(diff_text) + + if (diff_type == self.DIFF_EQUAL and + len(diff_text) >= 2 * self.Patch_Margin): + # Time for a new patch. + if len(patch.diffs) != 0: + self.patch_addContext(patch, prepatch_text) + patches.append(patch) + patch = patch_obj() + # Unlike Unidiff, our patch lists have a rolling context. + # https://github.com/google/diff-match-patch/wiki/Unidiff + # Update prepatch text & pos to reflect the application of the + # just completed patch. + prepatch_text = postpatch_text + char_count1 = char_count2 + + # Update the current character count. + if diff_type != self.DIFF_INSERT: + char_count1 += len(diff_text) + if diff_type != self.DIFF_DELETE: + char_count2 += len(diff_text) + + # Pick up the leftover patch if not empty. + if len(patch.diffs) != 0: + self.patch_addContext(patch, prepatch_text) + patches.append(patch) + return patches + + def patch_deepCopy(self, patches): + """Given an array of patches, return another array that is identical. + + Args: + patches: Array of Patch objects. + + Returns: + Array of Patch objects. + """ + patchesCopy = [] + for patch in patches: + patchCopy = patch_obj() + # No need to deep copy the tuples since they are immutable. + patchCopy.diffs = patch.diffs[:] + patchCopy.start1 = patch.start1 + patchCopy.start2 = patch.start2 + patchCopy.length1 = patch.length1 + patchCopy.length2 = patch.length2 + patchesCopy.append(patchCopy) + return patchesCopy + + def patch_apply(self, patches, text): + """Merge a set of patches onto the text. Return a patched text, as well + as a list of true/false values indicating which patches were applied. + + Args: + patches: Array of Patch objects. + text: Old text. + + Returns: + Two element Array, containing the new text and an array of boolean values. + """ + if not patches: + return (text, []) + + # Deep copy the patches so that no changes are made to originals. + patches = self.patch_deepCopy(patches) + + nullPadding = self.patch_addPadding(patches) + text = nullPadding + text + nullPadding + self.patch_splitMax(patches) + + # delta keeps track of the offset between the expected and actual location + # of the previous patch. If there are patches expected at positions 10 and + # 20, but the first patch was found at 12, delta is 2 and the second patch + # has an effective expected position of 22. + delta = 0 + results = [] + for patch in patches: + expected_loc = patch.start2 + delta + text1 = self.diff_text1(patch.diffs) + end_loc = -1 + if len(text1) > self.Match_MaxBits: + # patch_splitMax will only provide an oversized pattern in the case of + # a monster delete. + start_loc = self.match_main(text, text1[:self.Match_MaxBits], + expected_loc) + if start_loc != -1: + end_loc = self.match_main(text, text1[-self.Match_MaxBits:], + expected_loc + len(text1) - self.Match_MaxBits) + if end_loc == -1 or start_loc >= end_loc: + # Can't find valid trailing context. Drop this patch. + start_loc = -1 + else: + start_loc = self.match_main(text, text1, expected_loc) + if start_loc == -1: + # No match found. :( + results.append(False) + # Subtract the delta for this failed patch from subsequent patches. + delta -= patch.length2 - patch.length1 + else: + # Found a match. :) + results.append(True) + delta = start_loc - expected_loc + if end_loc == -1: + text2 = text[start_loc : start_loc + len(text1)] + else: + text2 = text[start_loc : end_loc + self.Match_MaxBits] + if text1 == text2: + # Perfect match, just shove the replacement text in. + text = (text[:start_loc] + self.diff_text2(patch.diffs) + + text[start_loc + len(text1):]) + else: + # Imperfect match. + # Run a diff to get a framework of equivalent indices. + diffs = self.diff_main(text1, text2, False) + if (len(text1) > self.Match_MaxBits and + self.diff_levenshtein(diffs) / float(len(text1)) > + self.Patch_DeleteThreshold): + # The end points match, but the content is unacceptably bad. + results[-1] = False + else: + self.diff_cleanupSemanticLossless(diffs) + index1 = 0 + for (op, data) in patch.diffs: + if op != self.DIFF_EQUAL: + index2 = self.diff_xIndex(diffs, index1) + if op == self.DIFF_INSERT: # Insertion + text = text[:start_loc + index2] + data + text[start_loc + + index2:] + elif op == self.DIFF_DELETE: # Deletion + text = text[:start_loc + index2] + text[start_loc + + self.diff_xIndex(diffs, index1 + len(data)):] + if op != self.DIFF_DELETE: + index1 += len(data) + # Strip the padding off. + text = text[len(nullPadding):-len(nullPadding)] + return (text, results) + + def patch_addPadding(self, patches): + """Add some padding on text start and end so that edges can match + something. Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + + Returns: + The padding string added to each side. + """ + paddingLength = self.Patch_Margin + nullPadding = "" + for x in range(1, paddingLength + 1): + nullPadding += chr(x) + + # Bump all the patches forward. + for patch in patches: + patch.start1 += paddingLength + patch.start2 += paddingLength + + # Add some padding on start of first diff. + patch = patches[0] + diffs = patch.diffs + if not diffs or diffs[0][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.insert(0, (self.DIFF_EQUAL, nullPadding)) + patch.start1 -= paddingLength # Should be 0. + patch.start2 -= paddingLength # Should be 0. + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[0][1]): + # Grow first equality. + extraLength = paddingLength - len(diffs[0][1]) + newText = nullPadding[len(diffs[0][1]):] + diffs[0][1] + diffs[0] = (diffs[0][0], newText) + patch.start1 -= extraLength + patch.start2 -= extraLength + patch.length1 += extraLength + patch.length2 += extraLength + + # Add some padding on end of last diff. + patch = patches[-1] + diffs = patch.diffs + if not diffs or diffs[-1][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.append((self.DIFF_EQUAL, nullPadding)) + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[-1][1]): + # Grow last equality. + extraLength = paddingLength - len(diffs[-1][1]) + newText = diffs[-1][1] + nullPadding[:extraLength] + diffs[-1] = (diffs[-1][0], newText) + patch.length1 += extraLength + patch.length2 += extraLength + + return nullPadding + + def patch_splitMax(self, patches): + """Look through the patches and break up any which are longer than the + maximum limit of the match algorithm. + Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + """ + patch_size = self.Match_MaxBits + if patch_size == 0: + # Python has the option of not splitting strings due to its ability + # to handle integers of arbitrary precision. + return + for x in range(len(patches)): + if patches[x].length1 <= patch_size: + continue + bigpatch = patches[x] + # Remove the big old patch. + del patches[x] + x -= 1 + start1 = bigpatch.start1 + start2 = bigpatch.start2 + precontext = '' + while len(bigpatch.diffs) != 0: + # Create one of several smaller patches. + patch = patch_obj() + empty = True + patch.start1 = start1 - len(precontext) + patch.start2 = start2 - len(precontext) + if precontext: + patch.length1 = patch.length2 = len(precontext) + patch.diffs.append((self.DIFF_EQUAL, precontext)) + + while (len(bigpatch.diffs) != 0 and + patch.length1 < patch_size - self.Patch_Margin): + (diff_type, diff_text) = bigpatch.diffs[0] + if diff_type == self.DIFF_INSERT: + # Insertions are harmless. + patch.length2 += len(diff_text) + start2 += len(diff_text) + patch.diffs.append(bigpatch.diffs.pop(0)) + empty = False + elif (diff_type == self.DIFF_DELETE and len(patch.diffs) == 1 and + patch.diffs[0][0] == self.DIFF_EQUAL and + len(diff_text) > 2 * patch_size): + # This is a large deletion. Let it pass in one chunk. + patch.length1 += len(diff_text) + start1 += len(diff_text) + empty = False + patch.diffs.append((diff_type, diff_text)) + del bigpatch.diffs[0] + else: + # Deletion or equality. Only take as much as we can stomach. + diff_text = diff_text[:patch_size - patch.length1 - + self.Patch_Margin] + patch.length1 += len(diff_text) + start1 += len(diff_text) + if diff_type == self.DIFF_EQUAL: + patch.length2 += len(diff_text) + start2 += len(diff_text) + else: + empty = False + + patch.diffs.append((diff_type, diff_text)) + if diff_text == bigpatch.diffs[0][1]: + del bigpatch.diffs[0] + else: + bigpatch.diffs[0] = (bigpatch.diffs[0][0], + bigpatch.diffs[0][1][len(diff_text):]) + + # Compute the head context for the next patch. + precontext = self.diff_text2(patch.diffs) + precontext = precontext[-self.Patch_Margin:] + # Append the end context for this patch. + postcontext = self.diff_text1(bigpatch.diffs)[:self.Patch_Margin] + if postcontext: + patch.length1 += len(postcontext) + patch.length2 += len(postcontext) + if len(patch.diffs) != 0 and patch.diffs[-1][0] == self.DIFF_EQUAL: + patch.diffs[-1] = (self.DIFF_EQUAL, patch.diffs[-1][1] + + postcontext) + else: + patch.diffs.append((self.DIFF_EQUAL, postcontext)) + + if not empty: + x += 1 + patches.insert(x, patch) + + def patch_toText(self, patches): + """Take a list of patches and return a textual representation. + + Args: + patches: Array of Patch objects. + + Returns: + Text representation of patches. + """ + text = [] + for patch in patches: + text.append(str(patch)) + return "".join(text) + + def patch_fromText(self, textline): + """Parse a textual representation of patches and return a list of patch + objects. + + Args: + textline: Text representation of patches. + + Returns: + Array of Patch objects. + + Raises: + ValueError: If invalid input. + """ + patches = [] + if not textline: + return patches + text = textline.split('\n') + while len(text) != 0: + m = re.match("^@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@$", text[0]) + if not m: + raise ValueError("Invalid patch string: " + text[0]) + patch = patch_obj() + patches.append(patch) + patch.start1 = int(m.group(1)) + if m.group(2) == '': + patch.start1 -= 1 + patch.length1 = 1 + elif m.group(2) == '0': + patch.length1 = 0 + else: + patch.start1 -= 1 + patch.length1 = int(m.group(2)) + + patch.start2 = int(m.group(3)) + if m.group(4) == '': + patch.start2 -= 1 + patch.length2 = 1 + elif m.group(4) == '0': + patch.length2 = 0 + else: + patch.start2 -= 1 + patch.length2 = int(m.group(4)) + + del text[0] + + while len(text) != 0: + if text[0]: + sign = text[0][0] + else: + sign = '' + line = urllib.parse.unquote(text[0][1:]) + if sign == '+': + # Insertion. + patch.diffs.append((self.DIFF_INSERT, line)) + elif sign == '-': + # Deletion. + patch.diffs.append((self.DIFF_DELETE, line)) + elif sign == ' ': + # Minor equality. + patch.diffs.append((self.DIFF_EQUAL, line)) + elif sign == '@': + # Start of next patch. + break + elif sign == '': + # Blank line? Whatever. + pass + else: + # WTF? + raise ValueError("Invalid patch mode: '%s'\n%s" % (sign, line)) + del text[0] + return patches + + +class patch_obj: + """Class representing one patch operation. + """ + + def __init__(self): + """Initializes with an empty list of diffs. + """ + self.diffs = [] + self.start1 = None + self.start2 = None + self.length1 = 0 + self.length2 = 0 + + def __str__(self): + """Emulate GNU diff's format. + Header: @@ -382,8 +481,9 @@ + Indices are printed as 1-based, not 0-based. + + Returns: + The GNU diff string. + """ + if self.length1 == 0: + coords1 = str(self.start1) + ",0" + elif self.length1 == 1: + coords1 = str(self.start1 + 1) + else: + coords1 = str(self.start1 + 1) + "," + str(self.length1) + if self.length2 == 0: + coords2 = str(self.start2) + ",0" + elif self.length2 == 1: + coords2 = str(self.start2 + 1) + else: + coords2 = str(self.start2 + 1) + "," + str(self.length2) + text = ["@@ -", coords1, " +", coords2, " @@\n"] + # Escape the body of the patch with %xx notation. + for (op, data) in self.diffs: + if op == diff_match_patch.DIFF_INSERT: + text.append("+") + elif op == diff_match_patch.DIFF_DELETE: + text.append("-") + elif op == diff_match_patch.DIFF_EQUAL: + text.append(" ") + # High ascii will raise UnicodeDecodeError. Use Unicode instead. + data = data.encode("utf-8") + text.append(urllib.parse.quote(data, "!~*'();/?:@&=+$,# ") + "\n") + return "".join(text) diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/OTA_INFO.py b/examples/OTA-lorawan/firmware/1.17.0/flash/OTA_INFO.py new file mode 100644 index 0000000..092afa1 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/OTA_INFO.py @@ -0,0 +1 @@ +1.17.0 diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/diff_match_patch.py b/examples/OTA-lorawan/firmware/1.17.0/flash/diff_match_patch.py new file mode 100644 index 0000000..f156f86 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/diff_match_patch.py @@ -0,0 +1,1613 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +#from __future__ import division + +"""Diff Match and Patch +Copyright 2018 The diff-match-patch Authors. +https://github.com/google/diff-match-patch + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +"""Functions for diff, match and patch. + +Computes the difference between two texts to create a patch. +Applies the patch onto another text, allowing for errors. +""" + +__author__ = 'fraser@google.com (Neil Fraser)' + +import math +import ure +import sys +import time +#import urllib + +class diff_match_patch: + """Class containing the diff, match and patch methods. + + Also contains the behaviour settings. + """ + + def __init__(self): + """Inits a diff_match_patch object with default settings. + Redefine these in your program to override the defaults. + """ + + # Number of seconds to map a diff before giving up (0 for infinity). + self.Diff_Timeout = 1.0 + # Cost of an empty edit operation in terms of edit characters. + self.Diff_EditCost = 4 + # At what point is no match declared (0.0 = perfection, 1.0 = very loose). + self.Match_Threshold = 0.5 + # How far to search for a match (0 = exact location, 1000+ = broad match). + # A match this many characters away from the expected location will add + # 1.0 to the score (0.0 is a perfect match). + self.Match_Distance = 1000 + # When deleting a large block of text (over ~64 characters), how close do + # the contents have to be to match the expected contents. (0.0 = perfection, + # 1.0 = very loose). Note that Match_Threshold controls how closely the + # end points of a delete need to match. + self.Patch_DeleteThreshold = 0.5 + # Chunk size for context length. + self.Patch_Margin = 4 + + # The number of bits in an int. + # Python has no maximum, thus to disable patch splitting set to 0. + # However to avoid long patches in certain pathological cases, use 32. + # Multiple short patches (using native ints) are much faster than long ones. + self.Match_MaxBits = 32 + + self._hexdig = '0123456789ABCDEFabcdef' + self._hextochr = dict((a+b, chr(int(a+b,16))) + for a in self._hexdig for b in self._hexdig) + + # DIFF FUNCTIONS + + # The data structure representing a diff is an array of tuples: + # [(DIFF_DELETE, "Hello"), (DIFF_INSERT, "Goodbye"), (DIFF_EQUAL, " world.")] + # which means: delete "Hello", add "Goodbye" and keep " world." + DIFF_DELETE = -1 + DIFF_INSERT = 1 + DIFF_EQUAL = 0 + + def diff_main(self, text1, text2, checklines=True, deadline=None): + """Find the differences between two texts. Simplifies the problem by + stripping any common prefix or suffix off the texts before diffing. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Optional speedup flag. If present and false, then don't run + a line-level diff first to identify the changed areas. + Defaults to true, which does a faster, slightly less optimal diff. + deadline: Optional time when the diff should be complete by. Used + internally for recursive calls. Users should set DiffTimeout instead. + + Returns: + Array of changes. + """ + # Set a deadline by which time the diff must be complete. + if deadline == None: + # Unlike in most languages, Python counts time in seconds. + if self.Diff_Timeout <= 0: + deadline = sys.maxsize + else: + deadline = time.time() + self.Diff_Timeout + + # Check for null inputs. + if text1 == None or text2 == None: + raise ValueError("Null inputs. (diff_main)") + + # Check for equality (speedup). + if text1 == text2: + if text1: + return [(self.DIFF_EQUAL, text1)] + return [] + + # Trim off common prefix (speedup). + commonlength = self.diff_commonPrefix(text1, text2) + commonprefix = text1[:commonlength] + text1 = text1[commonlength:] + text2 = text2[commonlength:] + + # Trim off common suffix (speedup). + commonlength = self.diff_commonSuffix(text1, text2) + if commonlength == 0: + commonsuffix = '' + else: + commonsuffix = text1[-commonlength:] + text1 = text1[:-commonlength] + text2 = text2[:-commonlength] + + # Compute the diff on the middle block. + diffs = self.diff_compute(text1, text2, checklines, deadline) + + # Restore the prefix and suffix. + if commonprefix: + diffs[:0] = [(self.DIFF_EQUAL, commonprefix)] + if commonsuffix: + diffs.append((self.DIFF_EQUAL, commonsuffix)) + self.diff_cleanupMerge(diffs) + return diffs + + def diff_compute(self, text1, text2, checklines, deadline): + """Find the differences between two texts. Assumes that the texts do not + have any common prefix or suffix. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Speedup flag. If false, then don't run a line-level diff + first to identify the changed areas. + If true, then run a faster, slightly less optimal diff. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + if not text1: + # Just add some text (speedup). + return [(self.DIFF_INSERT, text2)] + + if not text2: + # Just delete some text (speedup). + return [(self.DIFF_DELETE, text1)] + + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + i = longtext.find(shorttext) + if i != -1: + # Shorter text is inside the longer text (speedup). + diffs = [(self.DIFF_INSERT, longtext[:i]), (self.DIFF_EQUAL, shorttext), + (self.DIFF_INSERT, longtext[i + len(shorttext):])] + # Swap insertions for deletions if diff is reversed. + if len(text1) > len(text2): + diffs[0] = (self.DIFF_DELETE, diffs[0][1]) + diffs[2] = (self.DIFF_DELETE, diffs[2][1]) + return diffs + + if len(shorttext) == 1: + # Single character string. + # After the previous speedup, the character can't be an equality. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + # Check to see if the problem can be split in two. + hm = self.diff_halfMatch(text1, text2) + if hm: + # A half-match was found, sort out the return data. + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + # Send both pairs off for separate processing. + diffs_a = self.diff_main(text1_a, text2_a, checklines, deadline) + diffs_b = self.diff_main(text1_b, text2_b, checklines, deadline) + # Merge the results. + return diffs_a + [(self.DIFF_EQUAL, mid_common)] + diffs_b + + if checklines and len(text1) > 100 and len(text2) > 100: + return self.diff_lineMode(text1, text2, deadline) + + return self.diff_bisect(text1, text2, deadline) + + def diff_lineMode(self, text1, text2, deadline): + """Do a quick line-level diff on both strings, then rediff the parts for + greater accuracy. + This speedup can produce non-minimal diffs. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + + # Scan the text on a line-by-line basis first. + (text1, text2, linearray) = self.diff_linesToChars(text1, text2) + + diffs = self.diff_main(text1, text2, False, deadline) + + # Convert the diff back to original text. + self.diff_charsToLines(diffs, linearray) + # Eliminate freak matches (e.g. blank lines) + self.diff_cleanupSemantic(diffs) + + # Rediff any replacement blocks, this time character-by-character. + # Add a dummy entry at the end. + diffs.append((self.DIFF_EQUAL, '')) + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete >= 1 and count_insert >= 1: + # Delete the offending records and add the merged ones. + a = self.diff_main(text_delete, text_insert, False, deadline) + diffs[pointer - count_delete - count_insert : pointer] = a + pointer = pointer - count_delete - count_insert + len(a) + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + pointer += 1 + + diffs.pop() # Remove the dummy entry at the end. + + return diffs + + def diff_bisect(self, text1, text2, deadline): + """Find the 'middle snake' of a diff, split the problem in two + and return the recursively constructed diff. + See Myers 1986 paper: An O(ND) Difference Algorithm and Its Variations. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + max_d = (text1_length + text2_length + 1) // 2 + v_offset = max_d + v_length = 2 * max_d + v1 = [-1] * v_length + v1[v_offset + 1] = 0 + v2 = v1[:] + delta = text1_length - text2_length + # If the total number of characters is odd, then the front path will + # collide with the reverse path. + front = (delta % 2 != 0) + # Offsets for start and end of k loop. + # Prevents mapping of space beyond the grid. + k1start = 0 + k1end = 0 + k2start = 0 + k2end = 0 + for d in range(max_d): + # Bail out if deadline is reached. + if time.time() > deadline: + break + + # Walk the front path one step. + for k1 in range(-d + k1start, d + 1 - k1end, 2): + k1_offset = v_offset + k1 + if k1 == -d or (k1 != d and + v1[k1_offset - 1] < v1[k1_offset + 1]): + x1 = v1[k1_offset + 1] + else: + x1 = v1[k1_offset - 1] + 1 + y1 = x1 - k1 + while (x1 < text1_length and y1 < text2_length and + text1[x1] == text2[y1]): + x1 += 1 + y1 += 1 + v1[k1_offset] = x1 + if x1 > text1_length: + # Ran off the right of the graph. + k1end += 2 + elif y1 > text2_length: + # Ran off the bottom of the graph. + k1start += 2 + elif front: + k2_offset = v_offset + delta - k1 + if k2_offset >= 0 and k2_offset < v_length and v2[k2_offset] != -1: + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - v2[k2_offset] + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Walk the reverse path one step. + for k2 in range(-d + k2start, d + 1 - k2end, 2): + k2_offset = v_offset + k2 + if k2 == -d or (k2 != d and + v2[k2_offset - 1] < v2[k2_offset + 1]): + x2 = v2[k2_offset + 1] + else: + x2 = v2[k2_offset - 1] + 1 + y2 = x2 - k2 + while (x2 < text1_length and y2 < text2_length and + text1[-x2 - 1] == text2[-y2 - 1]): + x2 += 1 + y2 += 1 + v2[k2_offset] = x2 + if x2 > text1_length: + # Ran off the left of the graph. + k2end += 2 + elif y2 > text2_length: + # Ran off the top of the graph. + k2start += 2 + elif not front: + k1_offset = v_offset + delta - k2 + if k1_offset >= 0 and k1_offset < v_length and v1[k1_offset] != -1: + x1 = v1[k1_offset] + y1 = v_offset + x1 - k1_offset + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - x2 + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Diff took too long and hit the deadline or + # number of diffs equals number of characters, no commonality at all. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + def diff_bisectSplit(self, text1, text2, x, y, deadline): + """Given the location of the 'middle snake', split the diff in two parts + and recurse. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + x: Index of split point in text1. + y: Index of split point in text2. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + text1a = text1[:x] + text2a = text2[:y] + text1b = text1[x:] + text2b = text2[y:] + + # Compute both diffs serially. + diffs = self.diff_main(text1a, text2a, False, deadline) + diffsb = self.diff_main(text1b, text2b, False, deadline) + + return diffs + diffsb + + def diff_linesToChars(self, text1, text2): + """Split two texts into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + + Args: + text1: First string. + text2: Second string. + + Returns: + Three element tuple, containing the encoded text1, the encoded text2 and + the array of unique strings. The zeroth element of the array of unique + strings is intentionally blank. + """ + lineArray = [] # e.g. lineArray[4] == "Hello\n" + lineHash = {} # e.g. lineHash["Hello\n"] == 4 + + # "\x00" is a valid character, but various debuggers don't like it. + # So we'll insert a junk entry to avoid generating a null character. + lineArray.append('') + + def diff_linesToCharsMunge(text): + """Split a text into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + Modifies linearray and linehash through being a closure. + + Args: + text: String to encode. + + Returns: + Encoded string. + """ + chars = [] + # Walk the text, pulling out a substring for each line. + # text.split('\n') would would temporarily double our memory footprint. + # Modifying text would create many large strings to garbage collect. + lineStart = 0 + lineEnd = -1 + while lineEnd < len(text) - 1: + lineEnd = text.find('\n', lineStart) + if lineEnd == -1: + lineEnd = len(text) - 1 + line = text[lineStart:lineEnd + 1] + + if line in lineHash: + chars.append(chr(lineHash[line])) + else: + if len(lineArray) == maxLines: + # Bail out at 65535 because unichr(65536) throws. - not in micropython + line = text[lineStart:] + lineEnd = len(text) + lineArray.append(line) + lineHash[line] = len(lineArray) - 1 + chars.append(chr(len(lineArray) - 1)) + lineStart = lineEnd + 1 + return "".join(chars) + + # Allocate 2/3rds of the space for text1, the rest for text2. + maxLines = 40000 + chars1 = diff_linesToCharsMunge(text1) + maxLines = 65535 + chars2 = diff_linesToCharsMunge(text2) + return (chars1, chars2, lineArray) + + def diff_charsToLines(self, diffs, lineArray): + """Rehydrate the text in a diff from a string of line hashes to real lines + of text. + + Args: + diffs: Array of diff tuples. + lineArray: Array of unique strings. + """ + for x in range(len(diffs)): + text = [] + for char in diffs[x][1]: + text.append(lineArray[ord(char)]) + diffs[x] = (diffs[x][0], "".join(text)) + + def diff_commonPrefix(self, text1, text2): + """Determine the common prefix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the start of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[0] != text2[0]: + return 0 + # Binary search. + # Performance analysis: http://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerstart = 0 + while pointermin < pointermid: + if text1[pointerstart:pointermid] == text2[pointerstart:pointermid]: + pointermin = pointermid + pointerstart = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonSuffix(self, text1, text2): + """Determine the common suffix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the end of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[-1] != text2[-1]: + return 0 + # Binary search. + # Performance analysis: http://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerend = 0 + while pointermin < pointermid: + if (text1[-pointermid:len(text1) - pointerend] == + text2[-pointermid:len(text2) - pointerend]): + pointermin = pointermid + pointerend = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonOverlap(self, text1, text2): + """Determine if the suffix of one string is the prefix of another. + + Args: + text1 First string. + text2 Second string. + + Returns: + The number of characters common to the end of the first + string and the start of the second string. + """ + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + # Eliminate the null case. + if text1_length == 0 or text2_length == 0: + return 0 + # Truncate the longer string. + if text1_length > text2_length: + text1 = text1[-text2_length:] + elif text1_length < text2_length: + text2 = text2[:text1_length] + text_length = min(text1_length, text2_length) + # Quick check for the worst case. + if text1 == text2: + return text_length + + # Start by looking for a single character match + # and increase length until no match is found. + # Performance analysis: http://neil.fraser.name/news/2010/11/04/ + best = 0 + length = 1 + while True: + pattern = text1[-length:] + found = text2.find(pattern) + if found == -1: + return best + length += found + if found == 0 or text1[-length:] == text2[:length]: + best = length + length += 1 + + def diff_halfMatch(self, text1, text2): + """Do the two texts share a substring which is at least half the length of + the longer text? + This speedup can produce non-minimal diffs. + + Args: + text1: First string. + text2: Second string. + + Returns: + Five element Array, containing the prefix of text1, the suffix of text1, + the prefix of text2, the suffix of text2 and the common middle. Or None + if there was no match. + """ + if self.Diff_Timeout <= 0: + # Don't risk returning a non-optimal diff if we have unlimited time. + return None + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + if len(longtext) < 4 or len(shorttext) * 2 < len(longtext): + return None # Pointless. + + def diff_halfMatchI(longtext, shorttext, i): + """Does a substring of shorttext exist within longtext such that the + substring is at least half the length of longtext? + Closure, but does not reference any external variables. + + Args: + longtext: Longer string. + shorttext: Shorter string. + i: Start index of quarter length substring within longtext. + + Returns: + Five element Array, containing the prefix of longtext, the suffix of + longtext, the prefix of shorttext, the suffix of shorttext and the + common middle. Or None if there was no match. + """ + seed = longtext[i:i + len(longtext) // 4] + best_common = '' + j = shorttext.find(seed) + while j != -1: + prefixLength = self.diff_commonPrefix(longtext[i:], shorttext[j:]) + suffixLength = self.diff_commonSuffix(longtext[:i], shorttext[:j]) + if len(best_common) < suffixLength + prefixLength: + best_common = (shorttext[j - suffixLength:j] + + shorttext[j:j + prefixLength]) + best_longtext_a = longtext[:i - suffixLength] + best_longtext_b = longtext[i + prefixLength:] + best_shorttext_a = shorttext[:j - suffixLength] + best_shorttext_b = shorttext[j + prefixLength:] + j = shorttext.find(seed, j + 1) + + if len(best_common) * 2 >= len(longtext): + return (best_longtext_a, best_longtext_b, + best_shorttext_a, best_shorttext_b, best_common) + else: + return None + + # First check if the second quarter is the seed for a half-match. + hm1 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 3) // 4) + # Check again based on the third quarter. + hm2 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 1) // 2) + if not hm1 and not hm2: + return None + elif not hm2: + hm = hm1 + elif not hm1: + hm = hm2 + else: + # Both matched. Select the longest. + if len(hm1[4]) > len(hm2[4]): + hm = hm1 + else: + hm = hm2 + + # A half-match was found, sort out the return data. + if len(text1) > len(text2): + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + else: + (text2_a, text2_b, text1_a, text1_b, mid_common) = hm + return (text1_a, text1_b, text2_a, text2_b, mid_common) + + def diff_cleanupSemantic(self, diffs): + """Reduce the number of edits by eliminating semantically trivial + equalities. + + Args: + diffs: Array of diff tuples. + """ + changes = False + equalities = [] # Stack of indices where equalities are found. + lastequality = None # Always equal to diffs[equalities[-1]][1] + pointer = 0 # Index of current position. + # Number of chars that changed prior to the equality. + length_insertions1, length_deletions1 = 0, 0 + # Number of chars that changed after the equality. + length_insertions2, length_deletions2 = 0, 0 + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_EQUAL: # Equality found. + equalities.append(pointer) + length_insertions1, length_insertions2 = length_insertions2, 0 + length_deletions1, length_deletions2 = length_deletions2, 0 + lastequality = diffs[pointer][1] + else: # An insertion or deletion. + if diffs[pointer][0] == self.DIFF_INSERT: + length_insertions2 += len(diffs[pointer][1]) + else: + length_deletions2 += len(diffs[pointer][1]) + # Eliminate an equality that is smaller or equal to the edits on both + # sides of it. + if (lastequality and (len(lastequality) <= + max(length_insertions1, length_deletions1)) and + (len(lastequality) <= max(length_insertions2, length_deletions2))): + # Duplicate record. + diffs.insert(equalities[-1], (self.DIFF_DELETE, lastequality)) + # Change second copy to insert. + diffs[equalities[-1] + 1] = (self.DIFF_INSERT, + diffs[equalities[-1] + 1][1]) + # Throw away the equality we just deleted. + equalities.pop() + # Throw away the previous equality (it needs to be reevaluated). + if len(equalities): + equalities.pop() + if len(equalities): + pointer = equalities[-1] + else: + pointer = -1 + # Reset the counters. + length_insertions1, length_deletions1 = 0, 0 + length_insertions2, length_deletions2 = 0, 0 + lastequality = None + changes = True + pointer += 1 + + # Normalize the diff. + if changes: + self.diff_cleanupMerge(diffs) + self.diff_cleanupSemanticLossless(diffs) + + # Find any overlaps between deletions and insertions. + # e.g: abcxxxxxxdef + # -> abcxxxdef + # e.g: xxxabcdefxxx + # -> defxxxabc + # Only extract an overlap if it is as big as the edit ahead or behind it. + pointer = 1 + while pointer < len(diffs): + if (diffs[pointer - 1][0] == self.DIFF_DELETE and + diffs[pointer][0] == self.DIFF_INSERT): + deletion = diffs[pointer - 1][1] + insertion = diffs[pointer][1] + overlap_length1 = self.diff_commonOverlap(deletion, insertion) + overlap_length2 = self.diff_commonOverlap(insertion, deletion) + if overlap_length1 >= overlap_length2: + if (overlap_length1 >= len(deletion) / 2.0 or + overlap_length1 >= len(insertion) / 2.0): + # Overlap found. Insert an equality and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, + insertion[:overlap_length1])) + diffs[pointer - 1] = (self.DIFF_DELETE, + deletion[:len(deletion) - overlap_length1]) + diffs[pointer + 1] = (self.DIFF_INSERT, + insertion[overlap_length1:]) + pointer += 1 + else: + if (overlap_length2 >= len(deletion) / 2.0 or + overlap_length2 >= len(insertion) / 2.0): + # Reverse overlap found. + # Insert an equality and swap and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, deletion[:overlap_length2])) + diffs[pointer - 1] = (self.DIFF_INSERT, + insertion[:len(insertion) - overlap_length2]) + diffs[pointer + 1] = (self.DIFF_DELETE, deletion[overlap_length2:]) + pointer += 1 + pointer += 1 + pointer += 1 + + def diff_cleanupSemanticLossless(self, diffs): + """Look for single edits surrounded on both sides by equalities + which can be shifted sideways to align the edit to a word boundary. + e.g: The cat came. -> The cat came. + + Args: + diffs: Array of diff tuples. + """ + + def diff_cleanupSemanticScore(one, two): + """Given two strings, compute a score representing whether the + internal boundary falls on logical boundaries. + Scores range from 6 (best) to 0 (worst). + Closure, but does not reference any external variables. + + Args: + one: First string. + two: Second string. + + Returns: + The score. + """ + if not one or not two: + # Edges are the best. + return 6 + + # Each port of this function behaves slightly differently due to + # subtle differences in each language's definition of things like + # 'whitespace'. Since this function's purpose is largely cosmetic, + # the choice has been made to use each language's native features + # rather than force total conformity. + char1 = one[-1] + char2 = two[0] + nonAlphaNumeric1 = not char1.isalpha() + nonAlphaNumeric2 = not char2.isalpha() + whitespace1 = nonAlphaNumeric1 and char1.isspace() + whitespace2 = nonAlphaNumeric2 and char2.isspace() + lineBreak1 = whitespace1 and (char1 == "\r" or char1 == "\n") + lineBreak2 = whitespace2 and (char2 == "\r" or char2 == "\n") + blankLine1 = lineBreak1 and self.BLANKLINEEND.search(one) + blankLine2 = lineBreak2 and self.BLANKLINESTART.match(two) + + if blankLine1 or blankLine2: + # Five points for blank lines. + return 5 + elif lineBreak1 or lineBreak2: + # Four points for line breaks. + return 4 + elif nonAlphaNumeric1 and not whitespace1 and whitespace2: + # Three points for end of sentences. + return 3 + elif whitespace1 or whitespace2: + # Two points for whitespace. + return 2 + elif nonAlphaNumeric1 or nonAlphaNumeric2: + # One point for non-alphanumeric. + return 1 + return 0 + + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + equality1 = diffs[pointer - 1][1] + edit = diffs[pointer][1] + equality2 = diffs[pointer + 1][1] + + # First, shift the edit as far left as possible. + commonOffset = self.diff_commonSuffix(equality1, edit) + if commonOffset: + commonString = edit[-commonOffset:] + equality1 = equality1[:-commonOffset] + edit = commonString + edit[:-commonOffset] + equality2 = commonString + equality2 + + # Second, step character by character right, looking for the best fit. + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + bestScore = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + while edit and equality2 and edit[0] == equality2[0]: + equality1 += edit[0] + edit = edit[1:] + equality2[0] + equality2 = equality2[1:] + score = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + # The >= encourages trailing rather than leading whitespace on edits. + if score >= bestScore: + bestScore = score + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + + if diffs[pointer - 1][1] != bestEquality1: + # We have an improvement, save it back to the diff. + if bestEquality1: + diffs[pointer - 1] = (diffs[pointer - 1][0], bestEquality1) + else: + del diffs[pointer - 1] + pointer -= 1 + diffs[pointer] = (diffs[pointer][0], bestEdit) + if bestEquality2: + diffs[pointer + 1] = (diffs[pointer + 1][0], bestEquality2) + else: + del diffs[pointer + 1] + pointer -= 1 + pointer += 1 + + # Define some regex patterns for matching boundaries. + BLANKLINEEND = ure.compile(r"\n\r?\n$"); + BLANKLINESTART = ure.compile(r"^\r?\n\r?\n"); + + def diff_cleanupMerge(self, diffs): + """Reorder and merge like edit sections. Merge equalities. + Any edit section can move as long as it doesn't cross an equality. + + Args: + diffs: Array of diff tuples. + """ + diffs.append((self.DIFF_EQUAL, '')) # Add a dummy entry at the end. + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete + count_insert > 1: + if count_delete != 0 and count_insert != 0: + # Factor out any common prefixies. + commonlength = self.diff_commonPrefix(text_insert, text_delete) + if commonlength != 0: + x = pointer - count_delete - count_insert - 1 + if x >= 0 and diffs[x][0] == self.DIFF_EQUAL: + diffs[x] = (diffs[x][0], diffs[x][1] + + text_insert[:commonlength]) + else: + diffs.insert(0, (self.DIFF_EQUAL, text_insert[:commonlength])) + pointer += 1 + text_insert = text_insert[commonlength:] + text_delete = text_delete[commonlength:] + # Factor out any common suffixies. + commonlength = self.diff_commonSuffix(text_insert, text_delete) + if commonlength != 0: + diffs[pointer] = (diffs[pointer][0], text_insert[-commonlength:] + + diffs[pointer][1]) + text_insert = text_insert[:-commonlength] + text_delete = text_delete[:-commonlength] + # Delete the offending records and add the merged ones. + if count_delete == 0: + diffs[pointer - count_insert : pointer] = [ + (self.DIFF_INSERT, text_insert)] + elif count_insert == 0: + diffs[pointer - count_delete : pointer] = [ + (self.DIFF_DELETE, text_delete)] + else: + diffs[pointer - count_delete - count_insert : pointer] = [ + (self.DIFF_DELETE, text_delete), + (self.DIFF_INSERT, text_insert)] + pointer = pointer - count_delete - count_insert + 1 + if count_delete != 0: + pointer += 1 + if count_insert != 0: + pointer += 1 + elif pointer != 0 and diffs[pointer - 1][0] == self.DIFF_EQUAL: + # Merge this equality with the previous one. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer][1]) + del diffs[pointer] + else: + pointer += 1 + + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + if diffs[-1][1] == '': + diffs.pop() # Remove the dummy entry at the end. + + # Second pass: look for single edits surrounded on both sides by equalities + # which can be shifted sideways to eliminate an equality. + # e.g: ABAC -> ABAC + changes = False + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + if diffs[pointer][1].endswith(diffs[pointer - 1][1]): + # Shift the edit over the previous equality. + diffs[pointer] = (diffs[pointer][0], + diffs[pointer - 1][1] + + diffs[pointer][1][:-len(diffs[pointer - 1][1])]) + diffs[pointer + 1] = (diffs[pointer + 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + del diffs[pointer - 1] + changes = True + elif diffs[pointer][1].startswith(diffs[pointer + 1][1]): + # Shift the edit over the next equality. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + diffs[pointer] = (diffs[pointer][0], + diffs[pointer][1][len(diffs[pointer + 1][1]):] + + diffs[pointer + 1][1]) + del diffs[pointer + 1] + changes = True + pointer += 1 + + # If shifts were made, the diff needs reordering and another shift sweep. + if changes: + self.diff_cleanupMerge(diffs) + + def diff_xIndex(self, diffs, loc): + """loc is a location in text1, compute and return the equivalent location + in text2. e.g. "The cat" vs "The big cat", 1->1, 5->8 + + Args: + diffs: Array of diff tuples. + loc: Location within text1. + + Returns: + Location within text2. + """ + chars1 = 0 + chars2 = 0 + last_chars1 = 0 + last_chars2 = 0 + for x in range(len(diffs)): + (op, text) = diffs[x] + if op != self.DIFF_INSERT: # Equality or deletion. + chars1 += len(text) + if op != self.DIFF_DELETE: # Equality or insertion. + chars2 += len(text) + if chars1 > loc: # Overshot the location. + break + last_chars1 = chars1 + last_chars2 = chars2 + + if len(diffs) != x and diffs[x][0] == self.DIFF_DELETE: + # The location was deleted. + return last_chars2 + # Add the remaining len(character). + return last_chars2 + (loc - last_chars1) + + def diff_text1(self, diffs): + """Compute and return the source text (all equalities and deletions). + + Args: + diffs: Array of diff tuples. + + Returns: + Source text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_INSERT: + text.append(data) + return "".join(text) + + def diff_text2(self, diffs): + """Compute and return the destination text (all equalities and insertions). + + Args: + diffs: Array of diff tuples. + + Returns: + Destination text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_DELETE: + text.append(data) + return "".join(text) + + def diff_levenshtein(self, diffs): + """Compute the Levenshtein distance; the number of inserted, deleted or + substituted characters. + + Args: + diffs: Array of diff tuples. + + Returns: + Number of changes. + """ + levenshtein = 0 + insertions = 0 + deletions = 0 + for (op, data) in diffs: + if op == self.DIFF_INSERT: + insertions += len(data) + elif op == self.DIFF_DELETE: + deletions += len(data) + elif op == self.DIFF_EQUAL: + # A deletion and an insertion is one substitution. + levenshtein += max(insertions, deletions) + insertions = 0 + deletions = 0 + levenshtein += max(insertions, deletions) + return levenshtein + + # MATCH FUNCTIONS + + def match_main(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc'. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Check for null inputs. + if text == None or pattern == None: + raise ValueError("Null inputs. (match_main)") + + loc = max(0, min(loc, len(text))) + if text == pattern: + # Shortcut (potentially not guaranteed by the algorithm) + return 0 + elif not text: + # Nothing to match. + return -1 + elif text[loc:loc + len(pattern)] == pattern: + # Perfect match at the perfect spot! (Includes case of null pattern) + return loc + else: + # Do a fuzzy compare. + match = self.match_bitap(text, pattern, loc) + return match + + def match_bitap(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc' using the + Bitap algorithm. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Python doesn't have a maxint limit, so ignore this check. + #if self.Match_MaxBits != 0 and len(pattern) > self.Match_MaxBits: + # raise ValueError("Pattern too long for this application.") + + # Initialise the alphabet. + s = self.match_alphabet(pattern) + + def match_bitapScore(e, x): + """Compute and return the score for a match with e errors and x location. + Accesses loc and pattern through being a closure. + + Args: + e: Number of errors in match. + x: Location of match. + + Returns: + Overall score for match (0.0 = good, 1.0 = bad). + """ + accuracy = float(e) / len(pattern) + proximity = abs(loc - x) + if not self.Match_Distance: + # Dodge divide by zero error. + return proximity and 1.0 or accuracy + return accuracy + (proximity / float(self.Match_Distance)) + + # Highest score beyond which we give up. + score_threshold = self.Match_Threshold + # Is there a nearby exact match? (speedup) + best_loc = text.find(pattern, loc) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + # What about in the other direction? (speedup) + best_loc = text.rfind(pattern, loc + len(pattern)) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + + # Initialise the bit arrays. + matchmask = 1 << (len(pattern) - 1) + best_loc = -1 + + bin_max = len(pattern) + len(text) + # Empty initialization added to appease pychecker. + last_rd = None + for d in range(len(pattern)): + # Scan for the best match each iteration allows for one more error. + # Run a binary search to determine how far from 'loc' we can stray at + # this error level. + bin_min = 0 + bin_mid = bin_max + while bin_min < bin_mid: + if match_bitapScore(d, loc + bin_mid) <= score_threshold: + bin_min = bin_mid + else: + bin_max = bin_mid + bin_mid = (bin_max - bin_min) // 2 + bin_min + + # Use the result from this iteration as the maximum for the next. + bin_max = bin_mid + start = max(1, loc - bin_mid + 1) + finish = min(loc + bin_mid, len(text)) + len(pattern) + + rd = [0] * (finish + 2) + rd[finish + 1] = (1 << d) - 1 + for j in range(finish, start - 1, -1): + if len(text) <= j - 1: + # Out of range. + charMatch = 0 + else: + charMatch = s.get(text[j - 1], 0) + if d == 0: # First pass: exact match. + rd[j] = ((rd[j + 1] << 1) | 1) & charMatch + else: # Subsequent passes: fuzzy match. + rd[j] = (((rd[j + 1] << 1) | 1) & charMatch) | ( + ((last_rd[j + 1] | last_rd[j]) << 1) | 1) | last_rd[j + 1] + if rd[j] & matchmask: + score = match_bitapScore(d, j - 1) + # This match will almost certainly be better than any existing match. + # But check anyway. + if score <= score_threshold: + # Told you so. + score_threshold = score + best_loc = j - 1 + if best_loc > loc: + # When passing loc, don't exceed our current distance from loc. + start = max(1, 2 * loc - best_loc) + else: + # Already passed loc, downhill from here on in. + break + # No hope for a (better) match at greater error levels. + if match_bitapScore(d + 1, loc) > score_threshold: + break + last_rd = rd + return best_loc + + def match_alphabet(self, pattern): + """Initialise the alphabet for the Bitap algorithm. + + Args: + pattern: The text to encode. + + Returns: + Hash of character locations. + """ + s = {} + for char in pattern: + s[char] = 0 + for i in range(len(pattern)): + s[pattern[i]] |= 1 << (len(pattern) - i - 1) + return s + + # PATCH FUNCTIONS + def patch_deepCopy(self, patches): + """Given an array of patches, return another array that is identical. + + Args: + patches: Array of Patch objects. + + Returns: + Array of Patch objects. + """ + patchesCopy = [] + for patch in patches: + patchCopy = patch_obj() + # No need to deep copy the tuples since they are immutable. + patchCopy.diffs = patch.diffs[:] + patchCopy.start1 = patch.start1 + patchCopy.start2 = patch.start2 + patchCopy.length1 = patch.length1 + patchCopy.length2 = patch.length2 + patchesCopy.append(patchCopy) + return patchesCopy + + def patch_apply(self, patches, text): + """Merge a set of patches onto the text. Return a patched text, as well + as a list of true/false values indicating which patches were applied. + + Args: + patches: Array of Patch objects. + text: Old text. + + Returns: + Two element Array, containing the new text and an array of boolean values. + """ + if not patches: + return (text, []) + + # Deep copy the patches so that no changes are made to originals. + patches = self.patch_deepCopy(patches) + + nullPadding = self.patch_addPadding(patches) + text = nullPadding + text + nullPadding + self.patch_splitMax(patches) + + # delta keeps track of the offset between the expected and actual location + # of the previous patch. If there are patches expected at positions 10 and + # 20, but the first patch was found at 12, delta is 2 and the second patch + # has an effective expected position of 22. + delta = 0 + results = [] + for patch in patches: + expected_loc = patch.start2 + delta + text1 = self.diff_text1(patch.diffs) + end_loc = -1 + if len(text1) > self.Match_MaxBits: + # patch_splitMax will only provide an oversized pattern in the case of + # a monster delete. + start_loc = self.match_main(text, text1[:self.Match_MaxBits], + expected_loc) + if start_loc != -1: + end_loc = self.match_main(text, text1[-self.Match_MaxBits:], + expected_loc + len(text1) - self.Match_MaxBits) + if end_loc == -1 or start_loc >= end_loc: + # Can't find valid trailing context. Drop this patch. + start_loc = -1 + else: + start_loc = self.match_main(text, text1, expected_loc) + if start_loc == -1: + # No match found. :( + results.append(False) + # Subtract the delta for this failed patch from subsequent patches. + delta -= patch.length2 - patch.length1 + else: + # Found a match. :) + results.append(True) + delta = start_loc - expected_loc + if end_loc == -1: + text2 = text[start_loc : start_loc + len(text1)] + else: + text2 = text[start_loc : end_loc + self.Match_MaxBits] + if text1 == text2: + # Perfect match, just shove the replacement text in. + text = (text[:start_loc] + self.diff_text2(patch.diffs) + + text[start_loc + len(text1):]) + else: + # Imperfect match. + # Run a diff to get a framework of equivalent indices. + diffs = self.diff_main(text1, text2, False) + if (len(text1) > self.Match_MaxBits and + self.diff_levenshtein(diffs) / float(len(text1)) > + self.Patch_DeleteThreshold): + # The end points match, but the content is unacceptably bad. + results[-1] = False + else: + self.diff_cleanupSemanticLossless(diffs) + index1 = 0 + for (op, data) in patch.diffs: + if op != self.DIFF_EQUAL: + index2 = self.diff_xIndex(diffs, index1) + if op == self.DIFF_INSERT: # Insertion + text = text[:start_loc + index2] + data + text[start_loc + + index2:] + elif op == self.DIFF_DELETE: # Deletion + text = text[:start_loc + index2] + text[start_loc + + self.diff_xIndex(diffs, index1 + len(data)):] + if op != self.DIFF_DELETE: + index1 += len(data) + # Strip the padding off. + text = text[len(nullPadding):-len(nullPadding)] + return (text, results) + + def patch_addPadding(self, patches): + """Add some padding on text start and end so that edges can match + something. Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + + Returns: + The padding string added to each side. + """ + paddingLength = self.Patch_Margin + nullPadding = "" + for x in range(1, paddingLength + 1): + nullPadding += chr(x) + + # Bump all the patches forward. + for patch in patches: + patch.start1 += paddingLength + patch.start2 += paddingLength + + # Add some padding on start of first diff. + patch = patches[0] + diffs = patch.diffs + if not diffs or diffs[0][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.insert(0, (self.DIFF_EQUAL, nullPadding)) + patch.start1 -= paddingLength # Should be 0. + patch.start2 -= paddingLength # Should be 0. + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[0][1]): + # Grow first equality. + extraLength = paddingLength - len(diffs[0][1]) + newText = nullPadding[len(diffs[0][1]):] + diffs[0][1] + diffs[0] = (diffs[0][0], newText) + patch.start1 -= extraLength + patch.start2 -= extraLength + patch.length1 += extraLength + patch.length2 += extraLength + + # Add some padding on end of last diff. + patch = patches[-1] + diffs = patch.diffs + if not diffs or diffs[-1][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.append((self.DIFF_EQUAL, nullPadding)) + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[-1][1]): + # Grow last equality. + extraLength = paddingLength - len(diffs[-1][1]) + newText = diffs[-1][1] + nullPadding[:extraLength] + diffs[-1] = (diffs[-1][0], newText) + patch.length1 += extraLength + patch.length2 += extraLength + + return nullPadding + + def patch_splitMax(self, patches): + """Look through the patches and break up any which are longer than the + maximum limit of the match algorithm. + Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + """ + patch_size = self.Match_MaxBits + if patch_size == 0: + # Python has the option of not splitting strings due to its ability + # to handle integers of arbitrary precision. + return + for x in range(len(patches)): + if patches[x].length1 <= patch_size: + continue + bigpatch = patches[x] + # Remove the big old patch. + del patches[x] + x -= 1 + start1 = bigpatch.start1 + start2 = bigpatch.start2 + precontext = '' + while len(bigpatch.diffs) != 0: + # Create one of several smaller patches. + patch = patch_obj() + empty = True + patch.start1 = start1 - len(precontext) + patch.start2 = start2 - len(precontext) + if precontext: + patch.length1 = patch.length2 = len(precontext) + patch.diffs.append((self.DIFF_EQUAL, precontext)) + + while (len(bigpatch.diffs) != 0 and + patch.length1 < patch_size - self.Patch_Margin): + (diff_type, diff_text) = bigpatch.diffs[0] + if diff_type == self.DIFF_INSERT: + # Insertions are harmless. + patch.length2 += len(diff_text) + start2 += len(diff_text) + patch.diffs.append(bigpatch.diffs.pop(0)) + empty = False + elif (diff_type == self.DIFF_DELETE and len(patch.diffs) == 1 and + patch.diffs[0][0] == self.DIFF_EQUAL and + len(diff_text) > 2 * patch_size): + # This is a large deletion. Let it pass in one chunk. + patch.length1 += len(diff_text) + start1 += len(diff_text) + empty = False + patch.diffs.append((diff_type, diff_text)) + del bigpatch.diffs[0] + else: + # Deletion or equality. Only take as much as we can stomach. + diff_text = diff_text[:patch_size - patch.length1 - + self.Patch_Margin] + patch.length1 += len(diff_text) + start1 += len(diff_text) + if diff_type == self.DIFF_EQUAL: + patch.length2 += len(diff_text) + start2 += len(diff_text) + else: + empty = False + + patch.diffs.append((diff_type, diff_text)) + if diff_text == bigpatch.diffs[0][1]: + del bigpatch.diffs[0] + else: + bigpatch.diffs[0] = (bigpatch.diffs[0][0], + bigpatch.diffs[0][1][len(diff_text):]) + + # Compute the head context for the next patch. + precontext = self.diff_text2(patch.diffs) + precontext = precontext[-self.Patch_Margin:] + # Append the end context for this patch. + postcontext = self.diff_text1(bigpatch.diffs)[:self.Patch_Margin] + if postcontext: + patch.length1 += len(postcontext) + patch.length2 += len(postcontext) + if len(patch.diffs) != 0 and patch.diffs[-1][0] == self.DIFF_EQUAL: + patch.diffs[-1] = (self.DIFF_EQUAL, patch.diffs[-1][1] + + postcontext) + else: + patch.diffs.append((self.DIFF_EQUAL, postcontext)) + + if not empty: + x += 1 + patches.insert(x, patch) + + def unquote(self, s): + """unquote('abc%20def') -> 'abc def'.""" + res = s.split('%') + # fastpath + if len(res) == 1: + return s + s = res[0] + for item in res[1:]: + try: + s += self._hextochr[item[:2]] + item[2:] + except KeyError: + s += '%' + item +# except UnicodeDecodeError: +# s += unichr(int(item[:2], 16)) + item[2:] + return s + + def patch_fromText(self, textline): + """Parse a textual representation of patches and return a list of patch + objects. + + Args: + textline: Text representation of patches. + + Returns: + Array of Patch objects. + + Raises: + ValueError: If invalid input. + """ + if type(textline) == bytes: + # Patches should be composed of a subset of ascii chars, Unicode not + # required. If this encode raises UnicodeEncodeError, patch is invalid. + textline = textline.encode("ascii") + patches = [] + if not textline: + return patches + text = textline.split('\n') + while len(text) != 0: + m = ure.match("^@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@$", text[0]) + if not m: + raise ValueError("Invalid patch string: " + text[0]) + patch = patch_obj() + patches.append(patch) + patch.start1 = int(m.group(1)) + if m.group(2) == '': + patch.start1 -= 1 + patch.length1 = 1 + elif m.group(2) == '0': + patch.length1 = 0 + else: + patch.start1 -= 1 + patch.length1 = int(m.group(2)) + + patch.start2 = int(m.group(3)) + if m.group(4) == '': + patch.start2 -= 1 + patch.length2 = 1 + elif m.group(4) == '0': + patch.length2 = 0 + else: + patch.start2 -= 1 + patch.length2 = int(m.group(4)) + + del text[0] + + while len(text) != 0: + if text[0]: + sign = text[0][0] + else: + sign = '' + line = self.unquote(text[0][1:]) + #line = line.decode("utf-8") + if sign == '+': + # Insertion. + patch.diffs.append((self.DIFF_INSERT, line)) + elif sign == '-': + # Deletion. + patch.diffs.append((self.DIFF_DELETE, line)) + elif sign == ' ': + # Minor equality. + patch.diffs.append((self.DIFF_EQUAL, line)) + elif sign == '@': + # Start of next patch. + break + elif sign == '': + # Blank line? Whatever. + pass + else: + # WTF? + raise ValueError("Invalid patch mode: '%s'\n%s" % (sign, line)) + del text[0] + return patches + + +class patch_obj: + """Class representing one patch operation. + """ + + def __init__(self): + """Initializes with an empty list of diffs. + """ + self.diffs = [] + self.start1 = None + self.start2 = None + self.length1 = 0 + self.length2 = 0 + + def __str__(self): + """Emmulate GNU diff's format. + Header: @@ -382,8 +481,9 @@ + Indicies are printed as 1-based, not 0-based. + + Returns: + The GNU diff string. + """ + if self.length1 == 0: + coords1 = str(self.start1) + ",0" + elif self.length1 == 1: + coords1 = str(self.start1 + 1) + else: + coords1 = str(self.start1 + 1) + "," + str(self.length1) + if self.length2 == 0: + coords2 = str(self.start2) + ",0" + elif self.length2 == 1: + coords2 = str(self.start2 + 1) + else: + coords2 = str(self.start2 + 1) + "," + str(self.length2) + text = ["@@ -", coords1, " +", coords2, " @@\n"] + # Escape the body of the patch with %xx notation. + for (op, data) in self.diffs: + if op == diff_match_patch.DIFF_INSERT: + text.append("+") + elif op == diff_match_patch.DIFF_DELETE: + text.append("-") + elif op == diff_match_patch.DIFF_EQUAL: + text.append(" ") + # High ascii will raise UnicodeDecodeError. Use Unicode instead. + data = data.encode("utf-8") + text.append(urllib.quote(data, "!~*'();/?:@&=+$,# ") + "\n") + return "".join(text) diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/loranet.py b/examples/OTA-lorawan/firmware/1.17.0/flash/loranet.py new file mode 100644 index 0000000..1cdc070 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/loranet.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from network import LoRa +import socket +import binascii +import struct +import time +import _thread + +class LoraNet: + def __init__(self, frequency, dr, region, device_class=LoRa.CLASS_C, activation = LoRa.OTAA, auth = None): + self.frequency = frequency + self.dr = dr + self.region = region + self.device_class = device_class + self.activation = activation + self.auth = auth + self.sock = None + self._exit = False + self.s_lock = _thread.allocate_lock() + self.lora = LoRa(mode=LoRa.LORAWAN, region = self.region, device_class = self.device_class) + + self._msg_queue = [] + self.q_lock = _thread.allocate_lock() + self._process_ota_msg = None + + def stop(self): + self._exit = True + + def init(self, process_msg_callback): + self._process_ota_msg = process_msg_callback + + def receive_callback(self, lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + rx, port = self.sock.recvfrom(256) + if rx: + if '$OTA' in rx: + print("OTA msg received: {}".format(rx)) + self._process_ota_msg(rx.decode()) + else: + self.q_lock.acquire() + self._msg_queue.append(rx) + self.q_lock.release() + + def connect(self): + if self.activation != LoRa.OTAA and self.activation != LoRa.ABP: + raise ValueError("Invalid Lora activation method") + if len(self.auth) < 3: + raise ValueError("Invalid authentication parameters") + + self.lora.callback(trigger=LoRa.RX_PACKET_EVENT, handler=self.receive_callback) + + # set the 3 default channels to the same frequency + self.lora.add_channel(0, frequency=self.frequency, dr_min=0, dr_max=5) + self.lora.add_channel(1, frequency=self.frequency, dr_min=0, dr_max=5) + self.lora.add_channel(2, frequency=self.frequency, dr_min=0, dr_max=5) + + # remove all the non-default channels + for i in range(3, 16): + self.lora.remove_channel(i) + + # authenticate with abp or ota + if self.activation == LoRa.OTAA: + self._authenticate_otaa(self.auth) + else: + self._authenticate_abp(self.auth) + + # create socket to server + self._create_socket() + + def _authenticate_otaa(self, auth_params): + + # create an OTAA authentication params + self.dev_eui = binascii.unhexlify(auth_params[0]) + self.app_eui = binascii.unhexlify(auth_params[1]) + self.app_key = binascii.unhexlify(auth_params[2]) + + self.lora.join(activation=LoRa.OTAA, auth=(self.dev_eui, self.app_eui, self.app_key), timeout=0, dr=self.dr) + + while not self.lora.has_joined(): + time.sleep(2.5) + print('Not joined yet...') + + def has_joined(self): + return self.lora.has_joined() + + def _authenticate_abp(self, auth_params): + # create an ABP authentication params + self.dev_addr = struct.unpack(">l", binascii.unhexlify(auth_params[0]))[0] + self.nwk_swkey = binascii.unhexlify(auth_params[1]) + self.app_swkey = binascii.unhexlify(auth_params[2]) + + self.lora.join(activation=LoRa.ABP, auth=(self.dev_addr, self.nwk_swkey, self.app_swkey)) + + def _create_socket(self): + + # create a LoRa socket + self.sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + + # set the LoRaWAN data rate + self.sock.setsockopt(socket.SOL_LORA, socket.SO_DR, self.dr) + + # make the socket non blocking + self.sock.setblocking(False) + + time.sleep(2) + + def send(self, packet): + with self.s_lock: + self.sock.send(packet) + + def receive(self, bufsize): + with self.q_lock: + if len(self._msg_queue) > 0: + return self._msg_queue.pop(0) + return '' + + def get_dev_eui(self): + return binascii.hexlify(self.lora.mac()).decode('ascii') + + def change_to_multicast_mode(self, mcAuth): + print('Start listening for firmware updates ...........') + + if self.device_class != LoRa.CLASS_C: + self.lora = LoRa(mode=LoRa.LORAWAN, region = self.region, device_class=LoRa.CLASS_C) + self.connect() + + mcAddr = struct.unpack(">l", binascii.unhexlify(mcAuth[0]))[0] + mcNwkKey = binascii.unhexlify(mcAuth[1]) + mcAppKey = binascii.unhexlify(mcAuth[2]) + + self.lora.join_multicast_group(mcAddr, mcNwkKey, mcAppKey) diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/main.py b/examples/OTA-lorawan/firmware/1.17.0/flash/main.py new file mode 100644 index 0000000..add8093 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/main.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from loranet import LoraNet +from ota import LoraOTA +from network import LoRa +import machine +import utime + +def main(): + LORA_FREQUENCY = 868100000 + LORA_NODE_DR = 5 + LORA_REGION = LoRa.EU868 + LORA_DEVICE_CLASS = LoRa.CLASS_C + LORA_ACTIVATION = LoRa.OTAA + LORA_CRED = ('240ac4fffe0bf998', '948c87eff87f04508f64661220f71e3f', '5e6795a5c9abba017d05a2ffef6ba858') + + lora = LoraNet(LORA_FREQUENCY, LORA_NODE_DR, LORA_REGION, LORA_DEVICE_CLASS, LORA_ACTIVATION, LORA_CRED) + lora.connect() + + ota = LoraOTA(lora) + + while True: + rx = lora.receive(256) + if rx: + print('Received user message: {}'.format(rx)) + + utime.sleep(2) + +main() + +#try: +# main() +#except Exception as e: +# print('Firmware exception: Reverting to old firmware') +# LoraOTA.revert() diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/ota.py b/examples/OTA-lorawan/firmware/1.17.0/flash/ota.py new file mode 100644 index 0000000..dd20853 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/ota.py @@ -0,0 +1,484 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import diff_match_patch as dmp_module +from watchdog import Watchdog +from machine import RTC +import ubinascii +import uhashlib +import _thread +import utime +import uos +import machine +import json + +class LoraOTA: + + MSG_HEADER = b'$OTA' + MSG_TAIL = b'*' + + FULL_UPDATE = b'F' + DIFF_UPDATE = b'D' + NO_UPDATE = b'N' + + UPDATE_INFO_MSG = 1 + UPDATE_INFO_REPLY = 2 + + MULTICAST_KEY_REQ = 3 + MULTICAST_KEY_REPLY = 4 + + LISTENING_MSG = 5 + LISTENING_REPLY = 6 + + UPDATE_TYPE_FNAME = 7 + UPDATE_TYPE_PATCH = 8 + UPDATE_TYPE_CHECKSUM = 9 + + DELETE_FILE_MSG = 10 + MANIFEST_MSG = 11 + + def __init__(self, lora): + self.lora = lora + self.is_updating = False + self.version_file = '/flash/OTA_INFO.py' + self.update_version = '0.0.0' + self.update_time = -1 + self.update_type = None + self.resp_received = False + self.update_in_progress = False + self.operation_timeout = 10 + self.max_send = 5 + self.listen_before_sec = uos.urandom(1)[0] % 180 + self.updates_check_period = 6 * 3600 + + self.mcAddr = None + self.mcNwkSKey = None + self.mcAppSKey = None + + self.patch = '' + self.file_to_patch = None + self.patch_list = dict() + self.checksum_failure = False + self.device_mainfest = None + + self._exit = False + _thread.start_new_thread(self._thread_proc, ()) + + self.inactivity_timeout = 120 + self.wdt = Watchdog() + + self.lora.init(self.process_message) + + def stop(self): + self.lora.stop() + self._exit = True + + def _thread_proc(self): + updates_check_time = utime.time() + self.device_mainfest = self.create_device_manifest() + + while not self._exit: + if utime.time() > updates_check_time and self.update_time < 0: + self.synch_request(self.check_firmware_updates) + updates_check_time = utime.time() + self.updates_check_period + + if self.update_time > 0 and not self.update_in_progress: + if self.update_time - utime.time() < self.listen_before_sec: + self.update_in_progress = True + self.updating_proc() + + if self.update_failed(): + print('Update failed: No data received') + machine.reset() + + utime.sleep(2) + + def updating_proc(self): + self.synch_request(self.get_mulitcast_keys) + + if self.mcAddr is not None: + mulitcast_auth = (self.mcAddr, self.mcNwkSKey, self.mcAppSKey) + self.lora.change_to_multicast_mode(mulitcast_auth) + + wdt_timeout = self.listen_before_sec + self.inactivity_timeout + self.wdt.enable(wdt_timeout) + + self.synch_request(self.send_listening_msg) + else: + self.reset_update_params() + + def create_device_manifest(self): + + manifest = dict() + manifest["delete"] = 0 + manifest["update"] = 0 + manifest["new"] = 0 + + return manifest + + def reset_update_params(self): + self.mcAddr = None + self.mcNwkSKey = None + self.mcAppSKey = None + + self.update_in_progress = False + self.update_time = -1 + self.update_version = '0.0.0' + + def get_mulitcast_keys(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.MULTICAST_KEY_REQ).encode()) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + + def synch_request(self, func): + attempt_num = 0 + self.resp_received = False + + while attempt_num < self.max_send and not self.resp_received: + func() + + count_10ms = 0 + while(count_10ms <= self.operation_timeout * 100 and not self.resp_received): + count_10ms += 1 + utime.sleep(0.01) + + attempt_num += 1 + + def check_firmware_updates(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.UPDATE_INFO_MSG).encode()) + + version = self.get_current_version().encode() + msg.extend(b',' + version) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + print("Lora OTA: Request for info sent") + + def get_current_version(self): + version = '0.0.0' + if self.file_exists(self.version_file): + with open(self.version_file, 'r') as fh: + version = fh.read().rstrip("\r\n\s") + else: + self._write_version_info(version) + + print("Version: {}", version) + + return version + + def send_listening_msg(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.LISTENING_MSG).encode()) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + + def _write_version_info(self, version): + try: + with open(self.version_file, 'w+') as fh: + fh.write(version) + except Exception as e: + print("Exception creating OTA version file") + + def file_exists(self, file_path): + exists = False + try: + if uos.stat(file_path)[6] > 0: + exists = True + except Exception as e: + exists = False + return exists + + def get_msg_type(self, msg): + msg_type = -1 + try: + msg_type = int(msg.split(",")[1]) + except Exception as ex: + print("Exception getting message type") + + return msg_type + + def sync_clock(self, epoc): + try: + rtc = RTC() + rtc.init(utime.gmtime(epoc)) + except Exception as ex: + print("Exception setting system data/time: {}".format(ex)) + return False + + return True + + def parse_update_info_reply(self, msg): + self.resp_received = True + + try: + token_msg = msg.split(",") + self.update_type = token_msg[3].encode() + if self.update_type in [self.FULL_UPDATE, self.DIFF_UPDATE]: + self.update_version = token_msg[2] + self.update_time = int(token_msg[4]) + + if utime.time() < 1550000000: + self.sync_clock(int(token_msg[5])) + + except Exception as ex: + print("Exception getting update information: {}".format(ex)) + return False + + return True + + def parse_multicast_keys(self, msg): + + try: + token_msg = msg.split(",") + print(token_msg) + + if len(token_msg[2]) > 0: + self.mcAddr = token_msg[2] + self.mcNwkSKey = token_msg[3] + self.mcAppSKey = token_msg[4] + + print("mcAddr: {}, mcNwkSKey: {}, mcAppSKey: {}".format(self.mcAddr, self.mcNwkSKey, self.mcAppSKey)) + + self.resp_received = True + except Exception as ex: + print("Exception getting multicast keys: {}".format(ex)) + return False + + return True + + def parse_listening_reply(self, msg): + self.resp_received = True + + def _data_start_idx(self, msg): + # Find first index + i = msg.find(",") + + #Find second index + return msg.find(",", i + 1) + + def _data_stop_idx(self, msg): + return msg.rfind(",") + + def get_msg_data(self, msg): + data = None + try: + start_idx = self._data_start_idx(msg) + 1 + stop_idx = self._data_stop_idx(msg) + data = msg[start_idx:stop_idx] + except Exception as ex: + print("Exception getting msg data: {}".format(ex)) + return data + + def process_patch_msg(self, msg): + partial_patch = self.get_msg_data(msg) + + if partial_patch: + self.patch += partial_patch + + def verify_patch(self, patch, received_checksum): + h = uhashlib.sha1() + h.update(patch) + checksum = ubinascii.hexlify(h.digest()).decode() + print("Computed checksum: {}".format(checksum)) + print("Received checksum: {}".format(received_checksum)) + + if checksum != received_checksum: + self.checksum_failure = True + return False + + return True + + def process_checksum_msg(self, msg): + checksum = self.get_msg_data(msg) + verified = self.verify_patch(self.patch, checksum) + if verified: + self.patch_list[self.file_to_patch] = self.patch + + self.file_to_patch = None + self.patch = '' + + def backup_file(self, filename): + bak_path = "{}.bak".format(filename) + + # Delete previous backup if it exists + try: + uos.remove(bak_path) + except OSError: + pass # There isnt a previous backup + + # Backup current file + uos.rename(filename, bak_path) + + def process_delete_msg(self, msg): + filename = self.get_msg_data(msg) + + if self.file_exists('/flash/' + filename): + self.backup_file('/flash/' + filename) + self.device_mainfest["delete"] += 1 + + def get_tmp_filename(self, filename): + idx = filename.rfind(".") + return filename[:idx + 1] + "tmp" + + def _read_file(self, filename): + + try: + with open('/flash/' + filename, 'r') as fh: + return fh.read() + except Exception as ex: + print("Error reading file: {}".format(ex)) + + return None + + def backup_file(self, filename): + bak_path = "{}.bak".format(filename) + + # Delete previous backup if it exists + try: + uos.remove(bak_path) + except OSError: + pass # There isnt a previous backup + + # Backup current file + uos.rename(filename, bak_path) + + def _write_to_file(self, filename, text): + tmp_file = self.get_tmp_filename('/flash/' + filename) + + try: + with open(tmp_file, 'w+') as fh: + fh.write(text) + except Exception as ex: + print("Error writing to file: {}".format(ex)) + return False + + if self.file_exists('/flash/' + filename): + self.backup_file('/flash/' + filename) + uos.rename(tmp_file, '/flash/' + filename) + + return True + + def apply_patches(self): + for key, value in self.patch_list.items(): + self.dmp = dmp_module.diff_match_patch() + self.patch_list = self.dmp.patch_fromText(value) + + to_patch = '' + print('Updating file: {}'.format(key)) + if self.update_type == self.DIFF_UPDATE and \ + self.file_exists('/flash/' + key): + to_patch = self._read_file(key) + + patched_text, success = self.dmp.patch_apply(self.patch_list, to_patch) + if False in success: + return False + + if not self._write_to_file(key, patched_text): + return False + + return True + + @staticmethod + def find_backups(): + backups = [] + for file in uos.listdir("/flash"): + if file.endswith(".bak"): + backups.append(file) + return backups + + @staticmethod + def revert(): + backup_list = LoraOTA.find_backups() + for backup in backup_list: + idx = backup.find('.bak') + new_filename = backup[:idx] + uos.rename(backup, new_filename) + print('Error: Reverting to old firmware') + machine.reset() + + def manifest_failure(self, msg): + + try: + start_idx = msg.find("{") + stop_idx = msg.find("}") + + recv_manifest = json.loads(msg[start_idx:stop_idx]) + + print("Received manifest: {}".format(recv_manifest)) + print("Actual manifest: {}".format(self.device_mainfest)) + + if (recv_manifest["update"] != self.device_mainfest["update"]) or \ + (recv_manifest["new"] != self.device_mainfest["new"]) or \ + (recv_manifest["delete"] != self.device_mainfest["delete"]): + return True + except Exception as ex: + print("Error in manifest: {}".format(ex)) + return True + + return False + + def process_manifest_msg(self, msg): + + if self.manifest_failure(msg): + print('Manifest failure: Discarding update ...') + self.reset_update_params() + if self.checksum_failure: + print('Failed checksum: Discarding update ...') + self.reset_update_params() + elif not self.apply_patches(): + LoraOTA.revert() + else: + print('Update Success: Restarting .... ') + self._write_version_info(self.update_version) + machine.reset() + + def process_filename_msg(self, msg): + self.file_to_patch = self.get_msg_data(msg) + + if self.update_type == self.DIFF_UPDATE and \ + self.file_exists('/flash/' + self.file_to_patch): + self.device_mainfest["update"] += 1 + print("Update file: {}".format(self.file_to_patch)) + else: + self.device_mainfest["new"] += 1 + print("Create new file: {}".format(self.file_to_patch)) + + self.wdt.enable(self.inactivity_timeout) + + def update_failed(self): + return self.wdt.update_failed() + + def process_message(self, msg): + self.wdt.ack() + + msg_type = self.get_msg_type(msg) + if msg_type == self.UPDATE_INFO_REPLY: + self.parse_update_info_reply(msg) + elif msg_type == self.MULTICAST_KEY_REPLY: + self.parse_multicast_keys(msg) + elif msg_type == self.LISTENING_REPLY: + self.parse_listening_reply(msg) + elif msg_type == self.UPDATE_TYPE_FNAME: + self.process_filename_msg(msg) + elif msg_type == self.UPDATE_TYPE_PATCH: + self.process_patch_msg(msg) + elif msg_type == self.UPDATE_TYPE_CHECKSUM: + self.process_checksum_msg(msg) + elif msg_type == self.DELETE_FILE_MSG: + self.process_delete_msg(msg) + elif msg_type == self.MANIFEST_MSG: + self.process_manifest_msg(msg) diff --git a/examples/OTA-lorawan/firmware/1.17.0/flash/watchdog.py b/examples/OTA-lorawan/firmware/1.17.0/flash/watchdog.py new file mode 100644 index 0000000..299c736 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.0/flash/watchdog.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from machine import Timer +import _thread + +class Watchdog: + + def __init__(self): + self.failed = False + self.acknowledged = 0 + self._alarm = None + self._lock = _thread.allocate_lock() + + def enable(self, timeout = 120): + if self._alarm: + self._alarm.cancel() + self._alarm = None + + self._alarm = Timer.Alarm(self._check, s = timeout, periodic = True) + + def _check(self, alarm): + with self._lock: + if self.acknowledged > 0: + self.failed = False + self.acknowledged = 0 + else: + self.failed = True + + def ack(self): + with self._lock: + self.acknowledged += 1 + + def update_failed(self): + with self._lock: + return self.failed diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/OTA_INFO.py b/examples/OTA-lorawan/firmware/1.17.1/flash/OTA_INFO.py new file mode 100644 index 0000000..511a76e --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/OTA_INFO.py @@ -0,0 +1 @@ +1.17.1 diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/diff_match_patch.py b/examples/OTA-lorawan/firmware/1.17.1/flash/diff_match_patch.py new file mode 100644 index 0000000..f156f86 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/diff_match_patch.py @@ -0,0 +1,1613 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +#from __future__ import division + +"""Diff Match and Patch +Copyright 2018 The diff-match-patch Authors. +https://github.com/google/diff-match-patch + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +"""Functions for diff, match and patch. + +Computes the difference between two texts to create a patch. +Applies the patch onto another text, allowing for errors. +""" + +__author__ = 'fraser@google.com (Neil Fraser)' + +import math +import ure +import sys +import time +#import urllib + +class diff_match_patch: + """Class containing the diff, match and patch methods. + + Also contains the behaviour settings. + """ + + def __init__(self): + """Inits a diff_match_patch object with default settings. + Redefine these in your program to override the defaults. + """ + + # Number of seconds to map a diff before giving up (0 for infinity). + self.Diff_Timeout = 1.0 + # Cost of an empty edit operation in terms of edit characters. + self.Diff_EditCost = 4 + # At what point is no match declared (0.0 = perfection, 1.0 = very loose). + self.Match_Threshold = 0.5 + # How far to search for a match (0 = exact location, 1000+ = broad match). + # A match this many characters away from the expected location will add + # 1.0 to the score (0.0 is a perfect match). + self.Match_Distance = 1000 + # When deleting a large block of text (over ~64 characters), how close do + # the contents have to be to match the expected contents. (0.0 = perfection, + # 1.0 = very loose). Note that Match_Threshold controls how closely the + # end points of a delete need to match. + self.Patch_DeleteThreshold = 0.5 + # Chunk size for context length. + self.Patch_Margin = 4 + + # The number of bits in an int. + # Python has no maximum, thus to disable patch splitting set to 0. + # However to avoid long patches in certain pathological cases, use 32. + # Multiple short patches (using native ints) are much faster than long ones. + self.Match_MaxBits = 32 + + self._hexdig = '0123456789ABCDEFabcdef' + self._hextochr = dict((a+b, chr(int(a+b,16))) + for a in self._hexdig for b in self._hexdig) + + # DIFF FUNCTIONS + + # The data structure representing a diff is an array of tuples: + # [(DIFF_DELETE, "Hello"), (DIFF_INSERT, "Goodbye"), (DIFF_EQUAL, " world.")] + # which means: delete "Hello", add "Goodbye" and keep " world." + DIFF_DELETE = -1 + DIFF_INSERT = 1 + DIFF_EQUAL = 0 + + def diff_main(self, text1, text2, checklines=True, deadline=None): + """Find the differences between two texts. Simplifies the problem by + stripping any common prefix or suffix off the texts before diffing. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Optional speedup flag. If present and false, then don't run + a line-level diff first to identify the changed areas. + Defaults to true, which does a faster, slightly less optimal diff. + deadline: Optional time when the diff should be complete by. Used + internally for recursive calls. Users should set DiffTimeout instead. + + Returns: + Array of changes. + """ + # Set a deadline by which time the diff must be complete. + if deadline == None: + # Unlike in most languages, Python counts time in seconds. + if self.Diff_Timeout <= 0: + deadline = sys.maxsize + else: + deadline = time.time() + self.Diff_Timeout + + # Check for null inputs. + if text1 == None or text2 == None: + raise ValueError("Null inputs. (diff_main)") + + # Check for equality (speedup). + if text1 == text2: + if text1: + return [(self.DIFF_EQUAL, text1)] + return [] + + # Trim off common prefix (speedup). + commonlength = self.diff_commonPrefix(text1, text2) + commonprefix = text1[:commonlength] + text1 = text1[commonlength:] + text2 = text2[commonlength:] + + # Trim off common suffix (speedup). + commonlength = self.diff_commonSuffix(text1, text2) + if commonlength == 0: + commonsuffix = '' + else: + commonsuffix = text1[-commonlength:] + text1 = text1[:-commonlength] + text2 = text2[:-commonlength] + + # Compute the diff on the middle block. + diffs = self.diff_compute(text1, text2, checklines, deadline) + + # Restore the prefix and suffix. + if commonprefix: + diffs[:0] = [(self.DIFF_EQUAL, commonprefix)] + if commonsuffix: + diffs.append((self.DIFF_EQUAL, commonsuffix)) + self.diff_cleanupMerge(diffs) + return diffs + + def diff_compute(self, text1, text2, checklines, deadline): + """Find the differences between two texts. Assumes that the texts do not + have any common prefix or suffix. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + checklines: Speedup flag. If false, then don't run a line-level diff + first to identify the changed areas. + If true, then run a faster, slightly less optimal diff. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + if not text1: + # Just add some text (speedup). + return [(self.DIFF_INSERT, text2)] + + if not text2: + # Just delete some text (speedup). + return [(self.DIFF_DELETE, text1)] + + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + i = longtext.find(shorttext) + if i != -1: + # Shorter text is inside the longer text (speedup). + diffs = [(self.DIFF_INSERT, longtext[:i]), (self.DIFF_EQUAL, shorttext), + (self.DIFF_INSERT, longtext[i + len(shorttext):])] + # Swap insertions for deletions if diff is reversed. + if len(text1) > len(text2): + diffs[0] = (self.DIFF_DELETE, diffs[0][1]) + diffs[2] = (self.DIFF_DELETE, diffs[2][1]) + return diffs + + if len(shorttext) == 1: + # Single character string. + # After the previous speedup, the character can't be an equality. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + # Check to see if the problem can be split in two. + hm = self.diff_halfMatch(text1, text2) + if hm: + # A half-match was found, sort out the return data. + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + # Send both pairs off for separate processing. + diffs_a = self.diff_main(text1_a, text2_a, checklines, deadline) + diffs_b = self.diff_main(text1_b, text2_b, checklines, deadline) + # Merge the results. + return diffs_a + [(self.DIFF_EQUAL, mid_common)] + diffs_b + + if checklines and len(text1) > 100 and len(text2) > 100: + return self.diff_lineMode(text1, text2, deadline) + + return self.diff_bisect(text1, text2, deadline) + + def diff_lineMode(self, text1, text2, deadline): + """Do a quick line-level diff on both strings, then rediff the parts for + greater accuracy. + This speedup can produce non-minimal diffs. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time when the diff should be complete by. + + Returns: + Array of changes. + """ + + # Scan the text on a line-by-line basis first. + (text1, text2, linearray) = self.diff_linesToChars(text1, text2) + + diffs = self.diff_main(text1, text2, False, deadline) + + # Convert the diff back to original text. + self.diff_charsToLines(diffs, linearray) + # Eliminate freak matches (e.g. blank lines) + self.diff_cleanupSemantic(diffs) + + # Rediff any replacement blocks, this time character-by-character. + # Add a dummy entry at the end. + diffs.append((self.DIFF_EQUAL, '')) + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete >= 1 and count_insert >= 1: + # Delete the offending records and add the merged ones. + a = self.diff_main(text_delete, text_insert, False, deadline) + diffs[pointer - count_delete - count_insert : pointer] = a + pointer = pointer - count_delete - count_insert + len(a) + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + pointer += 1 + + diffs.pop() # Remove the dummy entry at the end. + + return diffs + + def diff_bisect(self, text1, text2, deadline): + """Find the 'middle snake' of a diff, split the problem in two + and return the recursively constructed diff. + See Myers 1986 paper: An O(ND) Difference Algorithm and Its Variations. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + max_d = (text1_length + text2_length + 1) // 2 + v_offset = max_d + v_length = 2 * max_d + v1 = [-1] * v_length + v1[v_offset + 1] = 0 + v2 = v1[:] + delta = text1_length - text2_length + # If the total number of characters is odd, then the front path will + # collide with the reverse path. + front = (delta % 2 != 0) + # Offsets for start and end of k loop. + # Prevents mapping of space beyond the grid. + k1start = 0 + k1end = 0 + k2start = 0 + k2end = 0 + for d in range(max_d): + # Bail out if deadline is reached. + if time.time() > deadline: + break + + # Walk the front path one step. + for k1 in range(-d + k1start, d + 1 - k1end, 2): + k1_offset = v_offset + k1 + if k1 == -d or (k1 != d and + v1[k1_offset - 1] < v1[k1_offset + 1]): + x1 = v1[k1_offset + 1] + else: + x1 = v1[k1_offset - 1] + 1 + y1 = x1 - k1 + while (x1 < text1_length and y1 < text2_length and + text1[x1] == text2[y1]): + x1 += 1 + y1 += 1 + v1[k1_offset] = x1 + if x1 > text1_length: + # Ran off the right of the graph. + k1end += 2 + elif y1 > text2_length: + # Ran off the bottom of the graph. + k1start += 2 + elif front: + k2_offset = v_offset + delta - k1 + if k2_offset >= 0 and k2_offset < v_length and v2[k2_offset] != -1: + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - v2[k2_offset] + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Walk the reverse path one step. + for k2 in range(-d + k2start, d + 1 - k2end, 2): + k2_offset = v_offset + k2 + if k2 == -d or (k2 != d and + v2[k2_offset - 1] < v2[k2_offset + 1]): + x2 = v2[k2_offset + 1] + else: + x2 = v2[k2_offset - 1] + 1 + y2 = x2 - k2 + while (x2 < text1_length and y2 < text2_length and + text1[-x2 - 1] == text2[-y2 - 1]): + x2 += 1 + y2 += 1 + v2[k2_offset] = x2 + if x2 > text1_length: + # Ran off the left of the graph. + k2end += 2 + elif y2 > text2_length: + # Ran off the top of the graph. + k2start += 2 + elif not front: + k1_offset = v_offset + delta - k2 + if k1_offset >= 0 and k1_offset < v_length and v1[k1_offset] != -1: + x1 = v1[k1_offset] + y1 = v_offset + x1 - k1_offset + # Mirror x2 onto top-left coordinate system. + x2 = text1_length - x2 + if x1 >= x2: + # Overlap detected. + return self.diff_bisectSplit(text1, text2, x1, y1, deadline) + + # Diff took too long and hit the deadline or + # number of diffs equals number of characters, no commonality at all. + return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)] + + def diff_bisectSplit(self, text1, text2, x, y, deadline): + """Given the location of the 'middle snake', split the diff in two parts + and recurse. + + Args: + text1: Old string to be diffed. + text2: New string to be diffed. + x: Index of split point in text1. + y: Index of split point in text2. + deadline: Time at which to bail if not yet complete. + + Returns: + Array of diff tuples. + """ + text1a = text1[:x] + text2a = text2[:y] + text1b = text1[x:] + text2b = text2[y:] + + # Compute both diffs serially. + diffs = self.diff_main(text1a, text2a, False, deadline) + diffsb = self.diff_main(text1b, text2b, False, deadline) + + return diffs + diffsb + + def diff_linesToChars(self, text1, text2): + """Split two texts into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + + Args: + text1: First string. + text2: Second string. + + Returns: + Three element tuple, containing the encoded text1, the encoded text2 and + the array of unique strings. The zeroth element of the array of unique + strings is intentionally blank. + """ + lineArray = [] # e.g. lineArray[4] == "Hello\n" + lineHash = {} # e.g. lineHash["Hello\n"] == 4 + + # "\x00" is a valid character, but various debuggers don't like it. + # So we'll insert a junk entry to avoid generating a null character. + lineArray.append('') + + def diff_linesToCharsMunge(text): + """Split a text into an array of strings. Reduce the texts to a string + of hashes where each Unicode character represents one line. + Modifies linearray and linehash through being a closure. + + Args: + text: String to encode. + + Returns: + Encoded string. + """ + chars = [] + # Walk the text, pulling out a substring for each line. + # text.split('\n') would would temporarily double our memory footprint. + # Modifying text would create many large strings to garbage collect. + lineStart = 0 + lineEnd = -1 + while lineEnd < len(text) - 1: + lineEnd = text.find('\n', lineStart) + if lineEnd == -1: + lineEnd = len(text) - 1 + line = text[lineStart:lineEnd + 1] + + if line in lineHash: + chars.append(chr(lineHash[line])) + else: + if len(lineArray) == maxLines: + # Bail out at 65535 because unichr(65536) throws. - not in micropython + line = text[lineStart:] + lineEnd = len(text) + lineArray.append(line) + lineHash[line] = len(lineArray) - 1 + chars.append(chr(len(lineArray) - 1)) + lineStart = lineEnd + 1 + return "".join(chars) + + # Allocate 2/3rds of the space for text1, the rest for text2. + maxLines = 40000 + chars1 = diff_linesToCharsMunge(text1) + maxLines = 65535 + chars2 = diff_linesToCharsMunge(text2) + return (chars1, chars2, lineArray) + + def diff_charsToLines(self, diffs, lineArray): + """Rehydrate the text in a diff from a string of line hashes to real lines + of text. + + Args: + diffs: Array of diff tuples. + lineArray: Array of unique strings. + """ + for x in range(len(diffs)): + text = [] + for char in diffs[x][1]: + text.append(lineArray[ord(char)]) + diffs[x] = (diffs[x][0], "".join(text)) + + def diff_commonPrefix(self, text1, text2): + """Determine the common prefix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the start of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[0] != text2[0]: + return 0 + # Binary search. + # Performance analysis: http://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerstart = 0 + while pointermin < pointermid: + if text1[pointerstart:pointermid] == text2[pointerstart:pointermid]: + pointermin = pointermid + pointerstart = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonSuffix(self, text1, text2): + """Determine the common suffix of two strings. + + Args: + text1: First string. + text2: Second string. + + Returns: + The number of characters common to the end of each string. + """ + # Quick check for common null cases. + if not text1 or not text2 or text1[-1] != text2[-1]: + return 0 + # Binary search. + # Performance analysis: http://neil.fraser.name/news/2007/10/09/ + pointermin = 0 + pointermax = min(len(text1), len(text2)) + pointermid = pointermax + pointerend = 0 + while pointermin < pointermid: + if (text1[-pointermid:len(text1) - pointerend] == + text2[-pointermid:len(text2) - pointerend]): + pointermin = pointermid + pointerend = pointermin + else: + pointermax = pointermid + pointermid = (pointermax - pointermin) // 2 + pointermin + return pointermid + + def diff_commonOverlap(self, text1, text2): + """Determine if the suffix of one string is the prefix of another. + + Args: + text1 First string. + text2 Second string. + + Returns: + The number of characters common to the end of the first + string and the start of the second string. + """ + # Cache the text lengths to prevent multiple calls. + text1_length = len(text1) + text2_length = len(text2) + # Eliminate the null case. + if text1_length == 0 or text2_length == 0: + return 0 + # Truncate the longer string. + if text1_length > text2_length: + text1 = text1[-text2_length:] + elif text1_length < text2_length: + text2 = text2[:text1_length] + text_length = min(text1_length, text2_length) + # Quick check for the worst case. + if text1 == text2: + return text_length + + # Start by looking for a single character match + # and increase length until no match is found. + # Performance analysis: http://neil.fraser.name/news/2010/11/04/ + best = 0 + length = 1 + while True: + pattern = text1[-length:] + found = text2.find(pattern) + if found == -1: + return best + length += found + if found == 0 or text1[-length:] == text2[:length]: + best = length + length += 1 + + def diff_halfMatch(self, text1, text2): + """Do the two texts share a substring which is at least half the length of + the longer text? + This speedup can produce non-minimal diffs. + + Args: + text1: First string. + text2: Second string. + + Returns: + Five element Array, containing the prefix of text1, the suffix of text1, + the prefix of text2, the suffix of text2 and the common middle. Or None + if there was no match. + """ + if self.Diff_Timeout <= 0: + # Don't risk returning a non-optimal diff if we have unlimited time. + return None + if len(text1) > len(text2): + (longtext, shorttext) = (text1, text2) + else: + (shorttext, longtext) = (text1, text2) + if len(longtext) < 4 or len(shorttext) * 2 < len(longtext): + return None # Pointless. + + def diff_halfMatchI(longtext, shorttext, i): + """Does a substring of shorttext exist within longtext such that the + substring is at least half the length of longtext? + Closure, but does not reference any external variables. + + Args: + longtext: Longer string. + shorttext: Shorter string. + i: Start index of quarter length substring within longtext. + + Returns: + Five element Array, containing the prefix of longtext, the suffix of + longtext, the prefix of shorttext, the suffix of shorttext and the + common middle. Or None if there was no match. + """ + seed = longtext[i:i + len(longtext) // 4] + best_common = '' + j = shorttext.find(seed) + while j != -1: + prefixLength = self.diff_commonPrefix(longtext[i:], shorttext[j:]) + suffixLength = self.diff_commonSuffix(longtext[:i], shorttext[:j]) + if len(best_common) < suffixLength + prefixLength: + best_common = (shorttext[j - suffixLength:j] + + shorttext[j:j + prefixLength]) + best_longtext_a = longtext[:i - suffixLength] + best_longtext_b = longtext[i + prefixLength:] + best_shorttext_a = shorttext[:j - suffixLength] + best_shorttext_b = shorttext[j + prefixLength:] + j = shorttext.find(seed, j + 1) + + if len(best_common) * 2 >= len(longtext): + return (best_longtext_a, best_longtext_b, + best_shorttext_a, best_shorttext_b, best_common) + else: + return None + + # First check if the second quarter is the seed for a half-match. + hm1 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 3) // 4) + # Check again based on the third quarter. + hm2 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 1) // 2) + if not hm1 and not hm2: + return None + elif not hm2: + hm = hm1 + elif not hm1: + hm = hm2 + else: + # Both matched. Select the longest. + if len(hm1[4]) > len(hm2[4]): + hm = hm1 + else: + hm = hm2 + + # A half-match was found, sort out the return data. + if len(text1) > len(text2): + (text1_a, text1_b, text2_a, text2_b, mid_common) = hm + else: + (text2_a, text2_b, text1_a, text1_b, mid_common) = hm + return (text1_a, text1_b, text2_a, text2_b, mid_common) + + def diff_cleanupSemantic(self, diffs): + """Reduce the number of edits by eliminating semantically trivial + equalities. + + Args: + diffs: Array of diff tuples. + """ + changes = False + equalities = [] # Stack of indices where equalities are found. + lastequality = None # Always equal to diffs[equalities[-1]][1] + pointer = 0 # Index of current position. + # Number of chars that changed prior to the equality. + length_insertions1, length_deletions1 = 0, 0 + # Number of chars that changed after the equality. + length_insertions2, length_deletions2 = 0, 0 + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_EQUAL: # Equality found. + equalities.append(pointer) + length_insertions1, length_insertions2 = length_insertions2, 0 + length_deletions1, length_deletions2 = length_deletions2, 0 + lastequality = diffs[pointer][1] + else: # An insertion or deletion. + if diffs[pointer][0] == self.DIFF_INSERT: + length_insertions2 += len(diffs[pointer][1]) + else: + length_deletions2 += len(diffs[pointer][1]) + # Eliminate an equality that is smaller or equal to the edits on both + # sides of it. + if (lastequality and (len(lastequality) <= + max(length_insertions1, length_deletions1)) and + (len(lastequality) <= max(length_insertions2, length_deletions2))): + # Duplicate record. + diffs.insert(equalities[-1], (self.DIFF_DELETE, lastequality)) + # Change second copy to insert. + diffs[equalities[-1] + 1] = (self.DIFF_INSERT, + diffs[equalities[-1] + 1][1]) + # Throw away the equality we just deleted. + equalities.pop() + # Throw away the previous equality (it needs to be reevaluated). + if len(equalities): + equalities.pop() + if len(equalities): + pointer = equalities[-1] + else: + pointer = -1 + # Reset the counters. + length_insertions1, length_deletions1 = 0, 0 + length_insertions2, length_deletions2 = 0, 0 + lastequality = None + changes = True + pointer += 1 + + # Normalize the diff. + if changes: + self.diff_cleanupMerge(diffs) + self.diff_cleanupSemanticLossless(diffs) + + # Find any overlaps between deletions and insertions. + # e.g: abcxxxxxxdef + # -> abcxxxdef + # e.g: xxxabcdefxxx + # -> defxxxabc + # Only extract an overlap if it is as big as the edit ahead or behind it. + pointer = 1 + while pointer < len(diffs): + if (diffs[pointer - 1][0] == self.DIFF_DELETE and + diffs[pointer][0] == self.DIFF_INSERT): + deletion = diffs[pointer - 1][1] + insertion = diffs[pointer][1] + overlap_length1 = self.diff_commonOverlap(deletion, insertion) + overlap_length2 = self.diff_commonOverlap(insertion, deletion) + if overlap_length1 >= overlap_length2: + if (overlap_length1 >= len(deletion) / 2.0 or + overlap_length1 >= len(insertion) / 2.0): + # Overlap found. Insert an equality and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, + insertion[:overlap_length1])) + diffs[pointer - 1] = (self.DIFF_DELETE, + deletion[:len(deletion) - overlap_length1]) + diffs[pointer + 1] = (self.DIFF_INSERT, + insertion[overlap_length1:]) + pointer += 1 + else: + if (overlap_length2 >= len(deletion) / 2.0 or + overlap_length2 >= len(insertion) / 2.0): + # Reverse overlap found. + # Insert an equality and swap and trim the surrounding edits. + diffs.insert(pointer, (self.DIFF_EQUAL, deletion[:overlap_length2])) + diffs[pointer - 1] = (self.DIFF_INSERT, + insertion[:len(insertion) - overlap_length2]) + diffs[pointer + 1] = (self.DIFF_DELETE, deletion[overlap_length2:]) + pointer += 1 + pointer += 1 + pointer += 1 + + def diff_cleanupSemanticLossless(self, diffs): + """Look for single edits surrounded on both sides by equalities + which can be shifted sideways to align the edit to a word boundary. + e.g: The cat came. -> The cat came. + + Args: + diffs: Array of diff tuples. + """ + + def diff_cleanupSemanticScore(one, two): + """Given two strings, compute a score representing whether the + internal boundary falls on logical boundaries. + Scores range from 6 (best) to 0 (worst). + Closure, but does not reference any external variables. + + Args: + one: First string. + two: Second string. + + Returns: + The score. + """ + if not one or not two: + # Edges are the best. + return 6 + + # Each port of this function behaves slightly differently due to + # subtle differences in each language's definition of things like + # 'whitespace'. Since this function's purpose is largely cosmetic, + # the choice has been made to use each language's native features + # rather than force total conformity. + char1 = one[-1] + char2 = two[0] + nonAlphaNumeric1 = not char1.isalpha() + nonAlphaNumeric2 = not char2.isalpha() + whitespace1 = nonAlphaNumeric1 and char1.isspace() + whitespace2 = nonAlphaNumeric2 and char2.isspace() + lineBreak1 = whitespace1 and (char1 == "\r" or char1 == "\n") + lineBreak2 = whitespace2 and (char2 == "\r" or char2 == "\n") + blankLine1 = lineBreak1 and self.BLANKLINEEND.search(one) + blankLine2 = lineBreak2 and self.BLANKLINESTART.match(two) + + if blankLine1 or blankLine2: + # Five points for blank lines. + return 5 + elif lineBreak1 or lineBreak2: + # Four points for line breaks. + return 4 + elif nonAlphaNumeric1 and not whitespace1 and whitespace2: + # Three points for end of sentences. + return 3 + elif whitespace1 or whitespace2: + # Two points for whitespace. + return 2 + elif nonAlphaNumeric1 or nonAlphaNumeric2: + # One point for non-alphanumeric. + return 1 + return 0 + + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + equality1 = diffs[pointer - 1][1] + edit = diffs[pointer][1] + equality2 = diffs[pointer + 1][1] + + # First, shift the edit as far left as possible. + commonOffset = self.diff_commonSuffix(equality1, edit) + if commonOffset: + commonString = edit[-commonOffset:] + equality1 = equality1[:-commonOffset] + edit = commonString + edit[:-commonOffset] + equality2 = commonString + equality2 + + # Second, step character by character right, looking for the best fit. + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + bestScore = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + while edit and equality2 and edit[0] == equality2[0]: + equality1 += edit[0] + edit = edit[1:] + equality2[0] + equality2 = equality2[1:] + score = (diff_cleanupSemanticScore(equality1, edit) + + diff_cleanupSemanticScore(edit, equality2)) + # The >= encourages trailing rather than leading whitespace on edits. + if score >= bestScore: + bestScore = score + bestEquality1 = equality1 + bestEdit = edit + bestEquality2 = equality2 + + if diffs[pointer - 1][1] != bestEquality1: + # We have an improvement, save it back to the diff. + if bestEquality1: + diffs[pointer - 1] = (diffs[pointer - 1][0], bestEquality1) + else: + del diffs[pointer - 1] + pointer -= 1 + diffs[pointer] = (diffs[pointer][0], bestEdit) + if bestEquality2: + diffs[pointer + 1] = (diffs[pointer + 1][0], bestEquality2) + else: + del diffs[pointer + 1] + pointer -= 1 + pointer += 1 + + # Define some regex patterns for matching boundaries. + BLANKLINEEND = ure.compile(r"\n\r?\n$"); + BLANKLINESTART = ure.compile(r"^\r?\n\r?\n"); + + def diff_cleanupMerge(self, diffs): + """Reorder and merge like edit sections. Merge equalities. + Any edit section can move as long as it doesn't cross an equality. + + Args: + diffs: Array of diff tuples. + """ + diffs.append((self.DIFF_EQUAL, '')) # Add a dummy entry at the end. + pointer = 0 + count_delete = 0 + count_insert = 0 + text_delete = '' + text_insert = '' + while pointer < len(diffs): + if diffs[pointer][0] == self.DIFF_INSERT: + count_insert += 1 + text_insert += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_DELETE: + count_delete += 1 + text_delete += diffs[pointer][1] + pointer += 1 + elif diffs[pointer][0] == self.DIFF_EQUAL: + # Upon reaching an equality, check for prior redundancies. + if count_delete + count_insert > 1: + if count_delete != 0 and count_insert != 0: + # Factor out any common prefixies. + commonlength = self.diff_commonPrefix(text_insert, text_delete) + if commonlength != 0: + x = pointer - count_delete - count_insert - 1 + if x >= 0 and diffs[x][0] == self.DIFF_EQUAL: + diffs[x] = (diffs[x][0], diffs[x][1] + + text_insert[:commonlength]) + else: + diffs.insert(0, (self.DIFF_EQUAL, text_insert[:commonlength])) + pointer += 1 + text_insert = text_insert[commonlength:] + text_delete = text_delete[commonlength:] + # Factor out any common suffixies. + commonlength = self.diff_commonSuffix(text_insert, text_delete) + if commonlength != 0: + diffs[pointer] = (diffs[pointer][0], text_insert[-commonlength:] + + diffs[pointer][1]) + text_insert = text_insert[:-commonlength] + text_delete = text_delete[:-commonlength] + # Delete the offending records and add the merged ones. + if count_delete == 0: + diffs[pointer - count_insert : pointer] = [ + (self.DIFF_INSERT, text_insert)] + elif count_insert == 0: + diffs[pointer - count_delete : pointer] = [ + (self.DIFF_DELETE, text_delete)] + else: + diffs[pointer - count_delete - count_insert : pointer] = [ + (self.DIFF_DELETE, text_delete), + (self.DIFF_INSERT, text_insert)] + pointer = pointer - count_delete - count_insert + 1 + if count_delete != 0: + pointer += 1 + if count_insert != 0: + pointer += 1 + elif pointer != 0 and diffs[pointer - 1][0] == self.DIFF_EQUAL: + # Merge this equality with the previous one. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer][1]) + del diffs[pointer] + else: + pointer += 1 + + count_insert = 0 + count_delete = 0 + text_delete = '' + text_insert = '' + + if diffs[-1][1] == '': + diffs.pop() # Remove the dummy entry at the end. + + # Second pass: look for single edits surrounded on both sides by equalities + # which can be shifted sideways to eliminate an equality. + # e.g: ABAC -> ABAC + changes = False + pointer = 1 + # Intentionally ignore the first and last element (don't need checking). + while pointer < len(diffs) - 1: + if (diffs[pointer - 1][0] == self.DIFF_EQUAL and + diffs[pointer + 1][0] == self.DIFF_EQUAL): + # This is a single edit surrounded by equalities. + if diffs[pointer][1].endswith(diffs[pointer - 1][1]): + # Shift the edit over the previous equality. + diffs[pointer] = (diffs[pointer][0], + diffs[pointer - 1][1] + + diffs[pointer][1][:-len(diffs[pointer - 1][1])]) + diffs[pointer + 1] = (diffs[pointer + 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + del diffs[pointer - 1] + changes = True + elif diffs[pointer][1].startswith(diffs[pointer + 1][1]): + # Shift the edit over the next equality. + diffs[pointer - 1] = (diffs[pointer - 1][0], + diffs[pointer - 1][1] + diffs[pointer + 1][1]) + diffs[pointer] = (diffs[pointer][0], + diffs[pointer][1][len(diffs[pointer + 1][1]):] + + diffs[pointer + 1][1]) + del diffs[pointer + 1] + changes = True + pointer += 1 + + # If shifts were made, the diff needs reordering and another shift sweep. + if changes: + self.diff_cleanupMerge(diffs) + + def diff_xIndex(self, diffs, loc): + """loc is a location in text1, compute and return the equivalent location + in text2. e.g. "The cat" vs "The big cat", 1->1, 5->8 + + Args: + diffs: Array of diff tuples. + loc: Location within text1. + + Returns: + Location within text2. + """ + chars1 = 0 + chars2 = 0 + last_chars1 = 0 + last_chars2 = 0 + for x in range(len(diffs)): + (op, text) = diffs[x] + if op != self.DIFF_INSERT: # Equality or deletion. + chars1 += len(text) + if op != self.DIFF_DELETE: # Equality or insertion. + chars2 += len(text) + if chars1 > loc: # Overshot the location. + break + last_chars1 = chars1 + last_chars2 = chars2 + + if len(diffs) != x and diffs[x][0] == self.DIFF_DELETE: + # The location was deleted. + return last_chars2 + # Add the remaining len(character). + return last_chars2 + (loc - last_chars1) + + def diff_text1(self, diffs): + """Compute and return the source text (all equalities and deletions). + + Args: + diffs: Array of diff tuples. + + Returns: + Source text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_INSERT: + text.append(data) + return "".join(text) + + def diff_text2(self, diffs): + """Compute and return the destination text (all equalities and insertions). + + Args: + diffs: Array of diff tuples. + + Returns: + Destination text. + """ + text = [] + for (op, data) in diffs: + if op != self.DIFF_DELETE: + text.append(data) + return "".join(text) + + def diff_levenshtein(self, diffs): + """Compute the Levenshtein distance; the number of inserted, deleted or + substituted characters. + + Args: + diffs: Array of diff tuples. + + Returns: + Number of changes. + """ + levenshtein = 0 + insertions = 0 + deletions = 0 + for (op, data) in diffs: + if op == self.DIFF_INSERT: + insertions += len(data) + elif op == self.DIFF_DELETE: + deletions += len(data) + elif op == self.DIFF_EQUAL: + # A deletion and an insertion is one substitution. + levenshtein += max(insertions, deletions) + insertions = 0 + deletions = 0 + levenshtein += max(insertions, deletions) + return levenshtein + + # MATCH FUNCTIONS + + def match_main(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc'. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Check for null inputs. + if text == None or pattern == None: + raise ValueError("Null inputs. (match_main)") + + loc = max(0, min(loc, len(text))) + if text == pattern: + # Shortcut (potentially not guaranteed by the algorithm) + return 0 + elif not text: + # Nothing to match. + return -1 + elif text[loc:loc + len(pattern)] == pattern: + # Perfect match at the perfect spot! (Includes case of null pattern) + return loc + else: + # Do a fuzzy compare. + match = self.match_bitap(text, pattern, loc) + return match + + def match_bitap(self, text, pattern, loc): + """Locate the best instance of 'pattern' in 'text' near 'loc' using the + Bitap algorithm. + + Args: + text: The text to search. + pattern: The pattern to search for. + loc: The location to search around. + + Returns: + Best match index or -1. + """ + # Python doesn't have a maxint limit, so ignore this check. + #if self.Match_MaxBits != 0 and len(pattern) > self.Match_MaxBits: + # raise ValueError("Pattern too long for this application.") + + # Initialise the alphabet. + s = self.match_alphabet(pattern) + + def match_bitapScore(e, x): + """Compute and return the score for a match with e errors and x location. + Accesses loc and pattern through being a closure. + + Args: + e: Number of errors in match. + x: Location of match. + + Returns: + Overall score for match (0.0 = good, 1.0 = bad). + """ + accuracy = float(e) / len(pattern) + proximity = abs(loc - x) + if not self.Match_Distance: + # Dodge divide by zero error. + return proximity and 1.0 or accuracy + return accuracy + (proximity / float(self.Match_Distance)) + + # Highest score beyond which we give up. + score_threshold = self.Match_Threshold + # Is there a nearby exact match? (speedup) + best_loc = text.find(pattern, loc) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + # What about in the other direction? (speedup) + best_loc = text.rfind(pattern, loc + len(pattern)) + if best_loc != -1: + score_threshold = min(match_bitapScore(0, best_loc), score_threshold) + + # Initialise the bit arrays. + matchmask = 1 << (len(pattern) - 1) + best_loc = -1 + + bin_max = len(pattern) + len(text) + # Empty initialization added to appease pychecker. + last_rd = None + for d in range(len(pattern)): + # Scan for the best match each iteration allows for one more error. + # Run a binary search to determine how far from 'loc' we can stray at + # this error level. + bin_min = 0 + bin_mid = bin_max + while bin_min < bin_mid: + if match_bitapScore(d, loc + bin_mid) <= score_threshold: + bin_min = bin_mid + else: + bin_max = bin_mid + bin_mid = (bin_max - bin_min) // 2 + bin_min + + # Use the result from this iteration as the maximum for the next. + bin_max = bin_mid + start = max(1, loc - bin_mid + 1) + finish = min(loc + bin_mid, len(text)) + len(pattern) + + rd = [0] * (finish + 2) + rd[finish + 1] = (1 << d) - 1 + for j in range(finish, start - 1, -1): + if len(text) <= j - 1: + # Out of range. + charMatch = 0 + else: + charMatch = s.get(text[j - 1], 0) + if d == 0: # First pass: exact match. + rd[j] = ((rd[j + 1] << 1) | 1) & charMatch + else: # Subsequent passes: fuzzy match. + rd[j] = (((rd[j + 1] << 1) | 1) & charMatch) | ( + ((last_rd[j + 1] | last_rd[j]) << 1) | 1) | last_rd[j + 1] + if rd[j] & matchmask: + score = match_bitapScore(d, j - 1) + # This match will almost certainly be better than any existing match. + # But check anyway. + if score <= score_threshold: + # Told you so. + score_threshold = score + best_loc = j - 1 + if best_loc > loc: + # When passing loc, don't exceed our current distance from loc. + start = max(1, 2 * loc - best_loc) + else: + # Already passed loc, downhill from here on in. + break + # No hope for a (better) match at greater error levels. + if match_bitapScore(d + 1, loc) > score_threshold: + break + last_rd = rd + return best_loc + + def match_alphabet(self, pattern): + """Initialise the alphabet for the Bitap algorithm. + + Args: + pattern: The text to encode. + + Returns: + Hash of character locations. + """ + s = {} + for char in pattern: + s[char] = 0 + for i in range(len(pattern)): + s[pattern[i]] |= 1 << (len(pattern) - i - 1) + return s + + # PATCH FUNCTIONS + def patch_deepCopy(self, patches): + """Given an array of patches, return another array that is identical. + + Args: + patches: Array of Patch objects. + + Returns: + Array of Patch objects. + """ + patchesCopy = [] + for patch in patches: + patchCopy = patch_obj() + # No need to deep copy the tuples since they are immutable. + patchCopy.diffs = patch.diffs[:] + patchCopy.start1 = patch.start1 + patchCopy.start2 = patch.start2 + patchCopy.length1 = patch.length1 + patchCopy.length2 = patch.length2 + patchesCopy.append(patchCopy) + return patchesCopy + + def patch_apply(self, patches, text): + """Merge a set of patches onto the text. Return a patched text, as well + as a list of true/false values indicating which patches were applied. + + Args: + patches: Array of Patch objects. + text: Old text. + + Returns: + Two element Array, containing the new text and an array of boolean values. + """ + if not patches: + return (text, []) + + # Deep copy the patches so that no changes are made to originals. + patches = self.patch_deepCopy(patches) + + nullPadding = self.patch_addPadding(patches) + text = nullPadding + text + nullPadding + self.patch_splitMax(patches) + + # delta keeps track of the offset between the expected and actual location + # of the previous patch. If there are patches expected at positions 10 and + # 20, but the first patch was found at 12, delta is 2 and the second patch + # has an effective expected position of 22. + delta = 0 + results = [] + for patch in patches: + expected_loc = patch.start2 + delta + text1 = self.diff_text1(patch.diffs) + end_loc = -1 + if len(text1) > self.Match_MaxBits: + # patch_splitMax will only provide an oversized pattern in the case of + # a monster delete. + start_loc = self.match_main(text, text1[:self.Match_MaxBits], + expected_loc) + if start_loc != -1: + end_loc = self.match_main(text, text1[-self.Match_MaxBits:], + expected_loc + len(text1) - self.Match_MaxBits) + if end_loc == -1 or start_loc >= end_loc: + # Can't find valid trailing context. Drop this patch. + start_loc = -1 + else: + start_loc = self.match_main(text, text1, expected_loc) + if start_loc == -1: + # No match found. :( + results.append(False) + # Subtract the delta for this failed patch from subsequent patches. + delta -= patch.length2 - patch.length1 + else: + # Found a match. :) + results.append(True) + delta = start_loc - expected_loc + if end_loc == -1: + text2 = text[start_loc : start_loc + len(text1)] + else: + text2 = text[start_loc : end_loc + self.Match_MaxBits] + if text1 == text2: + # Perfect match, just shove the replacement text in. + text = (text[:start_loc] + self.diff_text2(patch.diffs) + + text[start_loc + len(text1):]) + else: + # Imperfect match. + # Run a diff to get a framework of equivalent indices. + diffs = self.diff_main(text1, text2, False) + if (len(text1) > self.Match_MaxBits and + self.diff_levenshtein(diffs) / float(len(text1)) > + self.Patch_DeleteThreshold): + # The end points match, but the content is unacceptably bad. + results[-1] = False + else: + self.diff_cleanupSemanticLossless(diffs) + index1 = 0 + for (op, data) in patch.diffs: + if op != self.DIFF_EQUAL: + index2 = self.diff_xIndex(diffs, index1) + if op == self.DIFF_INSERT: # Insertion + text = text[:start_loc + index2] + data + text[start_loc + + index2:] + elif op == self.DIFF_DELETE: # Deletion + text = text[:start_loc + index2] + text[start_loc + + self.diff_xIndex(diffs, index1 + len(data)):] + if op != self.DIFF_DELETE: + index1 += len(data) + # Strip the padding off. + text = text[len(nullPadding):-len(nullPadding)] + return (text, results) + + def patch_addPadding(self, patches): + """Add some padding on text start and end so that edges can match + something. Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + + Returns: + The padding string added to each side. + """ + paddingLength = self.Patch_Margin + nullPadding = "" + for x in range(1, paddingLength + 1): + nullPadding += chr(x) + + # Bump all the patches forward. + for patch in patches: + patch.start1 += paddingLength + patch.start2 += paddingLength + + # Add some padding on start of first diff. + patch = patches[0] + diffs = patch.diffs + if not diffs or diffs[0][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.insert(0, (self.DIFF_EQUAL, nullPadding)) + patch.start1 -= paddingLength # Should be 0. + patch.start2 -= paddingLength # Should be 0. + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[0][1]): + # Grow first equality. + extraLength = paddingLength - len(diffs[0][1]) + newText = nullPadding[len(diffs[0][1]):] + diffs[0][1] + diffs[0] = (diffs[0][0], newText) + patch.start1 -= extraLength + patch.start2 -= extraLength + patch.length1 += extraLength + patch.length2 += extraLength + + # Add some padding on end of last diff. + patch = patches[-1] + diffs = patch.diffs + if not diffs or diffs[-1][0] != self.DIFF_EQUAL: + # Add nullPadding equality. + diffs.append((self.DIFF_EQUAL, nullPadding)) + patch.length1 += paddingLength + patch.length2 += paddingLength + elif paddingLength > len(diffs[-1][1]): + # Grow last equality. + extraLength = paddingLength - len(diffs[-1][1]) + newText = diffs[-1][1] + nullPadding[:extraLength] + diffs[-1] = (diffs[-1][0], newText) + patch.length1 += extraLength + patch.length2 += extraLength + + return nullPadding + + def patch_splitMax(self, patches): + """Look through the patches and break up any which are longer than the + maximum limit of the match algorithm. + Intended to be called only from within patch_apply. + + Args: + patches: Array of Patch objects. + """ + patch_size = self.Match_MaxBits + if patch_size == 0: + # Python has the option of not splitting strings due to its ability + # to handle integers of arbitrary precision. + return + for x in range(len(patches)): + if patches[x].length1 <= patch_size: + continue + bigpatch = patches[x] + # Remove the big old patch. + del patches[x] + x -= 1 + start1 = bigpatch.start1 + start2 = bigpatch.start2 + precontext = '' + while len(bigpatch.diffs) != 0: + # Create one of several smaller patches. + patch = patch_obj() + empty = True + patch.start1 = start1 - len(precontext) + patch.start2 = start2 - len(precontext) + if precontext: + patch.length1 = patch.length2 = len(precontext) + patch.diffs.append((self.DIFF_EQUAL, precontext)) + + while (len(bigpatch.diffs) != 0 and + patch.length1 < patch_size - self.Patch_Margin): + (diff_type, diff_text) = bigpatch.diffs[0] + if diff_type == self.DIFF_INSERT: + # Insertions are harmless. + patch.length2 += len(diff_text) + start2 += len(diff_text) + patch.diffs.append(bigpatch.diffs.pop(0)) + empty = False + elif (diff_type == self.DIFF_DELETE and len(patch.diffs) == 1 and + patch.diffs[0][0] == self.DIFF_EQUAL and + len(diff_text) > 2 * patch_size): + # This is a large deletion. Let it pass in one chunk. + patch.length1 += len(diff_text) + start1 += len(diff_text) + empty = False + patch.diffs.append((diff_type, diff_text)) + del bigpatch.diffs[0] + else: + # Deletion or equality. Only take as much as we can stomach. + diff_text = diff_text[:patch_size - patch.length1 - + self.Patch_Margin] + patch.length1 += len(diff_text) + start1 += len(diff_text) + if diff_type == self.DIFF_EQUAL: + patch.length2 += len(diff_text) + start2 += len(diff_text) + else: + empty = False + + patch.diffs.append((diff_type, diff_text)) + if diff_text == bigpatch.diffs[0][1]: + del bigpatch.diffs[0] + else: + bigpatch.diffs[0] = (bigpatch.diffs[0][0], + bigpatch.diffs[0][1][len(diff_text):]) + + # Compute the head context for the next patch. + precontext = self.diff_text2(patch.diffs) + precontext = precontext[-self.Patch_Margin:] + # Append the end context for this patch. + postcontext = self.diff_text1(bigpatch.diffs)[:self.Patch_Margin] + if postcontext: + patch.length1 += len(postcontext) + patch.length2 += len(postcontext) + if len(patch.diffs) != 0 and patch.diffs[-1][0] == self.DIFF_EQUAL: + patch.diffs[-1] = (self.DIFF_EQUAL, patch.diffs[-1][1] + + postcontext) + else: + patch.diffs.append((self.DIFF_EQUAL, postcontext)) + + if not empty: + x += 1 + patches.insert(x, patch) + + def unquote(self, s): + """unquote('abc%20def') -> 'abc def'.""" + res = s.split('%') + # fastpath + if len(res) == 1: + return s + s = res[0] + for item in res[1:]: + try: + s += self._hextochr[item[:2]] + item[2:] + except KeyError: + s += '%' + item +# except UnicodeDecodeError: +# s += unichr(int(item[:2], 16)) + item[2:] + return s + + def patch_fromText(self, textline): + """Parse a textual representation of patches and return a list of patch + objects. + + Args: + textline: Text representation of patches. + + Returns: + Array of Patch objects. + + Raises: + ValueError: If invalid input. + """ + if type(textline) == bytes: + # Patches should be composed of a subset of ascii chars, Unicode not + # required. If this encode raises UnicodeEncodeError, patch is invalid. + textline = textline.encode("ascii") + patches = [] + if not textline: + return patches + text = textline.split('\n') + while len(text) != 0: + m = ure.match("^@@ -(\d+),?(\d*) \+(\d+),?(\d*) @@$", text[0]) + if not m: + raise ValueError("Invalid patch string: " + text[0]) + patch = patch_obj() + patches.append(patch) + patch.start1 = int(m.group(1)) + if m.group(2) == '': + patch.start1 -= 1 + patch.length1 = 1 + elif m.group(2) == '0': + patch.length1 = 0 + else: + patch.start1 -= 1 + patch.length1 = int(m.group(2)) + + patch.start2 = int(m.group(3)) + if m.group(4) == '': + patch.start2 -= 1 + patch.length2 = 1 + elif m.group(4) == '0': + patch.length2 = 0 + else: + patch.start2 -= 1 + patch.length2 = int(m.group(4)) + + del text[0] + + while len(text) != 0: + if text[0]: + sign = text[0][0] + else: + sign = '' + line = self.unquote(text[0][1:]) + #line = line.decode("utf-8") + if sign == '+': + # Insertion. + patch.diffs.append((self.DIFF_INSERT, line)) + elif sign == '-': + # Deletion. + patch.diffs.append((self.DIFF_DELETE, line)) + elif sign == ' ': + # Minor equality. + patch.diffs.append((self.DIFF_EQUAL, line)) + elif sign == '@': + # Start of next patch. + break + elif sign == '': + # Blank line? Whatever. + pass + else: + # WTF? + raise ValueError("Invalid patch mode: '%s'\n%s" % (sign, line)) + del text[0] + return patches + + +class patch_obj: + """Class representing one patch operation. + """ + + def __init__(self): + """Initializes with an empty list of diffs. + """ + self.diffs = [] + self.start1 = None + self.start2 = None + self.length1 = 0 + self.length2 = 0 + + def __str__(self): + """Emmulate GNU diff's format. + Header: @@ -382,8 +481,9 @@ + Indicies are printed as 1-based, not 0-based. + + Returns: + The GNU diff string. + """ + if self.length1 == 0: + coords1 = str(self.start1) + ",0" + elif self.length1 == 1: + coords1 = str(self.start1 + 1) + else: + coords1 = str(self.start1 + 1) + "," + str(self.length1) + if self.length2 == 0: + coords2 = str(self.start2) + ",0" + elif self.length2 == 1: + coords2 = str(self.start2 + 1) + else: + coords2 = str(self.start2 + 1) + "," + str(self.length2) + text = ["@@ -", coords1, " +", coords2, " @@\n"] + # Escape the body of the patch with %xx notation. + for (op, data) in self.diffs: + if op == diff_match_patch.DIFF_INSERT: + text.append("+") + elif op == diff_match_patch.DIFF_DELETE: + text.append("-") + elif op == diff_match_patch.DIFF_EQUAL: + text.append(" ") + # High ascii will raise UnicodeDecodeError. Use Unicode instead. + data = data.encode("utf-8") + text.append(urllib.quote(data, "!~*'();/?:@&=+$,# ") + "\n") + return "".join(text) diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/loranet.py b/examples/OTA-lorawan/firmware/1.17.1/flash/loranet.py new file mode 100644 index 0000000..1cdc070 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/loranet.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from network import LoRa +import socket +import binascii +import struct +import time +import _thread + +class LoraNet: + def __init__(self, frequency, dr, region, device_class=LoRa.CLASS_C, activation = LoRa.OTAA, auth = None): + self.frequency = frequency + self.dr = dr + self.region = region + self.device_class = device_class + self.activation = activation + self.auth = auth + self.sock = None + self._exit = False + self.s_lock = _thread.allocate_lock() + self.lora = LoRa(mode=LoRa.LORAWAN, region = self.region, device_class = self.device_class) + + self._msg_queue = [] + self.q_lock = _thread.allocate_lock() + self._process_ota_msg = None + + def stop(self): + self._exit = True + + def init(self, process_msg_callback): + self._process_ota_msg = process_msg_callback + + def receive_callback(self, lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + rx, port = self.sock.recvfrom(256) + if rx: + if '$OTA' in rx: + print("OTA msg received: {}".format(rx)) + self._process_ota_msg(rx.decode()) + else: + self.q_lock.acquire() + self._msg_queue.append(rx) + self.q_lock.release() + + def connect(self): + if self.activation != LoRa.OTAA and self.activation != LoRa.ABP: + raise ValueError("Invalid Lora activation method") + if len(self.auth) < 3: + raise ValueError("Invalid authentication parameters") + + self.lora.callback(trigger=LoRa.RX_PACKET_EVENT, handler=self.receive_callback) + + # set the 3 default channels to the same frequency + self.lora.add_channel(0, frequency=self.frequency, dr_min=0, dr_max=5) + self.lora.add_channel(1, frequency=self.frequency, dr_min=0, dr_max=5) + self.lora.add_channel(2, frequency=self.frequency, dr_min=0, dr_max=5) + + # remove all the non-default channels + for i in range(3, 16): + self.lora.remove_channel(i) + + # authenticate with abp or ota + if self.activation == LoRa.OTAA: + self._authenticate_otaa(self.auth) + else: + self._authenticate_abp(self.auth) + + # create socket to server + self._create_socket() + + def _authenticate_otaa(self, auth_params): + + # create an OTAA authentication params + self.dev_eui = binascii.unhexlify(auth_params[0]) + self.app_eui = binascii.unhexlify(auth_params[1]) + self.app_key = binascii.unhexlify(auth_params[2]) + + self.lora.join(activation=LoRa.OTAA, auth=(self.dev_eui, self.app_eui, self.app_key), timeout=0, dr=self.dr) + + while not self.lora.has_joined(): + time.sleep(2.5) + print('Not joined yet...') + + def has_joined(self): + return self.lora.has_joined() + + def _authenticate_abp(self, auth_params): + # create an ABP authentication params + self.dev_addr = struct.unpack(">l", binascii.unhexlify(auth_params[0]))[0] + self.nwk_swkey = binascii.unhexlify(auth_params[1]) + self.app_swkey = binascii.unhexlify(auth_params[2]) + + self.lora.join(activation=LoRa.ABP, auth=(self.dev_addr, self.nwk_swkey, self.app_swkey)) + + def _create_socket(self): + + # create a LoRa socket + self.sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + + # set the LoRaWAN data rate + self.sock.setsockopt(socket.SOL_LORA, socket.SO_DR, self.dr) + + # make the socket non blocking + self.sock.setblocking(False) + + time.sleep(2) + + def send(self, packet): + with self.s_lock: + self.sock.send(packet) + + def receive(self, bufsize): + with self.q_lock: + if len(self._msg_queue) > 0: + return self._msg_queue.pop(0) + return '' + + def get_dev_eui(self): + return binascii.hexlify(self.lora.mac()).decode('ascii') + + def change_to_multicast_mode(self, mcAuth): + print('Start listening for firmware updates ...........') + + if self.device_class != LoRa.CLASS_C: + self.lora = LoRa(mode=LoRa.LORAWAN, region = self.region, device_class=LoRa.CLASS_C) + self.connect() + + mcAddr = struct.unpack(">l", binascii.unhexlify(mcAuth[0]))[0] + mcNwkKey = binascii.unhexlify(mcAuth[1]) + mcAppKey = binascii.unhexlify(mcAuth[2]) + + self.lora.join_multicast_group(mcAddr, mcNwkKey, mcAppKey) diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/main.py b/examples/OTA-lorawan/firmware/1.17.1/flash/main.py new file mode 100644 index 0000000..3908377 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/main.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from loranet import LoraNet +from ota import LoraOTA +from network import LoRa +import machine +import utime + +def main(): + print('Booting with firmware version 1.17.1') + + LORA_FREQUENCY = 868100000 + LORA_NODE_DR = 5 + LORA_REGION = LoRa.EU868 + LORA_DEVICE_CLASS = LoRa.CLASS_C + LORA_ACTIVATION = LoRa.OTAA + LORA_CRED = ('240ac4fffe0bf998', '948c87eff87f04508f64661220f71e3f', '5e6795a5c9abba017d05a2ffef6ba858') + + lora = LoraNet(LORA_FREQUENCY, LORA_NODE_DR, LORA_REGION, LORA_DEVICE_CLASS, LORA_ACTIVATION, LORA_CRED) + lora.connect() + + ota = LoraOTA(lora) + + while True: + rx = lora.receive(256) + if rx: + print('Received user message: {}'.format(rx)) + + utime.sleep(2) + +main() + +#try: +# main() +#except Exception as e: +# print('Firmware exception: Reverting to old firmware') +# LoraOTA.revert() diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/ota.py b/examples/OTA-lorawan/firmware/1.17.1/flash/ota.py new file mode 100644 index 0000000..dd20853 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/ota.py @@ -0,0 +1,484 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import diff_match_patch as dmp_module +from watchdog import Watchdog +from machine import RTC +import ubinascii +import uhashlib +import _thread +import utime +import uos +import machine +import json + +class LoraOTA: + + MSG_HEADER = b'$OTA' + MSG_TAIL = b'*' + + FULL_UPDATE = b'F' + DIFF_UPDATE = b'D' + NO_UPDATE = b'N' + + UPDATE_INFO_MSG = 1 + UPDATE_INFO_REPLY = 2 + + MULTICAST_KEY_REQ = 3 + MULTICAST_KEY_REPLY = 4 + + LISTENING_MSG = 5 + LISTENING_REPLY = 6 + + UPDATE_TYPE_FNAME = 7 + UPDATE_TYPE_PATCH = 8 + UPDATE_TYPE_CHECKSUM = 9 + + DELETE_FILE_MSG = 10 + MANIFEST_MSG = 11 + + def __init__(self, lora): + self.lora = lora + self.is_updating = False + self.version_file = '/flash/OTA_INFO.py' + self.update_version = '0.0.0' + self.update_time = -1 + self.update_type = None + self.resp_received = False + self.update_in_progress = False + self.operation_timeout = 10 + self.max_send = 5 + self.listen_before_sec = uos.urandom(1)[0] % 180 + self.updates_check_period = 6 * 3600 + + self.mcAddr = None + self.mcNwkSKey = None + self.mcAppSKey = None + + self.patch = '' + self.file_to_patch = None + self.patch_list = dict() + self.checksum_failure = False + self.device_mainfest = None + + self._exit = False + _thread.start_new_thread(self._thread_proc, ()) + + self.inactivity_timeout = 120 + self.wdt = Watchdog() + + self.lora.init(self.process_message) + + def stop(self): + self.lora.stop() + self._exit = True + + def _thread_proc(self): + updates_check_time = utime.time() + self.device_mainfest = self.create_device_manifest() + + while not self._exit: + if utime.time() > updates_check_time and self.update_time < 0: + self.synch_request(self.check_firmware_updates) + updates_check_time = utime.time() + self.updates_check_period + + if self.update_time > 0 and not self.update_in_progress: + if self.update_time - utime.time() < self.listen_before_sec: + self.update_in_progress = True + self.updating_proc() + + if self.update_failed(): + print('Update failed: No data received') + machine.reset() + + utime.sleep(2) + + def updating_proc(self): + self.synch_request(self.get_mulitcast_keys) + + if self.mcAddr is not None: + mulitcast_auth = (self.mcAddr, self.mcNwkSKey, self.mcAppSKey) + self.lora.change_to_multicast_mode(mulitcast_auth) + + wdt_timeout = self.listen_before_sec + self.inactivity_timeout + self.wdt.enable(wdt_timeout) + + self.synch_request(self.send_listening_msg) + else: + self.reset_update_params() + + def create_device_manifest(self): + + manifest = dict() + manifest["delete"] = 0 + manifest["update"] = 0 + manifest["new"] = 0 + + return manifest + + def reset_update_params(self): + self.mcAddr = None + self.mcNwkSKey = None + self.mcAppSKey = None + + self.update_in_progress = False + self.update_time = -1 + self.update_version = '0.0.0' + + def get_mulitcast_keys(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.MULTICAST_KEY_REQ).encode()) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + + def synch_request(self, func): + attempt_num = 0 + self.resp_received = False + + while attempt_num < self.max_send and not self.resp_received: + func() + + count_10ms = 0 + while(count_10ms <= self.operation_timeout * 100 and not self.resp_received): + count_10ms += 1 + utime.sleep(0.01) + + attempt_num += 1 + + def check_firmware_updates(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.UPDATE_INFO_MSG).encode()) + + version = self.get_current_version().encode() + msg.extend(b',' + version) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + print("Lora OTA: Request for info sent") + + def get_current_version(self): + version = '0.0.0' + if self.file_exists(self.version_file): + with open(self.version_file, 'r') as fh: + version = fh.read().rstrip("\r\n\s") + else: + self._write_version_info(version) + + print("Version: {}", version) + + return version + + def send_listening_msg(self): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.LISTENING_MSG).encode()) + msg.extend(b',' + self.MSG_TAIL) + + self.lora.send(msg) + + def _write_version_info(self, version): + try: + with open(self.version_file, 'w+') as fh: + fh.write(version) + except Exception as e: + print("Exception creating OTA version file") + + def file_exists(self, file_path): + exists = False + try: + if uos.stat(file_path)[6] > 0: + exists = True + except Exception as e: + exists = False + return exists + + def get_msg_type(self, msg): + msg_type = -1 + try: + msg_type = int(msg.split(",")[1]) + except Exception as ex: + print("Exception getting message type") + + return msg_type + + def sync_clock(self, epoc): + try: + rtc = RTC() + rtc.init(utime.gmtime(epoc)) + except Exception as ex: + print("Exception setting system data/time: {}".format(ex)) + return False + + return True + + def parse_update_info_reply(self, msg): + self.resp_received = True + + try: + token_msg = msg.split(",") + self.update_type = token_msg[3].encode() + if self.update_type in [self.FULL_UPDATE, self.DIFF_UPDATE]: + self.update_version = token_msg[2] + self.update_time = int(token_msg[4]) + + if utime.time() < 1550000000: + self.sync_clock(int(token_msg[5])) + + except Exception as ex: + print("Exception getting update information: {}".format(ex)) + return False + + return True + + def parse_multicast_keys(self, msg): + + try: + token_msg = msg.split(",") + print(token_msg) + + if len(token_msg[2]) > 0: + self.mcAddr = token_msg[2] + self.mcNwkSKey = token_msg[3] + self.mcAppSKey = token_msg[4] + + print("mcAddr: {}, mcNwkSKey: {}, mcAppSKey: {}".format(self.mcAddr, self.mcNwkSKey, self.mcAppSKey)) + + self.resp_received = True + except Exception as ex: + print("Exception getting multicast keys: {}".format(ex)) + return False + + return True + + def parse_listening_reply(self, msg): + self.resp_received = True + + def _data_start_idx(self, msg): + # Find first index + i = msg.find(",") + + #Find second index + return msg.find(",", i + 1) + + def _data_stop_idx(self, msg): + return msg.rfind(",") + + def get_msg_data(self, msg): + data = None + try: + start_idx = self._data_start_idx(msg) + 1 + stop_idx = self._data_stop_idx(msg) + data = msg[start_idx:stop_idx] + except Exception as ex: + print("Exception getting msg data: {}".format(ex)) + return data + + def process_patch_msg(self, msg): + partial_patch = self.get_msg_data(msg) + + if partial_patch: + self.patch += partial_patch + + def verify_patch(self, patch, received_checksum): + h = uhashlib.sha1() + h.update(patch) + checksum = ubinascii.hexlify(h.digest()).decode() + print("Computed checksum: {}".format(checksum)) + print("Received checksum: {}".format(received_checksum)) + + if checksum != received_checksum: + self.checksum_failure = True + return False + + return True + + def process_checksum_msg(self, msg): + checksum = self.get_msg_data(msg) + verified = self.verify_patch(self.patch, checksum) + if verified: + self.patch_list[self.file_to_patch] = self.patch + + self.file_to_patch = None + self.patch = '' + + def backup_file(self, filename): + bak_path = "{}.bak".format(filename) + + # Delete previous backup if it exists + try: + uos.remove(bak_path) + except OSError: + pass # There isnt a previous backup + + # Backup current file + uos.rename(filename, bak_path) + + def process_delete_msg(self, msg): + filename = self.get_msg_data(msg) + + if self.file_exists('/flash/' + filename): + self.backup_file('/flash/' + filename) + self.device_mainfest["delete"] += 1 + + def get_tmp_filename(self, filename): + idx = filename.rfind(".") + return filename[:idx + 1] + "tmp" + + def _read_file(self, filename): + + try: + with open('/flash/' + filename, 'r') as fh: + return fh.read() + except Exception as ex: + print("Error reading file: {}".format(ex)) + + return None + + def backup_file(self, filename): + bak_path = "{}.bak".format(filename) + + # Delete previous backup if it exists + try: + uos.remove(bak_path) + except OSError: + pass # There isnt a previous backup + + # Backup current file + uos.rename(filename, bak_path) + + def _write_to_file(self, filename, text): + tmp_file = self.get_tmp_filename('/flash/' + filename) + + try: + with open(tmp_file, 'w+') as fh: + fh.write(text) + except Exception as ex: + print("Error writing to file: {}".format(ex)) + return False + + if self.file_exists('/flash/' + filename): + self.backup_file('/flash/' + filename) + uos.rename(tmp_file, '/flash/' + filename) + + return True + + def apply_patches(self): + for key, value in self.patch_list.items(): + self.dmp = dmp_module.diff_match_patch() + self.patch_list = self.dmp.patch_fromText(value) + + to_patch = '' + print('Updating file: {}'.format(key)) + if self.update_type == self.DIFF_UPDATE and \ + self.file_exists('/flash/' + key): + to_patch = self._read_file(key) + + patched_text, success = self.dmp.patch_apply(self.patch_list, to_patch) + if False in success: + return False + + if not self._write_to_file(key, patched_text): + return False + + return True + + @staticmethod + def find_backups(): + backups = [] + for file in uos.listdir("/flash"): + if file.endswith(".bak"): + backups.append(file) + return backups + + @staticmethod + def revert(): + backup_list = LoraOTA.find_backups() + for backup in backup_list: + idx = backup.find('.bak') + new_filename = backup[:idx] + uos.rename(backup, new_filename) + print('Error: Reverting to old firmware') + machine.reset() + + def manifest_failure(self, msg): + + try: + start_idx = msg.find("{") + stop_idx = msg.find("}") + + recv_manifest = json.loads(msg[start_idx:stop_idx]) + + print("Received manifest: {}".format(recv_manifest)) + print("Actual manifest: {}".format(self.device_mainfest)) + + if (recv_manifest["update"] != self.device_mainfest["update"]) or \ + (recv_manifest["new"] != self.device_mainfest["new"]) or \ + (recv_manifest["delete"] != self.device_mainfest["delete"]): + return True + except Exception as ex: + print("Error in manifest: {}".format(ex)) + return True + + return False + + def process_manifest_msg(self, msg): + + if self.manifest_failure(msg): + print('Manifest failure: Discarding update ...') + self.reset_update_params() + if self.checksum_failure: + print('Failed checksum: Discarding update ...') + self.reset_update_params() + elif not self.apply_patches(): + LoraOTA.revert() + else: + print('Update Success: Restarting .... ') + self._write_version_info(self.update_version) + machine.reset() + + def process_filename_msg(self, msg): + self.file_to_patch = self.get_msg_data(msg) + + if self.update_type == self.DIFF_UPDATE and \ + self.file_exists('/flash/' + self.file_to_patch): + self.device_mainfest["update"] += 1 + print("Update file: {}".format(self.file_to_patch)) + else: + self.device_mainfest["new"] += 1 + print("Create new file: {}".format(self.file_to_patch)) + + self.wdt.enable(self.inactivity_timeout) + + def update_failed(self): + return self.wdt.update_failed() + + def process_message(self, msg): + self.wdt.ack() + + msg_type = self.get_msg_type(msg) + if msg_type == self.UPDATE_INFO_REPLY: + self.parse_update_info_reply(msg) + elif msg_type == self.MULTICAST_KEY_REPLY: + self.parse_multicast_keys(msg) + elif msg_type == self.LISTENING_REPLY: + self.parse_listening_reply(msg) + elif msg_type == self.UPDATE_TYPE_FNAME: + self.process_filename_msg(msg) + elif msg_type == self.UPDATE_TYPE_PATCH: + self.process_patch_msg(msg) + elif msg_type == self.UPDATE_TYPE_CHECKSUM: + self.process_checksum_msg(msg) + elif msg_type == self.DELETE_FILE_MSG: + self.process_delete_msg(msg) + elif msg_type == self.MANIFEST_MSG: + self.process_manifest_msg(msg) diff --git a/examples/OTA-lorawan/firmware/1.17.1/flash/watchdog.py b/examples/OTA-lorawan/firmware/1.17.1/flash/watchdog.py new file mode 100644 index 0000000..299c736 --- /dev/null +++ b/examples/OTA-lorawan/firmware/1.17.1/flash/watchdog.py @@ -0,0 +1,43 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from machine import Timer +import _thread + +class Watchdog: + + def __init__(self): + self.failed = False + self.acknowledged = 0 + self._alarm = None + self._lock = _thread.allocate_lock() + + def enable(self, timeout = 120): + if self._alarm: + self._alarm.cancel() + self._alarm = None + + self._alarm = Timer.Alarm(self._check, s = timeout, periodic = True) + + def _check(self, alarm): + with self._lock: + if self.acknowledged > 0: + self.failed = False + self.acknowledged = 0 + else: + self.failed = True + + def ack(self): + with self._lock: + self.acknowledged += 1 + + def update_failed(self): + with self._lock: + return self.failed diff --git a/examples/OTA-lorawan/groupUpdater.py b/examples/OTA-lorawan/groupUpdater.py new file mode 100644 index 0000000..98bca0a --- /dev/null +++ b/examples/OTA-lorawan/groupUpdater.py @@ -0,0 +1,272 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import diff_match_patch as dmp_module +import threading +import hashlib +import binascii +import filecmp +import json +import time +import os + +class updateHandler: + + def __init__(self, dev_version, latest_version, clientApp, jwt, multicast_id, ota_obj): + self.tag = dev_version + ',' + latest_version + self.oper_dict = None + self.patch_dict = None + self.dev_version = dev_version + self.latest_version = latest_version + self._clientApp = clientApp + self.ota =ota_obj + self.max_send = 5 + + self._loraserver_jwt = jwt + self._multicast_group_id = multicast_id + + self._binary_ext = [] + + self._m_th = threading.Thread(target=self._multicast_proc) + self._m_th.start() + + def print_file_operations(self, oper_dict): + if 'delete_txt' in oper_dict: + print('Delete text: {}'.format(oper_dict['delete_txt'])) + if 'delete_bin' in oper_dict: + print('Delete bin: {}'.format(oper_dict['delete_bin'])) + if 'new_txt' in oper_dict: + print('New text: {}'.format(oper_dict['new_txt'])) + if 'new_bin' in oper_dict: + print('New binary: {}'.format(oper_dict['new_bin'])) + if 'update_txt' in oper_dict: + print('Update {}'.format(oper_dict['update_txt'])) + + def _create_manifest(self, oper_dict): + manifest = {"delete":0, "update":0, "new":0} + for key, value in oper_dict.items(): + if 'delete_txt' in key: + manifest['delete'] = len(value) + elif 'new_txt' in key: + manifest['new'] = len(value) + elif 'update_txt' in key: + manifest['update'] = len(value) + + return json.dumps(manifest) + + def get_all_paths(self, path, ignore=[]): + ignore = set(ignore) + paths = [] + for entry in os.walk(path): + d, _, files = entry + files = set(files).difference(ignore) + paths += [os.path.join(d, f) for f in files] + out = [d.replace('{}{}'.format(path, os.path.sep), '') for d in paths] + return set(out) + + def text_binary_lists(self, all_delete, all_new): + path_dict = dict() + + delete_dict = self.text_binary_separation(all_delete, 'delete') + path_dict.update(delete_dict) + + new_dict = self.text_binary_separation(all_new, 'new') + path_dict.update(new_dict) + + return path_dict + + def text_binary_separation(self, paths, key): + path_dict = dict() + + for path in paths: + filename, extension = os.path.splitext(path) + if extension in self._binary_ext: + if key + '_bin' not in path_dict: + path_dict[key + '_bin'] = [] + path_dict[key + '_bin'].append(path) + else: + if key + '_txt' not in path_dict: + path_dict[key + '_txt'] = [] + path_dict[key + '_txt'].append(path) + + return path_dict + + def get_diff_list(self, left, right, ignore=['.DS_Store', 'pymakr.conf']): + left_paths = self.get_all_paths(left, ignore=ignore) + right_paths = self.get_all_paths(right, ignore=ignore) + new_files = right_paths.difference(left_paths) + to_delete = left_paths.difference(right_paths) + common = left_paths.intersection(right_paths) + + paths_dict = self.text_binary_lists(to_delete, new_files) + + for f in common: + if not filecmp.cmp(os.path.join(left, f), + os.path.join(right, f), shallow=False): + filename, extension = os.path.splitext(f) + if extension in self._binary_ext: + # No diff update for binary files + if 'new_bin' not in paths_dict: + paths_dict['update_txt'] = [] + paths_dict['new_bin'].append(f) + else: + if 'update_txt' not in paths_dict: + paths_dict['update_txt'] = [] + paths_dict['update_txt'].append(f) + + return paths_dict + + def _read_firware_file(self, filename): + text = '' + try: + text = open(filename).read() + except Exception as e: + pass + return text + + def _create_hash(self, data): + h = hashlib.sha1() + h.update(data.encode()) + + return binascii.hexlify(h.digest()).decode() + + def chunkstring(self, string, length): + return list(string[0+i:length+i] for i in range(0, len(string), length)) + + def _send_delete_msg(self, filename): + msg = bytearray() + msg.extend(self.ota.MSG_HEADER) + msg.extend(b',' + str(self.ota.DELETE_FILE_MSG).encode()) + msg.extend(b',' + filename.encode()) + msg.extend(b',' + self.ota.MSG_TAIL) + + self._clientApp.send(self._loraserver_jwt, self._multicast_group_id, msg) + + def _send_delete_operations(self, oper_dict): + for key, value in oper_dict.items(): + if key in ['delete_txt', 'delete_bin']: + for filename in value: + self._send_delete_msg(filename[6:]) + + def _send_patches(self, patch_dict): + for fname in patch_dict: + #send file name to patch + self._send_multicast_msg(self.ota.UPDATE_TYPE_FNAME, fname) + time.sleep(3) + patch_list = self.chunkstring(patch_dict[fname][0], 200) + patch_idx = 0 + for p in patch_list: + patch_idx += len(p) + # send segmented patch + self._send_multicast_msg(self.ota.UPDATE_TYPE_PATCH, p) + time.sleep(3) + checksum = patch_dict[fname][1] + # Send checksum + self._send_multicast_msg(self.ota.UPDATE_TYPE_CHECKSUM, checksum) + time.sleep(3) + + def _send_multicast_msg(self, msg_type, data): + + msg = bytearray() + msg.extend(self.ota.MSG_HEADER) + msg.extend(b',' + str(msg_type).encode()) + msg.extend(b',' + data.encode()) + msg.extend(b',' + self.ota.MSG_TAIL) + + self._clientApp.send(self._loraserver_jwt, self._multicast_group_id, msg) + + def _send_manifest_msg(self): + msg = bytearray() + msg.extend(self.ota.MSG_HEADER) + msg.extend(b',' + str(self.ota.UPDATE_TYPE_RESTART).encode()) + msg.extend(b',' + self.ota.MSG_TAIL) + + self._clientApp.send(self._loraserver_jwt, self._multicast_group_id, msg) + + def _create_file_patch(self, left, right, fileList): + patch_dict = dict() + + for f in fileList: + left_text = self._read_firware_file(left + '/' + f) + right_text = self._read_firware_file(right + '/' + f) + + dmp = dmp_module.diff_match_patch() + + # Execute one reverse diff as a warmup. + patch_lst = dmp.patch_make(left_text, right_text) + patch_str = dmp.patch_toText(patch_lst) + + print("File name: {}".format(f)) + print("Patch : {}".format(patch_str)) + + idx = f.find('/flash') + 7 + hash = self._create_hash(patch_str) + patch_dict[f[idx:]] = (patch_str, hash) + + return patch_dict + + def file_operations(self, device_version, update_version): + oper_dict = dict() + + left = self.ota.firmware_dir + '/' + device_version + right = self.ota.firmware_dir + '/' + update_version + if os.path.isdir(left): + oper_dict = self.get_diff_list(left, right) + else: + oper_dict = self.get_diff_list('', right) + + self.print_file_operations(oper_dict) + + return oper_dict + + def _create_patches(self, device_version, update_version, oper_dict): + patch_dict = dict() + + left = self.ota.firmware_dir + '/' + device_version + right = self.ota.firmware_dir + '/' + update_version + + if 'update_txt' in oper_dict: + update_dict = self._create_file_patch(left, right, oper_dict['update_txt']) + patch_dict.update(update_dict) + + if 'new_txt' in oper_dict: + new_dict = self._create_file_patch(left, right, oper_dict['new_txt']) + patch_dict.update(new_dict) + + return patch_dict + + def _send_manifest_msg(self): + manifest = self._create_manifest(self.oper_dict) + print('Manifest: {}'.format(manifest)) + + for i in (0, self.max_send): + self._send_multicast_msg(self.ota.MANIFEST_MSG, manifest) + time.sleep(4) + + def _multicast_proc(self): + + self.oper_dict = self.file_operations(self.dev_version, self.latest_version) + self.patch_dict = self._create_patches(self.dev_version, self.latest_version, self.oper_dict) + + self._send_patches(self.patch_dict) + self._send_delete_operations(self.oper_dict) + self._send_manifest_msg() + + while not self.ota.is_empty_multicast_queue(self._loraserver_jwt, self._multicast_group_id): + time.sleep(1) + + self.ota.clear_multicast_group(self.tag) + + + + + + + diff --git a/examples/OTA-lorawan/ota.py b/examples/OTA-lorawan/ota.py new file mode 100644 index 0000000..f9482ad --- /dev/null +++ b/examples/OTA-lorawan/ota.py @@ -0,0 +1,281 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +from distutils.version import LooseVersion +from LoraServer import LoraServerClient +from groupUpdater import updateHandler +import threading +import json +import base64 +import os +import time +import config + + +class OTAHandler: + + MSG_HEADER = b'$OTA' + MSG_TAIL = b'*' + MSG_END = b'' + + FULL_UPDATE = b'F' + DIFF_UPDATE = b'D' + NO_UPDATE = b'N' + + UPDATE_INFO_MSG = 1 + UPDATE_INFO_REPLY = 2 + + MULTICAST_KEY_REQ = 3 + MULTICAST_KEY_REPLY = 4 + + LISTENING_MSG = 5 + LISTENING_REPLY = 6 + + UPDATE_TYPE_FNAME = 7 + UPDATE_TYPE_PATCH = 8 + UPDATE_TYPE_CHECKSUM = 9 + + DELETE_FILE_MSG = 10 + MANIFEST_MSG = 11 + + def __init__(self): + self._exit = False + self.p_client = None + self._latest_version = '0.0.0' + self._v_lock = threading.Lock() + self.firmware_dir = './firmware' + + self._next_update = -1 + self._update_timer = None + self._update_delay = config.UPDATE_DELAY + self._device_dict = dict() + self._keys_dict = dict() + + self._clientApp = LoraServerClient() + self._loraserver_jwt = None + + self._service_profile = config.LORASERVER_SERVICE_PROFILE + self._downlink_datarate = config.LORASERVER_DOWNLINK_DR + self._downlink_freq = config.LORASERVER_DOWNLINK_FREQ + + self._m_th = threading.Thread(target=self._firmware_monitor) + self._m_th.start() + + self.multicast_updaters = [] + self._updater_lock = threading.Lock() + + def stop(self): + self._exit = True + + def set_mqtt_client(self, client): + self.p_client = client + + def _firmware_monitor(self): + self._loraserver_jwt = self._clientApp.login() + + while not self._exit: + with self._v_lock: + self._latest_version = self._check_version() + + time.sleep(5) + + def process_rx_msg(self, payload): + + dev_eui = self.get_device_eui(payload) + dev_msg = self.decode_device_msg(payload) + if self.MSG_HEADER in dev_msg: + msg_type = self.get_msg_type(dev_msg.decode()) + if msg_type == self.UPDATE_INFO_MSG: + self._send_update_info(dev_eui, dev_msg.decode()) + elif msg_type == self.MULTICAST_KEY_REQ: + self._send_multicast_keys(dev_eui) + elif msg_type == self.LISTENING_MSG: + self._send_listening_reply(dev_eui) + + def _send_listening_reply(self, dev_eui): + + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.LISTENING_REPLY).encode()) + msg.extend(b',' + self.MSG_TAIL) + + self.send_payload(dev_eui, msg) + + def _send_multicast_keys(self, dev_eui): + + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.MULTICAST_KEY_REPLY).encode()) + + if dev_eui in self._device_dict: + multicast_param_key = self._device_dict[dev_eui] + multicast_param = self._keys_dict[multicast_param_key] + + msg.extend(b',' + multicast_param[1]) + msg.extend(b',' + multicast_param[2]) + msg.extend(b',' + multicast_param[3]) + else: + msg.extend(b',,,') + + msg.extend(b',' + self.MSG_TAIL) + + self.send_payload(dev_eui, msg) + + def get_device_eui(self, payload): + dev_eui = None + try: + dev_eui = json.loads(payload)["devEUI"] + except Exception as ex: + print("Exception extracting device eui") + + return dev_eui + + def get_msg_type(self, msg): + msg_type = -1 + + try: + msg_type = int(msg.split(",")[1]) + except Exception as ex: + print("Exception getting message type") + + return msg_type + + def decode_device_msg(self, payload): + dev_msg = None + try: + rx_pkt = json.loads(payload) + dev_msg = base64.b64decode(rx_pkt["data"]) + except Exception as ex: + print("Exception decoding device message") + return dev_msg + + def _create_multicast_group(self, update_info): + service_id = self._clientApp.request_service_profile_id(self._service_profile, self._loraserver_jwt) + + group_name = update_info.replace(',','-') + multicast_param = self._clientApp.create_multicast_group(self._downlink_datarate, self._downlink_freq, group_name, service_id, self._loraserver_jwt) + + return multicast_param + + def _init_update_params(self, dev_eui, dev_version, latest_version): + if self._next_update <= 0: + self._next_update = int(time.time()) + self._update_delay + self._update_timer = threading.Timer(self._update_delay, self.update_proc) + self._update_timer.start() + + update_info = dev_version.strip() + ',' + latest_version.strip() + self._device_dict[dev_eui] = update_info + if update_info not in self._keys_dict: + multicast_param = self._create_multicast_group(update_info) + self._keys_dict[update_info] = multicast_param + self._clientApp.add_device_multicast_group(dev_eui, multicast_param[0], self._loraserver_jwt) + + def _send_update_info(self, dev_eui, msg): + print(msg) + dev_version = self.get_device_version(msg) + print("Device eui: {}, Device Version: {}".format(dev_eui, dev_version)) + + if len(dev_version) > 0: + version = self.get_latest_version() + if LooseVersion(version) > LooseVersion(dev_version): + self._init_update_params(dev_eui, dev_version, version) + msg = self._create_update_info_msg(version, dev_version) + self.send_payload(dev_eui, msg) + + def get_device_version(self, msg): + dev_version = None + try: + dev_version = msg.split(",")[2] + except Exception as ex: + print("Exception extracting device version") + + return dev_version + + def _check_version(self): + latest = '0.0.0' + for d in os.listdir(self.firmware_dir): + if os.path.isfile(d): + continue + if latest is None or LooseVersion(latest) < LooseVersion(d): + latest = d + + return latest + + def get_latest_version(self): + with self._v_lock: + return self._latest_version + + def is_empty_multicast_queue(self, jwt, multicast_group_id): + queue_length =self._clientApp.multicast_queue_length(jwt, multicast_group_id) + if queue_length > 0: + return False + else: + return True + + def clear_multicast_group(self, dict_key): + with self._updater_lock: + if dict_key in self._keys_dict: + group_id = self._keys_dict[dict_key][0] + self._clientApp.delete_multicast_group(group_id) + del self._keys_dict[dict_key] + + self._device_dict = {key:val for key, val in self._device_dict.items() if val != dict_key} + + for updater in self.multicast_updaters: + if updater.tag == dict_key: + self.multicast_updaters.remove(updater) + + if len(self.multicast_updaters) == 0: + self._next_update = -1 + self._update_timer = None + + def update_proc(self): + + for dict_key in self._keys_dict: + dev_version = dict_key.split(',')[0] + latest_version = dict_key.split(',')[1] + multicast_group_id = self._keys_dict[dict_key][0] + upater = updateHandler(dev_version, latest_version, self._clientApp, self._loraserver_jwt, multicast_group_id, self) + + self.multicast_updaters.append(upater) + + def _get_update_type(self, need_updating, device_version): + update_type = b',' + self.NO_UPDATE + print(os.path.isdir(self.firmware_dir + '/' + device_version)) + if need_updating: + if os.path.isdir(self.firmware_dir + '/' + device_version): + return b',' + self.DIFF_UPDATE + else: + return b',' + self.FULL_UPDATE + + return update_type + + def _create_update_info_msg(self, version, device_version): + msg = bytearray() + msg.extend(self.MSG_HEADER) + msg.extend(b',' + str(self.UPDATE_INFO_REPLY).encode()) + msg.extend(b',' + version.encode()) + need_updating = self._next_update > 0 + update_type = self._get_update_type(need_updating, device_version) + msg.extend(update_type) + if need_updating: + msg.extend(b',' + str(int(self._next_update)).encode()) + else: + msg.extend(b',-1') + msg.extend(b',' + str(int(time.time())).encode()) + msg.extend(b',' + self.MSG_TAIL) + return msg + + def send_payload(self, dev_eui, data): + b64Data = base64.b64encode(data) + payload = '{"reference": "abcd1234" ,"fPort":1,"data": "' + b64Data.decode() + '"}' + topic = "application/" + str(config.LORASERVER_APP_ID) + "/device/" + dev_eui + "/command/down" + print('Publish to {} : {}'.format(topic, payload)) + self.p_client.publish(topic=topic,payload=payload) diff --git a/examples/OTA-lorawan/requirements.txt b/examples/OTA-lorawan/requirements.txt new file mode 100644 index 0000000..620ac93 --- /dev/null +++ b/examples/OTA-lorawan/requirements.txt @@ -0,0 +1 @@ +paho_mqtt==1.5.1 diff --git a/examples/OTA-lorawan/updaterService.py b/examples/OTA-lorawan/updaterService.py new file mode 100644 index 0000000..9da347a --- /dev/null +++ b/examples/OTA-lorawan/updaterService.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import paho.mqtt.client as paho +from ota import OTAHandler +import signal +import time +import config +import sys + +exit = False +client = None + +def sigint_handler(signum, frame): + global exit + exit = True + print("Terminating Lora OTA updater") + +def on_message(mosq, ota, msg): + print("{} {} {}".format(msg.topic, msg.qos, msg.payload)) + ota.process_rx_msg(msg.payload.decode()) + +def on_publish(mosq, obj, mid): + pass + +if __name__ == '__main__': + signal.signal(signal.SIGINT, sigint_handler) + + ota = OTAHandler() + + client = paho.Client(userdata=ota) + client.connect(config.LORASERVER_IP, config.LORASERVER_MQTT_PORT, 60) + + client.on_message = on_message + client.on_publish = on_publish + + client.subscribe("application/+/device/+/event/up", 0) + + ota.set_mqtt_client(client) + + while client.loop() == 0 and not exit: + pass + + ota.stop() + sys.exit(0) diff --git a/examples/OTA/1.0.0/flash/config.py b/examples/OTA/1.0.0/flash/config.py index 302f1d4..4d471cd 100644 --- a/examples/OTA/1.0.0/flash/config.py +++ b/examples/OTA/1.0.0/flash/config.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + WIFI_SSID = "ENTER_ME" WIFI_PW = "ENTER_ME" SERVER_IP = "ENTER_ME" diff --git a/examples/OTA/1.0.0/flash/get_id.py b/examples/OTA/1.0.0/flash/get_id.py index fa40144..f56041e 100644 --- a/examples/OTA/1.0.0/flash/get_id.py +++ b/examples/OTA/1.0.0/flash/get_id.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import binascii lora = LoRa(mode=LoRa.LORAWAN) diff --git a/examples/OTA/1.0.0/flash/lib/OTA.py b/examples/OTA/1.0.0/flash/lib/OTA.py index 7cedd1e..333ad91 100644 --- a/examples/OTA/1.0.0/flash/lib/OTA.py +++ b/examples/OTA/1.0.0/flash/lib/OTA.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import network import socket import machine @@ -154,7 +164,7 @@ def __init__(self, ssid, password, ip, port): def connect(self): self.wlan = network.WLAN(mode=network.WLAN.STA) - if not self.wlan.isconnected() or self.ssid() != self.SSID: + if not self.wlan.isconnected() or self.wlan.ssid() != self.SSID: for net in self.wlan.scan(): if net.ssid == self.SSID: self.wlan.connect(self.SSID, auth=(network.WLAN.WPA2, diff --git a/examples/OTA/1.0.0/flash/main.py b/examples/OTA/1.0.0/flash/main.py index a6512dc..8e93085 100644 --- a/examples/OTA/1.0.0/flash/main.py +++ b/examples/OTA/1.0.0/flash/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa, WLAN import socket import time @@ -23,7 +33,7 @@ w.deinit() # Initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) app_eui = binascii.unhexlify('ENTER_ME') app_key = binascii.unhexlify('ENTER_ME') diff --git a/examples/OTA/1.0.1/flash/.DS_Store b/examples/OTA/1.0.1/flash/.DS_Store deleted file mode 100644 index dcdebc5..0000000 Binary files a/examples/OTA/1.0.1/flash/.DS_Store and /dev/null differ diff --git a/examples/OTA/1.0.1/flash/config.py b/examples/OTA/1.0.1/flash/config.py index 302f1d4..780528e 100644 --- a/examples/OTA/1.0.1/flash/config.py +++ b/examples/OTA/1.0.1/flash/config.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2018, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + WIFI_SSID = "ENTER_ME" WIFI_PW = "ENTER_ME" SERVER_IP = "ENTER_ME" diff --git a/examples/OTA/1.0.1/flash/get_id.py b/examples/OTA/1.0.1/flash/get_id.py index fa40144..03160a0 100644 --- a/examples/OTA/1.0.1/flash/get_id.py +++ b/examples/OTA/1.0.1/flash/get_id.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2018, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import binascii lora = LoRa(mode=LoRa.LORAWAN) diff --git a/examples/OTA/1.0.1/flash/lib/OTA.py b/examples/OTA/1.0.1/flash/lib/OTA.py index 699e238..24beff8 100644 --- a/examples/OTA/1.0.1/flash/lib/OTA.py +++ b/examples/OTA/1.0.1/flash/lib/OTA.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2018, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import network import socket import machine @@ -153,7 +163,7 @@ def __init__(self, ssid, password, ip, port): def connect(self): self.wlan = network.WLAN(mode=network.WLAN.STA) - if not self.wlan.isconnected() or self.ssid() != self.SSID: + if not self.wlan.isconnected() or self.wlan.ssid() != self.SSID: for net in self.wlan.scan(): if net.ssid == self.SSID: self.wlan.connect(self.SSID, auth=(network.WLAN.WPA2, diff --git a/examples/OTA/1.0.1/flash/main.py b/examples/OTA/1.0.1/flash/main.py index 27b5fcc..8c87564 100644 --- a/examples/OTA/1.0.1/flash/main.py +++ b/examples/OTA/1.0.1/flash/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2018, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa, WLAN import socket import time @@ -23,7 +33,7 @@ w.deinit() # Initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) app_eui = binascii.unhexlify('ENTER_ME') app_key = binascii.unhexlify('ENTER_ME') diff --git a/examples/OTA/OTA_server.py b/examples/OTA/OTA_server.py index e49bb23..d03fc16 100644 --- a/examples/OTA/OTA_server.py +++ b/examples/OTA/OTA_server.py @@ -1,7 +1,14 @@ -#!/usr/bin/env python3 +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# # Firmware over the air update server -# Copyright Pycom Ltd. # # Version History # 1.0 - Initial release (Sebastian Goscik) @@ -257,6 +264,7 @@ def get_new_firmware(path, current_ver): # version - The version number of the file # host - The server address, used in URL formatting def generate_manifest_entry(host, path, version): + path = "/".join(path.split(os.path.sep)) entry = {} entry["dst_path"] = "/{}".format(path) entry["URL"] = "http://{}/%7B%7D/%7B%7D".format(host, version, path) diff --git a/examples/accelerometer_wake/main.py b/examples/accelerometer_wake/main.py index 5f589f1..ec0c495 100644 --- a/examples/accelerometer_wake/main.py +++ b/examples/accelerometer_wake/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from pytrack import Pytrack #from pysense import Pysense from LIS2HH12 import LIS2HH12 diff --git a/examples/adc/boot.py b/examples/adc/boot.py index c9ace5d..f767f86 100644 --- a/examples/adc/boot.py +++ b/examples/adc/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/adc/main.py b/examples/adc/main.py index db15e21..f292a62 100644 --- a/examples/adc/main.py +++ b/examples/adc/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import ADC import time diff --git a/examples/bluetooth/boot.py b/examples/bluetooth/boot.py index c9ace5d..f767f86 100644 --- a/examples/bluetooth/boot.py +++ b/examples/bluetooth/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/bluetooth/main.py b/examples/bluetooth/main.py index e3249b3..92a28e2 100644 --- a/examples/bluetooth/main.py +++ b/examples/bluetooth/main.py @@ -1,18 +1,32 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import Bluetooth +import binascii import time bt = Bluetooth() bt.start_scan(-1) while True: adv = bt.get_adv() - if adv + if adv: # try to get the complete name - print(bluetooth.resolve_adv_data(adv.data, Bluetooth.ADV_NAME_CMPL)) + print(bt.resolve_adv_data(adv.data, Bluetooth.ADV_NAME_CMPL)) # try to get the manufacturer data (Apple's iBeacon data is sent here) - print(binascii.hexlify(bluetooth.resolve_adv_data(adv.data, Bluetooth.ADV_MANUFACTURER_DATA))) + mfg_data = bt.resolve_adv_data(adv.data, Bluetooth.ADV_MANUFACTURER_DATA) + + if mfg_data: + # try to get the manufacturer data (Apple's iBeacon data is sent here) + print(binascii.hexlify(mfg_data)) - if bt.resolve_adv_data(adv.data, Bluetooth.ADV_NAME_CMPL) == 'Heart Rate': conn = bt.connect(adv.mac) services = conn.services() @@ -29,4 +43,4 @@ conn.disconnect() break else: - time.sleep(0.050) \ No newline at end of file + time.sleep(0.050) diff --git a/examples/deepsleep/main.py b/examples/deepsleep/main.py index 83d248c..be04712 100644 --- a/examples/deepsleep/main.py +++ b/examples/deepsleep/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from deepsleep import DeepSleep import deepsleep diff --git a/examples/https/boot.py b/examples/https/boot.py index c9ace5d..f767f86 100644 --- a/examples/https/boot.py +++ b/examples/https/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/https/main.py b/examples/https/main.py index 2466a13..15297de 100644 --- a/examples/https/main.py +++ b/examples/https/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import socket import ssl diff --git a/examples/i2c/bh1750fvi.py b/examples/i2c/bh1750fvi.py index 5236ef3..b1bbd91 100644 --- a/examples/i2c/bh1750fvi.py +++ b/examples/i2c/bh1750fvi.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + # Simple driver for the BH1750FVI digital light sensor class BH1750FVI: diff --git a/examples/i2c/boot.py b/examples/i2c/boot.py index c9ace5d..f767f86 100644 --- a/examples/i2c/boot.py +++ b/examples/i2c/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/i2c/main.py b/examples/i2c/main.py index d6e8538..d91364a 100644 --- a/examples/i2c/main.py +++ b/examples/i2c/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import socket import time import pycom diff --git a/examples/lopy-lopy/lopy-A/boot.py b/examples/lopy-lopy/lopy-A/boot.py index c9ace5d..f767f86 100644 --- a/examples/lopy-lopy/lopy-A/boot.py +++ b/examples/lopy-lopy/lopy-A/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/lopy-lopy/lopy-A/main.py b/examples/lopy-lopy/lopy-A/main.py index 6ff56ee..233b3c0 100644 --- a/examples/lopy-lopy/lopy-A/main.py +++ b/examples/lopy-lopy/lopy-A/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket import time diff --git a/examples/lopy-lopy/lopy-B/boot.py b/examples/lopy-lopy/lopy-B/boot.py index c9ace5d..f767f86 100644 --- a/examples/lopy-lopy/lopy-B/boot.py +++ b/examples/lopy-lopy/lopy-B/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/lopy-lopy/lopy-B/main.py b/examples/lopy-lopy/lopy-B/main.py index a29d847..5df238b 100644 --- a/examples/lopy-lopy/lopy-B/main.py +++ b/examples/lopy-lopy/lopy-B/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket import time diff --git a/examples/loraNanoGateway/gateway/boot.py b/examples/loraNanoGateway/gateway/boot.py index c9ace5d..f767f86 100644 --- a/examples/loraNanoGateway/gateway/boot.py +++ b/examples/loraNanoGateway/gateway/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/loraNanoGateway/gateway/main.py b/examples/loraNanoGateway/gateway/main.py index 80a2807..d641ac5 100644 --- a/examples/loraNanoGateway/gateway/main.py +++ b/examples/loraNanoGateway/gateway/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import socket import struct from network import LoRa @@ -8,7 +18,12 @@ _LORA_PKG_ACK_FORMAT = "BBB" # Open a LoRa Socket, use rx_iq to avoid listening to our own messages -lora = LoRa(mode=LoRa.LORA, rx_iq=True) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORA, rx_iq=True, region=LoRa.EU868) lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW) lora_sock.setblocking(False) diff --git a/examples/loraNanoGateway/node/main.py b/examples/loraNanoGateway/node/main.py index 9c29aa1..08afcbd 100644 --- a/examples/loraNanoGateway/node/main.py +++ b/examples/loraNanoGateway/node/main.py @@ -11,7 +11,12 @@ # Open a Lora Socket, use tx_iq to avoid listening to our own messages -lora = LoRa(mode=LoRa.LORA, tx_iq=True) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORA, tx_iq=True, region=LoRa.EU868) lora_sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW) lora_sock.setblocking(False) diff --git a/examples/loraabp/boot.py b/examples/loraabp/boot.py index c9ace5d..f767f86 100644 --- a/examples/loraabp/boot.py +++ b/examples/loraabp/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/loraabp/main.py b/examples/loraabp/main.py index 3fbce07..74e4cba 100644 --- a/examples/loraabp/main.py +++ b/examples/loraabp/main.py @@ -1,15 +1,30 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket import binascii import struct # Initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) # create an ABP authentication params -dev_addr = struct.unpack(">l", binascii.unhexlify('00 00 00 05'.replace(' ','')))[0] -nwk_swkey = binascii.unhexlify('2B 7E 15 16 28 AE D2 A6 AB F7 15 88 09 CF 4F 3C'.replace(' ','')) -app_swkey = binascii.unhexlify('2B 7E 15 16 28 AE D2 A6 AB F7 15 88 09 CF 4F 3C'.replace(' ','')) +dev_addr = struct.unpack(">l", binascii.unhexlify('00000005'))[0] +nwk_swkey = binascii.unhexlify('2B7E151628AED2A6ABF7158809CF4F3C') +app_swkey = binascii.unhexlify('2B7E151628AED2A6ABF7158809CF4F3C') # join a network using ABP (Activation By Personalization) lora.join(activation=LoRa.ABP, auth=(dev_addr, nwk_swkey, app_swkey)) diff --git a/examples/loramac/boot.py b/examples/loramac/boot.py index c9ace5d..f767f86 100644 --- a/examples/loramac/boot.py +++ b/examples/loramac/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/loramac/main.py b/examples/loramac/main.py index a6a274e..c71627b 100644 --- a/examples/loramac/main.py +++ b/examples/loramac/main.py @@ -1,9 +1,25 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket # Initialize LoRa in LORA mode. + # More params can be given, like frequency, tx power and spreading factor. -lora = LoRa(mode=LoRa.LORA) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORA, region=LoRa.EU868) # create a raw LoRa socket s = socket.socket(socket.AF_LORA, socket.SOCK_RAW) diff --git a/examples/lorawan-nano-gateway/abp_node.py b/examples/lorawan-nano-gateway/abp_node.py index c4d0170..15dadf9 100644 --- a/examples/lorawan-nano-gateway/abp_node.py +++ b/examples/lorawan-nano-gateway/abp_node.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket import binascii @@ -6,12 +16,17 @@ import config # initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) # create an ABP authentication params -dev_addr = struct.unpack(">l", binascii.unhexlify('26 01 14 7D'.replace(' ','')))[0] -nwk_swkey = binascii.unhexlify('3C 74 F4 F4 0C AE A0 21 30 3B C2 42 84 FC F3 AF'.replace(' ','')) -app_swkey = binascii.unhexlify('0F FA 70 72 CC 6F F6 9A 10 2A 0F 39 BE B0 88 0F'.replace(' ','')) +dev_addr = struct.unpack(">l", binascii.unhexlify('2601147D'))[0] +nwk_swkey = binascii.unhexlify('3C74F4F40CAEA021303BC24284FCF3AF') +app_swkey = binascii.unhexlify('0FFA7072CC6FF69A102A0F39BEB0880F') # remove all the non-default channels for i in range(3, 16): @@ -31,11 +46,13 @@ # set the LoRaWAN data rate s.setsockopt(socket.SOL_LORA, socket.SO_DR, config.LORA_NODE_DR) -# make the socket blocking +# make the socket non-blocking s.setblocking(False) for i in range (200): - s.send(b'PKT #' + bytes([i])) + pkt = b'PKT #' + bytes([i]) + print('Sending:', pkt) + s.send(pkt) time.sleep(4) rx, port = s.recvfrom(256) if rx: diff --git a/examples/lorawan-nano-gateway/abp_node_US915.py b/examples/lorawan-nano-gateway/abp_node_US915.py index 2cbf786..d5c567f 100644 --- a/examples/lorawan-nano-gateway/abp_node_US915.py +++ b/examples/lorawan-nano-gateway/abp_node_US915.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import LoRa import socket import binascii @@ -6,12 +16,17 @@ import config # initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.US915) # create an ABP authentication params -dev_addr = struct.unpack(">l", binascii.unhexlify('26 01 14 7D'.replace(' ','')))[0] -nwk_swkey = binascii.unhexlify('3C 74 F4 F4 0C AE A0 21 30 3B C2 42 84 FC F3 AF'.replace(' ','')) -app_swkey = binascii.unhexlify('0F FA 70 72 CC 6F F6 9A 10 2A 0F 39 BE B0 88 0F'.replace(' ','')) +dev_addr = struct.unpack(">l", binascii.unhexlify('2601147D'))[0] +nwk_swkey = binascii.unhexlify('3C74F4F40CAEA021303BC24284FCF3AF') +app_swkey = binascii.unhexlify('0FFA7072CC6FF69A102A0F39BEB0880F') # remove all the channels for channel in range(0, 72): @@ -30,11 +45,13 @@ # set the LoRaWAN data rate s.setsockopt(socket.SOL_LORA, socket.SO_DR, config.LORA_NODE_DR) -# make the socket blocking +# make the socket non-blocking s.setblocking(False) for i in range (200): - s.send(b'PKT #' + bytes([i])) + pkt = b'PKT #' + bytes([i]) + print('Sending:', pkt) + s.send(pkt) time.sleep(4) rx, port = s.recvfrom(256) if rx: diff --git a/examples/lorawan-nano-gateway/config.py b/examples/lorawan-nano-gateway/config.py index d28c0a8..552df79 100644 --- a/examples/lorawan-nano-gateway/config.py +++ b/examples/lorawan-nano-gateway/config.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + """ LoPy LoRaWAN Nano Gateway configuration options """ import machine @@ -23,5 +33,5 @@ # for US915 # LORA_FREQUENCY = 903900000 -# LORA_GW_DR = "SF7BW125" # DR_3 -# LORA_NODE_DR = 3 +# LORA_GW_DR = "SF10BW125" # DR_0 +# LORA_NODE_DR = 0 diff --git a/examples/lorawan-nano-gateway/main.py b/examples/lorawan-nano-gateway/main.py index 384fc89..c17920e 100644 --- a/examples/lorawan-nano-gateway/main.py +++ b/examples/lorawan-nano-gateway/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + """ LoPy LoRaWAN Nano Gateway example usage """ import config diff --git a/examples/lorawan-nano-gateway/nanogateway.py b/examples/lorawan-nano-gateway/nanogateway.py index 8ed532a..1810cd5 100644 --- a/examples/lorawan-nano-gateway/nanogateway.py +++ b/examples/lorawan-nano-gateway/nanogateway.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + """ LoPy LoRaWAN Nano Gateway. Can be used for both EU868 and US915. """ import errno @@ -31,7 +41,7 @@ TX_ERR_TX_POWER = 'TX_POWER' TX_ERR_GPS_UNLOCKED = 'GPS_UNLOCKED' -UDP_THREAD_CYCLE_MS = const(10) +UDP_THREAD_CYCLE_MS = const(20) STAT_PK = { 'stat': { @@ -157,7 +167,7 @@ def start(self): _thread.start_new_thread(self._udp_thread, ()) # initialize the LoRa radio in LORA mode - self._log('Setting up the LoRa radio at {:.1f} Mhz using {}', self._freq_to_float(self.frequency), self.datarate) + self._log('Setting up the LoRa radio at {} Mhz using {}', self._freq_to_float(self.frequency), self.datarate) self.lora = LoRa( mode=LoRa.LORA, frequency=self.frequency, @@ -245,6 +255,7 @@ def _lora_cb(self, lora): rx_data = self.lora_sock.recv(256) stats = lora.stats() packet = self._make_node_packet(rx_data, self.rtc.now(), stats.rx_timestamp, stats.sfrx, self.bw, stats.rssi, stats.snr) + packet = self.frequency_rounding_fix(packet, self.frequency) self._push_data(packet) self._log('Received packet: {}', packet) self.rxfw += 1 @@ -277,6 +288,16 @@ def _freq_to_float(self, frequency): frequency = frequency / (10 ** divider) return frequency + def frequency_rounding_fix(self, packet, frequency): + freq = str(frequency)[0:3] + '.' + str(frequency)[3] + + start = packet.find("freq\":") + end = packet.find(",", start) + + packet = packet[:start + 7] + freq + packet[end:] + + return packet + def _make_stat_packet(self): now = self.rtc.now() STAT_PK["stat"]["time"] = "%d-%02d-%02d %02d:%02d:%02d GMT" % (now[0], now[1], now[2], now[3], now[4], now[5]) @@ -340,17 +361,38 @@ def _send_down_link(self, data, tmst, datarate, frequency): coding_rate=LoRa.CODING_4_5, tx_iq=True ) - while utime.ticks_us() < tmst: - pass + #while utime.ticks_cpu() < tmst: + # pass self.lora_sock.send(data) self._log( - 'Sent downlink packet scheduled on {:.3f}, at {:.1f} Mhz using {}: {}', + 'Sent downlink packet scheduled on {:.3f}, at {:.3f} Mhz using {}: {}', tmst / 1000000, self._freq_to_float(frequency), datarate, data ) + def _send_down_link_class_c(self, data, datarate, frequency): + self.lora.init( + mode=LoRa.LORA, + frequency=frequency, + bandwidth=self._dr_to_bw(datarate), + sf=self._dr_to_sf(datarate), + preamble=8, + coding_rate=LoRa.CODING_4_5, + tx_iq=True, + device_class=LoRa.CLASS_C + ) + + self.lora_sock.send(data) + self._log( + 'Sent downlink packet scheduled on {:.3f}, at {:.3f} Mhz using {}: {}', + utime.time(), + self._freq_to_float(frequency), + datarate, + data + ) + def _udp_thread(self): """ UDP thread, reads data from the server and handles it. @@ -369,28 +411,38 @@ def _udp_thread(self): self.dwnb += 1 ack_error = TX_ERR_NONE tx_pk = ujson.loads(data[4:]) - tmst = tx_pk["txpk"]["tmst"] - t_us = tmst - utime.ticks_us() - 12500 - if t_us < 0: - t_us += 0xFFFFFFFF - if t_us < 20000000: + if "tmst" in data: + tmst = tx_pk["txpk"]["tmst"] + t_us = tmst - utime.ticks_cpu() - 15000 + if t_us < 0: + t_us += 0xFFFFFFFF + if t_us < 20000000: + self.uplink_alarm = Timer.Alarm( + handler=lambda x: self._send_down_link( + ubinascii.a2b_base64(tx_pk["txpk"]["data"]), + tx_pk["txpk"]["tmst"] - 50, tx_pk["txpk"]["datr"], + int(tx_pk["txpk"]["freq"] * 1000) * 1000 + ), + us=t_us + ) + else: + ack_error = TX_ERR_TOO_LATE + self._log('Downlink timestamp error!, t_us: {}', t_us) + else: self.uplink_alarm = Timer.Alarm( - handler=lambda x: self._send_down_link( + handler=lambda x: self._send_down_link_class_c( ubinascii.a2b_base64(tx_pk["txpk"]["data"]), - tx_pk["txpk"]["tmst"] - 50, tx_pk["txpk"]["datr"], - int(tx_pk["txpk"]["freq"] * 1000000) - ), - us=t_us + tx_pk["txpk"]["datr"], + int(tx_pk["txpk"]["freq"] * 1000) * 1000 + ), + us=50 ) - else: - ack_error = TX_ERR_TOO_LATE - self._log('Downlink timestamp error!, t_us: {}', t_us) self._ack_pull_rsp(_token, ack_error) self._log("Pull rsp") except usocket.timeout: pass except OSError as ex: - if ex.errno != errno.EAGAIN: + if ex.args[0] != errno.EAGAIN: self._log('UDP recv OSError Exception: {}', ex) except Exception as ex: self._log('UDP recv Exception: {}', ex) diff --git a/examples/lorawan-nano-gateway/otaa_node.py b/examples/lorawan-nano-gateway/otaa_node.py index 1735407..1abe66b 100644 --- a/examples/lorawan-nano-gateway/otaa_node.py +++ b/examples/lorawan-nano-gateway/otaa_node.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + """ OTAA Node example compatible with the LoPy Nano Gateway """ from network import LoRa @@ -8,12 +18,17 @@ import config # initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) # create an OTA authentication params -dev_eui = binascii.unhexlify('AA BB CC DD EE FF 77 78'.replace(' ','')) -app_eui = binascii.unhexlify('70 B3 D5 7E F0 00 3B FD'.replace(' ','')) -app_key = binascii.unhexlify('36 AB 76 25 FE 77 0B 68 81 68 3B 49 53 00 FF D6'.replace(' ','')) +dev_eui = binascii.unhexlify('AABBCCDDEEFF7778') +app_eui = binascii.unhexlify('70B3D57EF0003BFD') +app_key = binascii.unhexlify('36AB7625FE770B6881683B495300FFD6') # set the 3 default channels to the same frequency (must be before sending the OTAA join request) lora.add_channel(0, frequency=config.LORA_FREQUENCY, dr_min=0, dr_max=5) @@ -38,13 +53,15 @@ # set the LoRaWAN data rate s.setsockopt(socket.SOL_LORA, socket.SO_DR, config.LORA_NODE_DR) -# make the socket blocking +# make the socket non-blocking s.setblocking(False) time.sleep(5.0) for i in range (200): - s.send(b'PKT #' + bytes([i])) + pkt = b'PKT #' + bytes([i]) + print('Sending:', pkt) + s.send(pkt) time.sleep(4) rx, port = s.recvfrom(256) if rx: diff --git a/examples/lorawan-nano-gateway/otaa_node_US915.py b/examples/lorawan-nano-gateway/otaa_node_US915.py index 8b1415e..1008c20 100644 --- a/examples/lorawan-nano-gateway/otaa_node_US915.py +++ b/examples/lorawan-nano-gateway/otaa_node_US915.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + """ OTAA Node example compatible with the LoPy Nano Gateway """ from network import LoRa @@ -8,12 +18,17 @@ import config # initialize LoRa in LORAWAN mode. -lora = LoRa(mode=LoRa.LORAWAN) +# Please pick the region that matches where you are using the device: +# Asia = LoRa.AS923 +# Australia = LoRa.AU915 +# Europe = LoRa.EU868 +# United States = LoRa.US915 +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.US915) # create an OTA authentication params -dev_eui = binascii.unhexlify('AA BB CC DD EE FF 77 78'.replace(' ','')) -app_eui = binascii.unhexlify('70 B3 D5 7E F0 00 3B FD'.replace(' ','')) -app_key = binascii.unhexlify('36 AB 76 25 FE 77 0B 68 81 68 3B 49 53 00 FF D6'.replace(' ','')) +dev_eui = binascii.unhexlify('AABBCCDDEEFF7778') +app_eui = binascii.unhexlify('70B3D57EF0003BFD') +app_key = binascii.unhexlify('36AB7625FE770B6881683B495300FFD6') # remove all the channels for channel in range(0, 72): @@ -28,14 +43,14 @@ # wait until the module has joined the network join_wait = 0 -while True: +while True: time.sleep(2.5) if not lora.has_joined(): print('Not joined yet...') join_wait += 1 if join_wait == 5: lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_eui, app_key), timeout=0, dr=config.LORA_NODE_DR) - join_wait = 0 + join_wait = 0 else: break @@ -45,13 +60,15 @@ # set the LoRaWAN data rate s.setsockopt(socket.SOL_LORA, socket.SO_DR, config.LORA_NODE_DR) -# make the socket blocking +# make the socket non-blocking s.setblocking(False) time.sleep(5.0) for i in range (200): - s.send(b'PKT #' + bytes([i])) + pkt = b'PKT #' + bytes([i]) + print('Sending:', pkt) + s.send(pkt) time.sleep(4) rx, port = s.recvfrom(256) if rx: diff --git a/examples/lorawan-regional-examples/main_AS923.py b/examples/lorawan-regional-examples/main_AS923.py new file mode 100644 index 0000000..ce03f4f --- /dev/null +++ b/examples/lorawan-regional-examples/main_AS923.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +""" + OTAA Node example as per LoRaWAN AS923 regional specification + - compatible with the LoPy Nano Gateway and all other LoraWAN gateways + - tested works with a LoRaServer, shall works on TTN servers +""" + +from network import LoRa +import socket +import binascii +import struct +import time + +LORA_CHANNEL = 1 +LORA_NODE_DR = 4 + +''' + utility function to setup the lora channels +''' +def prepare_channels(lora, channel, data_rate): + + AS923_FREQUENCIES = [ + { "chan": 1, "fq": "923200000" }, + { "chan": 2, "fq": "923400000" }, + { "chan": 3, "fq": "922200000" }, + { "chan": 4, "fq": "922400000" }, + { "chan": 5, "fq": "922600000" }, + { "chan": 6, "fq": "922800000" }, + { "chan": 7, "fq": "923000000" }, + { "chan": 8, "fq": "922000000" }, + ] + + if not channel in range(0, 9): + raise RuntimeError("channels should be in 1-8 for AS923") + + if channel == 0: + import uos + channel = (struct.unpack('B',uos.urandom(1))[0] % 7) + 1 + + for i in range(0, 8): + lora.remove_channel(i) + + upstream = (item for item in AS923_FREQUENCIES if item["chan"] == channel).__next__() + + # set default channels frequency + lora.add_channel(int(upstream.get('chan')), frequency=int(upstream.get('fq')), dr_min=0, dr_max=data_rate) + + return lora + +''' + call back for handling RX packets +''' +def lora_cb(lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + if lora_socket is not None: + frame, port = lora_socket.recvfrom(512) # longuest frame is +-220 + print(port, frame) + if events & LoRa.TX_PACKET_EVENT: + print("tx_time_on_air: {} ms @dr {}", lora.stats().tx_time_on_air, lora.stats().sftx) + +''' + Main operations: this is sample code for LoRaWAN on AS923 +''' + +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.AS923, device_class=LoRa.CLASS_C, adr=False, tx_power=20) + +# create an OTA authentication params +dev_eui = binascii.unhexlify('0000000000000000') +app_key = binascii.unhexlify('a926e5bb85271f2d') # not used leave empty loraserver.io +nwk_key = binascii.unhexlify('a926e5bb85271f2da0440f2f4200afe3') + +# join a network using OTAA +lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_key, nwk_key), timeout=0, dr=2) # AS923 always joins at DR2 + +prepare_channels(lora, LORA_CHANNEL, LORA_NODE_DR) + +# wait until the module has joined the network +print('Over the air network activation ... ', end='') +while not lora.has_joined(): + time.sleep(2.5) + print('.', end='') +print('') + +# create a LoRa socket +lora_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + +# set the LoRaWAN data rate +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_DR, LORA_NODE_DR) + +# msg are confirmed at the FMS level +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_CONFIRMED, 0) + +# make the socket non blocking y default +lora_socket.setblocking(False) + +lora.callback(trigger=( LoRa.RX_PACKET_EVENT | + LoRa.TX_PACKET_EVENT | + LoRa.TX_FAILED_EVENT ), handler=lora_cb) + +time.sleep(4) # this timer is important and caused me some trouble ... + +for i in range(0, 1000): + pkt = struct.pack('>H', i) + print('Sending:', pkt) + lora_socket.send(pkt) + time.sleep(300) diff --git a/examples/lorawan-regional-examples/main_AU915.py b/examples/lorawan-regional-examples/main_AU915.py new file mode 100644 index 0000000..419217c --- /dev/null +++ b/examples/lorawan-regional-examples/main_AU915.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +""" + OTAA Node example as per LoRaWAN AU915 regional specification + - tested works with a LoRaServer, shall works on TTN servers + - This example uses 8 channels so you will need an 8 channel GW + (not a 1 channel GW like the NanoGateway) +""" + +from network import LoRa +import socket +import binascii +import struct +import time + +LORA_FREQUENCY = 915200000 # start of the 1st subband +LORA_NODE_DR = 4 +''' + utility function to setup the lora channels +''' +def prepare_channels(lora, channel, data_rate): + + AU915_FREQUENCIES = [ + { "chan": 64, "fq": "915200000" } + ] + if not channel in range(64,65): + raise RuntimeError("only channel 64 is implemented in this example)") + upstream = (item for item in AU915_FREQUENCIES if item["chan"] == channel).__next__() + + lora.add_channel(int(upstream.get('chan')), frequency=int(upstream.get('fq')), dr_min=0, dr_max=int(data_rate)) + print("*** Adding channel up %s %s" % (upstream.get('chan'), upstream.get('fq'))) + + for index in range(0, 71): + if index != upstream.get('chan'): + lora.remove_channel(index) + + return lora + +''' + call back for handling RX packets +''' +def lora_cb(lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + if lora_socket is not None: + frame, port = lora_socket.recvfrom(512) # longuest frame is +-220 + print(port, frame) + if events & LoRa.TX_PACKET_EVENT: + print("tx_time_on_air: {} ms @dr {}", lora.stats().tx_time_on_air, lora.stats().sftx) + + +''' + Main operations: this is sample code for LoRaWAN on AU915 +''' + +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.AU915, device_class=LoRa.CLASS_C) + +# create an OTA authentication params +dev_eui = binascii.unhexlify('0000000000000000') +app_key = binascii.unhexlify('a926e5bb85271f2d') # not used leave empty loraserver.io +nwk_key = binascii.unhexlify('a926e5bb85271f2da0440f2f4200afe3') + +prepare_channels(lora, 64, 5) + +# join a network using OTAA +lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_key, nwk_key), timeout=0, dr=0) # DR is 2 in v1.1rb but 0 worked for ne + +# wait until the module has joined the network +print('Over the air network activation ... ', end='') +while not lora.has_joined(): + time.sleep(2.5) + print('.', end='') +print('') + +for i in range(0, 8): + fq = LORA_FREQUENCY + (i * 200000) + lora.add_channel(i, frequency=fq, dr_min=0, dr_max=LORA_NODE_DR) + print("AU915 Adding channel up %s %s" % (i, fq)) + + +# create a LoRa socket +lora_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + +# set the LoRaWAN data rate +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_DR, LORA_NODE_DR) + +# msg are confirmed at the FMS level +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_CONFIRMED, 0) + +# make the socket non blocking y default +lora_socket.setblocking(False) + +lora.callback(trigger=( LoRa.RX_PACKET_EVENT | + LoRa.TX_PACKET_EVENT | + LoRa.TX_FAILED_EVENT ), handler=lora_cb) + +time.sleep(4) # this timer is important and caused me some trouble ... + +for i in range(0, 1000): + pkt = struct.pack('>H', i) + print('Sending:', pkt) + lora_socket.send(pkt) + time.sleep(300) diff --git a/examples/lorawan-regional-examples/main_EU868.py b/examples/lorawan-regional-examples/main_EU868.py new file mode 100644 index 0000000..646dad6 --- /dev/null +++ b/examples/lorawan-regional-examples/main_EU868.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +""" + OTAA Node example as per LoRaWAN EU868 regional specification + - compatible with the LoPy Nano Gateway and all other LoraWAN gateways + - tested works with a LoRaServer and TTN servers +""" + +from network import LoRa +import socket +import binascii +import struct +import time + +LORA_CHANNEL = 0 # zero = random +LORA_NODE_DR = 4 +''' + utility function to setup the lora channels +''' +def prepare_channels(lora, channel, data_rate): + EU868_FREQUENCIES = [ + { "chan": 1, "fq": "868100000" }, + { "chan": 2, "fq": "868300000" }, + { "chan": 3, "fq": "868500000" }, + { "chan": 4, "fq": "867100000" }, + { "chan": 5, "fq": "867300000" }, + { "chan": 6, "fq": "867500000" }, + { "chan": 7, "fq": "867700000" }, + { "chan": 8, "fq": "867900000" }, + ] + if not channel in range(0, 9): + raise RuntimeError("channels should be in 0-8 for EU868") + + if channel == 0: + import uos + channel = (struct.unpack('B',uos.urandom(1))[0] % 7) + 1 + + upstream = (item for item in EU868_FREQUENCIES if item["chan"] == channel).__next__() + + # set the 3 default channels to the same frequency + lora.add_channel(0, frequency=int(upstream.get('fq')), dr_min=0, dr_max=5) + lora.add_channel(1, frequency=int(upstream.get('fq')), dr_min=0, dr_max=5) + lora.add_channel(2, frequency=int(upstream.get('fq')), dr_min=0, dr_max=5) + + for i in range(3, 16): + lora.remove_channel(i) + + return lora + +''' + call back for handling RX packets +''' +def lora_cb(lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + if lora_socket is not None: + frame, port = lora_socket.recvfrom(512) # longuest frame is +-220 + print(port, frame) + if events & LoRa.TX_PACKET_EVENT: + print("tx_time_on_air: {} ms @dr {}", lora.stats().tx_time_on_air, lora.stats().sftx) + + +''' + Main operations: this is sample code for LoRaWAN on EU868 +''' + +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868, device_class=LoRa.CLASS_C) + +# create an OTA authentication params +dev_eui = binascii.unhexlify('0000000000000000') +app_key = binascii.unhexlify('a926e5bb85271f2d') # not used leave empty loraserver.io +nwk_key = binascii.unhexlify('a926e5bb85271f2da0440f2f4200afe3') + +prepare_channels(lora, 1, LORA_NODE_DR) + +# join a network using OTAA +lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_key, nwk_key), timeout=0, dr=LORA_NODE_DR) # DR is 2 in v1.1rb but 0 worked for ne + +# wait until the module has joined the network +print('Over the air network activation ... ', end='') +while not lora.has_joined(): + time.sleep(2.5) + print('.', end='') +print('') + +# create a LoRa socket +lora_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + +# set the LoRaWAN data rate +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_DR, LORA_NODE_DR) + +# msg are confirmed at the FMS level +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_CONFIRMED, 0) + +# make the socket non blocking y default +lora_socket.setblocking(False) + +lora.callback(trigger=( LoRa.RX_PACKET_EVENT | + LoRa.TX_PACKET_EVENT | + LoRa.TX_FAILED_EVENT ), handler=lora_cb) + +time.sleep(4) # this timer is important and caused me some trouble ... + +for i in range(0, 1000): + pkt = struct.pack('>H', i) + print('Sending:', pkt) + lora_socket.send(pkt) + time.sleep(300) diff --git a/examples/lorawan-regional-examples/main_US915.py b/examples/lorawan-regional-examples/main_US915.py new file mode 100644 index 0000000..52d0a4a --- /dev/null +++ b/examples/lorawan-regional-examples/main_US915.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +""" + OTAA Node example as per LoRaWAN US915 regional specification + - tested works with a LoRaServer, shall works on TTN servers + - This example uses 8 channels so you will need an 8 channel GW + (not a 1 channel GW like the NanoGateway) +""" + +from network import LoRa +import socket +import binascii +import struct +import time + +LORA_FREQUENCY = 916800000 # start of the 1st subband +LORA_NODE_DR = 4 +''' + utility function to setup the lora channels +''' +def prepare_channels(lora, channel, data_rate): + + US915_FREQUENCIES = [ + { "chan": 64, "fq": "902300000" } + ] + if not channel in range(64,65): + raise RuntimeError("channels should be in 64 for US915 (only subband 1 is implemented in this example)") + upstream = (item for item in US915_FREQUENCIES if item["chan"] == channel).__next__() + + lora.add_channel(int(upstream.get('chan')), frequency=int(upstream.get('fq')), dr_min=0, dr_max=int(data_rate)) + print("*** Adding channel up %s %s" % (upstream.get('chan'), upstream.get('fq'))) + + for index in range(0, 71): + if index != upstream.get('chan'): + lora.remove_channel(index) + + return lora + +''' + call back for handling RX packets +''' +def lora_cb(lora): + events = lora.events() + if events & LoRa.RX_PACKET_EVENT: + if lora_socket is not None: + frame, port = lora_socket.recvfrom(512) # longuest frame is +-220 + print(port, frame) + if events & LoRa.TX_PACKET_EVENT: + print("tx_time_on_air: {} ms @dr {}", lora.stats().tx_time_on_air, lora.stats().sftx) + + +''' + Main operations: this is sample code for LoRaWAN on US915 +''' + +lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.US915, device_class=LoRa.CLASS_C) + +# create an OTA authentication params +dev_eui = binascii.unhexlify('0000000000000000') +app_key = binascii.unhexlify('a926e5bb85271f2d') # not used leave empty loraserver.io +nwk_key = binascii.unhexlify('a926e5bb85271f2da0440f2f4200afe3') + +prepare_channels(lora, 64, 0) + +# join a network using OTAA +lora.join(activation=LoRa.OTAA, auth=(dev_eui, app_key, nwk_key), timeout=0, dr=0) # US915 always joins at DR2 + +for i in range(0, 8): + fq = LORA_FREQUENCY + (i * 200000) + lora.add_channel(i, frequency=fq, dr_min=0, dr_max=LORA_NODE_DR) + print("US915 Adding channel up %s %s" % (i, fq)) + +# wait until the module has joined the network +print('Over the air network activation ... ', end='') +while not lora.has_joined(): + time.sleep(2.5) + print('.', end='') +print('') + +lora_socket = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + +# set the LoRaWAN data rate +# create a LoRa socket +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_DR, LORA_NODE_DR) + +# msg are confirmed at the FMS level +lora_socket.setsockopt(socket.SOL_LORA, socket.SO_CONFIRMED, 0) + +# make the socket non blocking y default +lora_socket.setblocking(False) + +lora.callback(trigger=( LoRa.RX_PACKET_EVENT | + LoRa.TX_PACKET_EVENT | + LoRa.TX_FAILED_EVENT ), handler=lora_cb) + +time.sleep(4) # this timer is important and caused me some trouble ... + +for i in range(0, 1000): + pkt = struct.pack('>H', i) + print('Sending:', pkt) + lora_socket.send(pkt) + time.sleep(300) diff --git a/examples/mqtt/boot.py b/examples/mqtt/boot.py index c9ace5d..f767f86 100644 --- a/examples/mqtt/boot.py +++ b/examples/mqtt/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/mqtt/main.py b/examples/mqtt/main.py index 0e5a81f..04c0441 100644 --- a/examples/mqtt/main.py +++ b/examples/mqtt/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import WLAN from mqtt import MQTTClient import machine diff --git a/examples/mqtt/mqtt.py b/examples/mqtt/mqtt.py index 0436645..9a54d0c 100644 --- a/examples/mqtt/mqtt.py +++ b/examples/mqtt/mqtt.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import usocket as socket import ustruct as struct from ubinascii import hexlify @@ -136,7 +146,7 @@ def subscribe(self, topic, qos=0): #print(hex(len(pkt)), hexlify(pkt, ":")) self.sock.write(pkt) self._send_str(topic) - self.sock.write(qos.to_bytes(1)) + self.sock.write(qos.to_bytes(1, "little")) while 1: op = self.wait_msg() if op == 0x90: diff --git a/examples/onlineLog/boot.py b/examples/onlineLog/boot.py index 0b803ff..cf85552 100644 --- a/examples/onlineLog/boot.py +++ b/examples/onlineLog/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/onlineLog/main.py b/examples/onlineLog/main.py index 9cf772a..080515e 100644 --- a/examples/onlineLog/main.py +++ b/examples/onlineLog/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time import machine from onewire import DS18X20 diff --git a/examples/onlineLog/onewire.py b/examples/onlineLog/onewire.py index 935ac46..02768eb 100644 --- a/examples/onlineLog/onewire.py +++ b/examples/onlineLog/onewire.py @@ -1,4 +1,12 @@ -#!/usr/bin/env python3 +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# """ OneWire library for MicroPython diff --git a/examples/pytrack_pysense_accelerometer/.DS_Store b/examples/pytrack_pysense_accelerometer/.DS_Store deleted file mode 100644 index bfbb04c..0000000 Binary files a/examples/pytrack_pysense_accelerometer/.DS_Store and /dev/null differ diff --git a/examples/pytrack_pysense_accelerometer/main.py b/examples/pytrack_pysense_accelerometer/main.py index 9fc2c8d..90ceb88 100644 --- a/examples/pytrack_pysense_accelerometer/main.py +++ b/examples/pytrack_pysense_accelerometer/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time # From:https://github.com/pycom/pycom-libraries diff --git a/examples/pytrack_pysense_accelerometer/visualiser/.DS_Store b/examples/pytrack_pysense_accelerometer/visualiser/.DS_Store deleted file mode 100644 index ac75a33..0000000 Binary files a/examples/pytrack_pysense_accelerometer/visualiser/.DS_Store and /dev/null differ diff --git a/examples/sigfoxUplink/boot.py b/examples/sigfoxUplink/boot.py index c9ace5d..f767f86 100644 --- a/examples/sigfoxUplink/boot.py +++ b/examples/sigfoxUplink/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/sigfoxUplink/main.py b/examples/sigfoxUplink/main.py index d16d828..ff0d17c 100644 --- a/examples/sigfoxUplink/main.py +++ b/examples/sigfoxUplink/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from network import Sigfox import socket @@ -13,5 +23,5 @@ # configure it as uplink only s.setsockopt(socket.SOL_SIGFOX, socket.SO_RX, False) -# send some bytes -s.send(bytes([0x01, 0x02, 0x03]) +# send some bytes +s.send(bytes([0x01, 0x02, 0x03])) diff --git a/examples/threading/boot.py b/examples/threading/boot.py index c9ace5d..f767f86 100644 --- a/examples/threading/boot.py +++ b/examples/threading/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/examples/threading/main.py b/examples/threading/main.py index 8516af0..c0f23ff 100644 --- a/examples/threading/main.py +++ b/examples/threading/main.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import _thread import time diff --git a/lib/ADS1115/ADS1115.py b/lib/ADS1115/ADS1115.py index 97a07b7..9e7abc7 100644 --- a/lib/ADS1115/ADS1115.py +++ b/lib/ADS1115/ADS1115.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + # Interface for ADS1115 16-bit I2C ADC class ADS1115: diff --git a/lib/ADS1115/boot.py b/lib/ADS1115/boot.py index c9ace5d..f767f86 100644 --- a/lib/ADS1115/boot.py +++ b/lib/ADS1115/boot.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import UART import machine import os diff --git a/lib/ALSPT19/ALSPT19.py b/lib/ALSPT19/ALSPT19.py index 05835f7..0120c81 100644 --- a/lib/ALSPT19/ALSPT19.py +++ b/lib/ALSPT19/ALSPT19.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import ADC import time diff --git a/lib/mqtt/mqtt.py b/lib/mqtt/mqtt.py index 45a75c1..c183702 100644 --- a/lib/mqtt/mqtt.py +++ b/lib/mqtt/mqtt.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import usocket as socket import ustruct as struct from ubinascii import hexlify diff --git a/lib/mqtt_aws/MQTTClient.py b/lib/mqtt_aws/MQTTClient.py new file mode 100644 index 0000000..f1a5bff --- /dev/null +++ b/lib/mqtt_aws/MQTTClient.py @@ -0,0 +1,426 @@ +import MQTTConst as mqttConst +import MQTTMsgHandler as msgHandler +import time +import struct +import _thread + +class MQTTMessage: + def __init__(self): + self.timestamp = 0 + self.state = 0 + self.dup = False + self.mid = 0 + self.topic = "" + self.payload = None + self.qos = 0 + self.retain = False + +class MQTTClient: + + def __init__(self, clientID, cleanSession, protocol): + self.client_id = clientID + self._cleanSession = cleanSession + self._protocol = protocol + self._userdata = None + self._user = "" + self._password = "" + self._keepAliveInterval = 60 + self._will = False + self._will_topic = "" + self._will_message= None + self._will_qos = 0 + self._will_retain = False + self._connectdisconnectTimeout = 30 + self._mqttOperationTimeout = 5 + self._topic_callback_queue=[] + self._callback_mutex=_thread.allocate_lock() + self._pid = 0 + self._subscribeSent = False + self._unsubscribeSent = False + self._baseReconnectTimeSecond=1 + self._maximumReconnectTimeSecond=32 + self._minimumConnectTimeSecond=20 + self._msgHandler=msgHandler.MsgHandler(self._recv_callback, self.connect) + + def getClientID(self): + return self.client_id + + def configEndpoint(self, srcHost, srcPort): + self._msgHandler.setEndpoint(srcHost, srcPort) + + def configCredentials(self, srcCAFile, srcKey, srcCert): + self._msgHandler.setCredentials(srcCAFile, srcKey, srcCert) + + def setConnectDisconnectTimeoutSecond(self, srcConnectDisconnectTimeout): + self._connectdisconnectTimeout = srcConnectDisconnectTimeout + + def setMQTTOperationTimeoutSecond(self, srcMQTTOperationTimeout): + self._mqttOperationTimeout = srcMQTTOperationTimeout + self._msgHandler.setOperationTimeout(srcMQTTOperationTimeout) + + def clearLastWill(self): + self._will = False + self._will_topic = "" + self._will_message= None + self._will_qos = 0 + self._will_retain = False + + def setLastWill(self, topic, payload=None, QoS=0, retain=False): + self._will=True + self._will_qos = QoS + self._will_retain = retain + self._will_topic = topic.encode('utf-8') + + if isinstance(payload, bytearray): + self._will_message=payload + elif isinstance(payload, str): + self._will_message=payload.encode('utf-8') + elif isinstance(payload, int) or isinstance(payload, float): + self._will_message=str(payload) + + def configIAMCredentials(self, srcAWSAccessKeyID, srcAWSSecretAccessKey, srcAWSSessionToken): + raise NotImplementedError ('Websockets not supported') + + def setOfflinePublishQueueing(self, srcQueueSize, srcDropBehavior): + if srcDropBehavior != mqttConst.DROP_OLDEST and srcDropBehavior != mqttConst.DROP_NEWEST: + raise ValueError("Invalid packet drop behavior") + self._msgHandler.setOfflineQueueConfiguration(srcQueueSize, srcDropBehavior) + + def setDrainingIntervalSecond(self, srcDrainingIntervalSecond): + self._msgHandler.setDrainingInterval(srcDrainingIntervalSecond) + + def setBackoffTiming(self, srcBaseReconnectTimeSecond, srcMaximumReconnectTimeSecond, srcMinimumConnectTimeSecond): + self._baseReconnectTimeSecond=srcBaseReconnectTimeSecond + self._maximumReconnectTimeSecond=srcMaximumReconnectTimeSecond + self._minimumConnectTimeSecond=srcMinimumConnectTimeSecond + + def connect(self, keepAliveInterval=30): + self._keepAliveInterval = keepAliveInterval + + if not self._msgHandler.createSocketConnection(): + return False + + self._send_connect(self._keepAliveInterval, self._cleanSession) + + # delay to check the state + count_10ms = 0 + while(count_10ms <= self._connectdisconnectTimeout * 100 and not self._msgHandler.isConnected()): + count_10ms += 1 + time.sleep(0.01) + + return True if self._msgHandler.isConnected() else False + + def subscribe(self, topic, qos, callback): + if (topic is None or callback is None): + raise TypeError("Invalid subscribe values.") + topic = topic.encode('utf-8') + + header = mqttConst.MSG_SUBSCRIBE | (1<<1) + pkt = bytearray([header]) + + pkt_len = 2 + 2 + len(topic) + 1 # packet identifier + len of topic (16 bits) + topic len + QOS + pkt.extend(self._encode_varlen_length(pkt_len)) # len of the remaining + + self._pid += 1 + pkt.extend(self._encode_16(self._pid)) + pkt.extend(self._pascal_string(topic)) + pkt.append(qos) + + self._subscribeSent = False + self._msgHandler.push_on_send_queue(pkt) + + count_10ms = 0 + while(count_10ms <= self._mqttOperationTimeout * 100 and not self._subscribeSent): + count_10ms += 1 + time.sleep(0.01) + + if self._subscribeSent: + self._callback_mutex.acquire() + self._topic_callback_queue.append((topic, callback)) + self._callback_mutex.release() + return True + + return False + + def publish(self, topic, payload, qos, retain, dup=False): + topic = topic.encode('utf-8') + payload = payload.encode('utf-8') + + header = mqttConst.MSG_PUBLISH | (dup << 3) | (qos << 1) | retain + pkt_len = (2 + len(topic) + + (2 if qos else 0) + + (len(payload))) + + pkt = bytearray([header]) + pkt.extend(self._encode_varlen_length(pkt_len)) # len of the remaining + pkt.extend(self._pascal_string(topic)) + if qos: + self._pid += 1 #todo: I don't think this is the way to deal with the packet id + pkt.extend(self._encode_16(self._pid)) + + pkt = pkt + payload + self._msgHandler.push_on_send_queue(pkt) + + def _encode_16(self, x): + return struct.pack("!H", x) + + def _pascal_string(self, s): + return struct.pack("!H", len(s)) + s + + def _encode_varlen_length(self, length): + i = 0 + buff = bytearray() + while 1: + buff.append(length % 128) + length = length // 128 + if length > 0: + buff[i] = buff[i] | 0x80 + i += 1 + else: + break + + return buff + + def _topic_matches_sub(self, sub, topic): + result = True + multilevel_wildcard = False + + slen = len(sub) + tlen = len(topic) + + if slen > 0 and tlen > 0: + if (sub[0] == '$' and topic[0] != '$') or (topic[0] == '$' and sub[0] != '$'): + return False + + spos = 0 + tpos = 0 + + while spos < slen and tpos < tlen: + if sub[spos] == topic[tpos]: + if tpos == tlen-1: + # Check for e.g. foo matching foo/# + if spos == slen-3 and sub[spos+1] == '/' and sub[spos+2] == '#': + result = True + multilevel_wildcard = True + break + + spos += 1 + tpos += 1 + + if tpos == tlen and spos == slen-1 and sub[spos] == '+': + spos += 1 + result = True + break + else: + if sub[spos] == '+': + spos += 1 + while tpos < tlen and topic[tpos] != '/': + tpos += 1 + if tpos == tlen and spos == slen: + result = True + break + + elif sub[spos] == '#': + multilevel_wildcard = True + if spos+1 != slen: + result = False + break + else: + result = True + break + + else: + result = False + break + + if not multilevel_wildcard and (tpos < tlen or spos < slen): + result = False + + return result + + def _remove_topic_callback(self, topic): + deleted=False + + self._callback_mutex.acquire() + for i in range(0, len(self._topic_callback_queue)): + if self._topic_callback_queue[i][0] == topic: + self._topic_callback_queue.pop(i) + deleted=True + self._callback_mutex.release() + + return deleted + + def unsubscribe(self, topic): + self._unsubscribeSent = False + self._send_unsubscribe(topic, False) + + count_10ms = 0 + while(count_10ms <= self._mqttOperationTimeout * 100 and not self._unsubscribeSent): + count_10ms += 1 + time.sleep(0.01) + + if self._unsubscribeSent: + topic = topic.encode('utf-8') + return self._remove_topic_callback(topic) + + return False + + def disconnect(self): + pkt = struct.pack('!BB', mqttConst.MSG_DISCONNECT, 0) + self._msgHandler.push_on_send_queue(pkt) + + time.sleep(self._connectdisconnectTimeout) + self._msgHandler.disconnect() + + return True + + def _send_connect(self, keepalive, clean_session): + msg_sent = False + + pkt_len = (12 + len(self.client_id) + # 10 + 2 + len(client_id) + (2 + len(self._user) if self._user else 0) + + (2 + len(self._password) if self._password else 0)) + + flags = (0x80 if self._user else 0x00) | (0x40 if self._password else 0x00) | (0x02 if clean_session else 0x00) + + if self._will_message: + flags |= (self._will_retain << 3 | self._will_qos << 1 | 1) << 2 + pkt_len += 4 + len(self._will_topic) + len(self._will_message) + + pkt = bytearray([mqttConst.MSG_CONNECT]) # connect + pkt.extend(self._encode_varlen_length(pkt_len)) # len of the remaining + pkt.extend(b'\x00\x04MQTT\x04') # len of "MQTT" (16 bits), protocol name, and protocol version + pkt.append(flags) + pkt.extend(b'\x00\x00') # disable keepalive + pkt.extend(self._pascal_string(self.client_id)) + if self._will_message: + pkt.extend(self._pascal_string(self._will_topic)) + pkt.extend(self._pascal_string(self._will_message)) + if self._user: + pkt.extend(self._pascal_string(self._user)) + if self._password: + pkt.extend(self._pascal_string(self._password)) + + return self._msgHandler.priority_send(pkt) + + def _send_unsubscribe(self, topic, dup=False): + pkt = bytearray() + msg_type = mqttConst.MSG_UNSUBSCRIBE | (dup<<3) | (1<<1) + pkt.extend(struct.pack("!B", msg_type)) + + remaining_length = 2 + 2 + len(topic) + pkt.extend(self._encode_varlen_length(remaining_length)) + + self._pid += 1 + pkt.extend(self._encode_16(self._pid)) + pkt.extend(self._pascal_string(topic)) + + return self._msgHandler.push_on_send_queue(pkt) + + def _send_puback(self, msg_id): + remaining_length = 2 + pkt = struct.pack('!BBH', mqttConst.MSG_PUBACK, remaining_length, msg_id) + + return self._msgHandler.push_on_send_queue(pkt) + + def _send_pubrec(self, msg_id): + remaining_length = 2 + pkt = struct.pack('!BBH', mqttConst.MSG_PUBREC, remaining_length, msg_id) + + return self._msgHandler.push_on_send_queue(pkt) + + def _parse_connack(self, payload): + if len(payload) != 2: + return False + + (flags, result) = struct.unpack("!BB", payload) + + if result == 0: + self._msgHandler.setConnectionState(mqttConst.STATE_CONNECTED) + return True + else: + self._msgHandler.setConnectionState(mqttConst.STATE_DISCONNECTED) + return False + + def _parse_suback(self, payload): + self._subscribeSent = True + print('Subscribed to topic') + + return True + + def _parse_puback(self, payload): + return True + + def _notify_message(self, message): + notified = False + self._callback_mutex.acquire() + for t_obj in self._topic_callback_queue: + if self._topic_matches_sub(t_obj[0], message.topic): + t_obj[1](self, self._userdata, message) + notified = True + self._callback_mutex.release() + + return notified + + def _parse_publish(self, cmd, packet): + msg = MQTTMessage() + msg.dup = (cmd & 0x08)>>3 + msg.qos = (cmd & 0x06)>>1 + msg.retain = (cmd & 0x01) + + pack_format = "!H" + str(len(packet)-2) + 's' + (slen, packet) = struct.unpack(pack_format, packet) + pack_format = '!' + str(slen) + 's' + str(len(packet)-slen) + 's' + (msg.topic, packet) = struct.unpack(pack_format, packet) + + if len(msg.topic) == 0: + return False + + if msg.qos > 0: + pack_format = "!H" + str(len(packet)-2) + 's' + (msg.mid, packet) = struct.unpack(pack_format, packet) + + msg.payload = packet + + if msg.qos == 0: + self._notify_message(msg) + elif msg.qos == 1: + self._send_puback(msg.mid) + self._notify_message(msg) + elif msg.qos == 2: + self._send_pubrec(msg.mid) + self._notify_message(msg) + else: + return False + + return True + + def _parse_unsuback(self, payload): + self._unsubscribeSent = True + return True + + def _parse_pingresp(self): + self._msgHandler.setPingFlag(True) + return True + + def _recv_callback(self, cmd, payload): + msg_type = cmd & 0xF0 + + if msg_type == mqttConst.MSG_CONNACK: + return self._parse_connack(payload) + elif msg_type == mqttConst.MSG_SUBACK: + return self._parse_suback(payload) + elif msg_type == mqttConst.MSG_PUBACK: + return self._parse_puback(payload) + elif msg_type == mqttConst.MSG_PUBLISH: + return self._parse_publish(cmd, payload) + elif msg_type == mqttConst.MSG_UNSUBACK: + return self._parse_unsuback(payload) + elif msg_type == mqttConst.MSG_PINGRESP: + return self._parse_pingresp() + else: + print('Unknown message type: %d' % msg_type) + return False + + def insertShadowCallback(self, callback, payload, status, token): + self._msgHandler.insertShadowCallback(callback, payload, status, token) diff --git a/lib/mqtt_aws/MQTTConst.py b/lib/mqtt_aws/MQTTConst.py new file mode 100644 index 0000000..4c6d07b --- /dev/null +++ b/lib/mqtt_aws/MQTTConst.py @@ -0,0 +1,46 @@ + +# - Protocol types +MQTTv3_1 = 3 +MQTTv3_1_1 = 4 + +# - OfflinePublishQueueing drop behavior +DROP_OLDEST = 0 +DROP_NEWEST = 1 + +# Message types +MSG_CONNECT = 0x10 +MSG_CONNACK = 0x20 +MSG_PUBLISH = 0x30 +MSG_PUBACK = 0x40 +MSG_PUBREC = 0x50 +MSG_PUBREL = 0x60 +MSG_PUBCOMP = 0x70 +MSG_SUBSCRIBE = 0x80 +MSG_SUBACK = 0x90 +MSG_UNSUBSCRIBE = 0xA0 +MSG_UNSUBACK = 0xB0 +MSG_PINGREQ = 0xC0 +MSG_PINGRESP = 0xD0 +MSG_DISCONNECT = 0xE0 + +# Connection state +STATE_CONNECTED = 0x01 +STATE_CONNECTING = 0x02 +STATE_DISCONNECTED = 0x03 + +class UUID: + int_ = int + bytes_ = bytes + + def __init__(self, bytes=None, version=None): + + self._int = UUID.int_.from_bytes(bytes, 'big') + + self._int &= ~(0xc000 << 48) + self._int |= 0x8000 << 48 + self._int &= ~(0xf000 << 64) + self._int |= version << 76 + + @property + def urn(self): + return 'urn:uuid:' + str(self) diff --git a/lib/mqtt_aws/MQTTDeviceShadow.py b/lib/mqtt_aws/MQTTDeviceShadow.py new file mode 100644 index 0000000..6ea1489 --- /dev/null +++ b/lib/mqtt_aws/MQTTDeviceShadow.py @@ -0,0 +1,233 @@ +import MQTTConst as mqttConst +from machine import Timer +import json +import os +import _thread + +class _basicJSONParser: + + def setString(self, srcString): + self._rawString = srcString + self._dictionObject = None + + def regenerateString(self): + return json.dumps(self._dictionaryObject) + + def getAttributeValue(self, srcAttributeKey): + return self._dictionaryObject.get(srcAttributeKey) + + def setAttributeValue(self, srcAttributeKey, srcAttributeValue): + self._dictionaryObject[srcAttributeKey] = srcAttributeValue + + def validateJSON(self): + try: + self._dictionaryObject = json.loads(self._rawString) + except ValueError: + return False + return True + +class deviceShadow: + def __init__(self, srcShadowName, srcIsPersistentSubscribe, srcShadowManager): + + if srcShadowName is None or srcIsPersistentSubscribe is None or srcShadowManager is None: + raise TypeError("None type inputs detected.") + self._shadowName = srcShadowName + # Tool handler + self._shadowManagerHandler = srcShadowManager + self._basicJSONParserHandler = _basicJSONParser() + # Properties + self._isPersistentSubscribe = srcIsPersistentSubscribe + self._lastVersionInSync = -1 # -1 means not initialized + self._isGetSubscribed = False + self._isUpdateSubscribed = False + self._isDeleteSubscribed = False + self._shadowSubscribeCallbackTable = dict() + self._shadowSubscribeCallbackTable["get"] = None + self._shadowSubscribeCallbackTable["delete"] = None + self._shadowSubscribeCallbackTable["update"] = None + self._shadowSubscribeCallbackTable["delta"] = None + self._shadowSubscribeStatusTable = dict() + self._shadowSubscribeStatusTable["get"] = 0 + self._shadowSubscribeStatusTable["delete"] = 0 + self._shadowSubscribeStatusTable["update"] = 0 + self._tokenPool = dict() + self._dataStructureLock = _thread.allocate_lock() + + def _doNonPersistentUnsubscribe(self, currentAction): + self._shadowManagerHandler.shadowUnsubscribe(self._shadowName, currentAction) + + def _generalCallback(self, client, userdata, message): + # In Py3.x, message.payload comes in as a bytes(string) + # json.loads needs a string input + self._dataStructureLock.acquire() + currentTopic = message.topic + currentAction = self._parseTopicAction(currentTopic) # get/delete/update/delta + currentType = self._parseTopicType(currentTopic) # accepted/rejected/delta + payloadUTF8String = message.payload.decode('utf-8') + # get/delete/update: Need to deal with token, timer and unsubscribe + if currentAction in ["get", "delete", "update"]: + # Check for token + self._basicJSONParserHandler.setString(payloadUTF8String) + if self._basicJSONParserHandler.validateJSON(): # Filter out invalid JSON + currentToken = self._basicJSONParserHandler.getAttributeValue(u"clientToken") + if currentToken is not None and currentToken in self._tokenPool.keys(): # Filter out JSON without the desired token + # Sync local version when it is an accepted response + if currentType == "accepted": + incomingVersion = self._basicJSONParserHandler.getAttributeValue(u"version") + # If it is get/update accepted response, we need to sync the local version + if incomingVersion is not None and incomingVersion > self._lastVersionInSync and currentAction != "delete": + self._lastVersionInSync = incomingVersion + # If it is a delete accepted, we need to reset the version + else: + self._lastVersionInSync = -1 # The version will always be synced for the next incoming delta/GU-accepted response + # Cancel the timer and clear the token + self._tokenPool[currentToken].cancel() + del self._tokenPool[currentToken] + # Need to unsubscribe? + self._shadowSubscribeStatusTable[currentAction] -= 1 + if not self._isPersistentSubscribe and self._shadowSubscribeStatusTable.get(currentAction) <= 0: + self._shadowSubscribeStatusTable[currentAction] = 0 + self._doNonPersistentUnsubscribe(currentAction) + # Custom callback + if self._shadowSubscribeCallbackTable.get(currentAction) is not None: + self._shadowManagerHandler.insertShadowCallback(self._shadowSubscribeCallbackTable[currentAction], payloadUTF8String, currentType, currentToken) + # delta: Watch for version + else: + currentType += "/" + self._parseTopicShadowName(currentTopic) + # Sync local version + self._basicJSONParserHandler.setString(payloadUTF8String) + if self._basicJSONParserHandler.validateJSON(): # Filter out JSON without version + incomingVersion = self._basicJSONParserHandler.getAttributeValue(u"version") + if incomingVersion is not None and incomingVersion > self._lastVersionInSync: + self._lastVersionInSync = incomingVersion + # Custom callback + if self._shadowSubscribeCallbackTable.get(currentAction) is not None: + self._shadowManagerHandler.insertShadowCallback(self._shadowSubscribeCallbackTable[currentAction], payloadUTF8String, currentType, None) + self._dataStructureLock.release() + + def _parseTopicAction(self, srcTopic): + ret = None + fragments = srcTopic.decode('utf-8').split('/') + if fragments[5] == "delta": + ret = "delta" + else: + ret = fragments[4] + return ret + + def _parseTopicType(self, srcTopic): + fragments = srcTopic.decode('utf-8').split('/') + return fragments[5] + + def _parseTopicShadowName(self, srcTopic): + fragments = srcTopic.decode('utf-8').split('/') + return fragments[2] + + def _timerHandler(self, args): + srcActionName = args[0] + srcToken = args[1] + + self._dataStructureLock.acquire() + # Remove the token + del self._tokenPool[srcToken] + # Need to unsubscribe? + self._shadowSubscribeStatusTable[srcActionName] -= 1 + if not self._isPersistentSubscribe and self._shadowSubscribeStatusTable.get(srcActionName) <= 0: + self._shadowSubscribeStatusTable[srcActionName] = 0 + self._shadowManagerHandler.shadowUnsubscribe(self._shadowName, srcActionName) + # Notify time-out issue + if self._shadowSubscribeCallbackTable.get(srcActionName) is not None: + self._shadowSubscribeCallbackTable[srcActionName]("REQUEST TIME OUT", "timeout", srcToken) + self._dataStructureLock.release() + + def shadowGet(self, srcCallback, srcTimeout): + self._dataStructureLock.acquire() + # Update callback data structure + self._shadowSubscribeCallbackTable["get"] = srcCallback + # Update number of pending feedback + self._shadowSubscribeStatusTable["get"] += 1 + # clientToken + currentToken = mqttConst.UUID(bytes=os.urandom(16), version=4).urn[9:] + self._tokenPool[currentToken] = None + self._basicJSONParserHandler.setString("{}") + self._basicJSONParserHandler.validateJSON() + self._basicJSONParserHandler.setAttributeValue("clientToken", currentToken) + currentPayload = self._basicJSONParserHandler.regenerateString() + self._dataStructureLock.release() + # Two subscriptions + if not self._isPersistentSubscribe or not self._isGetSubscribed: + self._shadowManagerHandler.shadowSubscribe(self._shadowName, "get", self._generalCallback) + self._isGetSubscribed = True + # One publish + self._shadowManagerHandler.shadowPublish(self._shadowName, "get", currentPayload) + # Start the timer + self._tokenPool[currentToken] = Timer.Alarm(self._timerHandler, srcTimeout,arg=("get", currentToken),periodic=False) + return currentToken + + def shadowDelete(self, srcCallback, srcTimeout): + self._dataStructureLock.acquire() + # Update callback data structure + self._shadowSubscribeCallbackTable["delete"] = srcCallback + # Update number of pending feedback + self._shadowSubscribeStatusTable["delete"] += 1 + # clientToken + currentToken = mqttConst.UUID(bytes=os.urandom(16), version=4).urn[9:] + self._tokenPool[currentToken] = None + self._basicJSONParserHandler.setString("{}") + self._basicJSONParserHandler.validateJSON() + self._basicJSONParserHandler.setAttributeValue("clientToken", currentToken) + currentPayload = self._basicJSONParserHandler.regenerateString() + self._dataStructureLock.release() + # Two subscriptions + if not self._isPersistentSubscribe or not self._isDeleteSubscribed: + self._shadowManagerHandler.shadowSubscribe(self._shadowName, "delete", self._generalCallback) + self._isDeleteSubscribed = True + # One publish + self._shadowManagerHandler.shadowPublish(self._shadowName, "delete", currentPayload) + # Start the timer + self._tokenPool[currentToken] = Timer.Alarm(self._timerHandler,srcTimeout, arg=("delete", currentToken), periodic=False) + return currentToken + + def shadowUpdate(self, srcJSONPayload, srcCallback, srcTimeout): + # Validate JSON + JSONPayloadWithToken = None + currentToken = None + self._basicJSONParserHandler.setString(srcJSONPayload) + if self._basicJSONParserHandler.validateJSON(): + self._dataStructureLock.acquire() + # clientToken + currentToken = mqttConst.UUID(bytes=os.urandom(16), version=4).urn[9:] + self._tokenPool[currentToken] = None + self._basicJSONParserHandler.setAttributeValue("clientToken", currentToken) + JSONPayloadWithToken = self._basicJSONParserHandler.regenerateString() + # Update callback data structure + self._shadowSubscribeCallbackTable["update"] = srcCallback + # Update number of pending feedback + self._shadowSubscribeStatusTable["update"] += 1 + self._dataStructureLock.release() + # Two subscriptions + if not self._isPersistentSubscribe or not self._isUpdateSubscribed: + self._shadowManagerHandler.shadowSubscribe(self._shadowName, "update", self._generalCallback) + self._isUpdateSubscribed = True + # One publish + self._shadowManagerHandler.shadowPublish(self._shadowName, "update", JSONPayloadWithToken) + # Start the timer + self._tokenPool[currentToken] = Timer.Alarm(self._timerHandler, srcTimeout, arg=("update", currentToken), periodic=False) + else: + raise ValueError("Invalid JSON file.") + return currentToken + + def shadowRegisterDeltaCallback(self, srcCallback): + self._dataStructureLock.acquire() + # Update callback data structure + self._shadowSubscribeCallbackTable["delta"] = srcCallback + self._dataStructureLock.release() + # One subscription + self._shadowManagerHandler.shadowSubscribe(self._shadowName, "delta", self._generalCallback) + + def shadowUnregisterDeltaCallback(self): + self._dataStructureLock.acquire() + # Update callback data structure + del self._shadowSubscribeCallbackTable["delta"] + self._dataStructureLock.release() + # One unsubscription + self._shadowManagerHandler.shadowUnsubscribe(self._shadowName, "delta") diff --git a/lib/mqtt_aws/MQTTLib.py b/lib/mqtt_aws/MQTTLib.py new file mode 100644 index 0000000..49dc0a5 --- /dev/null +++ b/lib/mqtt_aws/MQTTLib.py @@ -0,0 +1,119 @@ +import MQTTConst as mqttConst +import MQTTClient as mqttClient +import MQTTShadowManager as shadowManager +import MQTTDeviceShadow as deviceShadow + +class AWSIoTMQTTClient: + + def __init__(self, clientID, protocolType=mqttConst.MQTTv3_1_1, useWebsocket=False, cleanSession=True): + self._mqttClient = mqttClient.MQTTClient(clientID, cleanSession, protocolType) + + # Configuration APIs + def configureLastWill(self, topic, payload, QoS): + self._mqttClient.setLastWill(topic, payload, QoS) + + def clearLastWill(self): + self._mqttClient.clearLastWill() + + def configureEndpoint(self, hostName, portNumber): + self._mqttClient.configEndpoint(hostName, portNumber) + + def configureIAMCredentials(self, AWSAccessKeyID, AWSSecretAccessKey, AWSSessionToken=""): + self._mqttClient.configIAMCredentials(AWSAccessKeyID, AWSSecretAccessKey, AWSSessionToken) + + def configureCredentials(self, CAFilePath, KeyPath="", CertificatePath=""): # Should be good for MutualAuth certs config and Websocket rootCA config + self._mqttClient.configCredentials(CAFilePath, KeyPath, CertificatePath) + + def configureAutoReconnectBackoffTime(self, baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond): + self._mqttClient.setBackoffTiming(baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond) + + def configureOfflinePublishQueueing(self, queueSize, dropBehavior=mqttConst.DROP_NEWEST): + self._mqttClient.setOfflinePublishQueueing(queueSize, dropBehavior) + + def configureDrainingFrequency(self, frequencyInHz): + self._mqttClient.setDrainingIntervalSecond(1/float(frequencyInHz)) + + def configureConnectDisconnectTimeout(self, timeoutSecond): + self._mqttClient.setConnectDisconnectTimeoutSecond(timeoutSecond) + + def configureMQTTOperationTimeout(self, timeoutSecond): + self._mqttClient.setMQTTOperationTimeoutSecond(timeoutSecond) + + # MQTT functionality APIs + def connect(self, keepAliveIntervalSecond=30): + return self._mqttClient.connect(keepAliveIntervalSecond) + + def disconnect(self): + return self._mqttClient.disconnect() + + def publish(self, topic, payload, QoS): + return self._mqttClient.publish(topic, payload, QoS, False) # Disable retain for publish by now + + def subscribe(self, topic, QoS, callback): + return self._mqttClient.subscribe(topic, QoS, callback) + + def unsubscribe(self, topic): + return self._mqttClient.unsubscribe(topic) + + +class AWSIoTMQTTShadowClient: + + def __init__(self, clientID, protocolType=mqttConst.MQTTv3_1_1, useWebsocket=False, cleanSession=True): + # AWSIOTMQTTClient instance + self._AWSIoTMQTTClient = AWSIoTMQTTClient(clientID, protocolType, useWebsocket, cleanSession) + # Configure it to disable offline Publish Queueing + self._AWSIoTMQTTClient.configureOfflinePublishQueueing(0) # Disable queueing, no queueing for time-sentive shadow messages + self._AWSIoTMQTTClient.configureDrainingFrequency(10) + # Now retrieve the configured mqttCore and init a shadowManager instance + self._shadowManager = shadowManager.shadowManager(self._AWSIoTMQTTClient._mqttClient) + + # Configuration APIs + def configureLastWill(self, topic, payload, QoS): + self._AWSIoTMQTTClient.configureLastWill(topic, payload, QoS) + + def clearLastWill(self): + self._AWSIoTMQTTClient.clearLastWill() + + def configureEndpoint(self, hostName, portNumber): + self._AWSIoTMQTTClient.configureEndpoint(hostName, portNumber) + + def configureIAMCredentials(self, AWSAccessKeyID, AWSSecretAccessKey, AWSSTSToken=""): + # AWSIoTMQTTClient.configureIAMCredentials + self._AWSIoTMQTTClient.configureIAMCredentials(AWSAccessKeyID, AWSSecretAccessKey, AWSSTSToken) + + def configureCredentials(self, CAFilePath, KeyPath="", CertificatePath=""): # Should be good for MutualAuth and Websocket + self._AWSIoTMQTTClient.configureCredentials(CAFilePath, KeyPath, CertificatePath) + + def configureAutoReconnectBackoffTime(self, baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond): + self._AWSIoTMQTTClient.configureAutoReconnectBackoffTime(baseReconnectQuietTimeSecond, maxReconnectQuietTimeSecond, stableConnectionTimeSecond) + + def configureConnectDisconnectTimeout(self, timeoutSecond): + self._AWSIoTMQTTClient.configureConnectDisconnectTimeout(timeoutSecond) + + def configureMQTTOperationTimeout(self, timeoutSecond): + self._AWSIoTMQTTClient.configureMQTTOperationTimeout(timeoutSecond) + + # Start the MQTT connection + def connect(self, keepAliveIntervalSecond=30): + return self._AWSIoTMQTTClient.connect(keepAliveIntervalSecond) + + # End the MQTT connection + def disconnect(self): + return self._AWSIoTMQTTClient.disconnect() + + # Shadow management API + def createShadowHandlerWithName(self, shadowName, isPersistentSubscribe): + # Create and return a deviceShadow instance + return deviceShadow.deviceShadow(shadowName, isPersistentSubscribe, self._shadowManager) + # Shadow APIs are accessible in deviceShadow instance": + ### + # deviceShadow.shadowGet + # deviceShadow.shadowUpdate + # deviceShadow.shadowDelete + # deviceShadow.shadowRegisterDelta + # deviceShadow.shadowUnregisterDelta + + # MQTT connection management API + def getMQTTConnection(self): + # Return the internal AWSIoTMQTTClient instance + return self._AWSIoTMQTTClient diff --git a/lib/mqtt_aws/MQTTMsgHandler.py b/lib/mqtt_aws/MQTTMsgHandler.py new file mode 100644 index 0000000..273f739 --- /dev/null +++ b/lib/mqtt_aws/MQTTMsgHandler.py @@ -0,0 +1,279 @@ +import MQTTConst as mqttConst +import time +import socket +import ssl +import _thread +import select +import struct + +class MsgHandler: + + def __init__(self, receive_callback, connect_helper): + self._host = "" + self._port = -1 + self._cafile = "" + self._key = "" + self._cert = "" + self._sock = None + self._output_queue_size=-1 + self._output_queue_dropbehavior=-1 + self._mqttOperationTimeout = 5 + self._connection_state = mqttConst.STATE_DISCONNECTED + self._conn_state_mutex=_thread.allocate_lock() + self._poll = select.poll() + self._output_queue=[] + self._out_packet_mutex=_thread.allocate_lock() + _thread.stack_size(8192) + _thread.start_new_thread(self._io_thread_func, ()) + self._recv_callback = receive_callback + self._connect_helper = connect_helper + self._pingSent=False + self._ping_interval=20 + self._waiting_ping_resp=False + self._ping_cutoff=3 + self._receive_timeout=3000 + self._draining_interval=2 + self._draining_cutoff=3 + self._shadow_cb_queue=[] + self._shadow_cb_mutex=_thread.allocate_lock() + + def setOfflineQueueConfiguration(self, queueSize, dropBehavior): + self._output_queue_size = queueSize + self._output_queue_dropbehavior = dropBehavior + + def setCredentials(self, srcCAFile, srcKey, srcCert): + self._cafile = srcCAFile + self._key = srcKey + self._cert = srcCert + + def setEndpoint(self, srcHost, srcPort): + self._host = srcHost + self._port = srcPort + + def setOperationTimeout(self, timeout): + self._mqttOperationTimeout=timeout + + def setDrainingInterval(self, srcDrainingIntervalSecond): + self._draining_interval=srcDrainingIntervalSecond + + def insertShadowCallback(self, callback, payload, status, token): + self._shadow_cb_mutex.acquire() + self._shadow_cb_queue.append((callback, payload, status, token)) + self._shadow_cb_mutex.release() + + def _callShadowCallback(self): + self._shadow_cb_mutex.acquire() + if len(self._shadow_cb_queue) > 0: + cbObj = self._shadow_cb_queue.pop(0) + cbObj[0](cbObj[1],cbObj[2], cbObj[3]) + self._shadow_cb_mutex.release() + + def createSocketConnection(self): + self._conn_state_mutex.acquire() + self._connection_state = mqttConst.STATE_CONNECTING + self._conn_state_mutex.release() + try: + if self._sock: + self._poll.unregister(self._sock) + self._sock.close() + self._sock = None + + self._sock = socket.socket() + self._sock.settimeout(30) + if self._cafile: + self._sock = ssl.wrap_socket( + self._sock, + certfile=self._cert, + keyfile=self._key, + ca_certs=self._cafile, + cert_reqs=ssl.CERT_REQUIRED) + + self._sock.connect(socket.getaddrinfo(self._host, self._port)[0][-1]) + self._poll.register(self._sock, select.POLLIN) + except socket.error as err: + print("Socket create error: {0}".format(err)) + + self._conn_state_mutex.acquire() + self._connection_state = mqttConst.STATE_DISCONNECTED + self._conn_state_mutex.release() + + return False + + return True + + def disconnect(self): + if self._sock: + self._sock.close() + self._sock = None + + def isConnected(self): + connected=False + self._conn_state_mutex.acquire() + if self._connection_state == mqttConst.STATE_CONNECTED: + connected = True + self._conn_state_mutex.release() + + return connected + + def setConnectionState(self, state): + self._conn_state_mutex.acquire() + self._connection_state = state + self._conn_state_mutex.release() + + def _drop_message(self): + if self._output_queue_size == -1: + return False + elif (self._output_queue_size == 0) and (self._connection_state == mqttConst.STATE_CONNECTED): + return False + else: + return True if len(self._output_queue) >= self._output_queue_size else False + + def push_on_send_queue(self, packet): + if self._drop_message(): + if self._output_queue_dropbehavior == mqttConst.DROP_OLDEST: + self._out_packet_mutex.acquire() + if self._out_packet_mutex.locked(): + self._output_queue.pop(0) + self._out_packet_mutex.release() + else: + return False + + self._out_packet_mutex.acquire() + if self._out_packet_mutex.locked(): + self._output_queue.append(packet) + self._out_packet_mutex.release() + + return True + + def priority_send(self, packet): + msg_sent = False + self._out_packet_mutex.acquire() + msg_sent = self._send_packet(packet) + self._out_packet_mutex.release() + + return msg_sent + + def _receive_packet(self): + try: + if not self._poll.poll(self._receive_timeout): + return False + except Exception as err: + print("Poll error: {0}".format(err)) + return False + + # Read message type + try: + self._sock.setblocking(False) + msg_type = self._sock.recv(1) + except socket.error as err: + print("Socket receive error: {0}".format(err)) + return False + else: + if len(msg_type) == 0: + return False + msg_type = struct.unpack("!B", msg_type)[0] + self._sock.setblocking(True) + + # Read payload length + multiplier = 1 + bytes_read = 0 + bytes_remaining = 0 + while True: + try: + if self._sock: + byte = self._sock.recv(1) + except socket.error as err: + print("Socket receive error: {0}".format(err)) + return False + else: + bytes_read = bytes_read + 1 + if bytes_read > 4: + return False + + byte = struct.unpack("!B", byte)[0] + bytes_remaining += (byte & 127) * multiplier + multiplier += 128 + + if (byte & 128) == 0: + break + + # Read payload + try: + if self._sock: + if bytes_remaining > 0: + payload = self._sock.recv(bytes_remaining) + else: + payload = b'' + except socket.error as err: + print("Socket receive error: {0}".format(err)) + return False + + return self._recv_callback(msg_type, payload) + + def _send_pingreq(self): + pkt = struct.pack('!BB', mqttConst.MSG_PINGREQ, 0) + return self.priority_send(pkt) + + def setPingFlag(self, flag): + self._pingSent=flag + + def _send_packet(self, packet): + written = -1 + try: + if self._sock: + written = self._sock.write(packet) + if(written == None): + written = -1 + else: + print('Packet sent. (Length: %d)' % written) + except socket.error as err: + print('Socket send error {0}'.format(err)) + return False + + return True if len(packet) == written else False + + def _verify_connection_state(self): + elapsed = time.time() - self._start_time + if not self._waiting_ping_resp and elapsed > self._ping_interval: + if self._connection_state == mqttConst.STATE_CONNECTED: + self._pingSent=False + self._send_pingreq() + self._waiting_ping_resp=True + elif self._connection_state == mqttConst.STATE_DISCONNECTED: + self._connect_helper() + + self._start_time = time.time() + elif self._waiting_ping_resp and (self._connection_state == mqttConst.STATE_CONNECTED or elapsed > self._mqttOperationTimeout): + if not self._pingSent: + if self._ping_failures <= self._ping_cutoff: + self._ping_failures+=1 + else: + self._connect_helper() + else: + self._ping_failures=0 + + self._start_time = time.time() + self._waiting_ping_resp = False + + def _io_thread_func(self): + time.sleep(5.0) + + self._start_time = time.time() + self._ping_failures=0 + while True: + + self._verify_connection_state() + + self._out_packet_mutex.acquire() + if self._ping_failures == 0: + if self._out_packet_mutex.locked() and len(self._output_queue) > 0: + packet=self._output_queue[0] + if self._send_packet(packet): + self._output_queue.pop(0) + self._out_packet_mutex.release() + + self._receive_packet() + self._callShadowCallback() + + if len(self._output_queue) >= self._draining_cutoff: + time.sleep(self._draining_interval) \ No newline at end of file diff --git a/lib/mqtt_aws/MQTTShadowManager.py b/lib/mqtt_aws/MQTTShadowManager.py new file mode 100644 index 0000000..a846317 --- /dev/null +++ b/lib/mqtt_aws/MQTTShadowManager.py @@ -0,0 +1,55 @@ +import _thread +import time + +class shadowManager: + + def __init__(self, MQTTClient): + if MQTTClient is None: + raise ValueError("MQTT Client is none") + + self._mqttClient = MQTTClient + self._subscribe_mutex = _thread.allocate_lock() + + def getClientID(self): + return self._mqttClient.getClientID() + + def _getDeltaTopic(self, shadowName): + return "$aws/things/" + str(shadowName) + "/shadow/update/delta" + + def _getNonDeltaTopics(self, shadowName, actionName): + generalTopic = "$aws/things/" + str(shadowName) + "/shadow/" + str(actionName) + acceptTopic = "$aws/things/" + str(shadowName) + "/shadow/" + str(actionName) + "/accepted" + rejectTopic = "$aws/things/" + str(shadowName) + "/shadow/" + str(actionName) + "/rejected" + + return (generalTopic, acceptTopic, rejectTopic) + + def shadowPublish(self, shadowName, shadowAction, payload): + (generalTopic, acceptTopic, rejectTopic) = self._getNonDeltaTopics(shadowName, shadowAction) + self._mqttClient.publish(generalTopic, payload, 0, False) + + def shadowSubscribe(self, shadowName, shadowAction, callback): + self._subscribe_mutex.acquire() + if shadowAction == "delta": + deltaTopic = self._getDeltaTopic(shadowName) + self._mqttClient.subscribe(deltaTopic, 0, callback) + else: + (generalTopic, acceptTopic, rejectTopic) = self._getNonDeltaTopics(shadowName, shadowAction) + self._mqttClient.subscribe(acceptTopic, 0, callback) + self._mqttClient.subscribe(rejectTopic, 0, callback) + time.sleep(2) + self._subscribe_mutex.release() + + def shadowUnsubscribe(self, srcShadowName, srcShadowAction): + self._subscribe_mutex.acquire() + currentShadowAction = _shadowAction(srcShadowName, srcShadowAction) + if shadowAction == "delta": + deltaTopic = self._getDeltaTopic(shadowName) + self._mqttClient.unsubscribe(deltaTopic) + else: + (generalTopic, acceptTopic, rejectTopic) = self._getNonDeltaTopics(shadowName, shadowAction) + self._mqttClient.unsubscribe(acceptTopic) + self._mqttClient.unsubscribe(rejectTopic) + self._subscribe_mutex.release() + + def insertShadowCallback(self, callback, payload, status, token): + self._mqttClient.insertShadowCallback(callback, payload, status, token) diff --git a/lib/onewire/onewire.py b/lib/onewire/onewire.py index ddf14df..2a9b096 100644 --- a/lib/onewire/onewire.py +++ b/lib/onewire/onewire.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python3 """ OneWire library for MicroPython diff --git a/lib/sqnsupgrade/README.md b/lib/sqnsupgrade/README.md new file mode 100644 index 0000000..e6ec433 --- /dev/null +++ b/lib/sqnsupgrade/README.md @@ -0,0 +1,265 @@ +# Modem Firmware Update + +_Note: This article is only related to GPy, FiPy, and G01 boards_ + + +Please read the following instructions carefully as there are some significant changes compared to the previous updater version. + +Most importantly, the updater is now integrated in the latest stable firmware release \(we will also publish a new development and pybytes firmware in the coming days\), so you no longer need to upload any scripts to your module. The built-in updater will take precedence over any scripts uploaded. + +Please start with the following steps: + +1. Upgrade the Pycom Firmware Updater tool to latest version +2. Select Firmware Type `stable` in the communication window to upgrade to latest stable version + +You can find the different versions of firmwares available here: http://software.pycom.io/downloads/sequans2.html + +These files are password protected, to download them you should be a forum.pycom.io member and access to: +Announcements & News --> Announcements only for members --> Firmware Files for Sequans LTE modem now are secured, or clicking Here + +We are using `CATM1-39529.zip` and `NB1-40343.zip` as examples in this tutorial. + +After unpacking the zip archive, you will find each firmware packages contains two files, one being the firmware file \(e.g. `upgdiff_33080-to-39529.dup` or `upgdiff_33080-to-40343.dup`\) and the `updater.elf` file, which is required when using the **"recovery"** firmware update method or if a previous upgrade failed and the modem is in **recovery** mode. + +to Know if your modem is in recovery mode or not, please execute the following code via the REPEL + +``` +import sqnsupgrade +sqnsupgrade.info() +``` + +For modems in recovery mode this will be displayed: +`Your modem is in recovery mode! Use firmware.dup and updater.elf to flash new firmware.` + +`firmware.dup` => here refers to the full firmware image eg. `CATM1-39529.dup` **NOT** the differential upgrade file eg. `upgdiff_33080-to-39529.dup` + +Please note that you have to choose the right .dup file when updating the modem Firmware , example if you are going to update modem with current FW version 33080 to FW version 39529, you should use `upgdiff_33080-to-39529.dup` and similarly for other Firmware versions. + +`sqnsupgrade.info()` will show the current firmware version: + +``` +Your modem is in application mode. Here is the current version: +UE5.0.0.0d +LR5.1.1.0-39529 + +IMEI: 354346099225475 +``` +Here it is **39529** + + +Please note that the `updater.elf` file is only around 300K so you can also store it inside the flash file system of the module. The firmware dup files will NOT fit into the available `/flash` file system on the module, so you either need to use an SD card or upload it directly from your computer. + +If you modem is **not** in recovery mode , you **don't** need the "updater.elf" file + + +To upgrade from the previous CAT-M1 firmware 38638 you can simply upload the `upgdiff_38638-to-39529.dup` file (452K) from the CATM1-39529.zip archive into the /flash directory on your module and run: + +```python +import sqnsupgrade +sqnsupgrade.run('upgdiff_38638-to-39529.dup') +``` +Similar upgrade packages are available for the NB-IoT firmwares. + +>When using differential upgrade packages (ex: upgdiff_XXXX-to-XXXX.dup) you **CANNOT** use updater.elf file. + +## Via SD card + +To transfer the firmware files onto the SD card you have two options: + +1. Format your SD card as with the FAT file system and then copy the files onto the card using your computer +2. Make sure your SD card has an MBR and a single primary partition, the format it directly on the module, mount it and transfer the firmware files onto the SD card using FTP. Please ensure the transfer is successful and that each file on the module has the same size as the original file on your PC. + +```python +from machine import SD + +sd = SD() +os.mkfs(sd) # format SD card +os.mount(sd, '/sd') # mount it +os.listdir('/sd') # list its content +``` + +Once you copied/uploaded the firmware files on to the SD card you can flash the LTE modem using the following command: + +To flash the CAT-M1 firmware onto your device using the recovery method: + +```python +import sqnsupgrade +sqnsupgrade.run('/sd/CATM1-39529.dup', '/sd/updater.elf') +``` + +To flash the NB-IoT firmware onto your device using the recovery method: + +```python +import sqnsupgrade +sqnsupgrade.run('/sd/NB1-40343.dup', '/sd/updater.elf') +``` + +Please note you can directly flash the desired firmware onto your module, it is not necessary to upgrade to the latest CAT-M1 firmware before switching to NB-IoT. + +If you have already mounted the SD card, please use the path you used when mounting it. Otherwise, if an absolute path other than `/flash` is specified, the script will automatically mount the SD card using the path specified. + +Once update is finished successfully you will have a summary of new updated versions. The full output from the upgrade will looks similar to this: + +``` +<<< Welcome to the SQN3330 firmware updater >>> +Attempting AT wakeup... +Starting STP (DO NOT DISCONNECT POWER!!!) +Session opened: version 1, max transfer 8192 bytes +Sending 54854 bytes: [########################################] 100% +Bootrom updated successfully, switching to upgrade mode +Attempting AT auto-negotiation... +Session opened: version 1, max transfer 2048 bytes +Sending 306076 bytes: [########################################] 100% +Attempting AT wakeup... +Upgrader loaded successfully, modem is in upgrade mode +Attempting AT wakeup... +Starting STP ON_THE_FLY +Session opened: version 1, max transfer 8192 bytes +Sending 5996938 bytes: [########################################] 100% +Code download done, returning to user mode +Resetting (DO NOT DISCONNECT POWER!!!)................ +Upgrade completed! +Here's the current firmware version: + +SYSTEM VERSION +============== + FIRMWARE VERSION + Bootloader0 : 5.1.1.0 [33080] + Bootloader1 : 5.1.1.0 [38638] + Bootloader2* : 5.1.1.0 [38638] + NV Info : 1.1,0,0 + Software : 5.1.1.0 [38638] by robot-soft at 2018-08-20 09:51:46 + UE : 5.0.0.0d + COMPONENTS + ZSP0 : 1.0.99-13604 + ZSP1 : 1.0.99-12341 +``` + + +Please note that the firmware update may seem to "stall" around 7-10% and again at 99%. This is not an indication of a failure but the fact that the modem has to do some tasks during and the updater will wait for these tasks to be completed. Unless the upgrade process is hanging for more than 5 minutes, **do not interrupt the process** as you will have to start again if you don't finish it. It may also take several minutes for the updater to load before responding to the AT wakeup command. + + +After you have updated your modem once using the recovery method, you can now flash your modem again using just the `firmware.dup` or `updiff_XXXX_to_XXXX.dup` file without specifying the `updater.elf` file. However, should the upgrade fail, your modem may end up in recovery mode and you will need the `updater.elf` file again. The updater will check for this and prompt you if using the `updater.elf` file is necessary. + +Example output using just the firmware file: + +``` +<<< Welcome to the SQN3330 firmware updater >>> +Attempting AT wakeup... + +Starting STP ON_THE_FLY +Session opened: version 1, max transfer 8192 bytes +Sending 5996938 bytes: [########################################] 100% +Code download done, returning to user mode +Resetting (DO NOT DISCONNECT POWER!!!)............................................................................ +Upgrade completed! +Here's the current firmware version: + +SYSTEM VERSION +============== + FIRMWARE VERSION + Bootloader0 : 5.1.1.0 [33080] + Bootloader1* : 5.1.1.0 [38638] + Bootloader2 : 5.1.1.0 [38638] + NV Info : 1.1,0,0 + Software : 5.1.1.0 [38638] by robot-soft at 2018-08-20 09:51:46 + UE : 5.0.0.0d + COMPONENTS + ZSP0 : 1.0.99-13604 + ZSP1 : 1.0.99-12341 +``` + +## Via UART Serial Interface + +If you can't use an SD card to hold the firmware images, you can use the existing UART interface you have with the board to load these firmware files from your Computer. + +You will need the following software installed on your computer: + +1. [Python 3](https://www.python.org/downloads), if it's not directly available through your OS distributor +2. [PySerial](https://pythonhosted.org/pyserial/pyserial.html#installation) + +You will also need to download the following Python scripts: [https://github.com/pycom/pycom-libraries/tree/master/lib/sqnsupgrade](https://github.com/pycom/pycom-libraries/tree/master/lib/sqnsupgrade) + +**Important**: When upgrading your modem for the first time, even if you have updated it in the past with the old firmware update method, you **MUST** use the "recovery" upgrade method described below. Otherwise, you will risk breaking your module. + +You can upload the `updater.elf` file to the module's flash file system rather than uploading it via UART directly to the modem, which will slightly increase the speed of the upgrade. + +First, you need to prepare your modem for upgrade mode by using the following commands. + +### **Commands to run on the Pycom module** + +To use the recovery method: + +```python +import sqnsupgrade +sqnsupgrade.uart(True) +``` + +To use the recovery method using the `updater.elf` file on the module**:** + +```python + import sqnsupgrade + sqnsupgrade.uart(True,'/flash/updater.elf') +``` + +To use the normal method: + +```python + import sqnsupgrade + sqnsupgrade.uart() +``` + +After this command is executed a message will be displayed asking you to close the port. + +```text +Going into MIRROR mode... please close this terminal to resume the upgrade via UART +``` + +### **Commands to be run on your computer** + +You must close the terminal/Atom or Visual Studio Code console to run the following commands from your computer: + +Go to the directory where you saved the `sqnsupgrade` scripts and run the following commands in terminal: + +When using the recovery method: + +```python +$ python3 +Python 3.6.5 (default, Apr 25 2018, 14:23:58) +[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin +Type "help", "copyright", "credits" or "license" for more information. +>>> +>>> import sqnsupgrade +>>> sqnsupgrade.run('Serial_Port', '/path/to/CATM1-39529.dup', '/path/to/updater.elf') +``` + +When using the standard method \(or if the `updater.elf` was loaded on the module\): + +```python + $ python3 + Python 3.6.5 (default, Apr 25 2018, 14:23:58) + [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin + Type "help", "copyright", "credits" or "license" for more information. + >>> + >>> import sqnsupgrade + >>> sqnsupgrade.run('Serial_Port', '/path/to/CATM1-39529.dup') +``` + +Please note that the firmware update may seem to "stall" around 7-10% and again at 99%. This is not an indication of a failure but the fact that the modem has to do some tasks during and the updater will wait for these tasks to be completed. Unless the upgrade process is hanging for more than 5 minutes, **do not interrupt the process** as you will have to start again if you don't finish it. It may also take several minutes for the updater to load before responding to the AT wakeup command. + +## Retrying process + +In case of any failure or interruption to the process of LTE modem upgrade you can repeat the same steps **after doing a hard reset to the board \(i.e disconnecting and reconnecting power\), pressing the reset button is not enough.** + +## Sqnsupgrade class + +The latest version of the `sqnsupgrade` class has a few additional features that help with debugging or modem update. + + +#### sqnsupgrade.info\(\) + If the modem is in application mode, the current firmware version is displayed. This behaviour replaces the version() command which now is only available in uart() mode. Optional parameters are sqnsupgrade.info(verbose=False, debug=False) + +#### sqnsupgrade.run\(load_fff=True\) + New optional command line option load_fff for the sqnsupgrade.run() command. This is designed to be an internal flag. And should only applied when advised by pycom support. + + diff --git a/lib/sqnsupgrade/sqnsbr.py b/lib/sqnsupgrade/sqnsbr.py new file mode 100644 index 0000000..ba70db2 --- /dev/null +++ b/lib/sqnsupgrade/sqnsbr.py @@ -0,0 +1,52 @@ +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +class bootrom(object): + import binascii + + def __init__(self): + self.__fpointer = 0 + self.__size = len(self.BOOTROM) + + def open(self, filename='', mode='r'): + self.__fpointer = 0 + + def close(self): + self.__fpointer = 0 + + def tell(self): + return self.__fpointer + + def seek(self, offset, start=0): + if start == 1: + start = self.__fpointer + elif start == 2: + start = self.__size + if start + offset < 0: + raise OSError(22,'Invalid argument') + else: + self.__fpointer = start + offset + return self.__fpointer + + def get_size(self): + return self.__size + + def read(self, size=0): + if size == 0: + size = self.__size - self.__fpointer + if self.__fpointer >= self.__size: + return b'' + else: + fpointer = self.__fpointer + self.__fpointer = fpointer + size + + if fpointer + size >= self.__size: + return self.BOOTROM[fpointer:self.__size] + else: + return self.BOOTROM[fpointer:self.__fpointer] + + BOOTROM=binascii.a2b_base64(b"""VVBHUgMAAADk2AAAAQAEABgAYm9vdHJvbSB1cGdyYWRlIHBhY2thZ2UAdG9vYgAAAAIAAAABc2JwawAAvqgAAAB0AAAZrEG97ANJAAAYSAABM5IAkgBib29vc3FuMQAAAAAAAQABAAAAAAAAAAABI0VnAGAAAABgvphHCCACWQgMAGQFAAJCAAQLTgMACEYX//9YEI//QQgEAgEIAAFPAgFdhCJNAIFah7hB3vgARgAgAUQQEAASEAAKhCEQEAAYhCAQEAAZRBAhHxIQAAqEIRAQABiEIhAQABmEIa5EhCAQEAAIZBIAAmYQgAdkEgADZBMAAoQgZBMAA2QVAAJmEIADZBUAA0kAAXVkFAACRCgAAP5WWBCAFGQUAANJAAFIZBAAAkYgD/D+VpIwXBCAJE4SAApkFeACZhCAB1gQgAFkFeADZBUAAlgQgANkFQADZBIkAkYQAP9kEiQDRhP//1gQj/9JAAACQBD4Aowo3QEFDoAIQG7AAIdATwIAnYT81QNZvYAChgid/Ex4QAeBJlm9gAGBHdU0QQMcAAUIAAFPAv/zQIh0AAUUAARHJib2WSkPb00Zf+mACPo0SQAAogUEAAkElAAITQX/3YkoBBSAA/uEiDBApIQAgAlJAACStYpQBQAITQX/zgQVAAGeTECgBAC1SkkAAIVMpf/EBQSAA1FEgByPiE8HABiPjAU6AAIFKgABtbRRSgAMj+FPMgAHj+FPMgAHt9HV7xEogADV7BMogADV6QQkgARGAAYAWAAAAIAoRQD//4IwglBBkIgAUTCC0AVAgABA/MwGTvIAA4J5nEwVQAAAnARBWkAJibWJ0UFaQAIFQIAAibWJ0UwZ//JJAABdBVCAAOI5TvP/4kkAAFZBKUAIibJVDYABTVj/cYNITwMAB0cABgBZCAAI3RBHAAYLWQgOqEcQBhhZGIzMQSlIA7fQjYRNCP/+R/AG8Fn/gBBGAAAAWAAAAEIOACFH0AYRWd6MyEfAAACDnEcABgRZCA04gBqAO90QRQD//4IwglCYSIM+UCABaOIiTvIAA4BBAzAAAJwCibOJ0UwBf/tJAAAM4gFO8//wSQAAB0EpQAhAuMgA3RlBOMACQUjACUEZ0ABBOUACQUlACUEp0ADdHkkAAFNJAAEcZBQAAkQoAAD+VlgQgBRkFAADSQAAHWQQAAJGIA/w/laSMFwQgCROEgAKZBXgAmYQgAdYEIABZBXgA2QVAAJYEIADZBUAA4YAtbDdEWQAAOFlAEACRAAAQFUYAANBEEQMhAFVKAAYk8NBIEgMhARVOADAk+ZBMEwMQwjIJEMITCSLk2QIAgFkAAAJTwP/+92egR5JAAEJSQAABEgAADyAnkkAAA3vAgQxIHcEQAY1wkAGAaXAAQIEBf8AgD5GYCAAhoGGBEVQAFA5A1QIUVqAAYYAOQNUCFFagAE5A1QIUVqAATkDVAhFUAAwRRAAnzkTVAmGY0kAAHgCAwAJRCAA/6bITAGABowqTDF//N2IgOHdhIYEAROACYZAhmFFUABQOQNUCFFagAGGADkDVAhFUAAwORNUCVFagAI5I1QJSQAAUwIDAAmmef5GThMAJIYEhiaGYEkAADcBE4ADASOAAYZhSQAAMEkAAEimeEQgAO9MIP/RRRAAUIZgSQAAJEUQADGGQoZhSQAAHkkAADZI///CSQAAUUUQAOsREwAIAROAAhETAHIBE4AEERMAcIYhERMAcxETAHRkAAAIZAAACd2IhgRFUABQOQNUCFFagAE5A1QIRVAAMDkTVAlRWoACOSNUCREzAGQRQwBgAVMAYE9T//7dnoBehgSGJYZAhmFFUABQOQNUCFFagAGGADkDVAhFUAAwORNUCVFagAI5I1QJSf//4QIDAAlCEAALThP/+t2CgD5GYCAAhoFFUABQhgQ5A1QIUVqAAQEDgAg5A1QIUVqAATkDVAhRWoABOQNUCFFagAE5A1QIRVAAMAETgAU5E1QJAROAB1FagAI5E1QJUVqAAjkTVAlRWoACORNUCVFagAI5E1QJATOABkn//6PdgUcABgtZCA6oRxAGAFkYhb5HIAYAWSkDsIuyDT8AAR04AAGPpE8W//tHAAYLWQgOqN0QSAABRkAAAAlAAAAJQAAACTuP7DxFsAABSAABXEAAAAk7j+w8RbAAAkgAAVRAAAAJO4/sPEWwAANIAAFMQAAACTuP7DxFsAAESAABREAAAAk7j+w8RbAABUgAATxAAAAJO4/sPEWwAAZIAAE0QAAACTuP7DxFsAAHSAABLEAAAAk7j+w8RbAACEgAASRAAAAJO4/sPEWwAAlIAAEcQAAACTuP7DxFsAAKSAABFEAAAAk7j+w8RbAAC0gAAQxAAAAJO4/sPEWwAAxIAAEEQAAACTuP7DxFsAANSAAA/EAAAAk7j+w8RbAADkgAAPRAAAAJO4/sPEWwAA9IAADsQAAACTuP7DxFsAAQSAAA5EAAAAk7j+w8RbAAEUgAANxAAAAJO4/sPEWwABJIAADUQAAACTuP7DxFsAATSAAAzEAAAAk7j+w8RbAAFEgAAMRAAAAJO4/sPEWwABVIAAC8QAAACTuP7DxFsAAWSAAAtEAAAAk7j+w8RbAAF0gAAKxAAAAJO4/sPEWwABhIAACkQAAACTuP7DxFsAAZSAAAnEAAAAk7j+w8RbAAGkgAAJRAAAAJO4/sPEWwABtIAACMQAAACTuP7DxFsAAcSAAAhEAAAAk7j+w8RbAAHUgAAHxAAAAJO4/sPEWwAB5IAAB0QAAACTuP7DxFsAAfSAAAbEAAAAk7j+w8RbAAIEgAAGRAAAAJO4/sPEWwACFIAABcQAAACTuP7DxFsAAiSAAAVEAAAAk7j+w8RbAAI0gAAExAAAAJO4/sPEWwACRIAABEQAAACTuP7DxFsAAlSAAAPEAAAAk7j+w8RbAAJkgAADRAAAAJO4/sPEWwACdIAAAsQAAACTuP7DxFsAAoSAAAJGQCZAKWH04DAAhG8AYAWPeAAEoAPACEIUwAwAhG8AYAWPeAAEoAPABkAyQClhZOAwAIRvAGAFj3inhKADwAh2tIAAACg1+DPmWCAAJVjAAgT4MABkkAABQF/4AAOg/vvEkAABK1kIAb3TA6D++Eg9mD+juP7ARkAAAEkgBL8HgBAAAAAEsAeAEAAAAAQAAACUAAAAn8AIDAyAlGAAYGWAAMiEkAAU5J//tvRgAGDFgAAsA4EBoCwQVQA3/33SH8gIAmRgAGBlgADJBJAAE6ZEJEAmQiZAJkMqQCRgAGBlgADKiAJkkAAS3VAMAEZBIAQ92eZAIAQ92eZAIAApYE3Z78AI4BXAAAAUn///H8gPwAZCIAAmQzAAJAMYAS/kb+X2QTAAOWFEn//+38gPxARnAGAFhzhfBGkAYAWJSJCIsnRmAG8FhjAACASYAngAZJAA9ZgAaAKUkAAF2ABoApSQAAaZp3RiAGAFghCRBGAAYAWAAJBDgggApGAAYMWAACvIDhZgAAB0YQBgBYEIj8OAOECkQPAAD/hlhjQABkYiQDhABkAwADZAMEA0ZgBgtYYwywRnAGC1hzjMiFIUagBgxYpQLA4sfoDaAxQASADIAgSf//n6AxonKMCTgVAgrV82QSAAJEAwAA/g9kAgADZAAACGQAAAlkEgBD/MD8AEYABgZYAAzkSQAAnPyA+hDdnvwASQAAE/yA/ABJAAAa/ID8AEkAAC/8gPwASQAAKPyA/ABJAAAZ/ICIIGYAAB/iAegGZAABIVAAACDV+t2eiCBmAAAf4gHoBmQAAQFQAAAg1frdnoggZgAAH+IB6AZkAAMBUAAAINX63Z5kAAAM3Z5kAAHhZAAA4WQAAAzdnoAgPAwaTsAF/ABJAAmS/IDdnvwAgMAgAwAAwAtaCAoFhA1J///vKAMAAUn//+vV9PyA/EOAwIEhgOKAH4Qg+kVJAA6SxgOAH9UMRAAAMBAPgADVFMMTWpAKDlqQEAeMAUBjJHfO+dX35mrpBFBBgDfVA1BBgDCvANXzgB9JAA6HgMCgOcAOpnjiBkBgPBvBCZvGxwdEAAAwSf//tI7h1frGB47BOA+YEEn//6zV+vzDOh+UPO/8Om+svEYQBgZYEI0E7/Q7AMQAgUBEAABbOw/EIEn//5c8DBwASQAO74QqgF9J//+oRAAAXUn//4v6EIE/Sf//h7GLhOCFYSgFAAFOAgCoWgglDMcESf//e9VTEHSAABR0gAFIAACZTnIAjFoAZD5e8ABl6Ble8ABI6A9e8ABF6GBe8AAwTvMAiF7wADrpYFoAQVhIAACBWgBhVFoAYzZaCFh71UBaAG87XvAAcOgHXvAAaOlGWgBpGdVuWgBzN17wAHToDVoIcGid9LTGRgAGBlgADQBJ//9EgAbVLFoAdRtaAHgh1Vi05ozETnQADPodSf//Lf46hCqASUn//0GE4NWjgAfV+SADAAOd9En//x/VFZ30tAaEKtUOnfS0BoQo1Qqd9LQG1Qa0Bp30Sf//GNUF+iCASUn//yOAx9XhjMdmAwAHUGAACKAB1dZaCDALABSAAMkEELSAANUdBBSAAcEaBCSAAcIFhCr+VBQUgAGASaBRUBD/0IggqFHVDFoICgWEDUn//uMgBX//Sf/+30j//1uE4Uj//1jsDDpvrITsGN2ehAQ8D+qzhAA8D+qx3Z78AEkAAoiEIEAAgAb8gPxgPA3qs1oAARLAHFoAAgRIAAFUhOBGkAYMWJSDeEagBgtYpQzYSAABK4QARhAGDFgQh8w8M9V+PEPVfYDAgSHVTUYABgZYAA0cSf/++EYABgxYAAeYRhAGBlgQjwiER0kADWHAB0YABgZYAA00SAAA3Dwd6rtSEIA0PB/qrsEHRgAGBlgADVBJ//7ZPAPVfTwT1X7+REYABgZYAA1wPB/qr0n//sw8DeqvXPAEAekHRgAGBlgADYxIAAC3RgAGDFgAB8w8D+qwSAAA9cYKoUmgseKi6QaIJIwB4gPp+NUMtEFaKAEGoUygjdr21QNaKAT0gMHV8c4PRgAGBlgADaxJ//6dPA3qsjwd6rpJAAucSAAA1KAxPB3qsaH0igE8D+quPH/qr88NRgAGBlgADcxJ//6FhAE8D+qzPH/qrtV3LgeqtKBzyAZGMAYKWDGCpNUFRjAGBlgxjQyAR0YABgZYAA3kSf/+a7QGPC3qr1oIBB9c8QQB6Q9GAAYGWAAOAEn//l2EATwP6rM8DeqvPA/qrtVNRgAGDFgAA3g8D+qwhAI8L+qsPA/qs9VBoTM8DeqyPE/qsOKA6AM8T+qyRjP//1Axj/9AEgwCRvAAAFj3gACIROIv/p7pE0YACABYAAAA4gLpDUYABgBYAAAA4gLoDkbwBwBY94AA4i/oCEYABgZYAA4oSf/+GtUgPBPVfUADJAFAAAQWSQAB7cgIRgAGBlgADkxJ//4K1RCEALYG1USNaEy1ABm0q6Ay2PsEJYABgAygcd0iyPWEA9U0QGOkAEYABgZYAA5woHJJ//3vtEZaIAQUtAaMA2YQAAOgMYwDZgAAA4gBjAyI4Dwd6qxQA4AM4gHp5NURUAMADEYQBgZYEI6ESQAMRsjmRrAGC1i1jMhQwwAQ1cY8Deqz5gPoBIQBPA/qs/zg/GCFYIEggUGAwYUBTmIAuDwN6rPmA07yALM8LequwhXiRjwN6rHpCogGikY8D+qxPC/qroAKSAAAq4gCPA/qsYrCiSI8v+quPH3qr05yAJQuB6q04uZAczwawCqEHzwP6aI8DeqwPA/poYQiRgAGDFgAA0A8zemjPH/pnzyf6Z5JAA5N5gKAIOkMRgAGBlgADog8LemkSf/9dYQDPA/qszwN6aM8HeqwigyIAdUKPA3qsIApgEdJAAqCPA3qsIgHPB3qsTwP6rA8DeqviseKB4kniOE8D+qvPH/qsciYLheqtMEJPgeqtEYABgxYAANASQAVa+bE6SuACUYQBgZYEI6whERJAAuyyCI8D+mfPA/pnvowRgAGDFgAA0BGIAYGWCEOuEQwADg+h6q0SQANsoAgwA1GAAYGWAAOwDwt6aRJ//0ghAM8D+qz1RlJ//39LgeqtE4C/1w8DeqvjsSOBDwP6q88DeqxjSSMBDwP6rFI//9OSf/96Uj//0o8DeqzWgADBUAFGAHVAoQf/OD8QTx96rFGYAYMWGMHmECgnADnSPCBgSE8b+qw6QNSk4AHiMeASfEBgAaJJ0kAC0RGAAYMWAAHmEYQBgZYEI8IgElJAAtMyBznJ+gFPK/qsYQB1Rc8D+qzUnOANIQfPA/qsjx/6q88b+qwSQAAvEYABgZYAA7sSf/8voQC1QKEAPzB/CHwgYDBRgAGBlgADxDxAUn//LBQcwAgCBMAAUYABgZYAA8gSf/8pkxj//hGAAYKWAADvEn//J78ofxVgOCmAEaQBPSmeVCUiwBAAIEEFJ+AAVoIAgemOqZ7QACBBMgJRgAGBlgADyhJ//yEhADVXIQAsEb6UEkABdjIBkYABgZYAA9Y1fJQY4AURgAGBlgAD4iwRkn//7aAJkYABgZYAA+QSf//r7AOSQAjGoAmsA76UEkAIyiwDrBG+lBJACMjRmAGDFhjC9CERLAOsEFJACMasA6AJkkAIzlGAAYGWAAPmIAmSf//joAGRhAGDFgQi/BJAAo3sAKcfPpASQAJV0kACfr1AkxUgAdGAAYGWAAPpNWrRgAGBlgAD8hJ//wqPG/rwYQB/NX8AfCBPA3rwcgHSQAFeFYAAAGWANUR8QFGAAYGWAAP5En//BQ8DevBRhAGDFgQi/BJAAoEhAH8gfwASQAJyYQAPA/rwfyA/AA8PeQAtECKQ0YwBgpYMY7AtkCAw4RAPE3kfuJE6BahGZckxA+0gKFd4oXpBIqFtoDVCMEDhAC2AUADCKCMBNUGjEFQMYAg1eiEAPyA/GFGkAYMWJSMBDhUggKAwIDi0h2EYUChmAyEQIQGgIKBYRAPgASwQYAKSQAIAwAFgACEQBAPgASwQYAKhGKAghB/gAVJAAf2OHSaCvzh/CBGYAYKWGMO3ITgPA3kfuLg6BUEA3/6lgTADYQAgECAJkn//8iEADwd684EI3/+SQAH8IzhUGMAINXp/KD8IDxN5J+EoITMRnAGClhzjsDUHYBnQjKYcwUBgCJNAEAVwgQEAYAktgLBE4RMRgAGClgADsBCAohzBCAAIzwN5ACIArYB1QWModXkhADVAoQB/KD8ZfGB8IaBYLBHsAa2X0n//2KA4MBy8AdGEAYMWBCMBDgQggJQw4AY8YIAL4AYgCxJ//948AeFIUCUgAwE34AABI+AAYXGUK+AIIeFTtIARaG68AZAABg3wQKKwYRAQPaYBoAJgCqEYYCCQGa8GxDvgCBJAAdvoPmEIvAGEBUAAJ1ZxQY4BRQIkgiOodX7gEWACYAqjGKEgfWDSQAHW/UDgAmAKIBmgEWAhUkAB1PwBkDWmAGJBojA9oaACbBFgEqEYoSAEc+AFEkAB0QABQABlgTI9NW88gLwB4AsSf//IoALtD9J//plgAvxAbRfSQAJZ1wAAAHVAfzl/AO2HzwN5ADxgfCFsESwBUn//uTICEYABgdYAAAcPB3kANUfRA//nxAPgAjwBIQhQACADLCDsEKEZISASQAHDQAfgA1aEMIJWhDvE0YABgdYAAA81QZGEAYHWBCABNUMSf/6vUYQBgZYEI/81QVGEAYHWBCAELQf8gFJAAkR/IOcRoRAGCAAAUwA//7dnvwAPG3r0EkAJTWEKEAQBDb6FEJggHOABvyA/EGBIfCBSf//8IDA8AGE6EkAJSNAEBwWhOFAc4AMWpABDk6SAAlakAIESAAAkoFHgSfVBoVAgSrVA4EnhUCEIPABSQAlEkn/+VOmsaZwQCFACEAhBwSmckAhBQSmc/6PptCWeP7PrtCmtab0QCFACEAhDwSm9v8LQCENBKb3lyD+n6bQ/s+u0AAjABEAMwAQQCFACEAhDwQAMwASQBCkAkAhDQQAMwAT/p+m0P7mrtAAIwAVADMAFEAhQAhAIQ8EADMAFkAhDQQAMwAX/p+m0P8erxAAIwAJADMACEAhQAhAIQ8EADMACkAhDQQAMwAL/p+m0EBxnBJAdRwEl/iv0AAjABkAMwAYQCFACEAhDwQAMwAaQCENBAAzABv+n4RgrtAAIwARADMAEEAhQAhAIQ8EADMAEkAhDQQAMwAT/p+m0ECRhAQQkQAASf/41fzB/ENJACP5PA/r0MgIRgAGB1gAAFRJ//nn1WeEQICihN+YQgAwgBEAQIAQQDHACEAxkwQAQIASUCEAJEAxkQQAQIAT/uevWAAwgBUAQIAUQDHACEAxkwQAQIAWQDGRBABAgBf+56+YADCAGQBAgBhAMcAIQDGTBABAgBoAEIAbQDGRBP5fr0haKNjNgB9J//7+sAJJ//77sARJ//74RnAGC1hzjPBGkAYLWJSNAEx0gB20B0kAI1FaB/8WobqoO1pgAQ5aYAIMhCDOCkkAIy5aAAEEWggCBIAg1QKAJqA7SQAjeozw1eT8w/xABJAAAoDgWpgBKLQASQAkCIFAtAc8bevQSQAkAoSIQAAQFvo0gGZCMARzABGAHQABgBwAIYAeQBDACEAQgwQAAYAfQBCJBEBFEFb+R0BEiAymiP8XrwgEk4ADWpgBKLQHSQAj3oFAtAc8bevQSQAj2ISIQAAQFvo0gGZCMARzABGAIQABgCAAIYAiQBDACEAQgwQAAYAjQBCJBEBFEFb+R0BEiAymiP8XrwiEAfzA/AHwgUn//n2AwPABSQAjsQATAA0AIwAMQBDACEAQiwQAIwAOADMAHEAQiQQAIwAP/leESEAgCBaEQUABAAwAIwAdpwhAIUAIQCEPBAAzAB4AEwAfQCENBP6PplD+Zf4OhCBAAIAG/IH8QYEh8IFJ//5IgMDwAUkAI3wAIwAhADMAIEAhQAhAIQ8EhCgAMwAiQAAFVgADACNAIQ0E/oeE4aaQQHOoDP6+yinwAYApSQAjaKa1pvRAIUAIQCEPBKb2pjdAIQ0E/oemEEAUqAxAE4QSQAAcEv4PlgCuEKYxpnBAAEAIQAAHBKZyQAAFBKZz/g+mQP/Pl/ivwPzB/ACEIEn//7T8gPwhgOHwgUn//fmAwPABSQAjLYQoQDAEFoRBADMAHUAhAAxAE4AMAAMAHABDAB5AMcAIQDGDBAADAB9AMZEE/sem2AADACD+XQAzACEAQwAiQDHACEAxgwQAAwAjQDGRBP7HphgAQwAJQDCAAkBxiAL+xQADAAhAQkAIAFMACkBCAwQAAwALQEIVBP8HpiD+1kAACBL+jv6Hxw+mNaZ0QABACEAABwSmdkAABQSmd/4PpkD/z6/AAAMACQATAAhAAEAIQAAHBAATAAqWkEAABQQAEwAL/g+ugMMQpjWmdEAAQAhAAAcEpnZAAAUEpnf+D6ZAQDCMEq7A/KH8AfCBSf//gfABhCFJ//8u/IH8IoAfSf/9bTxN69CEAPq0gGRCMBRzABGAGQAhgBhAEMAIQBCLBAAhgBpAEIkEACGAG/5Xpkg4H4AIjAFaCAbqsAJJ//1OhMCE4bAEQAAYfAAAf/BUEwAHQAAEDpYEwA6ABkkAIoGwQkAggHxUEAAHphBAE4QM/keuUIzBWmgw50ZgBgtYYwzwRnAGC1hzjQBMY4AWoDNaB/8RoLHCDrBEQBCAfAAQ//hUMAAHQBCMDpZMwQO0Jt0ijNDV64QAPF3r0ICAmOgAEYAZACGAGEAQwAhAEIsEACGAGlAAACRAEIkEACGAG/5XrwhaCNjt/KI8LeQ5hKBGMAYKWDGNrNIKQBGUoKEJTEBABJwM3Z6ModX3hADdnkZQBgtYUo44RhAGC1gQjpjRCAAigBhMIAAHUFKAINX5hADdnoAF3Z78AEn//9igAfyAoIEAEAAIptBAEYQSrlCggQAQAAimEP4OyP7dnvwAgMBJ///woHEAAwAIpoj+F64IoLEAEwAIphD+DsD+/ID8IEZgBgtYYw44RnAGC1hzjphMY4AJoHTBA4AG3SFQYwAg1fj8oLQgAACAFJYEwP2mCJYC3Z60IAAAgAhYAAACEACACN2etCAAAIAUVAAAYFoIYPzdnvxAgSAAAAAYgUFJ//+FtMmA4AADADDABIAJSf//6wQkgAWEABADADBAESg3oLzCAvpAoPugOv6flpBaCAEFWCEACNUFWggCBFghABhYAQCAEAMADJYIrjBAAKAJlgCSMK40lmeEAhATACAQIwAMEAMAEKA9WggECAADABBYAAAgEAMAEIQGEAMACIQBEAMAMPzA/EBGYAYLWGMOOEZwBgtYc46YTGOAFgADABhJ//80oEfJDKCzBJAAAcIIgAbdIsAFgAaAKUn//51QYwAg1ev8wPwASf//lvyA/EC0wIEgAAMAEIDhli7ADQADABiWJsgJRgAGB1gAAIwAFIAYSf/2r4AJl/hJ//91r/D8wIQA3Z7IDDxd/APaC/wAgAE8HfwCSQAE/oQB/ICEAN2e3Z7IBzwN59u2ATwN59rdnoQA3Z78AYQEgD9AL4AASf/69MAJtB+0IFoX/waMBMADSQAALPyB5jDpJvwgRiP//1AhD/9AIQRcOHAKAo4kgMBJAB4uTHBAFIAGRhAGB1gQgMiEQ0kABNnADYAGRhAGB1gQgMSEQ0kABNDAA4Qf1QKEAfyghB/dnvxChOCAwIEh94NJ///SPE3r0eLkTAfAF6ZzoPKMIYxhUCT//FwxgAGMgbACQBMEgIhGED+ACDxP69FJAAHv8IHVAveB8AH8wvxBRmAGC1hjDQBGkAYLWJSNEExkgBCE4LRGgAewQd0iwAfxAUn//8rAA4zh1faMyNXx/MGEQIBgtkGAArRBjAGUl7ZBpxiXN/6ntkEoIYABTiX/9t2etCCmyKaJjCK2IEABDQTdngAA/GWA4EYAf/9QAA/A8IRGAAD9UAAP//GCgSK2f/SD8IW0H+MgTvIBmYVgtD8IBIAB4ilO8wGUQBAMCVTAAAdAsKykWsAH9MgFhAGuOUgAAYVaAAjohMBayAEJgAmwSUn//7f2CYkgiMnwAoUAUKAABfACpgTjAOgWUAUAAbBJSf//p/UJTFXABowBiArIB9UKAAUAAI0BiUDV7Fq4DxOmeMkQWsgBBEgAAVVawAIESAABVoDJKAMAAU4F//5IAAFKCKAAAfCHRgAGB1gAAMw4ADAAVNUAB7V/QAA0DpYEQLMYG04CATWwB0n//4PxA0CABABABRAJlg9aAAEdwBhaCAInsAdJ//91gcCwBwSjgAFJ//9v8QOIAUHFAACwSPAHSf//VvEHiAHwh9UThAHwiNUOsAdJ//9dgcCwSPAHSf//R/EHh4CIAfCH1QOFwIOO8AfwgYVB8AjiCk7zAO3wAfCHXPaABoQArjlO8gDdRvAGAlj3gUQ4B7UBQPA8AN0PALgADAEyATIBigAMsEnwAUn//x7xB4IJiAGEIPCHgKGAAQhIAAGUz0AxgzVVEgB/lyKMoYAjQAiA5E5F//SJJVrYBRmnOMwWUPB/wUBHgAaOIYiBzA/yBOJP6QxQ8GAAQEeABogkThMAn/EF4i9O8wCb9AmgeY6B5oiIKE7yAJRG8AYCWPeB2DhHkQFA8jwASgA8AAAQABYBEAAaARABEAEQAB6uCEgAAH+sCNV8tgHVerZhqAnVd7BJ8AFJ//7I8QeAqYgBhCDwh4CBgAEIMoABlE9VAYB/ltpAEIM1jIFACADkTjX/9ZbEwwP+S/4DiST0CaD5joGSAeaIQAAH5IhokCHoUEbwBgJY94JcOEeQAEDyPADdDwgMjBCMjIwUrhjVQawY1T+2A9U9tiOoGdU6QMWkATzt+9JQNgADRABAAGYxgAOKDuIDoHnpQGZHAANGAAYMWAAMEIgEgEM4AKAKgCnzgUkAAunzAUAhuAA8L/vSWtgCNQSTgAGwB0n//niJCTjEAArVK/ABgEmmwPACgIi0IIAHOBCOAoBrSf/+cYEg4yvoB1rIAQaJDo1BSP//E4AKT8IACqZ5wQbxCONB6ANQBQABthyBaUBlmBqBJkj//meACdUGtB/VBIAKgSvV6vzlkgD8QkagBgtYpQ3A8IGAwYEi4snoMoQgCAMAAeMm6StAIAwJQBEEpFQgAAdaIAf2wCRaAAjwRlAGC1hSjRBMVQAHtAVMAIAGjLDV+oTg1QKA5YAGsENJ//4IiMDzA8cJiGbwAaB6gEahOUn//hjzA4jD1dCEANUChAH8wvwARgggAAAQAAiAwFoYCwhGAAYHWAAA0En/8/6EIUYIIAEQEAwdhACuMvyA/CCU31gxgARGaCAARnQQAJbYhKBQYwBQjPhGSCAA0guZLq7gmS85ABQAlSETAgAAjKHV9J4RlgAQAgBkhAEQAgBgAFIAYJdozf3BDEYEEACMCNIImOiU2aTYODCUCIyh1fn8oPxBgSGBQoDj9IGAw5o+5tFAFQAAiAnpB/pAhGGO0En//7/V9YBG8wFJ//+6/MHdnjwN/AjACaACwAf8AN0ghAA8D/wI/ICEADwP/AjdnoQC3Z6AAd2e3Z48DfwEwCv8AEkAGLc8HfwFwROEwEYABgdYAAEMPG/kzUn/84aABjwd/AVGIAYKWCEP9EkAGK48HfwEwQ9GAAYHWAABDEn/83Q8DfwERhAGC1gQgAxJABie/IDdnvwASf//vYQAPgeTBIQAPA/8BDwP/AU+B/AcPA38BowBPA/8BvyA/ACEADwP/AU8D/wESf//pYQBPgeTBPyALieTBMpA/ECBIDwN/AiAwcgeRnAGC1hzjcBGoAYLWKUN4Ex1ABWhezwN/AbQDrRHgAmAJt0iWggCBTx//AjVB8gEPA38Bqg7jPDV7DwN/AiAJqCBgAndIk4EAA2EADwP/AU8D/wESf//bIQBPgeTBNUGmnDBBIgJSf//xC4nkwSAAvzAgALdnjwN/AXIBjwd/ARAAAQG3Z6EAd2e/ACAwUn//088b/wE/IAuB/AcyAX8AEn//1r8gN2ehCBGCCADEhAEh4QhEBB4Ed2e/GCBQYDigMBJ//JGgWBABSxXwgNAJYgB4uJAI7wbgSLCCIAGgCpJAAExiMmJSYrpPJ38Ck6TADD6EEkAAlw8D/wKwCNGGCADUBCJAIRBEiCAQxKQgAeAQaTOw/9GE///UBCP//4OtgKEAhIBAAiEAKwTRhggAoQBEgEAChIBAAysFxAAiBHVB4AGgCqAR0kAAP3VW49hQAMsAsAFgAaEIUn/8fyYN0AQLALBBIQhSf/x9Ua4IANQtYkARsP//0SA//9Qxg//gavHQOMHgcfoA0Tg//8CBYADQBUwAkCQABM8DfwKQcSQCIgcqEFAEzACqEI8HfwJrEAS4AAB+iBJ//HNPB38CoAKiDykSY0hSf/xxVqYAgOFIEn/8cxABIATEgaAAzwN/AmALmYAEAA8D/wJgAZJ//G2AlWAAkxU//6K7olOiM7VwfzgPB38CVQAkAFaCAEr/CFGeCADUHOJAKW7PA38CpexQAAYgIREZhCYAKyBWBCIALCBrECogaiC+iCMwUn/8YpaaAIDhMBJ//GRljGsO6V63v88DfwJZgAQATwP/An8oTwN/AlmABABPA/8Cd2e/AA8LfwKwj6AwTozDABQEQAMOjCMIDoTBACMXDoRBCBGKCADOjAMAFARCWQ6MIwgnEQ6EIQAUDEJTDoRhCBQEAAIUDEJNDoQhAA6EYQgjAxQIQkcOgAAADoBACBJ//+cPB38CaY0ZhCHxFQgAARYEJABlAP+V1QAB8D+DzwP/An8gEYABgdYAAEkRBAAxEYgBgdYIQEsSQAZxPwARjP//4OAUDGP/0BAjAK5lFAQAED+zryREiAAJ6jBgAH6MEn/8RL8gIhAgKDSBBgSgAHV/d2e/AHwgYhBTBEABygwgAEYMAAB1frwAfyBgCCaiCgwgAHL/YAC3Z6AoIhA0ggIMoABCACAAZoYwPrVAoQA3Z4IUAABCCCAAdIF4qLpBoQB3Z7N94AF3Z6EH92ehKDSC4BhKECAAThAFAggMYAAwwmModX2xQaIoIQgEBL//92e3Z78QIDAgSHAN2YRABCA4skSSf//xOYD6QwgAwAAWggwCSADAAFaCHgFjML64NUDzwKE6oCGhACAxCgyAAHDHJaYUBF/0OYq6ARQMf/Q1Q5QEX+f5jroBFAx/6nVB1Ahf7/mWugJUDH/yeJn6AVCMBxzgAPV4k6SAAO2yfzA/ACEYEYABgtYAA4gRBAD6EYgBgJYIQmaPD4cAEkAAFf8gPwAPAwcAEQQA+iMATwOHABGIAYCWCEJmkYABgtYAA4ghGBJAABD/IDdnkZQBgtYUo4gRhAGC1gQjjjRL6CpTCAABIy41fu0RaBtAAEADKAsQDAEBMsGEDEACKBqyRrdngAxAAiWzsMLtGKay+Ijig+oLIQBqO0QAQAI3Z5GL/8AiEFAEQQGjgGIAagsqK3dnvwAoCvdIfyA3Z60AIQgEBAACKBByf/dnvxggMCBYoFDgSG04En///OgMYSBQAIADIAgSf/vl0QBlidEMAPohEAUswACFKMAA0IEgGlJABIdgKBGMAD/gIFQIY//zQriQekIhGCEQKi0qPWEA7Yn1Q5AMggCyw5GL/8AiEGOAUARBAaIIKh0qLWEARADgAjVFkYgAQCZGpKBtoe0h5sM4iRAEDwBhAOodKk1EAOACKA5wP+MQYhDkkG2R/zgjB9mEAAfPA3kwkYgBhlYIQD/ikDiQekFiCA8H+TC3Z6EAN2e/ACEBIAgSf/vPUYoIAVGAe/KUBEGAFAACbioCYQBEAEGABABBgz8gPwARgAGB1gAATRJ//A/1QD8QKGHgUEEEwANgSLBDQQDAArIGYOGugmEIUAQiAy5iriMuIvVEAQTAAmDgIThuwi4CkAThAyAR90jFAMADcjogAfVO4OGugq4DeMi6QuDhkAVCAFJ//5zhAC5CriMuYvVLAQTAAyb0eLpQHS8GogBgEdAFSQBiydJ//5hTpIADoOGuA1AFSQBgElJ//5YFJMADLgKuIvVEIOGvQy4CoinvYzYAxSTAAwEIwAL4kDoBIjiFHMAC4QA/MDAKaBHwSeEQKiPqIWogqiGoIrCBJaUFCAADEQggACojVAghTAUIIAbFCCAFBQggBOEQYQAFCCG8IRftgGoCagLFACACBQAgA4UAIAPFCCG8d2ehB7dnsAMoEfBCvwAhECDgbqKuou6jEn//8z8gIQe3Z78QIEgwCqhx8cogMFOFAAF/4qFQNUJQKCQCl7wgDCNQegDl5/GBVADf/jmCOgWBBOADcELBFOACdYIg4m6CbgK3SKEABQDgA0Uo4ACFGOACYAJSf//ydUChB78wMI+ICEAAFooMTtaODg5wDn8QIRAqIYEIAAIygiDgEYwBgRYMYbgu4i6igQgAAnKB0YgBgRYIQdIFCAACYOAuwiA4YDAhCG4CkQgG8zdI4EghBxOkgAahUAUkwAHgCcUpIANgAZJ//+ggODADYOGugm4CoAp3SIUowAH1QWEGt2ehB7dnoAH/MAAAE4CB038ZATgAAdO4gdKBJAAA06SB0YFwAAAoIHyg0/DAAROIwc+tE5aKAsEhEy2TqHEgaCEAPGCBLcADgSHAA+BR/YD8IG0DuYfTvIHKkbwBgJY9420OAeBAUDwPADdDwA+AUIBugIcAmACyANGA8IEPASyBOIFGgUkBcYGHAYgBnAGugdsCUwJUAp+Cs4LigvSDFgMcA0SDVgNVA5CBAcAAsgMhAzVRE5iBooIHgABjsFAEKAMiWGNCOcQ6fZUEAACwRtEEIsfTLDAGIQAgCCAQEkACs36LxAfgBxEH/+LEB+AHRQHAAawR4RCSQAKwBQHAAaEAdVIhCAUFwAEBBcACMEEhF8UIIAMlgTACkAFoAiWAUAALRz6L0AABDfBCkYABgdYAAFMFAaABvoNtg7Vi1QFgA9aCAg+k2QEFwAJVAWAD4wIyQQUBwAJ1QniIOgHjwRGAAYHWAABgNXmhCFAAIAMFAcABYQAgCCAQEkACZwUBwAGFAaADFQFggDIA4QL1QKECYUAtg6BaEj//11OYgYWCA4AAY7BQAAgDIlgjQjnEOn2VAWA/xS3AARaAAgHRgAGB1gAAWTVt0QA4ABABYACwAZGAAYHWAABlNWtBAcACMAEQhWgC7YgVAWCAMAPQRWgCQQHAAawR4RCEL+AHBEfgB1JAAo7FAcABoQChQC2DoFo1QpOYgXaCA4AAY7BQAAgDIlgjQhc9AAg6fUEBwAIwAMUsAABBAcABFQAAgDAF0AFoAkQD4AdQAXACRAPgB5BFeAJBAcABrBHhEQQv4AcER+AH0kACgoUBwAGhAOFALYOgWjVCk5iBakIDgABjsFAACAMiWCNCOcQ6fYEBwAIwAdUFYD/qEJAFaAJqEMEBwAEVAACAMAPQRWgCQQHAAawR4RCEL+AHBEfgB1JAAneFAcABoQEhQC2DoFoBAcABFQQBADBJ+cQ6AtOYgV3CB4AAY7BQBCgDIlhjQjV9QQXAAgUtwAQwQMUsIAFVAACAMAWQRWgCQQHAAawR4RCEL+AHBEfgB1JAAmxFAcABoVg1QcEBwAIwAWoRNUDgWCBC4QFtg4EBwAEVAAEAMA2BCcAEOJGgYJAwzwaTsIAKwRHAAjEEaAkwA+gZZqKoGZARggA4iSIAugDmorVAoBMgDxJ//viBAcABFQAAgDACQQHAAaAPIBMSQAJeRQHAAYEBwAQisxAMDABQc4wABQ3ABAEBwAQTgMFEoQAFAcAEIQGtg4EBwAEVAAIAMAxTmIFBoBchKAEFwAIjKEIMQABwQ6hD8QMBAcAEAQQgAjiAegGnEEUFwAQODIACMMC3uwEBwAEVAACAMANgEUEBwAGgDzzhfWESQAJOhQHAAbzBfUEisVBzhQAwwdIAATYBBcACMECqA+EABQHABCEB7YOBAcABFQAEADAMk5iBMiAXISgBBcACIyhCDEAAcEPBECACcQMBAcAEAQQgAriAegGnEEUFwAQODIACMMC3usEBwAEVAACAMANgEUEBwAGgDzzhfWESQAI+xQHAAbzBfUEisVBzhQAwwhIAASZBBcACMEDFACACYQItg4EBwAEVBACAMEZ5xDoC05iBIkIHgABjsFAEKAMiWGNCNX1AhcADUywgAhGAAYHWAABsEj//i2FAIFoBBcACMEIQgAkCxQAgAuEARQAgAyEAIAggEBJAAi+FAcABhQGgAxIAALgTmIEXggOAAGOwUAAIAyJYI0IXPQAIOn1QBXgCUQA/wBAIC0eQBCvAEAFgAKIIkEQgQCFYIQKFRcABoELFRaADLYOBAcAA8gPgA0UloADqcQVxoAAqYEUtwAOFIcAD4QCSAAEpIQAgCCAQEkAB54UBwAGFAaADIQLtg7wAo4F5gJO8wQeBAcAAcATVAQAB0C1gA1mhAAH+gpI//3LTmIEEAgOAAGOwUAAIAyJYI0I5wPp9lQFgAFAFYQJFAcAAVQAgANRJH/9WgACIVoAAyFaAAEEhA3VI1AnAExGAAYHWAADtLYChAmoEkYABgdYAAM0qBGEBagT+gO2DvACWggGEJNjgRJIAAPc+gDVCEYABgdYAAHEFAaABvoNtg5AsIgJgRJI//0SVAQAB0C1gA1mhAAHXPQAIOgLTmIDwggOAAGOwUAAIAyJYI0I1fREAP//QFWAE0AALh/QB0YABgdYAAHYSP/9Y4QOtg7wAoUAFFcAEIFoWggGBEgAA6KED7YOBDcAEE4yAhziZ0AzvBriw0AzPBtOMgOUgAmAPIBD84RJ//pE8wQEBwAQisNBzgwAiuOJI5rDFDcAEEj//MZOYgN/CA4AAY7BQAAgDIlgjQjnDun2VCWAH0AFlAlAFagJUCEBAZYnll9QNwBcjAGMJFzxAR+omagatiOTbo8O6APmH+kHRgAGB1gAAfhI//0OhAAUBwAa+gG2DgQnABcEBwAa4gLoH+cD6AtOYgNJCB4AAY7BQBCgDIlhjQjV9ZxBFBcAGkYQBgdYEIu0OACBAVQVgAdQAAA4OBcBCZNjjwPV34QgBAcAGuYT6A+cgRQnABpGIAYHWCELtDgBAQFQAAA4OBcBCdXvUAcFMBQHABsUBwAThAcUBwAVUBcAcIQA+kNQNwBsUEcAVFBXAvBJAAOX8IHAB0YABgdYAAIcSP/8tfABFAcAGvoCtg4EJwAYBEcAGYbBiIIENwAa4mRO8gCRBBcAFQUHABNAGwQMjiFAVYQCQFgUQKYppWnjAOgLTmIC3ggOAAGOwUAAIAyJYI0I1e/msOgMQLWADYsAnBlQMYA4FAcAGjhXDQnV1lpYEB2dQuMF6AtOYgLCCB4AAY7BQBCgDIlhjQjV9UC1gA2LAMM/VBWAA1ABgDc4VwEBjCOTYo8C1TJaUBEEnUfVIZ1D4wXoC05iAqMIHgABjsFAEKAMiWGNCNX1QRWADVQYgAeMI0C4jAlSAH/91RVOYgKPCB4AAY7BQBCgDIlhjQjjBen2QRWADVQYgH+MK0C4nAlSAH/5iQCEoIhh4oPoCkYABgdYAAI4FAaABvoNtg7VEI4hWh//BEj//3gEBwAanMFQAAA4FDcAGjhXAQnV8rQOWggdBEj/+6ACBwE4yAdGAAYHWAACVEj//AhQBwUwUXcAbFFnAvAUBwAbFAcAE4QJFAcAFYB3gLaEAVAXAHBQRwBUFX+ABRVvgARJAALI8IEFb4AEBX+ABcAHRgAGB1gAAnxI//viUDcAUKBcoB+2A1AQgDiEBqgaoJ2EAkAXBCCAd1BHAFiAtkkAAqnwgcAHRgAGB1gAAphI//vH+gO2DvACWggGBEgAAgr6BLYO5sbpKlzzgQLpJxSWgAMUdoAEFcaAABRmgAGADRS3AA4UhwAPgCpJAAPbtA4EloADBHaABAXGgAAEZoABBLcADgSHAA9aAAsESP/7J4QfFAcG8Uj/+yKEAAQXABUUBwbxhAFAEAQMBFcAE44hQDWEAkAyjECmmaYY4wIDAYAB6AtOYgHGCA4AAY7BQAAgDIlgjQjV7YBigvDAJlQQAPDJI5kQhCFAQJAMjoFAFZACQBCIDYg3QBKEQKbJpggDAIABmFPjAegLTmIBowgOAAGOwUAAIAyJYI0I1elAtYgNiwIUJwbxBBcG8UC1jA2LA4hhFDcG8RUHABDIBPoJSP/7QVQQACDBB4QfFAcG8YQLSP/7OFQQAEDBB0YABgdYAAKwSP/7LJYfFAcAEvoFtg4EFwASwSDjAegLTmIBaggOAAGOwUAAIAyJYI0I1fWEAUAABAwEJwAQjgFABYACiAIUBwAQBAcG8UC1hA2LAYggFBcG8QQHABAUBwby+ga2DgQnABaEAUAgCAwEVwAUjkFANYgCQDKMQKZZphjjAQMBgAGAYegLTmIBNAgOAAGOwUAAIAyJYI0I1exUIADwgvDKJpkIhCFAQJAMjoFAJZACQCEMDYhXQCKIQKZRphADAQABmJnjAugLTmIBEwgOAAGOwUAAIAyJYI0I1ekEJwbxQLWMDYhDiwMUJwbxBCcG8UC1hA2LAYgiFBcG8VQQAEDBB0YABgdYAALMSP/6qJYfFAcAEvoHFQcAEbYOBBcAEsEg4wHoC05iAOQIDgABjsFAACAMiWCNCNX1hAFAAAQMBCcAEY4BQAWAAogCFAcAEQQHBvFAtYQNiwGIIBQXBvH6CLYOTnIAxQRXABFABRwB4gXoIAQXAAuaKOIg6AoEFwbwwQdGAAYHWAAC5Ej/+mcEVwAMBBcADeKg6ASKBQRXAAqKoIihBBcAEOIgQBA8GtUFQFSUAQQXABDi4QQHABBAE7wbigEUBwAQiuGACZipCDKAARgwAAHa/AQHABCJIU4D+cz6BEj/+j1OcgCCBAcAEBAEgAD6BI7htg6NIUj/+b0EBwACwE1c9AAg6ArGcAgOAAGOwUAAIAyJYI0I1fWALaANQCUcAYgCqA2AbqAfiAKoH8ITBAcABEAUiAHABgQHAAZJAASt1QUEBwAGSQADxRQHAAYUBoAMBAcABMgPRAD/AEBV4AlAEC0eQFKvAIihQAWAAkBSgQDVAoCrBAcABtAMRgAGB1gAAwQUBoAG+g22DoFHSP/5cYUAgUeBaPoLtg4EBwACwBwEBwAEwBlc9AAg6ArGHAgOAAGOwUAAIAyJYI0I1fUEBwAHTLAACEYABgdYAAMcSP/5wYUAgWj6DLYO1QOEHdUChAHwgYANFJaAA6nEFcaAAKmBBAcAChS3AA4UhwAPyAtMo4AUtA7mHegQ5hrpBPACWgAEDIANgClAJRwBSf/3/cAE+g62DtVSgC0Ej4ADoAmhzIsAoApAdRwBiAioCqANgE6IB6gNoBeIB6gXoBLAGMcXBAcABAQWgAPACAQHAAaKJ4BHSQAEGNUHBAcABoongEdJAAMuFAcABhQGgAyDjrgBuQ/AA0QAAEC0TogBhCBaKAsERBAAgIggWiATBYQAWigOBEQAAQCIARQGgAtOgwADxwTwAloIBAzwAcgJhBvVCIQe3Z6EHtUEhBzVAvAB/OSSAMAWoEfBFAQgAAnCEfwABBCADYDAwQQEAAAK3SKDhroJuAqgd90ihACoN/yAhB7dnvxohMCA5jh/mQmMwVpoEP2AwUEQiCBMaIAKCwMAATh/wQGM4Th/wQnV97XEhi84b8UBzgaPoU8T//xIAAE247LoAoJRhMFPEwAatANEUABAnES2I4RBhCCvQK6BrEG0A52EtsOvQK6BrEGEAbYESAABGDh/mQHPBIzBTGj//IThggfVBI2BWwAQCjk/wQGV+YrzTnT/+UgAAQbHB04CAQNbEAEESAAA/1CfgCCE4BJ0gAGGATh0wQE4r8EBQTTAIIjqjYESeYABWwgP9oTgTHEAEDkAnQFPAgAKOTTBAVCpgAE4pMEJOHLNCYzh1fHADVoAARGF30dQBgdZWovcR0AGB1lKDBzVEIKlgoVE4AAT1QtE4AEAR1AGB1lailpHQAYHWUoKmuLSgPJAczwahgFBCBwMtQNRiH//WggBCFz4A1XpBoQBSAAAtFoAAgqEQIJlh5+Ap4EihYGHPNUWXPgCUdXwTyIASlCpf/9AJQgCiFI5L5kBj8FBKQATOS+ZCU8iAD7i5ulCCqmAAUCzJAHhTlV1gP+FoOkOQPcoB+gIOSqpATiqKQFU2QD/1QSFQETQAGBBZhQMQSEkDUC2LAyJ1ovLQvXkJEEkSECD1kHvLAEQ2QAAEXkAARKpAAGJz0/j//dRI3//QSZIDECpCAJOov+6k8HV+4BS1bxMaIBCAmmAADhgmQHVvkChYAJMrj+9QJOkGkBTJAFAhFhAQSYUDEDfpCBAsqQA43HoBji2lQGLy08mAApBJhQMiZJaCAEJXPgDVdUJjKFBKQQI1exaCAIGXPgCUU7y/3a1w0C1CAg4WSoItcODionLEHkAAbXDiXJBJEgBkcITJYAB1YXCCkAkCEBEAABArhARcQABEyEAAbQDQQBAQLeDtuSEANUHhB/VBU8j/s5I//7Q/Oj8YQUQAAcFMAABtKCDkVAp//u8Fo6hoMSFQUEyiAAEiIAVoIMEmIAMBOiADQVIgApARRAMmlmOQVAx/v9AhSAMjoFB4QQAQSEMALkOuw8FeIALBViAEwVogBSPAfSBUNd//0GE0ABBlyQA5m/oDacpp6pAcgwMUEGACEBDEAyIh4gkjHCMokBAoAJASpBAAQIAAaegpeFAEMANinDOBK/RSAAAylRDABBOQgEHl5+Ah8YR4mboB6cpjKFAQgwMiCSMaEBFGAyOgf8OiIdAEJgNimbmb+gNp6mn6kEDDAxQYYAIQGOYDIjQiCaMcIyi9gFAcJgCQHscQKe4ALOAAaX5VQMAEEAQrA2Ka08CALqXn+JmgWPoEwECgAGNaEEIDAzjZogw6QOModUJAQKAAoyiQLgsDIgrULGAEEA1GAyOYf7OQQGcAEBxeAHi8EAQmA1ANZgB6HZAeBwBQPucBugKBGiG8MYHRkAGB1hCAuRIAACggM1OkwAUQGocAeLkiM3oRlDzAAGKh1CxAAGAxwnHgAGOwRnFgAHO+9U24yfoI0C8HAGK6eLkiW3pA4DL1S+Kh41hUcEAAYHnCMWAAVD3//8YzgAB6frjJIhH6CCKiYDOndEIswABGLOAAUxs//yISdUTQGScAeLkiM3oEFDzAAGKh1CxAAGAxwnHgAGOwRnFgAHO+4hHQGFAAeaD6Qun8a/Rp/Kv0ozDjEOn8K/QjoPV9YDixFoBAwABEQEAAVpAAgSMQdVSpzKMQq861U5AYUABp/Gv0afyr9KOg6fz5oNRAQADULMAA6/T6QSAy4BQ1fGA4oBQxDmmtK68WkACBJy81TOnNZy9rz3VL1UDAEBPAwASQGUYDI7B/45BAxwAQQtAQABoAAAAuAABAngAAUj//y9GQAYHWEICzKkG+o22kdUYVEMAQMwIQGUYDI7B/46ZN0j//uKXroSLzvJGQAYHWEICsNXr4rPoBOJSTvP+xUBBjAlAMZBhiqSEgUBCDAyOgf5mnSm2gEBZlAGdEUApCAGDkYylUCEBAakDqUGohLmOu4/84fxgQHBACZbBWigBFaYIiGBEAP/w4gPoBEQfAA+IYZhf4gHoBEQPAA+IIEABhgRIAADLhAFOEgDI5lDpBETg//HVYohBTBEABwgAgAGIYIjj1fpEAP/w4gPoBEQPAA+IYERA//FAQ5AXQAGCBEgAAKymCABggAxAgYAApgkAUIANQUQAAKYKiRRA2gAApguJDUDGgACmDIkMQLYAAKYNAECADkClgACmDoloQJUAAKYPiUtBNIAAAACACIkqQSmAAAAAgAmJ6UEZAAAAAIAKidNBCIAAAACAC4myiBCJkYjAiBCIpgAwgA+IwIiFiKaIZIiFiIOMMIjkTBf/vEABuHdAA7j3XPEVsOkGUCFqUFDwlbDVr4ChwlWaEYgF5hDpQKeIpgqZM6bJjDCIZJmjiGAAAP/zmTeIg4hgAAD/9IiDiGAAAP/1iIOIYAAA//aIg4hgAAD/94iDiGAAAP/4iIOIYAAA//mIg4hgAAD/+oiDiGAAAP/7iIOIYAAA//yIg4hgAAD//YiDiGAAAP/+iIOIYAAA//+Ig4hgmePVvmYBAA+IoJafiEXSBggCgAGIYIjj1ftEAP/xQBGAd0ADgPdAAZ4E/OD8IE4SAYNEMP8AQEBgCUBRgR5AQgMA/saIhUBCDQD/I0YwBgdYMYzcwhBUAIADwA0IAIABQAATH1AABAA4AYICQEARA45B1fGfzIDHUQF//EYwBgdYMYzcQAgYAYgBXPAAIE7zAP+gMVCDAIj/BZdgQSJgCVBShABRKQcAORGWAkACQAk4UcoCkoiXIEBYlAOWALOgUEIFADhBkgJQAAYAOAGCAkBSxAP/Zf9FlyhBIuAJUEIEAFEpBwA5EZICQALACThBygKSqJdoQEiQA5YAs6FQUoUAOFGWAlAABgA4AYICQEJEA/8t/wWXYEEiYAlQUoQAUSkHADkRlgJAAkAJOFHKApKIlyBAWJQDlgCzolBCBQA4QZICUAAGADgBggJAUsQD/2X/RZcoQSLgCVBCBABRKQcAORGSAkACwAk4QcoCkqiXaEBIkAOWALOjUFKFADhRlgJQAAYAOAGCAkBCRAP/Lf8Fl2BBImAJUFKEAFEpBwA5EZYCQAJACThRygKSiJcgQFiUA5YAs6RQQgUAOEGSAlAABgA4AYICQFLEA/9l/0WXKEEi4AlQQgQAUSkHADkRkgJAAsAJOEHKApKol2hASJADlgCzpVBShQA4UZYCUAAGADgBggJAQkQD/y3/BUACQAmXYEEiYAmWAFBShABRKQcAUAAGADgBggI5EZYCkog4UcoCUGMAIJcgQFiUA1BCBQC1pjhBkgJAUsQD/2X/RUACwAmWAFAABgA5MYICQSLgCZYokqhRKQcAURAEAJdoOEHKAjgBxgJQUoUAOFGWAv8F/y1AQkwDSP/+/mYxAB+I41QxAB+dvFEhgARGUAYHWFKM3EAJGAGIB+YE6SCicf5lQADACZYAUAAGADkyggJBEOAJlgiSKFEYhwBRAAQAlkg4QsYCOALCAlAQhQA4EoYC/wX/DUBCTAPV3ZaPVDGAHMITjGSI40YQBgdYEIzcCAOAAUAAEx9QAAQAOACCAo5BQEARA8r1/yNAEmAJRCD/AEAQkwBAMREeiCP/FkAAkQDVAoAB/KBGAAYQWAAM9IQgrkBEEF+oqEHdnvwARgAGEFgADPRCQIgkRmAGFlhjDKSnQKDBzQPiZOgSjGiIA+IG6fhGAAYJWAAM3EQwX7CEiERQG8xJ/+JQhADVE4QhrkCcY2YQgANQIIAM4kPoCVAggAiOaIhAimGvUKjRqEGMCPyA/ABG8AYQWPeM9FAA//jiD+kHRvAGFlj3jKTiD+kIRgAGCVgADSRJ/+Im1R+gwYRAECD/+FAhgAiIQKZQyQWgUYgjjCioQUYgBhBYIQz0oFGMKIgi4iDoA4BB1fqmUMkFoEGUSYwoqEH8gPxAhMBEkAIYRnAGFlhzjMBCEyQkmLeYD1AQgGAUAQZIiCdEIAgAjMRJ//A1WmgM8oQDPA4aSfzA/AG2H/GBSQAK9VoIAQe0H/EBSQAQftUHWggDBrQf8QFJAAaI/IH8AOYEgMDoCEYABglYAA+UOBAaAtUFRhAGCVgQjUhGAAYJWAANUEn/4cKEA8YFjsKEAEAAGAb8gPwhn8WAB0n/6iWAwMAnPA/kykYABgpYAA/USQAMl4AHSf/qLPCBgCdGAAYJWAANfPIBSf/hoDwMGk5MYEALRgAGCVgADaRJ/+GWhAA8DhpOgAZJ/+omhAHVAfyh/ACgB0n/6kv8gPwgoceAwcYHgAeOwUn/6jqXsdX6/KD8QLShBKAAB8UTpYqA4YjFQJMIAExkgAuACkn/6igYAwABpHqMIax61faEAdUCgAX8wAAA/ED6E0kACumAwE4FABZJAAsvgAZJAAs8wA9GAAYJWAAN6En/4VCEwUScf/9GoAYJWKUPlNUhSQAGt4DA1faABkkABwDmxMAn6Bw4FRoCRgAGCVgADgBJ/+E25sToFzgVGgJGAAYJWAAOREn/4SyABkn//1SAwFpoAeQ4FRoC1RBGEAYJWBCNSNXjRhAGCVgQjUjV6OnzRhAGCVgQjUhGAAYJWAAOcEn/4Q7mxOgkPA3k1kbwBgRY94m8OBeYAEDwvADdDz4EFB5EHH///g5CADwIPA/k1vzAQAAkAkIAQAjVBkAAJAJEEYAA/g88D+TWhOjVD0YABglYAA6AgCZJ/+DiPA3k1oTiQAAkAjwP5NaABkkABlDIB0YABglYAA6sSf/g0YAHSQANkdWi/AA8HBpJRiAGFlghDMDmI+kMRgAGCVgADshEEABnRiAGCVghDtzVEFAwhkg4QQ4CxA1GAAYJWAAOyEQQAGhGIAYJWCEO7EkACK+MITgBDgo8HhpJ/ID8AKAKSf//0/yA/EC0wYDhpEoEoAAHQJMEAExkgAiACigTAAFJ/+nN1fmgOkn//7+EAfzA/AA8bBpJRhAGFlgQjMDOCUYABglYAA8ESf/geoAG1SXmxOkMRgAGCVgADshEEABbRiAGCVghDxDVE54xUGMGRzwOGkk4AJoCyA1GAAYJWAAOyEQQAF5GIAYJWCEPIEkACF6EQDggmgr8gPxAgSGA4gSgAAdJ///GwBJQYABgEnSAArbJFASAAojmTGOACIAKSf/o+BgDAAHV+YQB/MD8IFzxCAGA4ukMgCJGAAYJWAAPJEQgCABJ/+ArhADVC4DBSf//oMAHUBAAYKgytiat8oQB/KAAAPwAPD3kOYSgRkAGClhCDazTHJStmZQAEAAYobFMYMAUgMA8DeT3jgHmBOgpRvAGBFj3i8Q4B4AAQPA8AEoAPAAMLjI4jKHV5YQA1SBGAAYKWAANrIhAoBfAEkYQBglYEI+kSf/uasgLPG4aTtUIwf3VBloYAQXV+VoQAvg8XBpO3gSABkn/6G+EAfyA/ABGAAYJWAAPrEn/39BJAAuT/ID8AEYABglYAA+4SQAJf04FAA1JAAvcwAlGAAYJWAAPrEn/37tJAAt+SQAL60kADbVJAAMgRgAGCVgAD8xJAAlmTgwJgEYABglYAA/YSQAJXk4MCXj8gDxMGk/EGPwAhFyEpERhAABAIRRWjEFAQghAtERUMQADywzgwukMnsHAA4AD1fK2QZwk1QaABN2ehADVAoAD/ID8AEQAAJRJAAqvyAhGAAYJWAAP5En/33bVA0kABVv8gPwELgBpSDxsGlHAB0YABgpYAAzItACIwIAf+jBJ/+R1PAwaUJYEwAmAP0YABgpYAAAgSf/fV9UJRgAGClgAAEiAP4BGSf/fTjwMGlCWDsAHRgAGClgAAGhJ/99E/IT8A4DAPB4aUMAdBBAACIgBoENmEIADiCCMODweGk9J/+kDRgAGClgADMi0ADwcGk+KwIoghAE8HhpPPG4aUT4AaUhJ/+g7Sf//Uy4AaUjAEEYABgpYAAzItAA8HBpRiCA8HhpRPBwaT4gBPA4aT4QrgEFEMKBpRgAGClgAAKBJ/98CSQAE1Dwd5NCWBEIQzAlAEIJkPB/k0Dwd5NZCEMgJQACCRDwP5NaEC7BFPgeTVD4Hk1WwBEkACYnwBEkADKjwgUkADLlJAAyj8IJJAASvyAZGYAYKWGMCpNUFRmAGClhjAJRJAAlv8IPxAfICgGb0A/UFRgAGClgAAMhJ/97DSQAM2loAAwxJAAmO8IHxAUYABgpYAAEMSf/etTwsGlHKCEYABgpYAAEkSf/erNUQBAEACIhAoFNGAAYKWAABOGYQgAOIIqBNoJZJ/96cPAwaT8AFPA/k0zwP5M5JAAr6RgAGClgAAUxEEADKRiAGClghAVxJAAaE/YD9IYACgCWAcYBQyGhBKBQGTyIAk0Ug//9BKRQGTyIA90YAD/9QAA//QSAUBvvg+ghACcgagkBAAoANRzAGClk5gXQ4CYAAiBJTIAAgTyIAC0AoSAxACIANQBLIDP6HQDjIDEEQwAlBQUSXQSCAE0EBwAlDOkgkQEJACEBSQARBAswGgBRPAgALiKFAIoQGjgHKBUAizAZOIwGgirNBAsSXltlAQkAIQyhIJP7nQEHIBoBQxAyIYUARhAaOQckHQDHIBo+CQCgMG5IAQABACP6HhgCAEIAi3Z6SAEA4AAZOMwCIRDD//0AxgAZOMgCHRjAP/1Axj/9AAYAG+8D6aEA5ABpQAYAAQDIMDUcgBgpZKQF0ODkMAIgDUzAAIE8zAIdAIggGTiMBRUAYhAZWIIABhgDVYJIAzQSEAUAQFDdEAP//QAAEBsBqRgAP/1AAD/9AQAQG+qD6CEACkBqAgEAAgA1GUAYKWFKBdDgCgACIgFMyACBPMwDDikFBEMAJQSCAE4YBkgBBUUZ3QFHACUNKyCRBOcAIQEmUBEBSUAaAFcUKiIFAIgQGjgHKBUAiUAZOIwENipRBMkS3ltlAUsAIQynIJP7vQFHIBoBTxQuIYUARhAaOQckGQDHIBo/iQCmMG0AAQAj+h4AigBDdnoRAhgCAEIAi3Z6ECF0iAQBAMEgagANI//+BkgBdIoEAhmhACcgagkBI//8QXECBAIQIhKBAApAbgIDVm0ASgA1BQkwMQUoEBEFoAA1BWkAJQZtW90AIgA1AKEwM/odBigATQkzgJEF7wAhBIUAJQBvIBEAAkAaAeUBSzAzADIg0QADQBo5hyAdAAJAGwARQPP/+iDSbDEFiVBeWkUMrYCRAAEAI/hdAIEgGgDbCCIgUQCBQBo4hTiIAiZIAQDHACP5fQUDACZbpQCLACUJaDCSXCYKl/uRDUghzQEHACYiVQFIUBooSQioIJMUFRFEAAIhFkgBAUkAJiEVAUAgGzV1MAQBSgEGGAEj//35AEMwMQVgQDUBQwAlBapR3QAhMDEBIkA3/B0EggBNAMcAIQCJACUMLSCRAAYgEQCBABoKWQDjMDMIOiAFAIAQGUUt//8oIQCBABsIFUUt//ogBkgBBEEABQTiUF5chQinIJEAAQAj+J0BACAaCE8QNiAFAQAQGj4HMCEBACAbEBVEJ//6IAZIAQUpACJqCQQhQBIIlSP/+/EBCQAiW2UE4zAyIZEE5jAZPMv+onomGAEj//ySEQYYASP//IEAgSAZOIv94UBt//ogUSP//c5IAUAr//oiBSP/+8pIAUAp//oihSP/+X5IA/CBGYAYLWGMMOEZwBgtYc4yQTGOAB7QGjMhL4AAD1fr8oPwg/TBMY4AFojLdINX8/KD8AoQogMCwAUkAB4DICEYABgpYAAJ0Sf/cb9UqxgzxAUYABgpYAAKgOgAAADoAgCCEBNUZRgAGClgAAqhJ/9xc8gFHAAYKWQgCwIAQOwBABIAiOwDAJKaAroimga6JpgKuCoQHEg+ABLABSQAHR/yC/ACEAcEQRgAGClgAAshGEAYKWBCELEYgBgpYIQRMSf/cM4QASf//tvyA/AJaGAEGIAAAAFoAPxJGAAYKWAAC3EYQBgpYEIQsRiAGClghBDRJ/9wahMDVJ7ABhCNJAAcegMDICEYABgpYAAJ0Sf/cDNUaRhAGCvABWBCC8KZIrkBGEAYKWBCC8aZIrkFGEAYKWBCC8qZIrkKEAxIPgASwAUkABvSABkn//3X8gvwigOGAwJ4VSf/kV+bi8IHpBSADAABaAD0RRhAGClgQhCxGIAYKWCEEPEYABgpYAAL0Sf/b09UTjMGABrBDhEBJ/+ptth/wAyAAAADACkYABgpYAAMUgCZJ/9vAhMDVAoTBgAZJ//9Axg+0P0YABgpYAANISf/bsvABSf/kefABtD9J/+Tm/KL8AYTB8oHBEEYABgpYAALIRhAGClgQhCxGIAYKWCEEREn/25iEwIAGSf//GsYF8AGEIEkACi78gfxhhMCBIPGBRsAGCljGBAw4hhsCQLMMCIAISf/p7LVJgOCAKoAIgEdJ/+nsyBgCFIACTHCACDgFHABUAAD9Wgg9D0YABgpYAAQMiWAENYABQAUcAIon8gHdI9UOjMFaaATZRgAGClgAA2S0KUn/21aEAEn//tn84fxEgOCOBUn/47zwgUQRwgBGAAYKWAADhEn/20TwAUQRwgBJ/+R6sAWEKkkABkXIBkYABgpYAAJ01R7zBUYgBgpYIQO0OwFEBIADOwBEJKZQrkCECRIPgAywBUkABiWwAkQQAMhJAAYohSDIJkYABgpYAAPASf/bFvzEsAKEIRKfgAZJAAYHpjBaAAr5WgAN97AChCFJAAX+Ai+ABpgyAAB//1oIDRJaKMgWRgAGClgAA+xJ/9r38AKEIEQgAMhJ/+lL9gLV21oACvBc8QDI6eHV645BlpGEABIvgAaAJzgDCAiwAkn//1LV5/wARgAGC1gADJBGEAYLWBCMsEn//lD8gN2e/AG2H/GBSf/aPLQf8QFJAAAD/IFlAwACRwAAAGUDAAO0waHJBICAAgSQgAMEoIAEBLCABQTAgAbdAPwBhAawQYRASf/fGYDAyAhGAAYKWAAEUEn/2qXVEvYBRhAGClgQhHyABoRISf/pD8AIRgAGClgABIhJ/9qUhMCABvyB/ABJ///dwAMAAAAI/ID8QoFASf//1YDAwDeFIUCUqAyMCoTgCBAAAUAQpALJBIzhWngD+gATAAlAAKgOlgTIA88m1SNacAMiOwNMADsPzCDPBwAfgAlAEKQSEB+ACYAnRgAGClgABLCI/0n/2loAA4AKgD9AkCQS+kCABhCTgApJ/97l1QvVCoQB1QhacAP+OwNMADsPzCDV4/zC/ACAwEn//47AEIQhQCCYDIwKhCAIMAAB/tbLBIwhWhgD+54LXAAAAfyARgu5pjxcHAFQAAyg2AxGAQhjPFwcAlAABe/YBTwMHASSH92ehADdnvwigOCwQ7ACSQAEt4DHhAOqMfACSQAH0/CB8QGABoRIRmAGGFhjDMxJ/+iSUAOADEYQBgpYEITYRCAAgEn/6IiABoQg+kBJ/+hPRgu5plAADKC2BkYBCGNQAAXvqDGEAagyFGOAI0YABgtYAAAMoEWpxkIQzAioRfyi/EDu4IRApsmnCEAxwAhAMZMEpwuMJP7nAED//kAxkQSxCDgyCAqMRFooQO+AZERgADCAowQhgA605QRCgAlAUUQLiIdAcUwLoFn/fUAiiV+IREBQyAtAQJwL/y1AEgR/jGSIIo7BFBGAD87kUAAAUDsAXACAPzsP3CBGkAYKWJSFCLCIOHSaAjghGgJAShgLiEdAeiwL/+VASmQL/+VAWtACQEtQEv8tiOKYvEAxXABAKUQDQCFAAkBJRAJAURADQEgIC0AoNAv+pUBIWAv+pYhFjMFAeYwAiEOC9oJyWmBACYJRgtWCMIK0ggKCh9XKtl8VYIAHqcwVQIAFFVCABhUAgAEVEIACFSCAA4RAtIA4MIoCjEGIZKrBWigI+u0g/MBQQABAhCC2JEYQBgpYEITohGCEQFAAAFA7ANwAqKKo4zsAXCDdnvxAhUD9MECQiABMdIAdg4a4EAgTgAE4EwAIjAG4kFoIQPaDhoAGgCZJ//9IuBO5ElAwAgBAAYAGiAG4kruTFKMAENXk/MD8IIDABAAAEIDhnUFEH/+AXPAAODgTAAiIpugIUAMAOIQg0BYYEoAB1f2EIJouXPAAQOgEGBKAAdX6gAaAJkn//xqABoQgRCAAOEn/50mDhrkQuBOUS7oSiAFAEAQGiCJAICAJuZK4kxADAD8QIwA+QCBACZIYEAMAPEAAoAkQEwA7EAMAOkAAwAmSOBAjAD0QAwA5EBMAOIAGgCZJ//7rgCeEQ4OGuxSUE0AxgA2uyLsVjkFAMYANrsy7FowhQDGADa7PuxdAMYANEDCAC7sYQDGADRAwgA+7GUAxgA0QMIATuxpAMYANEDCAF7sbQAGADRAAgBtaL//X/KDCPfwgp0WkwKUBxQ+nRAhggAFAUxUEiGWIg5dZQDKOHJdhQEKSHI5Bl9SvxZJBwhxc8QFpgKLpA0RQAWiKRUBQlCBMEoALp4gBAIABQGgZBIhmiIOMItX2l1lAMo4cl2FAQpIc1eXHA6ZIrkRAMY4cQEISHKzArQH8oN2epsWkQKSBwwumBEAQgQCIQZYJQBAGHJYRQCAKHJYJltFAIYocQAAGHEAACgTdnvwCth+EH/GBEg+ABRIPgASEABAPgA20P7AC8gFJ//+dsAJJ///Y/IL8IkZgBgpYYwYI84PygvGBgOBJ/9cBgAZJ/9ghRhAGClgQhnQ4AJ4CSf/YGfIC8QFGAAYKWAAGOEn/2BHxA0YABgpYAAZISf/YCoAGSf/YB0kACE/8AvCB8YLyg4QA8QHyAvMDSf//zvwC8IHxgvKDhAHxAfIC8wNJ///E5gToFkbwBgVY94v0OAeAAEDwPABKADwABAgMEIQA1QaEAdUEhALVAoQDPA4aVN2ePA3lBFoAAQ7ABVoIAg6ECdUJLgeUvMgDhALVBIQD1QKEBDwOGlU8DBubwAM8DhpT3Z78QITARpAGCliUh0xGcAYYWHOGIDgEmgJJAAE8OAOaCozBWmgE+fzAAAD8YEawBhhYtYs0RsAGGFjGBiDmDTwP5NjoXkbwBgVY94yUOAeAAEDwPADdDxqoHhokKCwwNDg8QA4ARqAGGFilCzSFINUqhAHVE0n//63VQ4QC1Q6EA9UMhATVCoQF1QiEBtUGhAfVBIQI1QKECTwOGlXVMTgGHgJOBQAISQABUEAAHAz/h5ewjOFaeAT1AFUAbNYKjSGNTDwMG7WEwOMg6BiA5tXoBAUAHVoIDAlGAAYKWAAGfEn/107V7YQMgMtCZIBzg4a4HEn//1a4HdWiPG4aVfzgkgD8ADwN5NjICDwMGlVaCAEFSf//XtUfPA3k2TwcG5yMAeIBPA/k2ekEhAA8D+TZPB3k2YQMRmAGGFhjCzRCYIBzoDQ8DhpToDNJ//8qoDJJ//9t/ID8AEn//9X8gPwAPCwaVTxMGlRGEAYKWBCHbEYwBgpYMYdcOBCKAjgxkgI8LBpTRgAGClgABpxJ/9b4PAwaVfyAPAwaU92ePAwaVN2e/ABJ//+vPCwaVTxMGlRGEAYKWBCHbEYwBgpYMYdcOBCKAjgxkgI8LBpTRgAGClgABsRJ/9bUPAwaVfyA/EBGcAYLWHOCdPo0gEdCIARzRmAGC1hjDNgEIQAIRhAGC1gQjPBMYIAJtKbSA4zY1fugcVof/wSEH9UbgSCgNLQgyQ76FEJ0gHOgcaA5iAFGEAYYWBCGMDhgggrVCvoUgEdCJIBzgAKMBN0hyOzV5fzA/ECE4EZgBgtYYwzYRpAGC1iUjPCFX0xkgBSgMogHXPABAOgKoHPJBaAyqfGI4NUGtAbdIcj6FKMAAYzY1e38wICgPB3la4QA+lRGMAYLWDGCdEwAgA2Ag0JACHOhItQDjAHV+PwASf//lPyAhB/dnvwAPD3la4Qg+pRGUAYLWFKCdEwRgAyARUIgkHOhkUwDQASgFdUEjCHV9YQA/ID8QTx95WvwgYTA+zRGoAYLWKUCdExjgBCAKkITJHPwAaBPSf/ktsgFgAZJ//9j1QSMwdXxhB/8wUYQBhhYEIYwOBCCAsEJoIygksIG/ACgSYoB3SL8gN2eRhAGGFgQhjA4IIICwgmgVKBNwQj8AKCRigLdIfyAgALdnoAB3Z5GIAYYWCEGMDghAgLCCqDUoRrEB/wAoRGgm4oE3SL8gN2e/CBGGCABUFCML0YABhhYAApAggVGcAYYWHOKOEZABhhYQgowUCCMK5xsqEFQEoAIqEJQEoAMqENQEoAQQDgUAahEUBKAFLagqEWZn1ASgByIZI6hqEaphxQwAAhGGCABUAAAJNriRgggBVAgAfk8LhuCUCAB/TwuG4NQQAH4UCACATwuG4Q8ThuLUCACBVBAAfw8LhuFPE4bjFAgAglQQAIAPC4bhjxOG41QIIxFUEACBFAAAgg8LhuHPA4bj1AgjE1QAIxEPC4biDwOG5BGMAYYWDGKPFAAjExGIAYYWCEKNDw+G4k8LhuKPA4bkYxhhACMQTxOG448PhuSPC4bkxQAgxASAIYiRgAGGFgACkD8oOQa6QKMBt2e5BrpAo4G3Z78AFwQgAFJAAFQ/IA8DBuV3Z5GKJq8PFwcBVAhDe/aEDwsHAY8LhuWPCwcBzwuG5WEQDwuHAU8LBwIPC4blDwsG5bCBrZAPAwblLYB3Z78IIDBgOBJAAMVPA4blkkAAws8HBuWtic8DhuUtgb8oEYQBgpYEIe8OACCAt2ePA4bld2e/ABGSJq8RjAGGFgxjNxQQg3vtoOoGahaqJtJ/9SJ/IA8DhuX3Z78AbYfPAwbl/GBtGC0P/IB3SP8gfwBth88DBuX8YGgwbQ/8gHdI/yB/AHwgTwMG5fxAaCE3SL8gfwBth88DBuX8YGgxbQ/8gHdI/yB/AHwgTwMG5fxAaCG3SL8gUYnJia0oFAhDWPaD+Ys6Q2gQaACPB4bmo4BiAE8DhuYPB4bmYQB3Z6EAN2egCA8DBuawBA8DBuZPCwbmIw/ZhCAH4pA4kHpBYggPB4bmd2ehADdnkYgX15QIQEAgCJGOCAFAEGBdMQDjkHK/EY4IAUAQYJaRiggBcQDjiHJ+hABAlSEABQBAJgEUQBcBAEAXND+RgBfXlAAAQBGGCAFACCCWsIDjgHI/EYIIAUEAACY3Z78QIFCgOGAw4EgQHOoDEn//8pGUF9eQAAcEkBjKAxQUoEA/4eAhUYIIAUAEAF0wQOOoc38RhggBQAgglpGCCAFwgOOgcz6EJACVBRgAJcEUABcBBAAXNH+RgBfXlAAAQBGGCAFACCCWsIDjgHI/PzA/CCA4YDAVBM//4QGhE+AZ0n//8BGCCAFg4C5bpWyQGCYEkBjHES+7rlu/ku58fyg/ACEQIBihAeEP0n//6tEA///hCBJ///ehECAYoQIRBA//0n//5+EBoQhhEyEYEn//5lGCCAFg4CEILnuuW7+S7nxRhEAALmIhCEQEAAo/IDmG+kr/ECA4IQGgUFQY//lSf//UYRBQGEYDFSTP/9ABIH+hEDiQFqoAQbpFoAGgCbVEegShAhJ//8+QAAkAsAIRgAGClgAB8yAJ0n/0+6ABoQgSf//l/zA3Z5GGCAFAhCBDIRAlklGQAYKWEIIDMEMlszDBzgyCgJaN/8ETAGAB4xBkiHV9YAB3Z6EAd2ePAwb5sAw/ABGaCAFAAMCEJYAWgABCUYABgpYAAhESf/TvYQAABMCEVoQAQlGAAYKWAAIaEn/07KEAAATAhJaEAEJRgAGClgACIxJ/9On1QLICUYABgpYAAiwSf/Tn0n//2L8gN2e/EFJ//yJgOBGkAYLWJSOIEn//J7wgUZgBgtYYw3gTGSAGKFz3xO0RsoKoLLxAYAH3SKgccEJgAfdIdUGgAfxAd0iyPTVA0n/4CGM0NXpSf/8hYDg1d78IEAAABSXgVpgAQ2EAcYpWmgCHkYABgpYAAksSf/TYNUdOHCIAoABQHOAFIAiSf/7FkBzwAuAIExwABOAR0YABgpYAAjkSf/TS9UIRgAGClgACVyAJkn/00OEANUCgAb8oPxCsEKwg/CBSf/XpcgHRgAGClgACYjxAdUQ9gJGEAYKWBCJsIAGhERJ/+GdwApGAAYKWAAJuLQmSf/TIUgAAOMAIwAdABMAHECRQAhAlIcEABMAHkCUhQQAEwAfQJCkBECUgBRAlMALWp//CEYABgpYAAncSAAAwgATAAkAIwAIQBDACEAQiwQAIwAKADMAIkAQiQQAIwALAHMAI/5XQBCAFEIQwAvBC0BzjQRAc4AUlrlQEwAk+viAYNVIQHONBAAzACUAAwAkQDHACEAxgwQAAwAmQHOAFEAxgQQAAwAnlzn+x0AxgBRAMcALWjf/wAADACoAcwArQHOBBEBzgBSX+YjkAEMALQATACxAQkAIAFMALgADAC9AQgcEQEIVBP8HQAIAFEAAQAtaB/+hABMAMgAjADNAIQUEQCEAFJaRiEdQEwA0RHAAOIhgiGmIZ6QIiEOAJkn//yTIBkYABgpYAAn41UrwAph3AgAAEIBJSf//F8gGRgAGClgAChjVPUn/3xX2AkYQBgpYEIo8iMeABoRESf/g58glpjWmdEAAQAhAAAcEpnYAIwAIQAAFBKZ3/g8AEwAJQAAAFEAQwAhAEIsEACMACkAAQAtAEIkEACMAC4gG/ldAEIAUQBDAC9UDgAaAKUn/3wFJ/99DyAhGAAYKWAAKREn/0kHVA0n/30r8wkYQBgpYEIqkOACCAt2eRgggAoQgUAAMAK5E1QBGCCACUAAMAKYB3Z48DBvnyCRGGCACUBCMAKZJhGGWSEAhgAz+jowBwgQ8Dhvn1QNaCAn4PBwb50YIIAVaGAEJBBAAcVobCgWEIzweG+eEIBQQAHE8DBvn3Z78AEn//9iAwIQASf/805405gLoD4QggAaAQUn//M5J/9qYRgggAoQgUAAMAK5E1QD8gEYYIAIEAIIAhEAUIIIARhTlYEBQBALZB5YBwAZaAAEFhALdnoQD3Z78AbYf8YG0H/EBSf/8vMD8/IH8Y1BxABCBYPGBsAOAJ4DCSf/8zMgJgCdGAAYKWAAKyEn/0brVThJ/gAguoG+mPHwb6MYM8APxAYBGjBBJ/+AN8AGAJkkAAfWBIPMDRAAAdK4YRAAAc64ZRAAAYa4aRAAAZq4bQAMgCa4eQAPgCRABgAhAA8AJEAGACUADoAmEIBABgApABKAJEAGADhCxgAQQoYAFr58QcYALEBGADBARgA0QkYAPsANJ//x4yApGAAYKWAAK9En/0W+wA0n//ID84/wAhCBEAAD/gEFJ//+a/ID8REYABgpYAAsURnZmF0n/0VpQc4N0hADwhFCfgBSwAYQhSf//ffABpkDwBEAAgQTwhLABSf/8W/UE3/KwAYQsSf//bvABOwBIALABOwTIIEn//E0Cb4AOxhWEABIPgA76ILAESQABfIAgTAMADIBGRgAGClgACyxJ/9ElSf//utXPAB+AFOYk6AxG8AYGWPeJlDgHhABA8DwA3Q8SOl74RgAGClgAC1xJ/9EN1bU8YDfSzghJ/92fPmBvpjxuG+jVAoTfhCBEAACAgEFJ//8xxqNIAAC7RgAGClgADEC0APCBAA+AFT4Ab6bwBjwOG+iwQUQAAIGERNVKAB+AFS4gb6ZMEUBSPCwb6PEGnBFMEEBMAm+AC1pgAghGAAYKWAALoIAm1VewAYAmSf/++fABpkAQH4AApgEQD4ABsAFJ//vWgCaAH0kAAQsCH4APgEBMEAAHRgAGClgAC7zVKQIfgABc8Ifx6QhGAAYKWAAL6EQgB/DVHTwMG+g8GDfSjAGEIDwOG+iAQUQAAIJJ//7RSP//QwAfgBUuIG+mhAASD4AATBEACUYABgpYAAuESf/QitVHPCwb6PEGnBFMEH/1Ah+ACzxQN9LRB0YABgpYAAughELV7YAlsAFJ//6hPBA30vABSQAAvAIfgA+AQEwQAAlGAAYKWAALvEn/0GTVGDwMG+iAP4wBPA4b6IRCRAAAg0n//o7wATwQN9JJAAAiwAtGAAYKWAAMGEn/0EywAUn/+13VBrABSf/7WUj//u1J//7YSP/+6Un/3TjAA0n/3UZGAAYKWAAMNEn/0DT8xPxCLmefUYEggOHGZLABSf/7NIFAyAmAJ0YABgpYAAxESf/QIdVnLhBv4MkfhAE+AG/gPB4b6jweG+tGAAYYWAAMcPowRiAGBlghDrhEMAA4Sf/gkYAgwAlGAAYKWAAMeEn/0AGAytVGPJ4b6jx+G+vwATwOG+0CD4AEPA4b7oQiRgAGGFgADHA8bBvvSf/guOYCgCDpFy4Hn1DACoQAPgefUD4Hn1GwAUn/+vHVE0YABgpYAAygPCwb8En/z9SEwdUZPHwb74QAPgefUASfgAGK5oAJgCdJ/9x9gMAuB59RzgXACDwcG+vJxMAEsAFJ//rOgAb8wvwASf/9gYRARED//44hlklMEgATQDEgCZbYQCGJBAgwAAH+nZbQQCEMn0AhCYOW0EAhDKPV7JYR3Z5SZXNldAoAAE5vIGNhbGxiYWNrIGZvciBJUlEgJWQKAEJMOiBleGNlcHRpb24gJWQgSVRZUEU6IDB4JTA4bHggUEM6IDB4JTA4bHggRVZBOiAweCUwOGx4CgAAAEVycm9yOiBVbmhhbmRsZWQgTk1JISEhCgAAAAAweAAAAQAAAAAAAAogKGNvbXByZXNzZWQpAAAAZWxmOiBIZWFkZXIgZmluaXNoZWQKAAAAZWxmOiBFcnJvcjogSW52YWxpZCBoZWFkZXIKAGVsZjogSWdub3JpbmcgbmV4dCAlZCBieXRlcwoAAAAAZWxmOiBXYWl0aW5nIGZvciAlZCBieXRlcwoAAGVsZjogRXJyb3I6IEhlYWRlcnMgdG9vIGxhcmdlCgAAZWxmOiBQcm9ncmFtIEhlYWRlciBmaW5pc2hlZAoAAABlbGY6IFNlY3Rpb24gaXMgZW1wdHkKAABlbGY6IFBIIDB4JTA4WCwgJWQgYnl0ZXMlcwoAZWxmOiBFcnJvcjogTm90ZSB0b28gbGFyZ2UgPT4gc2tpcHBpbmcKAGVsZjogRXJyb3I6IElsbGVnYWwgcmFuZ2UgWyVwLCAlcFsKAGVsZjogRXJyb3I6IENhbid0IGVuYWJsZSBjcnlwdG8KAAAAAGVsZjogTm90ZSAweCUwOHgKAAAAU1FOAGVsZjogRXJyb3I6IEluZmxhdGUgZmFpbGVkIDogKCVkKSAlcwoAAAB6bGliAAAAADEuMi44AAAAZWxmOiBFcnJvcjogSW5mbGF0ZSBpbml0IGZhaWxlZCA6ICVkICVzCgAAAABlbGY6IEVMRiBmb3JtYXQgc2VsZWN0ZWQKAAAAf0VMRgECAQBlbGY6IGtleSAlcyAAAAAAJTAyWAAAAABlbGY6IEVycm9yOiBVbnN1cHBvcnRlZCBjcnlwdG8gaGVhZGVyIHZlcnNpb24KAABlbGY6IEVycm9yOiBDYW4ndCByZXRyaWV2ZSBwcmltYXJ5IGJvb3Qga2V5CgAAAABib290AAAAAG1hc3RlcgAAZW5jcnlwdGlvbgAAZWxmOiBFcnJvcjogTm8gbWF0Y2hpbmcgYm9vdCBrZXlzCgAAZWxmOiBNYXN0ZXIga2V5IHZlcmlmaWVkCgAAAGVsZjogRGVjaXBoZXJpbmcgUEglZAoAAFVua25vd24ATWFjcm9uaXgAAAAAV2luZGJvbmQAAAAARXJyb3IgaWRlbnRpZnlpbmcgZmxhc2ggQDB4JXgKAABVbmtvd24gZmxhc2ggSUQgMHgleAoAAABbR1BJT10gRXJyb3I6IEhBTCBpbml0IGZhaWxlZAoAAABgGMAAYBtwAGAabABgFogAYBp2AGAZcFdBUk5JTkc6IEZsb3cgY29udHJvbCBwcmV2ZW50cyB0byBzZW5kIGRhdGEgb24gdWFydCVkCgAAcHNkAHBzaQAA/yMARXJyb3I6IEZsYXNoIGlzIG5vdCBpbiBxdWFkIG1vZGUsIGl0IG1heSBiZSB1bnN1cHBvcnRlZAoAAAAAc2JwOiBCb290aW5nIGF0ICVwLi4uCgAAc2RtYS5jAABfZmlmbwAAAEJMOiBXYXRjaGRvZyB0aW1lb3V0IQoAAGluY29ycmVjdCBoZWFkZXIgY2hlY2sAAHVua25vd24gY29tcHJlc3Npb24gbWV0aG9kAABpbnZhbGlkIHdpbmRvdyBzaXplAHVua25vd24gaGVhZGVyIGZsYWdzIHNldAAAAABoZWFkZXIgY3JjIG1pc21hdGNoAGludmFsaWQgYmxvY2sgdHlwZQAAaW52YWxpZCBzdG9yZWQgYmxvY2sgbGVuZ3RocwAAAAB0b28gbWFueSBsZW5ndGggb3IgZGlzdGFuY2Ugc3ltYm9scwBpbnZhbGlkIGNvZGUgbGVuZ3RocyBzZXQAAAAAaW52YWxpZCBiaXQgbGVuZ3RoIHJlcGVhdAAAAGludmFsaWQgY29kZSAtLSBtaXNzaW5nIGVuZC1vZi1ibG9jawAAAABpbnZhbGlkIGxpdGVyYWwvbGVuZ3RocyBzZXQAaW52YWxpZCBkaXN0YW5jZXMgc2V0AAAAaW52YWxpZCBsaXRlcmFsL2xlbmd0aCBjb2RlAGludmFsaWQgZGlzdGFuY2UgY29kZQAAAGludmFsaWQgZGlzdGFuY2UgdG9vIGZhciBiYWNrAAAAaW5jb3JyZWN0IGRhdGEgY2hlY2sAAAAAaW5jb3JyZWN0IGxlbmd0aCBjaGVjawAAEAUAARcFAQETBQARGwUQAREFAAUZBQQBFQUAQR0FQAEQBQADGAUCARQFACEcBSABEgUACRoFCAEWBQCBQAUAABAFAAIXBQGBEwUAGRsFGAERBQAHGQUGARUFAGEdBWABEAUABBgFAwEUBQAxHAUwARIFAA0aBQwBFgUAwUAFAABgBwAAAAgAUAAIABAUCABzEgcAHwAIAHAACAAwAAkAwBAHAAoACABgAAgAIAAJAKAACAAAAAgAgAAIAEAACQDgEAcABgAIAFgACAAYAAkAkBMHADsACAB4AAgAOAAJANARBwARAAgAaAAIACgACQCwAAgACAAIAIgACABIAAkA8BAHAAQACABUAAgAFBUIAOMTBwArAAgAdAAIADQACQDIEQcADQAIAGQACAAkAAkAqAAIAAQACACEAAgARAAJAOgQBwAIAAgAXAAIABwACQCYFAcAUwAIAHwACAA8AAkA2BIHABcACABsAAgALAAJALgACAAMAAgAjAAIAEwACQD4EAcAAwAIAFIACAASFQgAoxMHACMACAByAAgAMgAJAMQRBwALAAgAYgAIACIACQCkAAgAAgAIAIIACABCAAkA5BAHAAcACABaAAgAGgAJAJQUBwBDAAgAegAIADoACQDUEgcAEwAIAGoACAAqAAkAtAAIAAoACACKAAgASgAJAPQQBwAFAAgAVgAIABZACAAAEwcAMwAIAHYACAA2AAkAzBEHAA8ACABmAAgAJgAJAKwACAAGAAgAhgAIAEYACQDsEAcACQAIAF4ACAAeAAkAnBQHAGMACAB+AAgAPgAJANwSBwAbAAgAbgAIAC4ACQC8AAgADgAIAI4ACABOAAkA/GAHAAAACABRAAgAERUIAIMSBwAfAAgAcQAIADEACQDCEAcACgAIAGEACAAhAAkAogAIAAEACACBAAgAQQAJAOIQBwAGAAgAWQAIABkACQCSEwcAOwAIAHkACAA5AAkA0hEHABEACABpAAgAKQAJALIACAAJAAgAiQAIAEkACQDyEAcABAAIAFUACAAVEAgBAhMHACsACAB1AAgANQAJAMoRBwANAAgAZQAIACUACQCqAAgABQAIAIUACABFAAkA6hAHAAgACABdAAgAHQAJAJoUBwBTAAgAfQAIAD0ACQDaEgcAFwAIAG0ACAAtAAkAugAIAA0ACACNAAgATQAJAPoQBwADAAgAUwAIABMVCADDEwcAIwAIAHMACAAzAAkAxhEHAAsACABjAAgAIwAJAKYACAADAAgAgwAIAEMACQDmEAcABwAIAFsACAAbAAkAlhQHAEMACAB7AAgAOwAJANYSBwATAAgAawAIACsACQC2AAgACwAIAIsACABLAAkA9hAHAAUACABXAAgAF0AIAAATBwAzAAgAdwAIADcACQDOEQcADwAIAGcACAAnAAkArgAIAAcACACHAAgARwAJAO4QBwAJAAgAXwAIAB8ACQCeFAcAYwAIAH8ACAA/AAkA3hIHABsACABvAAgALwAJAL4ACAAPAAgAjwAIAE8ACQD+YAcAAAAIAFAACAAQFAgAcxIHAB8ACABwAAgAMAAJAMEQBwAKAAgAYAAIACAACQChAAgAAAAIAIAACABAAAkA4RAHAAYACABYAAgAGAAJAJETBwA7AAgAeAAIADgACQDREQcAEQAIAGgACAAoAAkAsQAIAAgACACIAAgASAAJAPEQBwAEAAgAVAAIABQVCADjEwcAKwAIAHQACAA0AAkAyREHAA0ACABkAAgAJAAJAKkACAAEAAgAhAAIAEQACQDpEAcACAAIAFwACAAcAAkAmRQHAFMACAB8AAgAPAAJANkSBwAXAAgAbAAIACwACQC5AAgADAAIAIwACABMAAkA+RAHAAMACABSAAgAEhUIAKMTBwAjAAgAcgAIADIACQDFEQcACwAIAGIACAAiAAkApQAIAAIACACCAAgAQgAJAOUQBwAHAAgAWgAIABoACQCVFAcAQwAIAHoACAA6AAkA1RIHABMACABqAAgAKgAJALUACAAKAAgAigAIAEoACQD1EAcABQAIAFYACAAWQAgAABMHADMACAB2AAgANgAJAM0RBwAPAAgAZgAIACYACQCtAAgABgAIAIYACABGAAkA7RAHAAkACABeAAgAHgAJAJ0UBwBjAAgAfgAIAD4ACQDdEgcAGwAIAG4ACAAuAAkAvQAIAA4ACACOAAgATgAJAP1gBwAAAAgAUQAIABEVCACDEgcAHwAIAHEACAAxAAkAwxAHAAoACABhAAgAIQAJAKMACAABAAgAgQAIAEEACQDjEAcABgAIAFkACAAZAAkAkxMHADsACAB5AAgAOQAJANMRBwARAAgAaQAIACkACQCzAAgACQAIAIkACABJAAkA8xAHAAQACABVAAgAFRAIAQITBwArAAgAdQAIADUACQDLEQcADQAIAGUACAAlAAkAqwAIAAUACACFAAgARQAJAOsQBwAIAAgAXQAIAB0ACQCbFAcAUwAIAH0ACAA9AAkA2xIHABcACABtAAgALQAJALsACAANAAgAjQAIAE0ACQD7EAcAAwAIAFMACAATFQgAwxMHACMACABzAAgAMwAJAMcRBwALAAgAYwAIACMACQCnAAgAAwAIAIMACABDAAkA5xAHAAcACABbAAgAGwAJAJcUBwBDAAgAewAIADsACQDXEgcAEwAIAGsACAArAAkAtwAIAAsACACLAAgASwAJAPcQBwAFAAgAVwAIABdACAAAEwcAMwAIAHcACAA3AAkAzxEHAA8ACABnAAgAJwAJAK8ACAAHAAgAhwAIAEcACQDvEAcACQAIAF8ACAAfAAkAnxQHAGMACAB/AAgAPwAJAN8SBwAbAAgAbwAIAC8ACQC/AAgADwAIAI8ACABPAAkA/wAQABEAEgAAAAgABwAJAAYACgAFAAsABAAMAAMADQACAA4AAQAPAAAAEAAQABAAEAARABEAEgASABMAEwAUABQAFQAVABYAFgAXABcAGAAYABkAGQAaABoAGwAbABwAHAAdAB0AQABAAAEAAgADAAQABQAHAAkADQARABkAIQAxAEEAYQCBAMEBAQGBAgEDAQQBBgEIAQwBEAEYASABMAFAAWABAAAAAAAQABAAEAAQABAAEAAQABAAEQARABEAEQASABIAEgASABMAEwATABMAFAAUABQAFAAVABUAFQAVABAASABOAAAAAwAEAAUABgAHAAgACQAKAAsADQAPABEAEwAXABsAHwAjACsAMwA7AEMAUwBjAHMAgwCjAMMA4wECAAAAAAAAAAAAAHcHMJbuDmEsmQlRugdtxBlwavSP6WOlNZ5klaMO24gyedy4pODV6R6X0tmICbZMK36xfL3nuC0HkL8dkR23EGRqsCDy87lxSIS+Qd4a2tR9bd3k6/TUtVGD04XHE2yYVmRrqMD9Yvl6imXJ7BQBXE9jBmzZ+g89Y40IDfU7biDITGkQXtVgQeSiZ3FyPAPk0UsE1EfSDYX9pQq1azW1qPpCsphs27vJ1qy8+UAy2GzjRd9cddzWDc+r0T1ZJtkwrFHeADrI11GAv9BhFiG09LVWs8Qjz7qVmbi9pQ8oArieXwWICMYM2bKxC+kkL298h1hoTBHBYR2rtmYtPXbcQZAB23EGmNIgvO/VECpxsYWJBra1H5+/5KXouNQzeAfJog8A+TSWCaiO4Q6YGH9qDbsIbT0tkWRsl+ZjXAFra1H0HGxhYoVlMNjyYgBObAaV7RsBpXuCCPTB9Q/EV2Ww2cYSt+lQi7646vy5iHxi3R3fFdotSYzTfPP71ExlTbJhWDq1Uc6jvAB01Lsw4krfpUE92JXXpNHEbdPW9PtDaelqNG7Z/K1niEbaYLjQRAQtczMDHeWqCkxf3Q18yVAFcTwnAkGqvgsQEMkMIIZXaLUlIG+Fs7lm1AnOYeSfXt75DinZyZiw0Jgix9eotFmzPRcutA2Bt71cO8C6bK3tuIMgmr+ztgO24gx0sdKa6tVHOZ3Sd68E2yYVc9wWg+NjCxKUZDuEDW1qPnpqWqjkDs8Lkwn/nQoArid9B56x8A+TRIcIo9IeAfJoaQbC/vdiV12AZWfLGWw2cW5rBuf+1Bt2idMr4BDaelpn3UrM+bnfb46+7/kXt75DYLCO1dbWo+ih0ZN+ONjCxE/f8lLRu2fxprxXZz+1Bt1IsjZL2A0r2q8KG0w2A0r2QQR6YN9g78OoZ99VMW6O70ZpvnnLYbOMvGaDGiVv0qBSaOI2zAx3lbsLRwMiAha5VQUmL8W6O76yvQsoK7RaklyzagTC1/+ntdDPMSzZnotb3q4dm2TCsOxj8iZ1aqOcAm2TCpwJBqnrDjY/cgdnhQUAVxOVv0qC4rh6FHuxK64Mths4ktKOm+XVvg183O+3C9vfIYbT0tTx1OJCaN2z+B/ag26BvhbN9rkmW2+wd+EYt0d3iAha5v8PanBmBjvKEQELXI9lnv/4Yq5pYWv/0xZsz0WgCuJ41w3S7k4Eg1Q5A7PCp2cmYdBgFvdJaUdNPm53267RakrZ1lrcQN8LZjfYO/CpvK5T3ruexUeyz38wtf/pvb3yHMq6wopTs5MwJLSjprrQNgXN1waTVN5XKSPZZ7+zZnouxGFKuF1oGwIqbyuUtAu+N8MMjqFaBd8bLQLvjQAAAAAZGzFBMjZigistU8NkbMUEfXf0RVZap4ZPQZbHyNmKCNHCu0n67+iK4/TZy6y1Twy1rn5NnoMtjoeYHM9KwhJRU9kjEHj0cNNh70GSLq7XVTe15hQcmLXXBYOEloIbmFmbAKkYsC3626k2y5rmd11d/2xsHNRBP9/NWg6elYQkooyfFeOnskYgvql3YfHo4abo89Dnw96DJNrFsmVdXa6qREaf629rzCh2cP1pOTFrriAqWu8LBwksEhw4bd9GNvPGXQey7XBUcfRrZTC7KvP3ojHCtokckXWQB6A0F5+8+w6Ejbolqd55PLLvOHPzef9q6Ei+QcUbfVjeKjzweU8F6WJ+RMJPLYfbVBzGlBWKAY0Ou0CmI+iDvzjZwjigxQ0hu/RMCpanjxONls5czAAJRdcxSG76Yot34VPKurtdVKOgbBWIjT/WkZYOl97XmFDHzKkR7OH60vX6y5NyYtdca3nmHUBUtd5ZT4SfFg4SWA8VIxkkOHDaPSNBm2X9a6d85lrmV8sJJU7QOGQBka6jGIqf4jOnzCEqvP1grSThr7Q/0O6fEoMthgmybMlIJKvQUxXq+35GKeJld2gvP3n2NiRItx0JG3QEEio1S1O88lJIjbN5Zd5wYH7vMefm8/7+/cK/1dCRfMzLoD2Dijb6mpEHu7G8VHiop2U5O4OYSyKYqQoJtfrJEK7LiF/vXU9G9GwObdk/zXTCDozzWhJD6kEjAsFscMHYd0GAlzbXR44t5galALXFvBuEhHFBihpoWrtbQ3fomFps2dkVLU8eDDZ+XycbLZw+ABzduZgAEqCDMVOLrmKQkrVT0d30xRbE7/RX78KnlPbZltWuB7zptxyNqJwx3muFKu8qymt57dNwSKz4XRtv4UYqLmbeNuF/xQegVOhUY03zZSICsvPlG6nCpDCEkWcpn6Am5MWuuP3en/nW88w6z+j9e4Cpa7yZslr9sp8JPquEOH8sHCSwNQcV8R4qRjIHMXdzSHDhtFFr0PV6RoM2Y12yd8v6107S4eYP+cy1zODXhI2vlhJKto0jC52gcMiEu0GJAyNdRho4bAcxFT/EKA4OhWdPmEJ+VKkDVXn6wExiy4GBOMUfmCP0XrMOp52qFZbc5VQAG/xPMVrXYmKZznlT2EnhTxdQ+n5We9ctlWLMHNQtjYoTNJa7Uh+76JEGoNnQXn7z7Edlwq1sSJFudVOgLzoSNugjCQepCCRUahE/ZSuWp3nkj7xIpaSRG2a9iion8su84OvQjaHA/d5i2ebvIxS84b0Np9D8JoqDPz+Rsn5w0CS5acsV+ELmRjtb/Xd63GVrtcV+WvTuUwk390g4drgJrrGhEp/wij/MM5Mk/XIAAAAAAcJqNwOE1G4CRr5ZBwmo3AbLwusEjXyyBU8WhQ4TUbgP0TuPDZeF1gxV7+EJGvlkCNiTUwqeLQoLXEc9HCajcB3kyUcfonceHmAdKRsvC6wa7WGbGKvfwhlptfUSNfLIE/eY/xGxJqYQc0yRFTxaFBT+MCMWuI56F3rkTThNRuA5jyzXO8mSjjoL+Lk/RO48PoaECzzAOlI9AlBlNl4XWDecfW812sM2NBipATFXv4QwldWzMtNr6jMRAd0ka+WQJamPpyfvMf4mLVvJI2JNTCKgJ3sg5pkiISTzFSp4tCgrut4fKfxgRig+CnEtcRz0LLN2wy71yJovN6KtcJqNwHFY5/dzHlmuctwzmXeTJRx2UU8rdBfxcnXVm0V+idx4f0u2T30NCBZ8z2IheYB0pHhCHpN6BKDKe8bK/Wy8LrBtfkSHbzj63m76kOlrtYZsanfsW2gxUgJp8zg1Yq9/CGNtFT9hK6tmYOnBUWWm19RkZL3jZiIDumfgaY1I18sgSRWhF0tTH05KkXV5T95j/E4cCctMWreSTZjdpUbEmphHBvCvRUBO9kSCJMFBzTJEQA9Yc0JJ5ipDi4wdVPFoUFUzAmdXdbw+VrfWCVP4wIxSOqq7UHwU4lG+ftVa4jnoWyBT31lm7YZYpIexXeuRNFwp+wNeb0VaX60vbeE1G4Dg93G34rHP7uNzpdnmPLNc5/7Za+W4ZzLkeg0F7yZKOO7kIA/sop5W7WD0Yegv4uTp7YjT66s2iuppXL39E7jw/NHSx/6XbJ7/VQap+hoQLPvYehv5nsRC+FyudfMA6UjywoN/8IQ9JvFGVxH0CUGU9csro/eNlfr2T//N2XhdYNi6N1fa/IkO2z7jOd5x9bzfs5+L3fUh0tw3S+XXawzY1qlm79Tv2LbVLbKB0GKkBNGgzjPT5nBq0iQaXcVe/hDEnJQnxtoqfscYQEnCV1bMw5U8+8HTgqLAEeiVy02vqMqPxZ/IyXvGyQsR8cxEB3TNhm1Dz8DTGs4CuS2Rr5ZAkG38d5IrQi6T6SgZlqY+nJdkVKuVIurylOCAxZ+8x/iefq3PnDgTlp36eaGYtW8kmXcFE5sxu0qa89F9jYk1MIxLXweODeFej8+LaYqAneyLQvfbiQRJgojGI7WDmmSIglgOv4AesOaB3NrRhJPMVIVRpmOHFxg6htVyDani0KCoILqXqmYEzqukbvmu63h8rykSS61vrBKsrcYlp/GBGKYz6y+kdVV2pbc/QaD4KcShOkPzo3z9qqK+l521xHPQtAYZ57ZAp763gs2Jss3bDLMPsTuxSQ9isItlVbvXImi6FUhfuFP2BrmRnDG83oq0vRzgg79aXtq+mDTtAAAAALi8Z2WqCciLErWv7o9il1c33vAyJWtf3J3XOLnFtCjvfQhPim+94GTXAYcBSta/uPJq2N3g33czWGMQVlAZV5/opTD6+hCfFEKs+HHfe8DIZ8enrXVyCEPNzm8mla1/cC0RGBU/pLf7hxjQnhrP6Ceic49CsMYgrAh6R8mgMq8+GI7IWwo7Z7WyhwDQL1A4aZfsXwyFWfDiPeWXh2WGh9HdOuC0z49PWnczKD/q5BCGUlh340Dt2A34Ub9o8Cv4oUiXn8RaIjAq4p5XT39Jb/bH9QiT1UCnfW38wBg1n9BOjSO3K5+WGMUnKn+guv1HGQJBIHwQ9I+SqEjo95sUWD0jqD9YMR2Qtomh99MUds9qrMqoD75/B+EGw2CEXqBw0uYcF7f0qbhZTBXfPNHC54VpfoDge8svDsN3SGvLDQ+ic7Fox2EExynZuKBMRG+Y9fzT/5DuZlB+Vto3Gw65J022BUAopLDvxhwMiKOB27AaOWfXfyvSeJGTbh/0Oyb3A4OakGaRLz+IKZNY7bREYFQM+AcxHk2o36bxz7r+kt/sRi64iVSbF2fsJ3ACcfBIu8lML97b+YAwY0XnVWs/oJzTg8f5wTZoF3mKD3LkXTfLXOFQrk5U/0D26JglrouIcxY37xYEgkD4vD4nnSHpHySZVXhBi+DXrzNcsMrtWbY7VeXRXkdQfrD/7BnVYjshbNqHRgnIMunncI6OgijtntSQUfmxguRWXzpYMTqnjwmDHzNu5g2GwQi1OqZtvUDhpAX8hsEXSSkvr/VOSjIidvOKnhGWmCu+eCCX2R149MlLwEiuLtL9AcBqQWal95ZeHE8qOXldn5aX5SPx8k1rGQX1135g52LRjl/etuvCCY5SerXpN2gARtnQvCG8iN8x6jBjVo8i1vlhmmqeBAe9pr2/AcHYrbRuNhUICVMdck6apc4p/7d7hhEPx+F0khDZzSqsvqg4GRFGgKV2I9jGZnVgegEQcs+u/spzyZtXpPEi7xiWR/2tOalFEV7Mdk3uBs7xiWPcRCaNZPhB6PkveVFBkx40Uyax2uua1r+z+cbpC0WhjBnwDmKhTGkHPJtRvoQnNtuWkpk1Li7+UCZUuZme6N78jF1xEjThFnepNi7OEYpJqwM/5kW7g4Eg4+CRdltc9hNJ6Vn98VU+mGyCBiHUPmFExovOqn43qc/Wf0E4bsMmXXx2ibPEyu7WWR3Wb+GhsQrzFB7kS6h5gRPLaderdw6yucKhXAF+xjmcqf6AJBWZ5TagNguOHFFuhmYWpz7accIsb94slNO5SQkEgfCxuOaVow1JexuxLh5D0j5I+25ZLenb9sNRZ5GmzLCpH3QMznpmuWGU3gUG8QAAAACWMAd3LGEO7rpRCZkZxG0Hj/RqcDWlY+mjlWSeMojbDqS43Hke6dXgiNnSlytMtgm9fLF+By2455Edv5BkELcd8iCwakhxufPeQb6EfdTaGuvk3W1RtdT0x4XTg1aYbBPAqGtkevli/ezJZYpPXAEU2WwGY2M9D/r1DQiNyCBuO14QaUzkQWDVcnFnotHkAzxH1ARL/YUN0mu1CqX6qLU1bJiyQtbJu9tA+bys42zYMnVc30XPDdbcWT3Rq6ww2SY6AN5RgFHXyBZh0L+19LQhI8SzVpmVus8Ppb24nrgCKAiIBV+y2QzGJOkLsYd8by8RTGhYqx1hwT0tZraQQdx2BnHbAbwg0pgqENXviYWxcR+1tgal5L+fM9S46KLJB3g0+QAPjqgJlhiYDuG7DWp/LT1tCJdsZJEBXGPm9FFra2JhbBzYMGWFTgBi8u2VBmx7pQEbwfQIglfED/XG2bBlUOm3Euq4vot8iLn83x3dYkkt2hXzfNOMZUzU+1hhsk3OUbU6dAC8o+Iwu9RBpd9K15XYPW3E0aT79NbTaulpQ/zZbjRGiGet0Lhg2nMtBETlHQMzX0wKqsl8Dd08cQVQqkECJxAQC76GIAzJJbVoV7OFbyAJ1Ga5n+Rhzg753l6YydkpIpjQsLSo18cXPbNZgQ20LjtcvbetbLrAIIO47bazv5oM4rYDmtKxdDlH1eqvd9KdFSbbBIMW3HMSC2PjhDtklD5qbQ2oWmp6C88O5J3/CZMnrgAKsZ4HfUSTD/DSowiHaPIBHv7CBmldV2L3y2dlgHE2bBnnBmtudhvU/uAr04laetoQzErdZ2/fufn5776OQ763F9WOsGDoo9bWfpPRocTC2DhS8t9P8We70WdXvKbdBrU/SzaySNorDdhMGwqv9koDNmB6BEHD72DfVd9nqO+ObjF5vmlGjLNhyxqDZryg0m8lNuJoUpV3DMwDRwu7uRYCIi8mBVW+O7rFKAu9spJatCsEarNcp//XwjHP0LWLntksHa7eW7DCZJsm8mPsnKNqdQqTbQKpBgmcPzYO64VnB3ITVwAFgkq/lRR6uOKuK7F7OBu2DJuO0pINvtXlt+/cfCHf2wvU0tOGQuLU8fiz3Whug9ofzRa+gVsmufbhd7Bvd0e3GOZaCIhwag//yjsGZlwLARH/nmWPaa5i+NP/a2FFz2wWeOIKoO7SDddUgwROwrMDOWEmZ6f3FmDQTUdpSdt3bj5KatGu3FrW2WYL30DwO9g3U668qcWeu95/z7JH6f+1MBzyvb2KwrrKMJOzU6ajtCQFNtC6kwbXzSlX3lS/Z9kjLnpms7hKYcQCG2hdlCtvKje+C7ShjgzDG98FWo3vAi0AAAAAQTEbGYJiNjLDUy0rBMVsZEX0d32Gp1pWx5ZBTwiK2chJu8LRiujv+svZ9OMMT7WsTX6utY4tg57PHJiHURLCShAj2VPTcPR4kkHvYVXXri4U5rU317WYHJaEgwVZmBuCGKkAm9v6LbCayzapXV135hxsbP/fP0HUng5azaIkhJXjFZ+MIEayp2F3qb6m4ejx59Dz6CSD3sNlssXaqq5dXeufRkQozGtvaf1wdq5rMTnvWiogLAkHC204HBLzNkbfsgddxnFUcO0wZWv09/Mqu7bCMaJ1kRyJNKAHkPu8nxe6jYQOed6pJTjvsjz/efNzvkjoan0bxUE8Kt5YBU958ER+YumHLU/CxhxU2wGKFZRAuw6Ng+gjpsLZOL8NxaA4TPS7IY+nlgrOlo0TCQDMXEgx10WLYvpuylPhd1Rdu7oVbKCj1j+NiJcOlpFQmNfeEanMx9L64eyTy/r1XNdich3meWvetVRAn4RPWVgSDhYZIxUP2nA4JJtBIz2na/1l5lrmfCUJy1dkONBOo66RAeKfihghzKczYP28Kq/hJK3u0D+0LYMSn2yyCYarJEjJ6hVT0ClGfvtod2Xi9nk/L7dIJDZ0GwkdNSoSBPK8U0uzjUhScN5leTHvfmD+8+bnv8L9/nyR0NU9oMvM+jaKg7sHkZp4VLyxOWWnqEuYgzsKqZgiyfq1CYjLrhBPXe9fDmz0Rs0/2W2MDsJ0QxJa8wIjQerBcGzBgEF32EfXNpcG5i2OxbUApYSEG7waikFxW7taaJjod0PZ2WxaHk8tFV9+NgycLRsn3RwAPhIAmLlTMYOgkGKui9FTtZIWxfTdV/TvxJSnwu/Vltn26bwHrqiNHLdr3jGcKu8qhe15a8qsSHDTbxtd+C4qRuHhNt5moAfFf2NU6FQiZfNN5fOyAqTCqRtnkYQwJqCfKbiuxeT5n979Oszz1nv96M+8a6mA/VqymT4Jn7J/OISrsCQcLPEVBzUyRioec3cxB7ThcEj10GtRNoNGeneyXWNO1/rLD+bh0sy1zPmNhNfgShKWrwsjjbbIcKCdiUG7hEZdIwMHbDgaxD8VMYUODihCmE9nA6lUfsD6eVWBy2JMH8U4gV70I5idpw6z3JYVqhsAVOVaMU/8mWJi19hTec4XT+FJVn76UJUt13vUHMxiE4qNLVK7ljSR6Lsf0NmgBuzzfl6twmVHbpFIbC+gU3XoNhI6qQcJI2pUJAgrZT8R5HmnlqVIvI9mG5GkJyqKveC8y/KhjdDrYt79wCPv5tm94bwU/NCnDT+DiiZ+spE/uSTQcPgVy2k7RuZCenf9W7VrZdz0Wn7FNwlT7nY4SPexrgm48J8SoTPMP4py/SSTAAAAADdqwgFu1IQDWb5GAtyoCQfrwssGsnyNBIUWTwW4URMOjzvRD9aFlw3h71UMZPkaCVOT2AgKLZ4KPUdcC3CjJhxHyeQdHneiHykdYB6sCy8bm2HtGsLfqxj1tWkZyPI1Ev+Y9xOmJrERkUxzEBRaPBUjMP4Ueo64Fk3kehfgRk041yyPOY6SyTu5+As6PO5EPwuEhj5SOsA8ZVACPVgXXjZvfZw3NsPaNQGpGDSEv1cxs9WVMOpr0zLdAREzkOVrJKePqSX+Me8nyVstJkxNYiN7J6AiIpnmIBXzJCEotHgqH966K0Zg/ClxCj4o9BxxLcN2syyayPUuraI3L8CNmnD351hxrlkec5kz3HIcJZN3K09RdnLxF3RFm9V1eNyJfk+2S38WCA19IWLPfKR0gHmTHkJ4yqAEev3KxnuwLrxsh0R+bd76OG/pkPpubIa1a1vsd2oCUjFoNTjzaQh/r2I/FW1jZqsrYVHB6WDU16Zl471kZLoDImaNaeBnIMvXSBehFUlOH1NLeXWRSvxj3k/LCRxOkrdaTKXdmE2YmsRGr/AGR/ZOQEXBJIJERDLNQXNYD0Aq5klCHYyLQ1Bo8VRnAjNVPrx1VwnWt1aMwPhTu6o6UuIUfFDVfr5R6DniWt9TIFuG7WZZsYekWDSR610D+ylcWkVvXm0vrV+AGzXht3H34O7PseLZpXPjXLM85mvZ/ucyZ7jlBQ165DhKJu8PIOTuVp6i7GH0YO3k4i/o04jt6Yo2q+u9XGnq8LgT/cfS0fyebJf+qQZV/ywQGvobetj7QsSe+XWuXPhI6QDzf4PC8iY9hPARV0bxlEEJ9KMry/X6lY33zf9P9mBdeNlXN7rYDon82jnjPtu89XHei5+z39Ih9d3lSzfc2Axr1+9mqda22O/UgbIt1QSkYtAzzqDRanDm010aJNIQ/l7FJ5ScxH4q2sZJQBjHzFZXwvs8lcOigtPBlegRwKivTcufxY/KxnvJyPERC8l0B0TMQ22GzRrTwM8tuQLOQJavkXf8bZAuQiuSGSjpk5w+pparVGSX8uoilcWA4JT4x7yfz61+npYTOJyhefqdJG+1mBMFd5lKuzGbfdHzmjA1iY0HX0uMXuENjmmLz4/snYCK2/dCi4JJBIm1I8aIiGSag78OWILmsB6A0drcgVTMk4RjplGFOhgXhw1y1Yag0OKpl7ogqM4EZqr5bqSrfHjrrksSKa8SrG+tJcatrBiB8acv6zOmdlV1pEE/t6XEKfig80M6oar9fKOdl76i0HPEtecZBrS+p0C2ic2CtwzbzbI7sQ+zYg9JsVVli7BoIte7X0gVugb2U7gxnJG5tIrevIPgHL3aXlq/7TSYvgAAAABlZ7y4i8gJqu6vtRJXl2KPMvDeN9xfayW5ONed7yi0xYpPCH1k4L1vAYcB17i/1krd2GryM3ff4FYQY1ifVxlQ+jCl6BSfEPpx+KxCyMB7362nx2dDCHJ1Jm/OzXB/rZUVGBEt+7ekP57QGIcn6M8aQo9zoqwgxrDJR3oIPq8yoFvIjhi1ZzsK0ACHsmk4UC8MX+yX4vBZhYeX5T3Rh4ZltOA63VpPj88/KDN3hhDk6uN3WFIN2O1AaL9R+KH4K/DEn5dIKjAiWk9XnuL2b0l/kwj1x32nQNUYwPxtTtCfNSu3I43FGJafoH8qJxlH/bp8IEECko/0EPfoSKg9WBSbWD+oI7aQHTHT96GJas92FA+oyqzhB3++hGDDBtJwoF63FxzmWbip9DzfFUyF58LR4IB+aQ4vy3trSHfDog8Ny8dosXMpxwRhTKC42fWYb0SQ/9P8flBm7hs32lZNJ7kOKEAFtsbvsKSjiAwcGrDbgX/XZzmReNIr9B9ukwP3JjtmkJqDiD8vke1YkylUYES0MQf4DN+oTR66z/Gm7N+S/om4LkZnF5tUAnAn7LtI8HHeL0zJMID521XnRWOcoD9r+ceD0xdoNsFyD4p5yzdd5K5Q4VxA/1ROJZjo9nOIi64W7zcW+ECCBJ0nPrwkH+khQXhVma/X4IvKsFwzO7ZZ7V7R5VWwflBH1Rns/2whO2IJRofa5+kyyIKOjnDUnu0osflRkF9W5II6MVg6gwmPp+ZuMx8IwYYNbaY6taThQL3BhvwFLylJF0pO9a/zdiIylhGeini+K5gd2ZcgS8n0eC6uSMDAAf3SpWZBahxelvd5OSpPl5afXfLxI+UFGWtNYH7X9Y7RYufrtt5fUo4JwjfptXrZRgBovCG80Oox34iPVmMwYfnWIgSeapq9pr0H2MEBvzZutK1TCQgVmk5yHf8pzqURhnu3dOHHD83ZEJKovqwqRhEZOCN2pYB1ZsbYEAF6YP6uz3KbyXPKIvGkV0eWGO+pOa39zF4RRQbuTXZjifHOjSZE3OhB+GRReS/5NB6TQdqxJlO/1prr6cb5s4yhRQtiDvAZB2lMob5RmzzbNieENZmSllD+Li6ZuVQm/N7onhJxXYx3FuE0zi42qatJihFF5j8DIIGDu3aR4OMT9lxb/VnpSZg+VfEhBoJsRGE+1KrOi8bPqTd+OEF/1l0mw26ziXZ81u7KxG/WHVkKsaHh5B4U84F5qEvXacsTsg53q1yhwrk5xn4BgP6pnOWZFSQLNqA2blEcjqcWZobCcdo+LN5vLEm505TwgQQJlea4sXtJDaMeLrEbSD7SQy1ZbvvD9tvpppFnUR+psMx6zgx0lGG5ZvEGBd5bemxpYmNdIEVycm9yOiBGYWlsZWQgdG8gYWxsb2MgJWQgKiAlZCBieXRlcyAocG9vbD0lbHUgaGVhZGVyPSVsdSkgJWx1CgBbemxpYmNdIEVycm9yOiBCYWQgZnJlZSBhZGRyZXNzCgAAAABVTktOT1dOAEZhaWxlZCB0byBib290ICVzLiBGYWxsYmFja2luZyBib290IG1vZGUKAAAAdWFydDogdXNpbmcgdWFydCVkIHdpdGggYmF1ZHJhdGUgJWQKAAAAAEVycm9yOiBjb25zb2xlIGFuZCBGRkggdXNlcyB0aGUgc2FtZSBVQVJULCBkZWFjdGl2YXRpbmcgY29uc29sZQoAAAAAYm9vdDogRm9yY2VkIHRvIEZGSAoAAAAAJXMgaW1hZ2UgZmFpbGVkIHRvIGNvbXBsZXRlIGl0cyAzIGZpcnN0IGJvb3RzIHNvIGlzIG5vdCByZWxpYWJsZS4KAABGbGFzaCBhIG5ldyAlcyBpbWFnZSB0byBzb2x2ZSB0aGUgcHJvYmxlbQoAAGJvb3Q6ICVzIG1vZGUKAABVbmtvd24gYm9vdCBtb2RlICVkLCBmYWxsYmFja2luZyB0byBGRkYKAAAAAEZhaWxlZCB0byBzZXQgZGlydHkgYm9vdAoAAABib290U3JjX2dlcmJpbC5jAAAAAF9wb29sLmZyZWUgPCAzAAAhX3Bvb2wucG9vbFtfcG9vbC5mcmVlXQBQb29sIGVtcHR5CgBfcG9vbC5mcmVlIDw9IDMAcAAAAEVycm9yOiBhbGxvY2F0aW9uIGZhaWx1cmUgKCVkIGJ5dGVzKSwgY2FuJ3QgYWxsb2NhdGUgYmxvY2tzIG9mIG1vcmUgdGhhdCAlZCBieXRlcwoAAEZGRgBGRkgAVVBEQVRFUgBSRUNPVkVSWQAAAAAAYJ94AGCffABgn4AAYJ+IY29uc29sZQBTaHV0ZG93bgoAAABNT0RVTEVfRlNUX1NIVVRET1dOAHVhcnQwX2N0c19uAHVhcnQxX2N0c19uAGJvb3Q6IENhbid0IGFsbG9jYXRlIG1lbW9yeSBmb3IgZXh0IGJvb3QgQUJJIGRlc2NyaXB0b3IKAAAAAFJ1bm5pbmcgb24gJXMgZmxhc2ggZmFpbHNhZmUgc2VjdG9yCgAAAABSdW5uaW5nIG9uICVzIGZsYXNoIHNlY3RvciAlcAoAAFdBUk5JTkc6IGNvcnJ1cHRlZCBib290IHNlY3RvcnMgcHJlc2VudAoAAAAAW2Z3IHBhbmljXQAAUkJHZXJiaWwgJWQuJWRAJWQgJzUuMS4xLjAgWzQxMDY1XScKAAAAAFJlc2V0IGNhdXNlICclcycocmVhbCAnJXMnICklcyAoYm9vdFdERyA6ICclZCcpIFtyYXdSc3QgJzB4JTA4eCddCgAATlZSQU0gQm9vdCBTdGF0dXMgJyVzJwoATG9hZGVkIHZpYSBKVEFHCgAAAAByZWdDb25maWcgJXBAJWQKAAAAAGJvb3RMb2FkZXIuYwAAAABOb3RoaW5nIHRvIGJvb3QgZnJvbQAAAAAAAQICAwMDAwQEBAQEBAQEBQUFBQUFBQUFBQUFBQUFBQYGBgYGBgYGBgYGBgYGBgYGBgYGBgYGBgYGBgYGBgYGBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwcHBwgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIOmF0OiBFcnJvcjogRmFpbGVkIHRvIGFsbG9jYXRlIG91dHB1dCBwb29sCgBPSw0KAAAAADphdDogRXJyb3I6IENNRCBGQUlMRUQKAEVSUk9SDQoAOmF0OiBFcnJvcjogJXMlcwoAAAA6YXQ6IEVycm9yOiAlcyVzPwoAADANCgA6YXQ6IEVycm9yOiAlcyVzPTxiYXVkcmF0ZT4KAAAAADphdDogRXJyb3I6IEZhaWxlZCB0byBjb252ZXJ0ICVzIGludG8gYW4gaW50ZWdlcgoAAABhdDogU2V0dGluZyBiYXVkcmF0ZSB0byAlZAoAOmF0OiBFcnJvcjogVW5rb3duIGNvbW1hbmQgJXMKAABhdDogRm9yY2luZyBiYXVkcmF0ZSB0byAlZCBmb3IgQVQgbmVnb2NpYXRpb24KAAArU1lTRkZIDQoAAAA6YXQ6IEVycm9yOiBGYWlsZWQgdG8gYWxsb2NhdGUgaW5wdXQgcG9vbAoAADphdDogRXJyb3I6IENvbW1hbmQgdG9vIGxhcmdlCgAAAGCkTABgUzYAYKREAGBUagBgpDwAYFPiAGCkNABgU2BVc2FnZTogAEFUK1NNT0QAQVQrSVBSAABBVCtTVFAAAEFUAABib290ZmxhZ3MgRXJyb3I6IEZhaWxlZCB0byByZWFkIGJvb3QgZmxhZ3MKAEJPT1RGTEFHAAAAAGJvb3RmbGFncyBFcnJvcjogSW52YWxpZCBib290IGZsYWdzCgAAAABib290ZmxhZ3M6IFNldHRpbmcgZGlydHkgYm9vdCBsZXZlbCAlZAoANS4xLjEuMCBbNDEwNjVdAGoJ5me7Z66FPG7zcqVP9TpRDlJ/mwVojB+D2atb4M0ZQoovmHE3RJG1wPvP6bXbpTlWwltZ8RHxkj+CpKscXtXYB6qYEoNbASQxhb5VDH3Dcr5ddIDesf6b3AanwZvxdOSbacHvvkeGD8GdxiQMocwt6SxvSnSEqlywqdx2+YjamD5RUqgxxm2wAyfIv1l/x8bgC/PVp5FHBspjURQpKWcntwqFLhshOE0sbfxTOA0TZQpzVHZqCruBwskuknIshaK/6KGoGmZLwkuLcMdsUaPRkugZ1pkGJPQONYUQaqBwGaTBFh43bAgnSHdMNLC8tTkcDLNO2KpKW5zKT2gub/N0j4LueKVjb4TIeBSMxwIIkL7/+qRQbOu++aP3xnF48i0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoAAEZpbGU6ICVzQCVkCgAAAAAnJXMnCgAAAEZhdGFsIGVycm9yCgAAAABBc3NlcnRpb24gZmFpbGVkCgAAAABgplAAYKZgYm9vdDogU2tpcHBpbmcgc3RyYXAgcmVjdXJzaW9uCgBib290OiBDdXJyZW50ICVzLCB0aW1lb3V0ICV1LCBwcm90byAlcwoAYm9vdDogU3dpdGNoZWQgdG8gJXMsIHRpbWVvdXQgJXUsIHByb3RvICVzCgB0aHAAc3RwAGR0cABhdAAAbm9uZQAAAABmbGFzaAAAAHVzYjAAAAAAdXNiMQAAAABldGgAdWFydDAAAAB1YXJ0MQAAAHVhcnQyAAAAdWFydDMAAABzZGlvAAAAAAAAADcAAAA4AAAAOQAAADoAYKbwAGCm9ABgpvgAYKb8AGCnAABgpwgAYKcQAGCnGABgpyAAYKckAGCnLABgpzQAYKc8AGCnRE5vcm1hbCBCb290AEZsYXNoIHJlc2V0AE5vIE5WUkFNIEJvb3QAAAAAYKeUAGCnoABgb/wAYKesUGluICVkIGlzIHVzZWQgYXMgd2FrZSBzb3VyY2UsIGNhbid0IGJlIGNvbmZpZ3VyZWQgZm9yIG91dHB1dC4KAAAAABsAAAAcAAAAHQAAAB4AAAAfAAAAIAAAACEAAAAiAAAAIwAAACQAAAAlAAAAJgAAACcAAAAoW1BPV0VSXSBFcnJvcjogREMgMVY4IGlzIG5vdCBnb29kCgAAW1BPV0VSXSBFcnJvcjogREMgM1YwIGlzIG5vdCBnb29kCgAAW1BPV0VSXSBFcnJvcjogREMgMVYxIGlzIG5vdCBnb29kCgAAW1BPV0VSXSBFcnJvcjogUG93ZXIgaXMgbm90IGdvb2QsIHNodXR0aW5nIGRvd24KAAAAAFtTRkZGXSBFcnJvcjogQ1JDIEVycm9yOiBjb21wdXRlZCBjcmMgPSAweCUwOHgsIGV4cGVjdGVkIGNyYyA9IDB4JTA4eAoAAFtTRkZGXSBFcnJvcjogRUNEU0FfMjU2IGlzIG5vdCBzdXBwb3J0ZWQgeWV0CgAAAFtTRkZGXSBFcnJvcjogVW5rbm93biBzaWduYXR1cmUgdHlwZSAlZAoAAAAAW1NGRkZdIEVycm9yOiBCb290IHJlZ2lvbiAlZCBub3QgZm91bmQKAEZGRiEAAAAAW1NGRkZdIEVycm9yOiBXcm9uZyBtYWdpYzogMHglMDh4CgAAW1NGRkZdIEVycm9yOiBXcm9uZyBoZWFkZXIKAFtTRkZGXSBFcnJvcjogQ29ycnVwdGVkIGltYWdlCgAAW1NGRkZdIEVycm9yOiBDb3JydXB0ZWQgYm9vdCBpbWFnZQoASUVMRgAAAABbU0ZGRl0gRXJyb3I6IEludmFsaWQgYm9vdCBlbnRyeQoAAAA/Pz8ARVhUAExQAABTVwAAV0RUMAAAAABXRFQxAAAAAExQV0RUAAAATFBDUFVSU1QAAAAATFBIV1JTVAAAYKpoAGCqbABgqnAAYKp0AGCqeABgqoAAYKqIAGCqkABgqpxzdHA6IGZhaWxlZCB0byBhbGxvY2F0ZSBJT0Igb2YgJWQgYnl0ZXMKAAAAAHN0cDogZmFpbGVkIHRvIHB1c2ggU1JTUCBJT0IKAAAAc3RwOiBVc2luZyBTVFAgc291cmNlCgAAc3RwOiBpbmNvcnJlY3QgTVJFUSBoZWFkZXIgQ1JDOiAweCUwOFghPTB4JTA4WAoAc3RwOiB1bmtub3duIE1SRVEgb3BlcmF0aW9uOiAweCUwMlgKAAAAAHN0cDogd3Jvbmcgc2lkOiAlZCAhPSAlZAoAAABzdHA6IHdyb25nIHBsZW46ICVkICE9ICVsZAoAc3RwOiB3cm9uZyBwYXlsb2FkIENSQzogMHglMDhYICE9IDB4JTA4WAoAAABzdHA6IHBlbmRpbmcgdHJhbnNhY3Rpb24gaXMgdG9vIGJpZzogJWQgPiAlbGQKAABzdHA6IFNCUCBwcm9jZXNzaW5nIGZhaWxlZAoAc3RwOiBGQVRBTAoAAQEIAFtaU1RQXSBFcnJvcjogRmFpbGVkIHRvIGFsbG9jYXRlIElPQiBvZiAlZCBieXRlcwoAAABbWlNUUF0gRXJyb3I6IEluZmxhdGUgaW5pdCBmYWlsZWQgOiAlZAoAW1pTVFBdIEVycm9yOiBJbmZsYXRlIGZhaWxlZCA6ICglZCkgJXMKABwAAAAAAAAABQYAAQAAAQYBAQABAQYCAQACAQYDAQADAQYEAQAEAQAAAAAAAQECAgAAAAAAYK0QAQsAJAAEAFwAAAIAAAAAAABgrRQAYK00AwYAAQAAAQYBAQAEBAsCJAAMABQACAQAAAAAAABgrTgAAAAABAYABQAABAYBAQAEBAYCAQAIBAcDAgAMAAQAAAAAAAAAYK1wAQsAJAAEACAAAAQAAAAAAABgrXQAAAAACAYAAQAABAYBAQAEBAYCBQAIBAYDBQAMBAYEBQAQBAYFBQAUBAUGAwAYBgcFABwEAAAAAAAAAAAAAAAAAAHCAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcIAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAABwgAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAHCAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAABgrlACCwAkAAQAIAAABAALASQAiAAMAIQIAQAAAAAAAGCudABgrlgAAAAAAwYABQAABAYBAQAEBAYCAQAIBAAAAAAAAAAAAAsGAAEAAAQGAgUABAQGAwEACAQGBAEADAQGBQEAEAQGBgEAFAQGBwEAGAEGCAEAGQEGCQEAGgEGCgEAGwEGCwEAHAEAAAAAAAAAAAAAAAAAAAAAAgAAAQAAAAAAAAAAAAAAUBTFZpkGBQAAAAAAAAAAAAACAAABAAAAAAAAAAAAAABQFMVmmQYFAAAAAAAAAAAAAAIAAAEAAAAAAAAAAAAAAFAUxWaZBgUAAAAAAAAAAAAAAgAAAQAAAAAAAAAAAAAAUBTFZpkGBQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYK+4AQYABAAAAAAAAAAAAGCvvAAAAAABBwACAAAABAAAAAABAAAAAGGNAABgSxgAYEjEAGBIrgBgSKQAYEqEAGBLTABgSnoAAAAAa2FiaQAAAAMAAAAAAAAAAAAAAAAAAAAAYkFiaQAAAAcAAAAAAAAAAAAAoGkAAAAAAAAAAAAAAAH/////AGCwVAMGAAEAAAQLASQACAAMAAQIAAsCJABsAAwAaBABAAAAAGCweABgsFwAAAAAAwYAAQAAAQYBBQAEBAYCBQAIBAAAAAAAAAAAAAMGAAUAAAQGAQUABAQGAgAACAQAAAAAAAAAAAABBgAFAAAEAAAAAAAAAAABAAAAAAEGAAUAAAQAAAAAAAAAAAAEBgAFAAAEBgEFAAQEBgIBAAgIBgMBABAIAAAAAAAAAAAAAAAAAAAWCAAAAAAAAAAAAAAAAAAAABIGAAEAAAIGAQEAAgIGAgEABAIGAwUACAQGBAUADAQGBQUAEAQGBgUAFAQGBwUAGAQGCAEAHAIGCQEAHgEGCgEAHwEGCwEAIAEGDAEAIQEFDQMAJAUOAwAoBRADACwGEQUAMAQGEgUANAQAAAAAAAAUjhEBoAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAASwAAQAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAgBgsdgFCwAkAAQAJAAAQAAGAQEJBAQLAiQJDAAMCQgEAQsDJAlAAAgJPAQCCwQkCWQAEAlgBAMAAAAAAABgsjQAYLIYAGCyBABgsegAAAAABAUAAwAABQEDAAQFAgMACAUDAwAMAAAAAAAAAAIGAAUAAAQGAQAABAQAAAAAAAAAAwYAAQAABAYBAQAEBAYCAQAIBAAAAAAAAAAAAAkGAAEAAAQGAQUABAQGAgUACAQGAwUADAQGBAUAEAQGBQUAFAQFBgMAGAYHBQAcBAYIAQAgBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABTUU5TAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAD/////AAAAAP////8AAAAA/////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQEAAAAAAACKxR1iXnElWSUv1TcLbA3iAAAAAAAAAAAAYB6gAAAAAABgI6YAAAAAAGBdlAAAAAAAYFw+AAAAAABgF9gAAAAAAGBHsgAAAAAAYEzaAAAAAABgXo4AAAAAAGAJqAAAAAAAYCl4AAAAAABgRswAAAAAAGAUVAAAAAAAYCXiAAAAAABgKwoAAAAAAGBMugAAAAAAYBuCAAAAAABgKcQAAAADAGArMgAAAAJjcnkAAGANcmNyegQAYGHyAAAAAAAAAAAAAAApAAAAAABgcHQAAAAAAGCfuABgTBAAAAAA/////wBgTHoAAAAAAGAejgAAAAAAAAAUAGCs9ABgrMwAAAAAAAAAEgBhDBQAYKz8AAAAAAAAACsAYK2sAGCtXAAAAAAAAAAAAGCuwABgrjAAAAAAAAAAFwBhDNAAYK+oAAAAAAAAACAAYYs0AGCwMAAAAAAAAAAvAGCwpABgsJQAAAAAAAAAMQBhjGAAYLCoAAAAAAAAAAQAYLDYAGCwuAAAAAAAAAAGAGCxZABgsPAAAAAAAAAACQBgsnQAYLGcAAAAAABgEcgAYBA6AGANZAAAAAAAYCSEAGAkiABgJIwAAAAAAAAAAAAAAAAAYEj4AAAAAQBgSEgAAAAAAGBH6gAAAAcAYEhIAAAAAABgR+oAAAAGAGBISAAAAAAAYEfqAAAABYIAVAAAAAADAAAAAAAAAAAAAAAAAAAAAIIADACCAFH2AQAAAABgS4QAYBzKBjKIWAIAAAAAAAAAggAIAIIAUfUBAAAAAGBLhABgHMoGMohYAQAAAAAAAACCAAQAggBR9AEAAAAAYEuEAGAcygYyiFgAAAAAAAAAADUuMS4xLjAgWzQxMDY1XQByYm1jAAAAAYIAUkEAAABUAAAAAQIAAAAAAAAAAAAAAoIAHPAAAAAJAAAAAYIAHPEAAAAQAAAAAYIAHPIAAABpAAAAAYIAHPMAAAAQAAAAAYIAUfgAAAAQAAAAAoIAUfwAAH/vAAAAAoIAUgAAAAAQAAAAAnBzaQBzcW4x/////wAAAAABTgERAgEihIAAKoiAgAAyhqxAOgAJOQKGk73edBKYgAAAAgYKmIAAEoSAAAACCAqcgAASoIAAAAICCryAABKDgIAAAAIDCoO8gAAShMSAABGdNRqH/39xnHhnRUZiV2x3YVZCbTFoYW05eUF6VElReHNMazVOTEs1S0JPQUdZNUJYQmhiWENRVjFjMmxrd1F4TlVnN3N5dHpJCjN1V0RHQmdZR0NJTTV1REt5c21BcklLMHR6UzZTQ01iYXlZbWJHcHFabXBBYkdwcVptcEFiR3BxWm1wQ0NOamcKMnlFTmpDNk1yY3h2T0NtQkFZSkJtVnVkR1Z5a0FuRVRBd01EQWdNREV5T0NBMk5UVXpOVWdFNGlZR0JnWWtCZwpZR0JnUUd4cWFtWnFNUk1EQXdNeUF3TURBd0lEWTFOVE0xR0ltQmdaSEJBWUdCZ2JrQnNhbXBtYW1JS3l2RFM2ClNBVGdLRlNDdURrM3M4Z3JFd3R6SW9BWUNPUVZ0YjJSbHdNd1NBTHlVeU5EVWdNalExSURJME5TQXlORFVnTWoKUTFJREkwTlNBeU5EVWdNalExSURFZ01FUVZ3WVdjd2tBVGlKZ1lHQmdRR0JpY0hCQWJHcHFabXFRQk9JbUJnWQpHSkFZR0JpYkVCc2FtcG1hakVUQXdNRE1nTURFeU9DQTJOVFV6TlJnTW5JSzRNTE9ZeUFKd0hFZ0NjUk1EQXdNClNBd01ERTNJRFkxTlRNMUdBOE1Ca3dncmd3czVsSUFuQWNTQUp4RXdNREF4SURBd016UWdOalUxTXpVWUM0WUMKazBncmd3czVuSUFuQVdKQUU0aVlHQmdZa0JnWUdacVFHeHFhbVpxTUJjTUJTY1FWd1lXYzBrQVRnTEVnQ2NSTQpEQXdNU0F3TURZNElEWTFOVE0xR0F1R0FwUElLNE1MT2F5QUp3RmlRQk9JbUJnWUdKQVlHQnVaa0JzYW1wbWFqCkVUQXdNRE1nTURFek9DQTJOVFV6TlJnS1RCQlhCaFp6YVFCT0FzU0FKd0dZd0dnd0ZKa2dyZ3dzNXZJQW5BV0oKQUU0RE1ZRFFZQ2swUVZ3WVdjNGtBVGlKZ1lHQmdRR0JrYW1aQWJHcHFabXFRQk9BcE5rSmMzRnVSSEpwZG1XUQpaeVlXNW5aY1pOalUxTXpVZ05qVTFNelVnTmpVMU16VWdOalUxTXpVdVFCNkJtUmdLbFlDdExBV0RBV2pBWERBClVtR0FzR0F2R0F1R0FwTHNCWU1CZ01CY01CU1dZQ3dZREVZQzRZQ2txd0Znd0dRd0Z3d0ZKUmdMQmlKZ1lHQmkKUUdCZ2JtSkFiR3BxWm1veEV3TURBeklEQXhNemNnTmpVMU16VVlDa213Rmd3SEF3SEl3RkpKZ0xCZ09CZ09SZwpLU0xBYWpBVWtHQTJMRURNekFWS3dGYVdBc0dBdEdBdUdBcE1NQllNQmVNQmNNQlNYWUN3WURBWUM0WUNrc3dGCmd3R0l3Rnd3RkpWZ0xCZ01oZ0xoZ0tTakFXREVUQXdNREVnTURFMU1DQTJOVFV6TlJpSmdZR0JtUUdCaVpteEEKYkdwcVptb3dGSk5nTEJnT3hnUEJnS1NUQVdEQWRqQWVEQVVrV0ExR0FwSU1Cc1dJQ0F3RlNzQldsZ0xCZ0xSZwpMaGdLVERBV0RBWGpBWERBVWwyQXNHQXdHQXVHQXBMTUJZTUJpTUJjTUJTVllDd1lESVlDNFlDa293Rmd3SFl3Ckhnd0ZKTmdMQmdPeGdQQmdLU1RBV0RBZGpBZURBVWtXQTFHQXBJTUJzV0lHYW1BcVZnSzBzUk1EQXdNQ0F3TVQKWTBJRFkxTlRNMUdBdEdBdUdBcE1NQjhNUk1EQXdNU0F3TURBeElEWTFOVE0xR0F1R0FwTHNCOE1STURBd01TQQp3TURNeklEWTFOVE0xR0FuREFTU3pBWGhnS0l3RTRZQ1NWWUM4TVJNREF3TVNBd01ETTNJRFkxTlRNMUdBbkRBClNTakFYaGdMQXhFd01EQXpJREF4TXprZ05qVTFNelVZQ1NUWUM4TUJZR0F4REFTU1RBWGhnTEF3R0lZQ1NSWUMKcU1CSklNQldMRURPREFTbFlDV2xnTHd3RTBZQ2NNQkpNTUJlR0F2akFUaGdKSmRnTHd3R0FZQ2NNQkpMTUJlRwpBb2pBVGhnSkpWZ0x3d0dFWUNjTUJKS01CZUdJbUJnWUdKQVlHQnFia0JzYW1wbWFqQVloZ0pKTmdMd3dHUVlECkVNQkpKTUJlR0F5REFZaGdKSkZnS293RWtnd0ZZc1FReE1qQVNsWUNXbGdMd3dFMFlDY01CSk1NQmVHQXZqQVQKaGdKSmRnTHd3R0FZQ2NNQkpMTUJlR0FvakFUaGdKSlZnTHd3R0VZQ2NNQkpLTUJlR0FzREFUaGdKSk5nTHd3RgpnWUNjTUJKSk1CZUdBc0RBVGhnSkpGZ0tvd0VrZ3dGWXNRUXhNekFTbFlDV2xnTHd3RTBZQ2NNQkpNTUJlR0F2CmpBVGhnSkpkZ0x3d0dBWUNjTUJKTE1CZUdBb2pBVGhnSkpWZ0x3d0dFWUNjTUJKS01CZUdBc0RBWWhnSkpOZ0wKd3dGZ1lERU1CSkpNQmVHQXNEQVloZ0pKRmdLb3dFa2d3RllzUVF4TkRBU2xZQ1dsZ0x3d0UwWUNjTUJKTU1CZQpHQXZqQVRoZ0pKZGdMd3dHQVlDY01CSkxNQmVHQW9qQVRoZ0pKVmdMd3dHRVlDY01CSktNQmVHQXNEQVloZ0pKCk5nTHd3RmdZREVNQkpKTUJlR0FzREFZaGdKSkZnS293RWtnd0ZZc1FReE56QVNsWUNXbGdMd3dFMFlDY01CSk0KTUJlR0F2akFUaGdKSmRnTHd3R0FZQ2NNQkpMTUJlR0FvakFUaGdKSlZnTHd3R0VZQ2NNQkpLTUJlR0FzREFUaApnSkpOZ0x3d0ZnWUNjTUJKSk1CZUdBc0RBVGhnSkpGZ0tvd0VrZ3dGWXNRUXhPREFTbFlDV2xnTHd3RTBZQ2NNCkJKTU1CZUdBdmpBVGhnSkpkZ0x3d0dBWUNjTUJKTE1CZUdBb2pBVGhnSkpWZ0x3d0dFWUNjTUJKS01CZUdBc0QKQVloZ0pKTmdMd3dGZ1lERU1CSkpNQmVHQXNEQVloZ0pKRmdLb3dFa2d3RllzUVF4T1RBU2xZQ1dsZ0x3d0UwWQpDY01CSk1NQmVHQXZqQVRoZ0pKZGdMd3dHQVlDY01CSkxNQmVHQW9qQVRoZ0pKVmdMd3dHRVlDY01CSktNQmVHCkFzREFZaGdKSk5nTHd3RmdZREVNQkpKTUJlR0FzREFZaGdKSkZnS293RWtnd0ZZc1FReU1EQVNsWUNXbGdMd3cKRTBZQ2NNQkpNTUJlR0F2akFUaGdKSmRnTHd3R0FZQ2NNQkpMTUJlR0FvakFUaGdKSlZnTHd3R0VZQ2NNQkpLTQpCZUdBc0RBWWhnSkpOZ0x3d0ZnWURFTUJKSk1CZUdBc0RBWWhnSkpGZ0tvd0VrZ3dGWXNRUXlOVEFTbFlDV2xnCkpnd0UwWUNjTUJKTU1CTUdBbmpBVGhnSkpkZ0pnd0ZBWUNjTUJKTE1CTUdBb2pBVGhnSkpWZ0pnd0ZJWUNjTUIKSktNQk1HQXNEQVdSZ0pKTmdKZ3dGZ1lDeU1CSkpNQk1HQXNEQVdSZ0pKRmdLb3dFa2d3RllzUVF5TmpBU2xZQwpXbGdMd3dFMFlDY01CSk1NQmVHQXZqQVRoZ0pKZGdMd3dHQVlDY01CSkxNQmVHQW9qQVRoZ0pKVmdMd3dHRVlDCmNNQkpLTUJlR0FzREFZaGdKSk5nTHd3RmdZREVNQkpKTUJlR0FzREFZaGdKSkZnS293RWtnd0ZZc1FReU9EQVMKbFlDV2xnTHd3RTBZQ2NNQkpNTUJlR0F2akFUaGdKSmRnTHd3R0FZQ2NNQkpMTUJlR0FvakFUaGdKSlZnTHd3RwpFWUNjTUJKS01CZUdBc0RBWWhnSkpOZ0x3d0ZnWURFTUJKSk1CZUdBc0RBWWhnSkpGZ0tvd0VrZ3dGWXNRUTJOCmpBU2xZQ1dsZ0pnd0UwWUNjTUJKTU1CTUdBbmpBVGhnSkpkZ0pnd0ZBWUNjTUJKTE1CTUdBb2pBVGhnSkpWZ0oKZ3dGSVlDY01CSktNQk1HQXRqQVhCZ0pKTmdKZ3dGc1lDNE1CSkpNQk1HQXRqQVhCZ0pKRmdLb3dFa2d3Rll0cwpnckMzT2J2SUFYQVFrZ0JzQkRSQUQ0Q0N5QUlZQ0lRZ0NtQWlsZ0loR0FpRlNBT0lDRFNBTHdFVmhBM1I0a0FUCmdKaVFCT0FrblNCdVR4SUFuRVRBd01EQWdNREUwTUNBMk5UVXpOVWdDY0JIR0FuREFTVHBBSzVnSW9XQW1EQVMKU0RBY0JnSTR3RTRZQ1NkSUJhTUJGQ3dFd1lDU1FZRGdNQkhHQW5EQVNUcEFJQmdJb1dBbURBU1NEQWNCZ0k0dwpFNFlDU2RJQmRNQkZDd0Y0WUNTUVlpWUdCZ1lFQmdZbXBzUUd4cWFtWnFNQkhHQW5EQVNUcEFNWmdJb1dBdkRBClNTREFjUmdJNHdFNFlDU2RJQmxNQkZDd0Y0WUNTUVlEaU1CSEdBbkRBU1RwQU14Z0lvV0F2REFTU0RBY1JnSTQKd0U0WUNTZElCbk1CRkN3RjRZQ1NRWURpTUJIR0FuREFTVHBBTkJnSW9XQXZEQVNTREFjUmdJNHdFNFlDU2RJQgpwTUJGQ3dGNFlDU1FZRGlNQkhHQW5EQVNUcEFOUmdJb1dBdkRBU1NEQWNSZ0k0d0U0WUNTZElCck1CRkN3RjRZCkNTUVlEaU1CSEdBbkRBU1RwQU5oZ0lvV0FtREFTU0RBY0JnSTR3RTRZQ1NkSUJ0TUJGQ3dGNFlDU1FZRGlNQkgKR0FuREFTVHBBTnhnSW9XQXZEQVNTREFjUmdJNHdFNFlDU2RJQnZNQkZDd0V3WUNTUVlEZ01CSEdBbkRBU1RyVak0ATICAQACAgACAwACBAACBQACCAACDAACDQACDgACEQACEgACEwACFAACGQACGgACHAACQhlJEUeAAQAFAAlSRURCRUxMWQCAAQASAAZQWUNPTQCAAQAWAA1HRVJCSUwtUkVEQkUAgAEABAAIU1FOMzMzMACAAQARAAVGaVB5ADk+ASABHgLAbgACwDEAAsAyAALAMwACwG8AAAAAAAAAAAAAAAkaARgAAsBtAAIDAAICAAIBAAAAAAAAAAAAAABJYQFfAiUSATEMdWFydDBfY3RzX24AAAIDEgExDHVhcnQwX3J0c19uAAACBxIBMQx1YXJ0MV9ydHNfbgAAAgwSATEMdWFydDJfcnRzX24AAAIeEgExDXVhcnRfMDBfcmluZwBxExkRArmOYjKDTgACgqwAAAKB8AAPAREJDwIDEoGcIAACBwoDEoepQA8RBgKIAAqIAA9BKAEFRmlQeQAJCFNRTjMzMzAAGQ1HRVJCSUwtUkVEQkUAIQZQWUNPTQAPWSsBKQq4oAAqBDEDYXQAAAIBKgUxCGNvbnNvbGUAAAICCrigACoEMQRkY3AAD2GSEAGQB2dFTWMyVnhkV0Z1Y3pNek16QlFadFlXcHZjZ014eUR1ekszTWplNWFBanBwZ1lxSmFwSXFzYUVERzN0ek0wczcKcTVNTG8wdDdjUUthaW5HWm1abUJBZ21LQ1prQ21sckpzY0dCZ1lscG1Za0JpYmtERXd0ekk1a0NrWUdhUWRqYgoyMXRiMjZROTBlRjluWVdsdVgzQnZjblJmTU5mTFRZNE9EQWdMVFkwT0RBZ0xUVTVNVEFnTFRVek5UQWdMVFE1Ck5qQWdMVFF6TnpBZ0xUTTNOREFnTFRNd09EQWdMVEkwTVRBZ0xUSXdOVEFnTFRFM016QWdMVEUxTWpBZ0xURXoKTWpBZ0xURXdNekFnTFRneU1DQXROakF3U0h1and2czdDMHR5KzRON2s2TDVqc0JhYkhCZ1lFQmFhbkppWUVCYQphbWhtWUVCYWFHNW1ZRUJhYUdKd1lFQmFabXhtWUVCYVptQmtZRUJhWkdwb1lFQmFaR1p1WUVCYVpHSmtZRUJhClluQndZRUJhWW01Z1lFQmFZbWhzWUVCYVltUmlZRUJhWW1Ca1lFQmFjR2hnaUh1and2czdDMHR5KzRON2s2TDUKbGdPeUh1VHd2czdDMHR5KzRON2s2TDVobEprWUdwZ1FHUnNhR0JBWm1SbVlFQm1jR1JnUUdob1ptQkFhbUJpWQpFQnFhbVpnUUd4Z1lHQ0VQY25oZloyRnBibDl3YjNKMFh6SEtURTNOREFnTWpNeU1DQXlPVEF3SURNMU1qQWdOCkRFeU1DQTBOekF3SURVeU16QWdOVGMyTUZJZTVQQyt6c0xTM0w3ZzN1VG92bVdVbUpvYW1CQVpHcGlZRUJtWW0KQmdRR1pzY21CQWFHUm1ZRUJvYm14Z1FHcG9abUJBYkdKZ1lMRUZZbUZ1WkZBbkFCR1Fsd2IzSjBYMjUxYmNCQwpRcDBlRjlrWVdOZlltL0F6QkVLZEhoZmIyWm1jMlYwd1V0TXpCa0ljR0ZmWjJGcGJza0xURTFPVEFnTVRJM01DCkF4T0Rrd0lESXlNREFnTWpVek1DQXlPVEF3SURBZ01FSWE0TUsrNk5Ea3l1YlEzdGpKa2hhWW5CbWFrQmFZbWgKcWFrQmFibXBnUUdSdWFrQndjR3BBYUdCZ1lFQm9ZR0JncEZuUjRYM1JsYlhCbGNtRjBkWEpsWDI5bVpuTmxkTQpFTmpGaUhPVHd2c1Rld3VUSXZ0amU1dWVDbVJnWU9SWnllRjkwWlcxd1pYSmhkSFZ5WlY5dlptWnpaWFRCREk1ClFRWjBabUZqZE1KTUNBd0lEQWdNRUNCbVJnSUh3RnpZREJja0xURTFPVEFnTVRJM01DQXhPRGt3SURJeU1EQWcKTWpVek1DQXlOamt3SURBZ01DWklXbUp3Wm1wQVdtSm9hbXBBV201cVlFQmtibXBBY0c1Z1FHaGdZR0JBYUdCZwpZRHdRMU9CWUN3T0F0QmdMa0NCbVpnSUQ0Q2MyQ2xwb1lGeVF0TVRVNU1DQXhNamN3SURFNE9UQWdNakl3TUNBCnlOVE13SURJNU1UQWdNQ0F3SmtoYVluQm1ha0JhWW1ocWFrQmFibXBnUUdSdWFrQnljR0JBYUdCZ1lFQm9ZR0IKZ1BCRGN5RmdMQTRDMEdBdVFJR2FHQWdQZ0p6WUtXbXhnWEFha3dHdzhFTnpBV0FzRGdMUVlDNUFnWnFZQ2MrQQpuTmdwYVpHNWNrTFRFM05UQWdNVEExTUNBeE5EQXdJREl4TnpBZ01qUTVNQ0F5TnpNd0lEQWdNQ1pJV21SZ2FtCkJBV21Kb1ltcEFXbXhzYWtCbWFtQkFibTVxUUdoZ1lHQkFhR0JnWUR3UTJNQllDd09DR0p1REFUa0NCbkJnSTUKOEJITmdwYWFtQmNrTFRFM05UQWdNVEExTUNBeE5EQXdJREl4TnpBZ01qUTVNQ0F5TnpBd0lEQWdNQ1pJV21SZwphbUJBV21Kb1ltcEFXbXhzYWtCbWFtQkFibVJxUUdoZ1lHQkFhR0JnWUR3UTJNeFlDWUhBWUFZQ2NnUVF4TWpBClJ6NENPYkFYQzVJV21KdWFtQkFZbUJxWUVCaWFHQmdRR1JpYm1CQVpHaHlZRUJrY0daZ1FHQkFZRXlRdE1qQTEKTUNBdE1UUXhOU0F0TmpZMUlETTFNQ0E0TWpVZ05EQXdNQ0EwTURBd0hnTDRzQk1EZ01BTUJPUUlJWW1aZ0k1OApCSE5ncGFZbXBjQmRKZ0x3OEJmRmdKZ2NCZ0JnSnlCQkRFME1CSFBnSTVzQnFMZ0xwTUJlSGdMNHNCTURnTUFNCkJPUUlEQVlDT2ZBUnpZQzRYQVp5WURRUEFYeFlDWUhBWUFZQ2NnUVF4T0RBUno0Q09iQWFpNEM2VEFYaDRDa0wKQVRBNERBREFUa0NDR0p5WUNPZkFSellDNFhBWFNZQzhQQVh4WUNZSEFZQVlDY2dRUXlNREFSejRDT2JBVkM0Qwo2VEFYaDRDK0xBVEE0REFEQVRrQ0NHUnFZQ0FmQVJ6WUNRWEFVQ1lDaVBBVWhZQ1lIQVRRWUNjZ1FReU5qQVJ6CjRDT2JBWEM0QzZUQVhoNENrTEFUQTREQURBVGtDQ0dSd1lDT2ZBUnpZQzRYQVhTWUM4UEFYeFlDWUhBWUFZQ2MKZ1FRMk5qQVFENENPYkFYQzRDcVRBVmg0Q3VMQVRBNENhREFUa1FBPQmCA1VsTk5VZ0FCQUFBQUt3QUNNZ0FBQkFBU0FBNEJCQUFWRjY4QUY3Z1RBQ0wvWmdBakNVQUFMVFJKQURPTUJBQTIKQUFBQU95TUJBRHdBQkFBL0FBTUFRSVVuQUVGamxRQkNJRUFBVG5MakFFOENEUUJTRDhBQVd3QS9BRndBUHdCZQpCQ3dBWHhLSUFHUGN5QUJ0TXA4QWNJUlFBSEVSQ3dDaWlPTUF1Q0FBQUxrZ0FBQzZJQUFBdXlBQUFMd2dBQUM5CkFBQUF2Z0FBQUw4QUFBREFFYU1BdzZncEFNWUNBQURRQURrQTRCR2pBT1lDQUFEd0FEa3lOalk1TURFeE9UY3oICAgTdnExMAleFA==""") diff --git a/lib/sqnsupgrade/sqnsbrz.py b/lib/sqnsupgrade/sqnsbrz.py new file mode 100644 index 0000000..a1d9923 --- /dev/null +++ b/lib/sqnsupgrade/sqnsbrz.py @@ -0,0 +1,56 @@ +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +class bootrom(object): + import binascii + try: + import uzlib as zlib + except: + import zlib + + def __init__(self): + self.__fpointer = 0 + self.__size = len(self.BOOTROM) + + def open(self, filename='', mode='r'): + self.__fpointer = 0 + + def close(self): + self.__fpointer = 0 + + def tell(self): + return self.__fpointer + + def seek(self, offset, start=0): + if start == 1: + start = self.__fpointer + elif start == 2: + start = self.__size + if start + offset < 0: + raise OSError(22,'Invalid argument') + else: + self.__fpointer = start + offset + return self.__fpointer + + def get_size(self): + return self.__size + + def read(self, size=0): + if size == 0: + size = self.__size - self.__fpointer + if self.__fpointer >= self.__size: + return b'' + else: + fpointer = self.__fpointer + self.__fpointer = fpointer + size + + if fpointer + size >= self.__size: + return self.BOOTROM[fpointer:self.__size] + else: + return self.BOOTROM[fpointer:self.__fpointer] + + BOOTROM=zlib.decompress(binascii.a2b_base64(b"""eJykfAt4FEW2cPVjZnomM0knmUAIgfQkEQKiJojrAKPWhCEPN4YBg4ZZXINEZJ+g4BV33aUm6YROGAJCNjwWYaIYwl4WooaIyu7thFdQRHS3vegaNwSMibo/E54Shsx/ambCRu/u9/3/d5NvuupUnapz6tSpU+d0d/V8d/48DiF0/gxCDOJRMnpi2bKVzyz7hfTs8qeeWVT+pLR80eKfLXrqSbRy2bInAJGFH7PiieU/Q+jPzZBfidCYvc4/fcMVIpRcgJi7N6KNtI9lK57+ZQ6ifwz8R/+YjNlPoTLIlP15S74gsQsEMyrXITYX8THFHBLyRodCpeL6kFPgWUZAzByWeUxOfwh5PWsOOj//FuUhiXGJIkoQkUm2QSZZluA6xiXa0kaUpdMy2bbPFa4VyhMQu0QkBki58kTEyhJcufKkcClNgXfm2XIrYl1ZCA09UioSK0C0tKBcRGyeFBcYemRj9kKRZBYnIFN5Une4P8BjIB/uqTTaU3lCJpsnohCkXF5iZDQgGRaL37K+rC5GF0sE/EsVrcFzWPSSHNS4BX8ibLXwUrBoFTZ4JwDEeMdp05xcKtKFJRC6hJWVSGdFfP6ECVcXTIpb9tCY1f1EGJwG/b6s45GRr0fCQ7pQV00WbyXcDVnJxrtkRIxQv/FArVsXrj3FJyFmRxH28+jAg1BTVfRq6IiOJ9xcF0ldr8wxoOT1Pt0MxOomI+ZA69wHkXn92TlTkSFyffND7UJ8FkHaN4n02s9nEj4P6VEpTCbJmo1CoYrsCrdzg4LmZlec1mGCcPCkvvgi4iqe316UhBHazjs92FhzoOZDSFmKQfNFY0IXgZvHdG6CeqYXXwr1APSIcxIWal6fbyHMQ6WhpysL5nDIkA/UFghI6BIhF7NAiG3OF/XJC5J9J52TCrg3T6+THxJCQ/kBfWBBiIh5wBflLTcW2fJP6+MXfO7rzFcRqtwOrfkFgsVOUsjMLnGY8y0Flfe7JWZpTzrlmTi5bOCYrXkDOGRW3wCezD0McBeAnIFyhw92oq4xTrvKOgtUo3PMaeScjllnITY6J51GXeMB72HQn9R/pVVo3P+vVlWjAy1d8eUInX0SYdaFEJ6fjDin6DLLzPwslLzpsFMqMMv8fDtSN/U6s4vMs4TOzFlCUebaTeUCy0BL4xwudKNrhxc4Y6hm8AUIOcgOyFkusHyO9ByP9fd0YD3zqsqwvC6EyP15ZRKq9lbzs93IPZ0rEeZ6CFONhnMjU8DIni2indMTS4zVi6HPVSyHjC4JhZo6ixii900uylkd7FLI2S65mmcSibEaVy/6n/3SfmgfUMJOzyihfD5Me2p6fiivOBFlVvPVE6rLoPRe6INjMqDdIoCy4VfQtAroXSiSQh8CJ+4wVibkcqpzwzh0Rn5QEAp1QDoXyr+OT0QC9MJC+gykPKTLq21wXQG/lSAxgUqtS/nu+P8Vl/F3o/L4WaiMeRiVzXk4NNS1g/wYOL3t/3GModBZOsZcEcUUJ4YGuyqGJU/bVvPDrRmOCP9K9iO5gnHoaN+QGsL9D9P5FylzN9ED7Ve6vCPWE1qQXPXnfAnSSVzL2tctDyBmnB0x63fNGRW6MYzXJYKdz4MlbRz+zVz/jWN2C5hLxCwcAbMAl4yAOYCLRsCgg4xrBKwD2DEC1gM8bQRsAHjKCFgAOHMEbAQ4dQRsAtg6Ao4B2DwCNgPMj4AtsCaCI+BYgC+PgOMA/mYEDHJA50fA8QB/NgJOAPivI+BEgE+OgK0AHxkBJwF8aAQ8CuDWEfBogPeOgGG/RbtGwGMA3j4CTgG4fgQ8FmDfCDgVYHkEPA7gF0bA4wFeOQJOA/jnI2AJ4PIRsA3gkfOfDvDI+c8AeOT8ZwI8cv5vozZpBDwB4JHzPxHgkfOfBXBmOVvONqSFvYcA7EfXCHoQOZBsK0Lqd0rKuUy2YdQ/8WpX0dI1P4M+2MrHK+9/sgKx831ImlOJ9GAjrLoQQTPiLhyCfMKBDWRsVzZAcuUnlYNAnQfbwG9EPwysCvs2P0SRdHg9BBFRO42wP+pLkVkBi1tcGLqxDGBzKWJVu5jCtuvc3OprXbYgIROieBsAb0Z5rostT4cxTd3FRsubyQSouUNDKg8ezKyuHeVs9Mo28F07gqiOWQiuFazlgSAJovJ0xJbfDR5HDkkYyht6nPo6DVao/QfU4rzlMPYVVYG8DZDW1whrJ+aV6QOli2EXLyQTwSKguAVETybRPTma/mTbc3lgEUptRjGy4xt5u0RM0dEcImeXwJ6YB7ajVFSCdk42ueIQClWXLsao/IlMTgbJg1/E8RxQiildbG4BHmJKV/g6q2x5fujjVVbtOd5n8edgnpiJBJzu9Oe8/IzPaE9iTdol6r25wEUdioMRc8N2mUoiqAZRVEbngc/tQTIoUnlAPpFKAtKUaHpXNM2KpmOCRJGA77Qepk9fjhibGyFJG+za8Z1S5l+VcrdKgQ8zvTJn6c4cgYjkMKcUqzpKxbgxSCg/RJWAfzXGI5h0sgXGdyGLC8/X19pl4GcWUb020kPSZGlwdiGK3XiMI2maGXb5bDEOfC3r4UTPBlOsZ4No8DF4ceZzp65r13p/2s+7neRejYNr9n6kXSJp0HYNUf3T1dimVT16XOYY2278/bHjBtoTUGutO6sNHjPUtdvjtogA79UGg4dnpNU7LgRnLNt7CGZQXyqu4y9cnomOIC+GVj+aGXeEzsdmhzkVxhN7QZ5MHge4Geoeg3TtoOh9ANI1r62Vu6sWZelA0VnU7BFuMx/nofzX2sPiSoKsKwnsDGhr8TPI50Hl9/84gJ7sGwPXgr44uM7uK4NrdvElpEA6o7/Mg5ylgO/1oEUlHrT4Bx6h9Nca9qBlM6F+eZ8Brkv78zzoJ2O0X3rQinsBXtln8QjLl750ufVYRCMsCOi7iF6b4kHPjvWgVTattLXXd6R4JTIPjoO6O4ZmwGgKIeeUu7VXiEG7DrPEvXQZStK0JOhJL0/WYsNplmaiqaZv1YfrkzXdoBRum0GOa2d9x5eAf+ouQ4Kf0T72CNkxyErQCV5shdkbBz45054C/jLToZMnD5VYASaF/rluMXRakZrnauZhvRg6J+lWhyD9O/gpP5LPwrX0GzPMjfxNctcOmXfEffWGjOD6WkTLWUWWMCJ60KEyh+WrNzyISVBTPYil+2qJ3A1r3FxaX7kKVllM6avmM1B6u4xgns2l4pqTjru13zpmab+h2qc9FJVaKlD/NrKyDVsiGrFekPMLkWWRaojiTKO7nGPcV+/ME8k0R9pX+9qHa9zQ+hMHp/3Gkaj9dsgVLV0OOPuh5iTwuH9hgGf6h/F90NObUWonYVwtAF85Zmos9L/W83K/Xsn0MT1c/7eaudXpyWL0jUX+dZ9e1ThPFn+ZtGsDp+Ki/eyF3l+C3l8Hrt4uRDHb6b7rzwHotcbLtQz0vM+x+qv9H1ii+CcBv0qm5W9A+T7tuTsNf2j1r+jU52XrTaU5Fbs0HeT0pTnrzCQ/2uY8tPlZq95xx1f7PQKftnAAxhGlHwu6NvRYtD8YI6UXliiMi1tFxyWzjru+2kvrNWfj3WFO53zV0kP6OEhfz7s7FHLnrA/hBDObF0DhHUtx9dw1tKM/MQ8J4Timh+23DMdbPWxfbF7AEMbruatPiHKRBVykaBLI/jeYy2Qw4kfBvvGPzuH6Iqg3aaKM2vSaa93SogNoTOt/+qeeucHfBhpp9j/dld55Rea0aXjxrqhdjV3uf6YwdPNCa55H4q2teh+3REScPwdShDiF8ZmVbpDyXjdHID7qP6/FuzlkjuhNnVwI2c7evBbQvgO+TvdhJGrHqJ72cn18RFrB7mBZVZlX8jpJexVT/AQ6GKmHGOwNkPS+jqSePCh5rd+k6GvzqOaDHPcR8OvQfyoshWs7atId/wXz+5uv9oN9qacz2dOLVzhS1MlymiOu/2Vo3wJpo5wenRHseL//Fcfq/p2Onf0Q/8Q+1MsSqd8cHbHiuKN/F4z5WZmLzCfgjvuqpdasMJqJ9kUmEVgPpgqaVwxUx+gM03mvPV5rqJmonAV4P+jVa51b7hz9h9Z24/3A0zDtQpT0s94j/bcTY1RKLbILNPb1znTgcSf8dgxmD+PCrqsvtcUedGUj+/1r/tAKq/B1IqlRLY5Vo5xKEU61MZC/ScdfzIYWUn7qjtTxYV5AgusyfTT/GtgU8EVu9kP6YMRmcDqsS2Y0Vk6D2XCCHF+DfdpcutiwBfu3oy8LAl6vzbHsq5Z+bt4mYlCOk8IBiOhqJgLXru9bCrCJKKaoM/XLiX06x/6vXpMZbTTlbd4KMo3OBug9rEPaG+zBh6Lj+KYwFPyzzAIHKNgetAW8pD1SEycOwC4ZbHGvQJKQCCFHpBR2pGBT0eIQtVMmkNMhgHcEG4PzSXcTytvAX2563l2/FnwxL2/dSRiPwBqaZjT9msLDPlkcrJWgLCNtoYxa8gbdhUh3BlZ/pK5Uu+heTKxRSGnJAzvfNuylxW0AaH9LbCHKSCETWmJp24yslthILxkZEdnFnJZdUOakWC2x1H/LmB5tvwWgUKiO6CN2eG0AdOneFnb7C4OgG8ZH4Td4hS0qIVEbGbdL+89orhN4nuxY9nW7zAS1IBPwOixft3cagPdVj4D/14C0+IFhGYG1Clpp/QgqsOaC3rBfcoLuIl+3U1/Icd95CDFqZ0UsX53ahslhGTseOv/bHlffqMYxmzOPxLWSxsd6qvr52qo2ognt4NG1MZgT/D5e0/uc7hwiaX0wdyS4KLLj+Hh7SQVLVNLz0Th5EW7cYpaxrCcV3kXgz/AtTmIqRAKHdASCj+ES+QlSIa4mukJkuGpfuc0UPBuUQJam0sWxn8ndDgvw092XxHOrBxt41SIjgsNy7ISRjPv6FJ+xeghaBnxn3YvBN+sP+oMSjGGn7JdPgrdpKl1Rp/51HHkqd+qWFTqGpD+EcFIHzzMks41tT5SLIpoUq+ayygpeQhlADylsG6PpfI3aedATFmTnDz454A1Ue8ta8lv0bdQPeoJ0q88EDBEZw6jFCtZ9mCQPVKC7SDKZAhirAoYqG64nZv7vBPHrCVN1zL2fSGuqij9CsxvfDugxSr63na1tlzG+ukVPjGSyvIhU4CWHxooXwBdGhmX+63J6QC8mIfTSgqN6u84qbBTqGrUbZDbF9j0he69UAt6PrnAAZ5ElUF4F8MPQ98dbmBq9ol6tJsaW2eRB+QmZxH9ArFDrQuCrNfCdl7VDF9mAIcxrOolpBW9u8EkSM8C0Pg568hSNKzQm+EWQa0ujMoHxV7W4WnSwk5wP7ysG2I9SHePOIy3NFRfaCbMpBHjZBl6JuaWyJVculwlQs6A0YvGIHUaPeCEx2sqh0RVA73bxmhmo/ilqQ4Kw90bKxda0iwxwER+s3J4n42QJMUWI3l0Brf3F16cL0W33yFlY5H8waM0tIyvCfpDTC/YDxhIgaoCR+wAnA4upo+SzeAUxezYwscUbkdGzgXpIaKM33ztR01dh72SN806swrIUAHq3JRSGrj/c9FrTcmzDArYZ+KZnsE3HN60YWt90umHV0Af7TjcdaLocqY3jm66GYrDNwjdd2ywN7Ww6TetRBopHdyNxGAfyCVjcxVI8yCeG8XrDeEkAW0fgjYrijKY4ofH7RcAxAiyMwDFFcWIoDn56ewJ+NpXf/O1+2t8YKE8egZsSxR07tFMu+zecjeAKvyjz4osIPJpvNbDGs8ByXQdbcfrWbJeAdL7UnpIxeVn++5ZclE3iESYizlEFnLOJh3yC24Yycc6LNJ849OX+UsBJgrx1BM6oaP1oqN8C9WMgnzyiPgWJZCzFGXp8f4En68z7EN2EhoZaWLjeaOHDXmokiqSRbExp/TpUtJKMawVrmDHXYwiNany7eaanjIn1lLFmWToFFifjTvCPedgNeLBXLJngnwllv/IFtPPBw0HMb0As6fZsYbJawUJmCl7caohoWSYrKxiJowankSW52fwKFE/GIYakIhsZj0XgWKzkAU7DYg2PZ4uPDOVjl2JuUkKj9wv8JsKF+6R8ff7PPjPO/I8+bdCHBH2mj+gz41/1CZZIpdYfZPAbquXQ22soEQKeDGSOtF3LQz4WZjSVtod83NCjcgGWhFGyEzPIDCXjdo/Up/HQPo1qAWi4e+jJodhoXOEdXk9DBVFKL0BbG7SQhlvLWQClY6R7BHEoI9zHGvls0wa8otk89Od3JwUYeg8hY+l3V0zTvRHMJhFbm804UU7AKDVhKK4B7RObcmDlIcBE4ZWHwisvrgmHPgANV0EnkUxj0lZY7TZylnJ383qUuzvAHmTzMErgaRy2gTwSiRn4SkWz0PiIdoFcORgr1azjTWcgLw09Btg2wEgfgZERxUiGciPOJix+WmGHjkKNgHOxgB6GNZjLUcwYnJvEhwxN0tDHGAkJQ3VDa47HNd3TtPLWGP4jPIbnImPYr0IbI8hbGK6HvKlhA8WBXMxQ3D5yWPx37XG2L2GfCr5OeP5DXrBxNkjvhJlKpyvk5i8cD319WkaDraQ8N9tK9WoM6FTyLb2wkZSwTtjI2KFHmwrsaUTwgYek/4qurJvFsiqfbeFhT3oBodWBkkRkgCgiFnbcWHpnKN3bkosl8kKJiAwwc4myeSh/n9vX7lma/WX03k4gsiphNS4mo/x3w0qM97/WEdviwiJ5AYmhb0uyoU/RF9tQ1M61TuhK953WvoY9/LGvTxOype9f8etGYE1u8bxf8Ahn/hF82XHH+emyP+K1rNv7kQnH1/sbjUUY89vNXTtg174mo64deW7gZl6dHXaUGPC1t3wooHSSXCRBtD4PrMB1ikN04Wg6FDrjZ4LE70UiEsAOxssJ+9xRSByK7RyK3Mmhe4v/aaoHTcrQ6H2Cn669MIY6BBopReQQa4/IoW4LyMHoX9nOEX2XLeybfBv0t0oIEStI9WaT0MB27QjDAthWVoQ0CltLECrzCGVBoIu9UIKSvU6gXtV6gnQD/WyVJ/TZwdd8JtHJSIQSHJ91r/9QBzuI/Tf8M4Z2NmzwCIyu1IYETUftHuSSSxlEANfcIOzLxshvbEAbs/dNa3hKZsVEJIlgQ6BW9N8HMa4AVETgSqIlsh6uYHsoHbA/+F+McxTgJwNH0/z5J8z+N8CiMh0CjDtd1YXvJIZCL4Ul8DW9bwdQA70f2arC2KB/crbhTtVCe2iYEPbS6V7jQ1YCPV7dT4ybv4UWz+4PBFU6Z51mx2NB7tMYmBHGMS7IFiJ+COwioXVQa3BYvvxbGwPXT7t20LIgI/PkAXwXAbqDl1Vja1qr5Bkd0vt4lT6PmxL09mb3T4DZy4BI3BYXwjZ+oX25ia3LhBlH4+8sWo6tZNhf6ZRhV+Q/US23So6ES06rnJwW8RTlNKCZK3fTeyvXKoHzj2Bdfthzvsigjm5a4b/os/kWuTNDwYU5hPF5W1icyBMlT3yACI45X38IUfuFgFdjr8H6DnYEnRFZW1B0pxOLyokod7fmEUOLE6RroFFS6F2V853Vrvo6tYEgeJOkrM1J2Fanj6nf3Obcnbz53qHdbc4sG2GKbwtdpTrW1NlU40tvkzBj4bt2IBR8knTnodUhN4pTA3IeQjchFxqo8Ka3rb5cGahqTTsnFV9ktlaVtT4g8ITpmVR8ianHotlYosKabtm7y6MaLnfqZGbf9ALEVHmQ0Cernk7GCN4nvVf25lVjjaScCLBVyO1HugDbxJ9DfaPc4ImG63dfMRbNV/U+RjF1GjQTeKhoHVODtW88B+MSm1adEKEvemdrvkcN3+F6hJyg91GLdbA9IeZBwY+YwJqo5py0o2xUoiHDgdUYTQMLht9IHlvMMve0GIBS5QCHCbgZOtHYEAdewTg1GdbHxHDds161xcC/QqhMlw1wCuM8ilBLQYDWPTJgUICGligzAUWLDeM/BviR2vwBwxoSrueq1Mq6gCHgrXIGhB5T8SX0jwCUL7xK9DLaN734IurKC+jZ0mtel91wgMEBB+qKQweRmZkK/7XI3FIYoPTHDxgqjAoDfuka0kgY+rir/gPYoe6Zn4BWb073NYKHIJDzxbNDl2tu85zRjdltPznKHVjdjvOJvs6meE/GXeR75vSb3YEyRMuUzOJEtHNA13MX8PT7y0b/83XeXkXJAo7qoxydsee/yOCLjvAzE7B+oxgRpTAi/Ufj9wELaPVeQXsBYqNftS1qNmrPRXgd6hwwkD1RXr3A61TC1M+Zz5DVDZ+CN3OPzwu75vnie0JXGo4c5oZ+OMTVZAL963XejUwv+EKG88rSDbY+d4SLioX2/A2IctEVJ5h9os/ns+5L1px7k7UH2jjtvraM5jHaDHx0F+P4x42P3D9AnAvi/SU5hKuN7eH8z/fjJfmIi9ydMIsKT2bZkd9EJl3yFiK2/xKDbQeR464bH3nOsPeAj8bQmRxaVWO0H0Em7XbwWQqb1ABLlFaJGOxiHUt+BvVPe6Vzt/cZQAv1NbHrnAWhUCIxzelApqbn2/UDwjlnH0d1uS3V+xP85JYU74SC0NBTxKjpW9M0npi8t2tfBb/YiIK5kfuxFpXe6fCm95zomypLEA8z5yb0344lsxHH87tKYI/ySIaraiaso0BkH4OVPx8ZWnVF4Jv5WrRBuRv82C+IvmUWcCco6iXuuFFZEmD8vyJ5jdOhLPkSpxzWTkfj144gyhMkOqcCUT3JMcMe/enC0KUh2QZ1jCiax4GGTqU7Wf3fS3MIn7dUQnkrRdRwRvaD9Xb7vs0rkNBHMVvv3Ne99a7pyIp+Z0tkwVQ3apd3xDcgkUXlsEuwqAzNQ2Wbl75/s92cx4vIJ3wkbOmr/2TXGXt2vQDY1yGeBu/Sm0vOXfaSw9vu7/0QQ4yrGPsNg1heVAdchf5Lu0LyLtGV8HawvWuHwxIUVKOfVQ1B1CXRexhBgdp8moKdh52UMNT+Ax6v3k7vdCS/CbuDDiJ8NTJWxuxYdv59GG810dOaPAn8B1vcZcDcBzDfHjeMBzgraT8Rz4GYAWNH5GkOcPMnGd1v2MSHKfPw091vCKQCNtgugGgcisJ1OloP+K/KDMUPkjsnbuLfxXQ/p2Mh7Z3jo/6SGtWJ7qJnUVLjr2lfp2Nb8yGiBx8JbJPOsTooaIZOntY0z4So5ZtwDxP8XmLsSi/mkeU7FH8eoajpty1v5xXqJxyhtAkbVAkblpCuk0oAfD1eD7Jjwt5NO+DNcSwLUk5hRJ268Gg9dNyyBPrBJYj8GvqO1ap4wC/zOkkP9Ygu5nnLsG7Kox0cvk1henpwxqGx3nS695PJsJflKCdqCmv7HS8FTcWbUPagCGtwIfBpUjPykiXOLdYg2ZkgkVkJG4iBOHedOhyib065xfWhodg2VmYTGPA80N5EwAY9BsgEP/Pe0SJS4jUDpULvyKKb2o/WL8LcFJb6G9QvHghuuReLU9h2PgxdyTsI1A7UoLzD9NEQUDgWF/L+53F8zuA93se5ukMhVkc4nJTN4g0oEWRkch7ZICipzU6cmM0254K8jHtxQjdiBiECGXgfYBMxKY5dheto/0c9W1iuitacxDxJTNATDvowkjuXwJKD8Rrp+xYDbex8whaVhIZq/09NsXJKaw92035L0Abww5nbg7a8VcDlihr06juUg82vgTdOZNcScQva6y0VFdTi3Yubvc0Vg5IP5mug1rOU5WSYhYEXG3L2znz1V5+HKFWgSXXRGGwcCYXvndwRNHXcT9pn3G1G7nhknpHtk2Yk8si3cEY8L+VlSdyMbFpjLKc1210zRBm5c4xFM+JlyQ22A/LTaBmFfWa3zZg6A5zTGQyiUdl2OpamaUvENUfAgvGl4gamnht6tAQZ1KG4MD8kusIyXSI6AuvPUGpjphSiMUfAOt0NWzMJPwnBPvateqCGh04dejFBQhOb2wkzmA2jTAgSBRP/R3xyAmG0m9S3C3gVZ1E8MmRlEyY5GzHaIPhNXiJtU2jJezepxhO/gj8S6K4kgM+4LVkdpPebu3YIbsQIEmE+0tEnTnQdvH+NRgTUg5P9H8WQRVmYMHZsFaQcgg4bwXJdPapX/LIkJoRC1NoEMfXy1HuXxIMX23Miga61Xq7fTJ/5eoRsI6Qws6t0vo7Bbo37gJW/ItUyIkeypiLmcGrDFnf86tO9k/t4d07otBYL0M7eGWFoj2Zw21b/V6+nzwjQiZ6n+nS52akrCKf1FG9EXNsJ6kfLZSDPmFIUK7lErg/kyZbajNsc99PntejR8B1ecyqidWCbYlPRMMZwK5m+JzWLrvBorCRFYyX7h3f590CkxPsOajdaZ/t/gWDN+adAfM2/pxdzkOD/6YkU8Bsh13DqcEzrE9ve68mojWueIjPN/xBhxUKPd4WQ4oTdTF/HKEzzlOY/Uv3z307fegAPFEXezvQ7T4SoNQEpPuGd5bW1doMEL/lzZC9mUfh9gAubXUzDRFc21ydj6xuItb6CuFye/KQQJYwjEAOiEPG6betD75t6nP2CXCbj5tbmKzLXNlGLxVMF9r3YCCd1DOVFkZpXNh+gkQwRtFF5EoO2pmz0tq1pXfN7cw9Emg5G5ppX7rmH1vunqyGfU5m10dmWH+z2pS0REb0P2gFSHAPRVKgWA0WdIjnSzndEIw0EkQbl+T5YSbo85sK77ng9ciPjwWYj0GT0sBL1Zjov0XUwrTAUeEBDQdy4xuvkE5HFm95u4Tlk6hxTWf22UbZhUTG/VXvQd3CtJkK9sZLIZ98RDppo/E3yuzKsHLJ09hGDNhPwTQct59L7YyqrcZJAvbIVMnrLdND31lptCrQ1//7Dnn688lCKwpB8nJTJrJ0IOItAm2Irqw9aaAl9qj1Uat2EzAdNlGJl9Z8gp+z+k+8MR0v5DBTTg/t4pce6AsXIKKiqk/z57RNB5uubq5ormqv9tR18Q71VQmaXRFDzOrdUlW2VyFj4WeGXKDtlZJWqA/LjbUyzsTnGCjEv/GLhFwflAyDH8V07VDP0agJp4krv27Vvr33bB1pxEnb78ZF4WJ3cePx4FmkvtiJdqLYKa0bs32D6cYBkrwM/bPPOY/T9mm97hb5RfCKxtMfwDxPjx0JlzdvGg6audOCAIxbrK4S1QnwejqNPgD0YH1Q77pdssG6zcmZ67Pbp6vQghpFV8xIS3hUqSV62ni/Nqe5+R3m7FsqM7xpAEyCuNhTAeGFm3hHIWaLKtoMmlzT2ZFeGV5JTQbopVRhkZyATrbuIJfzWnZ90qxY6vwfBK+1KB402aDo5JTJ2YgiqCBWzhoeC5Xw3MhT3GB6EaJor3mjI00GN33uxcs5hxBdnGO5vLfZkxfByUVtx4xGvX0YDFfybKJZfg+K8+Ve5gLc1tjet+KJhcsSvXtdqN3iH4437mVzmbTaVLWM7uTyug3fwr/M9uhRdpu6YPlUv6Zfr3zb83FhkdJt+azoVUxvzkbnUvNySYCm1lMTm8gbEdppls+YqfkJfK4xHTF07Fv3mmkXrhC/F/qslImLbx7rEtWlFLWoy2DuJ4EJken/wLjGNpLrSQmshHWc1IH1LvpwLNSrNy4xWIEvW0YjnRyMB9u3HQWPMDbxqwjq/0MBgdEfq4F3gs9zbboqunyKrnugHLW2x2toSHYnzCML9m8qhtZFCPuEED/0aNWOP1GdYz0fbEK038owC6nTDvBm3Uw6gN3OJrgJ1cnIMaISxCrXFepeCr/8YjHOUEBseJ5LMNWXRcepIyPom4sFLN0R7L9fedKFuiC4Jq+qjZfXaH0FigsrnJvlj2iRKQY1zJvmNfFQC4n+R1HiQCEhiZlgSbISyZip+Qvfpd+kuvIyk/ivh/jhrC2Igx5cgFqmjQUpGMY6MwzqVpuOdSd3DFFxRCmlAwRSmwI2gsOd/jizcv6EkiYSaczFw2jzrFp1/w7nx83C/fKTfMLbIo/aJX4p9MUDjue9qiXaFzjHITmyHURBduOdR/6bn12jPVWWaIcyVrtmlcd4yb4ysa4uNcMUj9Qf8RCT25Hkr8GFHSnEHup3PR8KReH+mGud/clutfwnOE1BPpsL2cdtqIXIqIo7C0I2eW6MK0yUOUgT0nqf0ABZrT+LsbMZ5KhtZ70UiLSnmdAlgPwz0ntgwdQGpOTBCPVko++mofI1CDmLaYxvjjphpG14kAn17bLsTdFu0T0XCYfbzb25RtpDZEdqXqq7IQH0GpX5Jd4WvPeo8ZUWHDQWIPxNeEWxzXJS2YZi2iNSpxU/wnd+jHcdjYrxF3fQ/qH/976gLN75HHWJ+fmuYOgc22igLUcoQ5LWPicwtX/P9uWVHI0tRC4nGmEwLxMJ3VCHQCtqPkIsyqd2PkRm4modXoPDn4RVYgNhu6PXH/0rvMei0C4UQlu4Yj8X94XWmpDtFL6oqk01Jo5HeG5M0ipjDXHKdcWDnGwi350jSMYL2eEHfYq1gHWUWRrVrmLJhxzBlOaYtNsDW6XrZ4kv8eOiBURNLeGTAB4hliYwMg6aC0M33gDfxeyuGC9sCBifJRuiLKUGEm5u5+qYHsTYP4mz02ZNs0TLcE1FRRCZcK8Q2xuaEKDStOV7WNScOcpS+R9CLmxZ7EwoQ99kg0oaleGTY1uEWxQi1oZsJI3mj8qGzwXV8lzftMsgrhOeTRIzuTDs9bKfOQPvFciylV4WsjyLRuxTo8kDzZTkOpAf6XjyVTe15Ct99KKXnML7bMbZ4KlcP+6WDzLokF4YGXZf48Bo57DxlRrXnajK2HaarpCAUPAZcrP6ehGJBQreRNKyrN+KkZqPbxoBX1/C4+1600Mf4MhcOMGnNW5tT2jI2/XJ9bB/Xm9Y/zOm3wGlsWO9TBhngbCIaDXRTeti+tC+58IgLv69/YV1PidyfXdtqR14GLJnBjZDdPpoxblq8ntP+Lku0l97EvrjtXutEwA5HIzGtdib85mcYU7vgNuiygfJY+CXKBrgmuUej5TIanAWc/9ydj0rcj7KBQsRtDnjVKMdsKsjgQIAJc8yGOU4GazSmul2pALmm9JQXX0QvwlpI0kGveCxvrrPh+TKLS624adKrPzmH6KjYz783jxd6W/rMdLbXou1j3DnETvu3P2oxah97SsVxL+We04XbdXxfGpE2hx8AGXBuhtxrf5RhfBmbnljPalM97nj+pXzN9tKsaOtXvt/amUQsJcnE4MvAB33GeWj1TS0J8NZ/F++crv9qFHO173Z8cDvFvA6xvV9Z1FPZF92vWfut/Vqss3nSQjzsravoPGxvpzIH/YERgdwvtsZ6hHFQe8PPGhh757BkS0CyAp2Tuc+hn899ig3cmhu67pLIc6RNZugM0ZlJWk10ScsIXwjOSsCrg5wOSm7N0gvQew/Motu/0J/WxrlFYpf1zSn+l2AmRvMSeQ76KCVt0HrPiLndAq2O31qnsF5Y0yDfFtt7rH/ywktetn8itTfW/yA8tTjWJYQBCxS1OvR+CPe31lgeMHjA0AEGDxjDPpsHxdART5TTrAb9AOTAU6VaQiGZwSJv5h9FiaAr9wCHU324aWtT8jmWYwhDZ4459j19+Qd5oiKgTgBfLHAiY6sIPg/eYK7z4qQNLEQVFuVenCDjphNNAgcx8ZaHz0V6eeV7vfTjA4plLWudqB/gR+sH8AGfZS2nLLLeqx9IgtXfyQ8agVMnUJHaDRHO5RgosUMJbh+WGuwAN6Y0QC1KGKS79miU0C5FKf70exSvyPTtRTPdzyFmo/adpXseb6DUZctaRgEfEbiBMqB2cVAfXmGjoJUkUAlZ65z4HiUioQUgIYZKiCwK05r2PVrflEgoUBF4d8JW4ZZ8bgP52MwW5VGcruCmuU0ixyBmy9ZzbLiHxO/Lh58YkYoyay0XltLEW3ymUz6/I4eTBaHB5qgcDCC/+O/KAp3/t7KIHyGL+O/LYhB25eJn0FEYfTzWpTI9uj4J+o3ZlgWerwnmLXCLg/PAwVOAZ4Z6S4+/j6/VAWSq9SuNUCL2SBADp2g6XFLPhOGzVM448dDYWmrPxNqzxLhlD72TQu+1fBqktTW2Yu76yUHQ3cH7gIsKWibyBA3ydWfbYtfZCkLX/0QjBvWh8B5lOrb8+6Mkd/gt+LZURmGbLeSX/jRI0zoSqZ+BrQqjUl9MX4j4P2q6SI47OrxjU5zOuLBHML/biEXwCubtR0ojlRWe50Xg6f0nbXPaHN1p+WEL5M0Hvp4GryTfu3QwJuwvsGoq7U8dE+Uz9ft8Qq2hqAVFd2QO7Pz1durXDIK/oXHyOHqnPeD9rs8BbUzDFqAzpugVYoWobFyf2JvSz4MNgdmFqHASHX1h6NpNlR+EPVmbR+7g1xPOb2w8uRb5TfhZkI3QbPJbSLFiaLb4R8N1tD9BTT5O90KeH0U4VaDjrJ1I74byyWFfOQpxdw5Lq7LuIPNWnMrRM4StxfR5DY0fXSIiiuSREiEa8mTF8i7EIIUB/JjiSsQdp1wKvDnAdBrlsZpAY1R5vMbLqRobYILnN4LfDtG6lUbEHfFBBP6mhajtPI8QxNqRKNf/HI26m++NRvBLZZX02ldvNdK3B8SbpN0pKlLRUmKK4RBjX93O+M7C1ahdO3Ck+i77sqPMKf36xjmJoWABYn5w7vU+tmKu3D4nEaW0ci43wttdbRmyU5b2433evc5W7iW57XAkLzNt9FlhMtBiPuB97UVLQ0H5bIVB49d5f4RE0/QH2pnfXa+9VLwydB3w9McNxSzD/UikTxpRyL2TSHJ3wkrCVDP2le2MfX8745ymSspX67wJzxPmR0LcVbm76GkkTkcvMXNYZJo+rZ1x7yGMfVe70f7M+0bfWW1AtYD3F1/193y33rDAs/azfAzpg+ZUTax4taLK1Y0StRhXN4Mi9bWeSL1pW89H5CJ9k7macQqp5gPcXGV1yCMwwsJvufn0XiU9HQb+pUnGFU+u2Ul2e9OrvGsc2qiF37JztcCcdPSge8/qEL5NYJV50+/ayqxvd05CiZAzQt39Pb39uSbgFL+RyZwtnv8sCVX5+2PxtSxDnzB98h7G/odJTMknKKTxVdh1GpU5l1jNTlumBbdNMdd8vPa93CvnM52ZBbjyY+eFKYz4CULxzyMmYQ9iaj6Ycy50bW7G6pBzQoEZ7xHY4pdDb29q126QedohmOVc9icE2cu2MtqfcWMZW7TvgT/hTbtS8MOZDJZLsXOC1Yz/vkvCr+9C557u09vbfsesfW/OBGSiNTUbQQZGKgPN6Gt0TuLBhnsEVk9HXXwx9B8HDuMDgmBfMFk4cLiytuY9Ebg6cLjmGeCVebEj8TbCaFUdJpwp0BMueJ8Y/zRiEm2IaeWcCOM3K9vo+56wj2m6ORlDpyBuOR3sCy7SiQh8DsDyV77onhS6cWhUXaP/SJXTOVVBvEKS/JX8FrBFfcSiKyAmPFs0b1tQ53TnDIVwlQR7ivMsj5w2M3or9p043SoSoyslibqlxLqeuex1/zdISj6NnJszUe+yPsvuSbv/gJ8xm91OIuBZollZo2T6lvtextjP4gc3YIZFzG7/q2exqFpql5/i938ImvBuySzw13MZw+adZM2x+J4lfYbdk3yNONdshtZL8exk4CIUq6zB4hZL7ZIwnT27v3JyQGcR0Fm8xaycViZQOlcZvHwLi3+dincfRG8Q5tXr8znYAcS9ltqfgY6/vXlnzxLv4r5EhiXMuqVOwWw+t0TJ7ud8jZoRyljg9OAUs3K7+zUi4nuA7qKhU05mO8JPr2J6ApQ+vmcL0/cfeFUqg29s18P+tLQ6cMyQB3pfmgv7E0J+8n7xJmTFP4Wd7Lzyfl+e+xJiate4X0MMOW48TsAmjzlKmFM3tB+cm9iXgQ+lMrX9PedrftHPkfe0u2rXrFs0tx0x3i8FwHJfC4WSTyGmf/BcppLfJ9XWkFMvfSi8AWYBRlf081BQKdQScfn2CC3x39FS8vEizPRW9sfsHtj/4e6L+z/yHfbN2h3Yf7quEnawniMeBqxXPPgKHszyPqc2b/dUX+7+GVoxbTfcpq5y96XeyrkM4txvIG7/X/p58h5xawOkh7iPTG9q3XeItt5+SLt79z3b/7T/Pu0ukD6ew6EE/CTIsj1UR78N4IzBGC1FCB1EDLsKMeBF3xWV38k9+sF1bS9qyaAR+KQw3GrrvaDLPZv3yWtPXYxitmhf97zRx/fMK740dBQ7fUacs2FR7S7ZS7Wmzju05KVJbQQvqGdeiseTBKbyRd+rNGrbw+1xNstv1b2zPng2WIaXY2NDuyeLSWoSlDLYhwM9XB/vSkNxyqItj/cwkI+DvISZampT35OZ4gTU2evu513doQHticizG/pURilTzmmDt3qgrcpcODSAZ20YjZkK2npvk4DKiBl7CWoyIjexOGERN5lqrPhTSGNqLBh23CZzjRm3AWxBmMTiV6EktmYp/h2UxNX80DmN0G8WCDWTnZPCOWNNv3MMPb1JTDV/cQrhspia1xWx5kVFVUSlCWWTOMhVKU1KOVwrfdnK+aLRoUOYOfgc5g5eWziQ1NKvd9t+6nYHftei7SeNHfO3xSu6XrEf71aaTFvvbjoBbcq3vqKUwaZyaeu9SmU4dzmaXommV6PptWj6bTS9Hk0Ho+mNaBqMpjej6VA0DdF06zntz0sYkKG/Yacy+yO9wEYlfAMkPIDjCfBOrmFmBx/sDkrFCUylKxt8KFxmxHO943Euh4aOKVU414JCGXnZoC85vs86RHpfQ7XQ2cIoMc2NeGRnKliM47k6pzaw8yQ5PpdZHRzGx0IyozALA0gqvoRC/hx3JVJCus1lzvQyo3uejOZOMqDp8Q0sZrHRPvdddqOyWcKl9VwDesPvztUhu3Mj60b6CI15R7jQk6HZm7Oc6d1Gdy4fbb0RWqtGuxNaN29eigs20NaN7nlVyD634Z+tc11c6I7/R9ov/69ov/K/or3rf0X71X9Hm1KK0G9AwxwM41BONiqUD/p+XoQTysWBJsrHMH1KrSHcZnpOBUt5acja2Ez7mSvyaPNSyomdOcYOcxDShe7AuUUcWJ2hJTkoTTlXAteXDs21ET4PPJ3SeaAfRtAPQy/fL7389NCTGN2iMBUoiEBB2Jg1N3kNmgt61lBgzz3G2tkO1i0ChYRqSsFCKWhdDetLckhqR6KvXDkXuefj+0zgRugoqmAhJgQtffdKKAMnlBldEui6uAnhnPjxSkZoFEYv0kiBCfrBsxdLkfmyLO3DLvHx5mYnfao3XJqLlcy8Mv2o0sXmXbuxv/19rqe8L8G3VOF69P30RJIRsD5zZT/eIisu99iThaEeN/gTibJtH96+mH5vxi0Rc8+sPiOkQt1SBdcu2u9u/rDZ6RPCTwUDQOea77IbVn9PXL8B4FEA74K8EOndkgl9TtDS/O0yFqXQt24bESBKdZ/Q+ecqGb6sZmeeBH3YzJf9c31ZSjrEfxxxaoNhDGd9IcWgb17KqmsDm5y3HPpf4VNzEzMzt7y5Jc4tkjIroy9QJrokAfmOFIYC93iWmi/KnCM2pZC+ed6WNuAtRKYr4IUZWtMGmEIk/lYzeAROH4H0Cj1l1MsTtS/KcVw9PfEePtdhLBXXFUTH4S4Mne2QuWO6ug4ZYZRM35ux7TxKDIWhr24jqjrREXf+3cjJnLi/FiLz5nDNFIiuJkZ7eOEiRExn/fTEdVEZjomW7oKyBhkBv8X0WdtXE2R6hqUxiPy0/Q/pu02Nx0n7MQMx1LVDyYzNr2mDQX8QtzbyfmQ4mvhqLTmrHMWbBFRUTmKICXCykmFn3/Urn23vr7Sr0B9LdPQZXRAPJoIs+olarEOjClHMXfQ96RiHGhflpQ94ccvtru2rQ3l+KHk1rl6zgYzeJKp2leIaUO8RdWJfqj0phY20oadJz/6g90jf6BFlLiibEn5yWEJUz1LmPK3TxGGJaudu5fr6L31XzrHLoW0s9JfpsJz/GPSJL71Wc8g+egvCgUOoK+5+3jrelbo6NBSbixwCyPzjoIpRJpuLsKDpac4VTxB9e+P8x3KfFh1ZLKHnr7p7aJ9yD8Wi9XREenf4Xh3F2QsYH8KsIcuL2stB5EhNKQTdHAW6qfZmhE9cUqxOiD6fgnJjqS32M010Z1cX2J2x7BHLiPql0fpvCpGw32ezM7Emx3jQR1gxfpif0F/C7xK3k7O7HqRziDfxdO4EYspKpCft+9/Xrvtn0DfKwqcnkOPnwIkInIg+9ZQxqqU8cPsrotdu6z3yHd5+FKEdJ2qJO3Lci/X5dCXY0Ta2cySHP45iScDhj2Vsl7aZKE/0ZD/lCGgfUxPcZagMokm27YSVJ6zSW7SY8lgY6vuW6pd2PcydtHBAYEhPv5mkRznLpGsRuLsdLElM+C0tv2pwi6iseWrbhD9epG8cgy7Ss3z0nXgMKybXsvcvqfV/3FqPRJTc+FpRmWolKszVtTqml++bFNGCtUfsBoLo02D6rqf5zql28OC/ABpSZNVZ9irYP1pNiOjTelhX/+ennTGOX6YUa0L7TU3vSWZ02nWPyH7rWJhS/DlP9bNvGX0PO2wvKeegAX8/DSti04iyg4XIuBrWiwXKP1ONIzDfgZLfwu9r0JgDhYiTonUnocWSYrORROEzAP8Y4FVB4ihKmXMkOYjkhfIu1yII6GzWR3xOnCvgVhfsN9x75u6OfvOOdpUjnHaxzbk9U9MTnr6rAWuYo1yBA1kPa3h/57C1Og+c/IfGFSLdj6CevxP9pAD0ZW74/i9IxdzZihSVpNG3kc4/C7bH3cCrRvJApJa+BfX3RzVjFCogD5A8KCkO48UO94GWQpkrKAc5mJXxKW51HC8iQWH8s+gOoUg+O5TOAb3l/kkTVs+cWrVWkhlaB7Mw937gDCQ+E/ThYcqlKn4He64iAeZc2k5hQGfnyLcTpyvb/5MoD37ggb45/1fHuPOnG/hc8aQRixXljrTzp6Hk41yx04hRhSu87mNaZt9v2FQCv/ktPEi/JgBXc3MAdgLzW/B7JVABPe3v1EfObLL0BH04R2VrXBaoHGAusmTJZe6KLkq9szD0+WFo+akHcWbAqQt4w6dXTZG3Lz8/4JiSMvddIVoCe97nezWRZ0BK2P9wtNQella6/yF/A9RvBxnPUXXA71/gdwrmdDCKVwTr893Iu57MQrBQ8k1y00ZYcht5mrg7lzqzrPo56WjTbCkUck4K56/loTj6JnzIKVn1N7oHBWzsTKnAmCWW/Gy9acF070q7kSAl4WEJSYAfg7MKzFggFpzQaR5ag+2dZqeoGp1O12anRBKdjGqcNaMgk54dwvMw72RP6okVYukYpRGny/o65l0dTj+pL85g/LVvONkjmxs+obizsgoyh77Ezk49xIZmZRGOB1znCQPO6dSvr8BZ5rEbET0nNLSmGhGRpHftANiO9MV3IwW8+RDOIfriqWhNXnZc5AsDDNHfUAeX4ukoxc3A2p9qtuSDXBZMYlbap5uRwj2cDeO5G63B6QLlZjZOlvWPSPTbVVrZRvQ+T59CWO8NP09EvF796bCcMOb1g36QE7shhRCMiAV8PFPpPJATC3IiD08N93u41hmWC8ikmtmInHPznsNzQTYPdmY6p6sCLqzn8Ty3niQdNSlenM5HJeOmvFhq651TXW+CZOapwqxJnZlDF/BckMzDR2NuSUYflkwPnuQbG5ELSScirHY8LB9ZeCydQTi7IIWAZxrybkSPpXtR9dLI/EKJuBB7kSzIfhjJWEK03+MEiDJzi8zOB3neuRRZ6FfsnL9/5Fp4trOKYLadtSgxt6g70/lrVXDasBGP7eQx2qAnz+N5J82qWZmG0Wl93aJOAy1VebcDIrVpvzc7nygZ3fDirNvLMsO8jsZSgZ78oENQrFhy6+tsxemoBuYzRxWGHndi8I37cbpqzPWYMzcbK14dOj9rnrACO1Wj8js8z6qvTcidLGQe1bnmIqTMhnbzsFGZjd2C/v3Hihg0jzirEYzvt1g8aXaWihbshnn4af1zWCgyQ/RgCRnorITPwKVj46wY0FZG4bGE9RUN2H7S3BGrMFji9XN/uDr0rkDLO3SQH1LoPIqYcdrrR2+25cK8hEczEWNBX5F4xAKtQDfWe08KtOSIbq4xFGnzIBa2VTgFN19xG8QIQarvDZ84gZJS7pzu08+ZGmreURPmOVOO8C5RCRWnh1a5xwJdK5Ss2IjcJujPCz1cpHkob4T84xvR8Fksc/SMkm8D7LaGVr2v84fdiAt7fNLNbCjTvTy1S9KCALNyFlFbwIc1kM6o5WFXFoY+W6ZNPmYetk6sP/y+KyKSzGtjomXNgLXwIpMP0AKBVYk4E2GepM9EamYT2ac0effVNLH7TLIhgZ53pxTygxWwazHtUZvNdsIuayoV5SkRO8UXQY93y/T96zbYhVjYa/X0EJgHPZAQbfHZ91pMgxYpsqpNbGHkDKAwnqjfGYVZo8/DTQGmVKwINBXsw5H2FQOQd0bzFyGfK3PDXOovh33OZ4HXdPpm244k2PMe7e0JePt14fdb0X3x3+PCEaV4uTD0t79oib52ej5BxuBD/6ItLcDRMaim6JcbrNSH/JsKPLNye5gSPhbXGt1JOdjb/vY6PVty/vkAQ0+mn+8Nvhxk5PaL3n8rNfCT/7ZFVsN9pRzTBRgZ/DETPR+5SFa90oA3TwW8Y7zZXj2WxW+YBSLALvvNgULSTSaDL5ZPoc5k1krYouVEsOtSUQlCNz3CfXERiry5poy/ByJLqKmdeJHpytBi6TMM/pMo1+Wtk4CHR+jMDX0SPBt0ke46XWHo3KGA1xXfgaJYMuC4AgwtgXH9qkUnTwZpzw5/gyI8X9r4S9HTCVzrTMbFE24mcmU2ufdh2QizY27RAf5tLSzsbJ2Qy6qSOidE+1ah71HBIy2sbEvYGfbIDU3ZHmS67kGWa7QUPJshFiKVLVMRok8ULAmerM5R0dbfFIY+vRZgZckl0T26/4dXWe1v0DqwcAB19p/Vvq5zNrwoowRoTybaOUGgJ0dD87Qvw/5dDP2OWeQNYl8LSMAdPjMRiRlDnzoiESLigt4nOcTmgy5AyrW2N57gCWH5DYTj/YTnW4iOhznsQjDb+hYn1Z6/j/mnNvMQOX76qpZwlYnO/gtEL4O29MepwxgKYNRTPQh/8SIU6lLpR25phJ3rhd5CGsSX99IvMzSbfSa5WxARQ8/nn+B9Zz2ruEGUSL/j1hzbwHdyH0zQMjzLufSZXBGaGXdS+sCA0gh4SrsSREjDsSil2KKEgCY4M8REHsAbMhMGMdGLm6h3//kXWoxmgqhRgH6GhvvRztETOeETpXUqfTYvbaG8SEI2YoY+fo/32TzJ3I0dMfQbDEGSF/NWk2NhKuNGZv8Zcx4jLAaIdSPdhTM6hzmV35gWff84nXS3zGqhft2b5LjM/SEnAHnDX6h3RaUENjEZbOJJ8Bs30q/2RCV4BmabQJkCONIgSKhvDqVIqbXpKTVKqTlHZpqnWheTjMhMQ+XsPcfAbxSaZ8PKxP+nm75NsFsYPse/O8YH3gvCoSF6mvo1wT5VMPlcnix8gZS7ylA2eYW3kdjWL/hckOdcV4yyBj9dFONfEPoNTq95XHFhd2cMxttjQnfgBH61r1xJr2u3xpO4U+fdCLlnooWIPDAz7jMpbwOMob5KaFHsK7exdlsKix9MjlHy8a+mxIS+wA+W06vnNIt/6E4I3aH0bDmEcxYiPMnFYRtmcaGLxXNFDhcI4MVNixl6FReUwlWZ7WvHz/uQMqviasUznjJsrJhboVVkV7RWsBVrtHfbHk8qI4Y9J5Mw0SW5iT4JESZJJGySRDgZtxJ7di3rcyrlf2j3ZAmD/5CCqhs2Rllqy4zKvI++qR4ZyWeo+eXmczAiib7dXIVvZuMNSviLBpXVB0UhkTD2RCT4mIMbPAK+WlkNa49+/aTgYOJbCe5sFlG/TmEObnxnk/UVJGrnaRRJVB7Rc7UvOV1pIbIwgOy0D6WpT3BzyC5Lp0dFTlDI0rY7oRb3RU5UDEb7TqGaAFphLwx9WVhZ/ZZ4MLH+h28nKPQtFL2SjiXJ+NbGg5tEDj0gZqD7wR8wbkwGyEFPFYuJaCbkZ9A7ehvtUH8fQNOh1B7pfehrMlGeVVn9jrUePA5i2df5TlKdM5w7+c4ony2c++Cd0TQVs0nMO8nRXNw7Y6K5xHdSornR74yF8VtERMZ67gr9d9DfcV9Q2j17l/oqczRut0soA2v9cBKvPKlUbl6Ap9albl6Eczem1jk3/3X/0Y3OjtSFA8xPyMv9nMvNLK0Fj6leKkogMbsVht7lXjqGV5Yolb507erI1toXxznYK104py4V5yak7lX/CFF4146mo7vwLu/hmCYei16kOBvAUuhTG+KxZEptMDZ8iG21qRjp4Wei325k29JksI1g0XV0z5WRGEcsrQ+0sPTOVuilsG09Q3ffSDSlFy5VXqwY8JLuwtB/M3SXO2OLaFP1SjvaAdhnxlxkhz0VPczdmfiBaAypBwt5xhRuYyhEwpwgCzah4mKljGhsdokDSqdulTG3yo708n2j8gJ6Xem1tZdH3i/gBbMIUTR9Z5sPf72J3iEsoWfLvuA9iIlV6ZnzWNmoGe801B/q5ACHlznA4wFvvsM89vcqbfEw1XhZjazhNUXgqyWXrqiW7Pw2sFuMw85tM4X31+vh+21leS1Qf2DtNNjHk0uP6aVeC8R6Z/p+HOHQVz/MYVdcSvP4lMysKdnT7A4ci/L8gP9qzLQqSZsMHCfCyP6ozQKeYoEnM3hyJlmnCbIexmMAPgX67jflU8ux68ezxTokADewgFPNoTWbW+guwV9B89HPPzats60rgtEckNVzUl8y6dX6eB0a5xHM0XsA+hdgroq1f8hm8l5uOVkBKzqVniU+OE57GaL4+cHujfTejeX8mU4BItj59Mw+1P9YoydNPnGkjt3uY3oYGOUn/eETh1A27vwnsjliyWOm0W/f+KdRSfrvhnaT/VPh+ovwdy5hnwufAJoC8i5KKYloypqfR76asGahXaxl7TkbWaiPRtT67YWhj7+lXAQJXGFu4FoSPfW4//+jnyPQz18j/YS/OQr+d8XKwWkkP1fiV0T98jO8DQlRXyFQVEaMrU0fcb4z2g3/0/S9QDlNG+uV/NNapROxg9bclTDKp/3TFbrnJ4O+Z9vLKkyaadAKfWaSFYT18V22zm+0L+jXCeTuYRqRM/O+QNXjReXE6p+qGBYGGNRn8q84ofNP3TOgdNOvOkLLQbCcDFD/R1Alfse4L34mo8ESGGFMaU7FyiKIZ0klBCgrGtP/yvkY7duwROqDJHzSHznuA3xpsD5yhgrw44mZzM6VNqxofLGIw7w/SYP9XLsS/kqW0/GbL34W8MrqjWmRU5/sSohCRDI5NzFzRYDxzwGPsK1TF/ZhF0O7dm1ATgu23xq3WMG2G/0+/8YOfRD5C2uZrvTw+a3heqmC7TD6S/wPtQtQ/2ItG/7KbPR0LPiUyaU2fbbdxrIdJv9fG1OOGIKoMd7/+1o+0g/ETckS43b77oK5TC5FJlyhi6zKWnsehjTXlO2WfLdv/3mz051AhOZcuJqbZ8FVxHYr0+yCnLXN3zx7607IpSrldY3NeXvWWLNhtmnPCGV+2pMnSDq3xFx33Dm2AtKbkFa6MfOtW2IZyMuO4rFrIa+DsiDAVQD7ADa6MYsArgZ4HXAxG2DejVgBytY4Yseuh7KH3MjnAlgBeAPMYHJpTq0DyooiYzdNc9w/tgbqa6H+Rd8iGfmc0FsdlG6E0k1WVCkmoOr04dEH/edT+lmfvmsHTev0dL4XioSh9oCukrG/A6lmbTsE/pjObbNc+FR0TEnVQ08NkBog/Z2MHXem6gCiXNY7poxt6NC3YWhZ30bP3UqkHew54pKAnwZIY2DNN7RNBKi+TR/0R1fbIfoMC9ZjLKUHvnbBtkORsfk+c+daLrRVNo9p9jT/vjD01xrgKnbs5ojfDVQ2D3hby1ofoLEKeMO3yvzt/ywLeMNljF8GHRiJc/Rf4FRTnLyJEya0+mG8iz+N653Sb/E7/axj/Nht9KQfUN8C+a30NGX4CysStNyminDdCqPf4vu/7L13XBNZ9zg8CT10BEVBjYoVUbpYYRIIBAgQOrERQgghFRIIsC6CoGIXC2IFK2JBRFFURMTesSBiAXUVcS2gYl2U35lJQGB3n93v+/383veP95nxcNs59557yp17J3OvrjFG6YO79s4BphKPRp49K2AIPHWH0lzIGgiaIT+rthy93K5MzWNhO+ogZ8ilb0YEYnAWYkpA8tSZyAx1AjKj5gcNAWqEgICNaSDkeaxqteWEi+2YncE8KA/fNUaBNeyZDDJ2NhF2YgYtYPYs7NwhN84ovYDADKRjYfp8DB8xIshPqy3feq29sy4sF+rLuPrNaCUx2DQSWQdGN0PdCJlx82/axWdgsGIO7uPakaWZ5Z/OgxarsHoy04+J15a6ReZBu5bulW+PiX/4HHvXXoDtdUuPytLKwk7U3euu1tGBn3PUgOdquxu5dmAnI0NdQ7J8syIhvl5ZWxb5mKoOmjGCHMvG/w+MUe3pTRYvrdvd0h9naWagAZyO50DBzELd2OZ6watdO9zUM35kuT11Y+0maL7si83Kao2bTbJgNdwxDfs1p3MlpXU1fSS94/ZbbDaInQTcXgV6gp4SjTL0stxy6cpfG7T1TuvlXj2j5WJPIrImdKj7EtK1lqKrYMTDxhzMDkD3TVW2YLexICU1olEuAnME1RNSG1btt09mIUgfojHLqCs3FnJL8VyTbrlLIbewlnixMw2rzNv52Fl8+NiF0jvaF6U/Vo76y7GTIre0ZqjOT3kMTwDzrbLGPodpl0kFpbAe0no0tCD+tA6EQ2o107XewUh6sa0WZj2PhyytqX0J1PPTH9c2tJPdEMR0XQYrkqCfRTg/mhVLHKRsXWcstB5ZO9BFmk2EuaIs3TR9KL3jj75usirddLKvFOmDnx+L8fkMMH1qVVLVmYHNgm9T8V8hwMfbKSWUkkzsTKe6nRdVvxHoZL8j1Bp9Iir9f1EJaAD7FXBz59sTnfLDWB3Ymeu/YSd6wZrV0i3HTdttzULsJKlBbmvmq+Pneq3coe62Jt3UbU2VLiu/o5ODh0BXja1zgVa727lhJPw8L+W5YbqIDBn2I8zNKN2UYlSle1rXTbZMHetl7rGAPojVty/pkbVeWB7QDEfUsFPfuk7QGoHhudllYPGR6yb+uACzdewEOF2wD+wMrDFQt7WbLAOvb93X7GcIFbEBfsaqztcaBzjj3Sha6srztWDYN4Xlky5Lq2MrYNkDdw7YqXxuQxDT3JxsT+DH0V2KuGRHZvOzeTu0s6n4Ksaq842Kzpda71ZinoKIrYmw02b6dZaQzGun0jsazTolvXhK9gWlrB+/uDi850lcuKxUp3Ep3xEAV91kh3HYKbtsTaXkoFy3Vk15AlIjzOobqZ3vMUjQxi0Us7dG7/ZqVes7lOM8+DYxiwzrb6TYvRY7t4OIxXcpPenFRSvsLIQAo6XILnoWO9fLbUi63o/lSwnVMLe2eFGrxtLW+QJPkRfYCMEyJ+jASBXPsiBpZA2D0fZFFtkU0lg9nSdfpVdh76vab29xbCI2G2aRgVvMk67TO+7n9eSkPR1rWR2ZB6OVKXkeQjN9HukWoE6s18olVGmCT2tg53BkqXW+9cHf+HS0V1S1wzOEExCPGGVEvssoUUsfmV4N+Vcvdr1BIV2kd9w8XutnMjdde1yBZNeUORbN5/Va1cBPaUuNQBv6rQT8JPyPGeQP2PfL8mJz+Csr7g9/2cUD4G9MsYWbGlmneJCb2mMdI0K6tptaFRbquKkV6GSRIUaCuRiWY2B0MF3daGu6xoF8o/h0XSOYrwHoG+WkG5aAPtqTL6q8jNQGXEnwvPT237Cz56CdDkw6HRsgx12JpWtKS4rpB5isAFmmPAtpzQrITzctUZ7O9msrYZdbqzp2/mxrVgkmjekf1RvfQyl26qa4lTAZ8UJKCJPVL2LjFYMoSTc4b5aFwHrQ4BsZ+7WNMAfGEjVEL52mag3obg7HTkWpvYEMTjdtsmrWg/WPZvjnRbD+yVKtf0wmzVKdkKs7A/D1a8umRE64dR1G+Ef50yJBvmKLZljlNOI9Ssd7ZHd+O4wKJ1S/IrodRlozEMN0s2mIZFerJlhWcwn2uV1GlnutN7RrNo4s2eVr7BYIz/Tmd5qbjH2N3HyBe11WJFFl4boF8FwJKyHga/6vmByMBqeDFcMql4BJ4o+76SOwHakEXeLgdMN0N18jRDX66VbUjoY8ZEbrwncvO2trdidrtdYOBMttnmI+4dZSQhYZ4yod42oetHDTq6OD2skZLkHE1xhRPSl0s+gdNYtrPX9yO/cjtKA7JWDCrZudrRZkUWrfpA/HePuxdYrRhFvYuYJIRRd3nXVVQF3RteYYJ+mu2MkKFs1ZFOAiE+iWtxIwSqAbWqWrkqU54PviPZ5Zq4mHEV4dP94A9j0I4UnzyKUKbOyRSsN6joDv2H62nTKOl8/MgJnE+WicyjHD7afH6MH4UTOkljfOSPL40uAsAujpMfj37wCvlLNXPek32+6nSIMX5aSTq1S90EsGekL65VralC1AlWbxqhXryRuiYTo2jrzNGtpZyxShRQvQluMnZfcbp5UfUEXCzsDJDwBgYpx9e1fbR1VrAUi4ld5x407W6dr+4MUtSkz1/HTC4ibsjSm94+Gv6VVQC/O6RpU2jFOvLp2tUsdruQ5Pw2psTPqekeXm7tbRsXxILt3XBOnjZkfWyb3nNmSROv5+dHNujdsQvXy3ITqZeGx77etc40dbArkyrpyEIH4SMoctFEaxOQJyjCSBTA9kkodHkxCK7yQyN5nDlcr5EjHkkOnBEQEek8i2ycNtXYTJ5ADqz7hHKNqVgCoRj4QEScIkcog4li2OFnKjyX4M+pAhQ7AixDYZ6fxPzkjkURyJSJrAlcm40aOxN9zCmElkLy47mptAjuGL+bJYbjSpM19VKV2cxBbyo8mxOBpJWUbniSUJfDGPLOYmyzFmo1LkXBneHl4exubLsWKsg91Ku9erbFZGlkskZCE7gcftLA9IkPAS2KK/5SuIy8FFxJeRuSKpPKWLzkspkvCxXU0Ol5F6tOknkXN/NkieOo0sE/ClUuC0Jx5dKOTy2EJyAlsMaNOHS6FK6fSeOFS2eKSczBWzo4RcMichRSqX/Ow/3hDODa6fIKZfL5nGCNmAEcPmY9qaRB41PHo0eTguwFQhH/u/7BC7cfbjXP6kCyUdiET+kxh6O7yb7D1gMQlyF7HlZBlXCMJSym4u5BOIBCWOgJsCNNgPXshwW/twpFc7IWJZolQqSQBSVddU6icngcpA+L10qZRFAleewOcmccnSBL6InZBCxv67PqwpnDcsgYUitkzOTQB6MV4zVIb00hEZWOfEYubTWUGX7TBwYpx94IQfw+9mF+5cDl8ay8XNMsALcykkRCwQSxRihMHmJEjE/GSs/TC+ODpKIo5GOv2GzI/miuX8mBTcYIVsWSzZDVSXrKQHclUm3Z2sykamewbQ/Wd2WTLq210jGEKkeRUSaSFFIgcIkci+2RAmIZH9pWFooB/dz3MSmSaUKMgciVieIBGCuEBoYjnmCqAxcTQ5mi1nk8HCE9kJcrwfUlk0AB+e7sM6fZ2G8wQ+IAYR8cXk+ER2NFkkieaOJQMnIjZIn0tO/KlHXAeyKOkkMgWEinUV7GO4dNy4cVAiixaxx3EQZHYMP0YCeNhYFIYpIVrCI8v5Iq4kUY6NJnwxR5KQACbVaQ6cWC5HgCCJSjmTO0cXzD1FXHmsJBqjUY4fCpA7dFrGT+V24atqAfnyZNB13D46a07gkEV8GW4KXXVECSUwaspTpNyf9crkkgRuZ5GQK+bJY2VYPZibi9jiFFUeGRQdzZfJ2WIOlyxLEUVJhLKuOjggt07aTj662gRxqqpI4Eq5bHm3MpzOxgZjVIaJFJRnI4mxwXnpXoeQD1bLFo7v3kRnWSdTnQ3/NQ3e1J9olLnIn/OxzsewE8jYUwbprjfctFRa657f2YqyxEgDIfTTIBD6aCDGFhpGBGMNRKO/hjrBTANBB2q4EaBczVyDSDDVQIZYapAJJhqIzgANbUJfDSTDTQOnJwJ9BtD3t9Awx+i1+mtoYvTsgRqRGL26uYYaRm9nqWGL0esP0NDD6E9j9JFa2C+iSACAkak2IjPRQgZDXApgC6vCKiMtWAtqI5EAZEgXQIjhpwO4QfoxlGvi/zmJNmIO6ZV9tJDJEE8GcIF0jbEWYgzxWIBRkC6BELuzAbwg3Qr06hDHDs8wNdNGfgN6a4jLARyh/CLQ60M8GsAK0rshxPCzANwh3Qz0WH0zACwhnWeqBatGbWQOwBRI34P+9IO4EGAspMsh1ANYCuAL6S9ArwbxQAATaH87tA+rOyQBwB7Kz0L7uhCPAhgK6R0QEgHmAVAg/QzoMQ5YAAMgvQbap0I8FWASpO9A+30gHgcwBtKHIcTkuRjAG9JtQK8B8VCAvrDiRaB9B4gnAThD+VVo3xDiMQAjIL0PQkzeCwBokH4N9DoQnwUwCGLY6TUciKcBTIP0Q2jfAuJigHGQroDQAGA5gB+k21X6ZwIYQ/8zVfqPB7CD8mqV/tkAQyC9DUICQAYACumnKv1HAPSH9CqV/lMAJkL6lkr/fIDRkC6FEON3EQAdYu9V+g8BMDPSJhBV+k8EcILyyyr9cwGGQ3oPhJi85gN4QPp3lf5nAgyE9AaV/n8FmArp+yr9iwBsIH0cQqy+ZQAMSH9T6R+j6QP9P6PSvwzAAcrPq/SPyRQ7gX8Xoo3jZwJQId2k0v90AAtI56r0/wvAZEjfVelfAGAN6SMQYvUtAfCB9CeV/sMA+nXTvwJgApRfV+mfBzAS0sUQYu0tBPCE9FuV/mcDDIbYFpX+5wK4QrpBpX8JwHhIV0KI1bcCwB/SP/7B/0/38v+tSE//f9LL/3N6+f/NXv5/EOnp/+/+wf8v9fL/IqSn/7/s5f/re/l/fS//P4b09P+v/+D/53r5/06kp/8/7+X/a3v5f20v/y9Devr/x3/w/2u9/H8/0tP/3/Ty/829/P9RL/8/ifT0/+//4P9nevn/dqSn///Wy/9X9/L/2738/xDS0/8//IP/X+nl/3uRnv7/qpf/b+zl/w96+f8JpKf///EP/n+hl/8XIj39/0Uv/1/Xy//revn/UaSn/3/+B/+/0cv/DyA9/b+ll//n9/L/xl7+fwrp6f8diBHoxwS3AawmTQQ7cVsXdIL9jwn6YOsGoG9DBDsvWHkb4/iwFofbFG4zuPvC3Q9uc7j7wz0Abgu4LeEeCLcb3ASoSw3q1cDb0Yda+oNF2YEVscGaThMIhAwiQY2gTtAkaBP0CEYEcwKZYEtwI0Qql9RGvW5j1W2iuvuoblPVbaa6jWCc8QN6ZduauPR0oJe6wIMh0PcBvi1AdsNAPw6gMyrYAgf0nwm2fgb5jUBEui6Flm3uWwP22PU6zONaorP9pXFtK15ydjptiV673eBBtn3Kw/Idj2tfDlp3qz5b54ivddrBOSdflNtorTw1MGfgUaPouBLy+w/H4r2yKtGGAffv/Cp69OxV250yZubt+Rf6CPNCowW7q75HfU1dzL302pQww5+jKaz/ZjiVs0xb/+NkMfmiL99oVm0k+mwbLz5hitqzmz7qdzxv6c//vpNUJnAq2/2NUponfHDi0t19FV/d7O8Jf/NonJH48K7+jb03p0aMqLfdx2xAJl2sY6afqmH3HXK4rSz00NlhN46vXV9+cqfhKGL5ltka2drn9epLD+q+tBovmbMwPNbX+DR74N4jMTZTkx6iKwkP4jXzbpErWmqNxsQfnL9I80jZ4PxTz3Y2l99xSNa6tM0Q+eqYq7N7+RODPPO5cfontEVTbXKiheuaODMIAgGzzVLIjprPtb33PgrxE2qufWNB2PnLPO220x8Nz4ZxS+rPmxx9GbCksvz39mPZc6IeDWw0u29DX3p7zoc/7vhyGaXs8EllzOvbKxD5nRO2T70bd6JT762t23HzrOj23bY/qPyXcY7i+vb9vGza/cjyGnd1G5mD2sDne0i+sx/pz7kUoBE/ZSQR3VOpa2R0SY+8ICy2bDhZMv/QsZg7OtfZz/JnNXw1GF1/Ka+kJm/ohbrdhyMOTe037rB+xtGTMyZXHRfuf1OeSd5w6tARtSNP9eQHb234vdZz4uZbigPqD0aYyR72zfyNo2uyJnpylr4oblpqHGv3M4Mbuqt1OjaTkOKRv2ptOdhquNp9ofb2W4MI72P5mtU/PkeFzUzn8q70FzrHiwWaL37csUhadNv6sdH9VBbvkffVr8caJcsrW772O1pJjSxZXnv37vbmrTdXp7ncqz7r3/g+8OYJ3rtdFWE81zLNR16lzj739K3vHyBZ+DqreX9C1VMjGyNbzuzmNYbYiZe30PiVKVfYh5ZWxGQOGC65VRAY+9T5qp5i7QldT7WhxL7HQjRGjD93fHJl6UndUdaHWatmHIpTr67rKCyruWE3tn7LkukNxQM3RleXvOa8H5EYt30TUbSatElHs+iVgbNrghZvvgYS1mftKe95T8tTTX85aF2sd8TCZdWt5Ruf11bqz3nYclT3QeOQBbdv3Xl35ykl9tGhL4PvZ4ozKvte+3RsxHRJieKJ+VFPRbY2q6nDME4aozn5sjFBd8YK7paOL1HFfLag43Zf4Q2PAtLT5Dr9W2/91DODJ6odqi7kjWDXRPb9TOd7MqaJFQ+Kb8Z5199lPXRr1I2ZcG9ya1FFcVDDiS3nPEtvzLUt63h58uR7y8vHqxcHHVpta3V4+67jNc4a1+o0Vwc3hI0eVs87dSgmddxZtnf5zFgL4hiJ9ZrDupUTzugt38rSaLSwIbYsw/y/v4Udau8cNc/aJuhMtPCc+q+KNo9QVuECfzT3wsX6xdo3q0/Qv7U0L/6trf7KvjJ/vbLiNMaWTJvlC/Msb3hXmzCD6ocZJbdJb7Nb0FXjiutCJpQ1mVrmldVpZGblzrPIi9iIFJmX2Hx7UOR8ZUOTYubMDqHQ8g7q2niNZbBlbZbVtqX5Zr8VltLIlUUK9rvmJ7uaP9S8ONOQaXX/XCl35sziPe60/FcSwdVRSdLv/Il2gmLyGFaLrpbOWBNLF1EjzfnD+ZlapW+kwfFtAq7tiTEfPm+zqz6yyDIncaVWgWO//Io/DLKWHR9e1JAypbTFRfYhpSOu2asSPWfxa3jDmCmtKf4aL6PS3Kv9bRY+CLY8v8ZsMWGZwQm3XcOaM0+51Fe7FJzTH3KizZeUW7iiz7Lc6zOuIjoedXZe4m9RSxRPgi4fPzEzeHuB0Cx7mevdnFyDdQ11eQEXrhYZv37y7dbHb1dWJ0TVzRCkNA10Cy5riPDPyu9rYBJuaDasv5WL9P7UYehG7ndB4ZwmVlPYFZ3hfjUu0YSc4u3mi/OfOhReHTKm4nvkfqsnBw671rzNN8m0WaBTKrzkZbW3Jsjs9z/SaKOfchWx411TPjlbeR0dqGMhVzcZ4+QTVPE+0GvZoRRugzQyrcXuRdOHHz++V5+qrcmZc/VKwdTMxc7fNuRonThYEZy8u5A7cXJmns/QvCKSTtm3S0bFV7Jnt8z0p7UJDUT1rtfk1QZLP7BMqL+jw4inhdLT9xRo+jrnOs/lNk2aO5GycxUWWVnx6OIBsawT06mK5jyWsL7ezMZ/kJ5z2uyRFjabpiGWj47lISYFmXZBS4qjVq4qC7r5qO1c37MtbWEt1YVrPtXn1hZrVbw8arls9ya7BsH8MS1jLgtS3tyWeu37MtNC8oQ2ZlxMg/OTuee0CoKbgzmMD9yhxNIPzy2KqnfYZuXwRucXjHh2rrj8e0P+17sfrk660fz9l/QiQcX6Utb30nydaXuzXOaOtbQqcdIyezdoDM1ey04h85I+OcwU1HxMpWU6c2aWKq58q/O79aTJ8OvVsquP67KWHcg18T6ybJju5gLpxawT6CK1YTNpA1yEWnZmrmdHGRjM5/nnUdKCi9RCUr5V+UZdychwOTc4b1jbrEMGhZv3mOU+fB6MWLT727HqoqLWX08Jukd/4t8v4Fta6C91NmujrlresVm2uI9j7onAwSeaczQL6mtmpX147cmt3i/0yhEnBhWMn2Ti3DxMR6tI2yo4ztiVa51bmPJsRYXXzh05FjEnF48Z+f5KxeNXNcu2Vn1viKpvahlmWvHkpH5hTfuIxZmurjmladIaq2P8K2ZfKE20ydO/K1IfcgVl59JYbW+DdCZ89nJJKtcpPrjVJL91setVh9VW3xMw/ydUx01Qy7ojJtIqI7R0dj/UvFL9Sn3ZnFIN/77zDfowyw1vTl6hv27+Xb2Qlic6A75Ga99bHUTaYkPSneE51XLEdunAZ5c8B29TDBoUOXC0xXjdfQPesDea722s7s8v+2ji9P5in895HcYHR+wykvnmmE1hmZr+sB3Wt3x5ar/UZwwXBu3xxBVj6yZfWrV8ku6XY67ub6dMW5ClO6VqUuBUYgDXeVa/8AmbfpU43T/j7GheRLALO5Vlu7b2kP1twe8OxoRHVoLnK4cXrSgc2WL3Y4TN9EvDohi+QwtG/kJuWj90iNUHszHJh0dZH28YPLo9kjZqGineJt6ybeyhpDPjPl7cMH7Ctv3SDcuq4sNffJYNiihOeOiwXrF6uGUS099a3u9dQmLtRo+0RQ+T5/oc8f9VX7vvnBtRQ1LS5TuSKYNWp6oXXP7l/OXvwopxJaI094USl28N4m8rXwrKFgjjFK+nx9oFEvkfXJyiDszV5ojMXNnWe2MiX55mcnfV3YmOPvlbzFC147zH/GVedVfIdLOt/XyCBvt55ySm+Ddw2v0sda74so6uYuQ92kk7uyHPU7P1gIeb3yf3eVan0Wv27m6G4TIKvWkMdcnSgcHvYgNCHIi8sMSKaaFH7+oEfalaGjhpz4mAOaZPmZVptaynE5unk4MaI2LeLAjfsfDgzFc5jjNG/6E2S+LBmr1/vOiJk0X648/xR58evPH2N9nO+qYph2a8+FEveF7Os3+Wqq/RMsLb5e0zsuHrbVtC30S2sZvHP3328k327Vd7nRf/zp9x8nuf8tb2m7cu/Fgn3NIRoln0bYDR2D/upVp83XKW8mVGceIH5KXX++rMua1ZU0e8o4UZt+mgaz5esd7+ednab5/8O67VJ8+MvHd8Qtj99kUGD6b9NrEh/mNF46H8JY8+Drn1cILP8zqB3r27RTEtd1ruHam1Kc2oidqhfrPgusPtJmncLasBM8/N+mF0dtOakefvj0m7YO5Grw4LvXpm7ZQ/Tt+et63KuHntFcaB3ZdXnMu/eOmX85d0jd9dddeSX1sgot6ouj3gOvGYTc6BXLeVonbFKmvKuNUvR/XP3TVt07ro4L1rh/7+fs3j9HP5FRe+bEnbf2OTS5/czd9StuaVSazWKzT6bLQ74b3hw81fly1ysl3qM1truf6TWStuLOEvTt/8egnl84NF6vR52eeHlWVuiM6eF25wKn1QSVPGw/s3s1ZfDZ7P3MVZ2M980oLaBP2ipzUFu8nH1+2JUb++d4f4a/Gr5DkHRpv47JfsM9m3//zwwncZ5rscXo3fkRiStPOoK1rwZfTZrZOoH7bP+b5nW+W6zWVnZTWHNfu/OOJWWHl03rVFpdce6B0yPDj5IN0wqmQJN+RE3dDY42Zes8uDPmkey9lkV9Gw+PBJy8eZp1iz7lfmOb7B/L+8gsfdo3NxiUnZgbcrotaFTWhotR8umP1wc53LsXOHR7X8qu2/WHLycXQdYSHB++6p8vdx9x49blQ4hHOMQgP6h+U377T99s0o35Sy70t84y9VF3kXCvcnJmhTr12XjFi7f67UxtjczHXH0T8WmtdsGXCjeeQ22QpKyXnyPu1Uz0sF9gemmS+/OJ00mVdWuhCpGR/gwl/3erbe/IjWp1Ofr1vIXbDw5qNJjw/fWOHPUjiMcv39mdGCwHDFb25v7ul/YZ6KbbX+stVrXf5Z1lDbMU+3hPnPpUs+XfiovbrWrfBXUXuVuVN+jd+yYUet83PNz40cM7fg+HfP/kSUPMeobcWq3V7Nnzeahk8dtts13G7gyiOLtn6+bZp0I27f5d2GlXO1nmieicyaVSC91WTZ72hbUXmEr1njlJvVL+bz09If/3JlvMEZhZfgir7hNtnB2Ats9Quj68sLfN0leR/bb3esfBsTkBZ6f4KFwbGRjCMabqN2lLSct9TL3p7xoGTARF7dXOtbyTmrxYPbJo/4rJa5YWVMznjX7NGrw98cdo8M1vuiZTeIsbtx17sbx3+sanxNG1e+KHhjP97rkVJifKvXiUu+4xsefE235Xi8CBG4Fmy6nXnh62nn2H4piw0Tns2ccGXGk4Biv+AOt0/NecOLl2TL+k5o6as+z+1LxbSRm4e8HGy1PiQZXfK47oDDjJLLbyKOTA55fnOWZ0BaScfr/rVRk4cI7y+k6Vy0f/lCunz5vFFvttxZyfx6cN6z0NmTwu0mFa7QyRzsIG7SX3Bau2zSLtFJtyc7NNoXnO5HHz3+wEc/b/uhSR8WbzHOzbOuTCavqx+Y3HbJp8qreNyt74SqODRm5+fcWZb+YyamzMzPXfd82Lv3DEF/jY91aZEvom4un91w5FW1zvLA1LKXE2IRWn1NxZCK7Ea73205oSuG3v3K3hC3RV3r5K6Tpwin7+0/LHY209YJGpjgt2Hn9dEdR39ZYGx44Yl8lVH9tTH7Kne79Dempe9MGnbvfExiZCrBKOFG8Y/Lsksbw3a8G9pinuv5ff/EIg/jWVeTGG81r79bxHnoPmJZ9Be0+ev4FCa6epBj0IiD919tuHvq0NfzL3U9ti7t32oQtdWXrzVlI7Mya6Tzg9xV653GjfsRMCL42PotzQ3tS2fGmzg+6asoch533Xgxfa+aa5PHicwM8m+Pc5Kmz/jUh/4y4vu7kGl5wnmaQ+5MY7ufX3J9T9qEoht356Iu4jMjZs5JWnTo7OW3dyMG3pU82XqQ9MF00DOf3SkZfa7w6/YqDEqPVW+dQUg7P3FT0Y90K7P1z50LnHWXWzLFC2L6Fk67H189VtIwds3tY3Qd9YzWg+VNa7fr03+xODhuEPXWNK8/xBE2Lx98OsPk5ey6WlI0WK53PTXmGHtNg4bmO8z/c221FGPZBm+PM3XW9z8r0lrRFid12sl5uX1t9Bb77AcGO8ofpgx6Wfs4u/7WOmvfIzon5xxM07Ipf5Ez8NTKaKOjA9+TS+K84o99aEArs369c3/Aq2ePRMyyO20X5t/ODM0T9qnaLYhO/Rr1/fUl7mL/GQTTeqEmhzPV8NtHfe1lF8niybOM+L7P0MjahHjetpvP1KZ43lH3+T5f/5agjLTz2+4yJ2FeKeXupRMP3L5W7PtNeM8+cUajxw39uw8jpt7cu8+2fsQkpIGZzqy72Jddc6qs7fCQYWcPha5fe/yG4c6T5VvKiaO0szVml9brnbd6qXtw4RzJeGPf2PC9A9mnp9rEHFmJPkzSjH9AqCDfyhtjVNuyaP7B+MFlRzR3PjuV73CnvHnbJa1kx6+I4fLdOrnmeQZPTujHzbWZKtJeJ4zOIczgNLUxBYIottDyni13vh8S9f7NWk3hLzsJFqfbtOeFnTX8eL6+hBvw8qjJ7+WVS+ZkH2tvHPgoim5z3+zDnNtLub53/ghnlzKuM8smyZGK7U9tT9xBdzZ61629N1V09uaOP9ru3o57yae214sdadm8/TXlkfdlNuruzweqOcz2Je25NEf/0ZR4jYA9KHGkkZFu5QKy3qXhZbFhh+ZLyDp3Yo7lP2NfN/jaMCvvUv3ooXk1JYd3113oN/VQRIb+4XGTZ5w8ul94vIqcWf7myKFTG/SeHlHbcOugfKJn7e8HFLc2m414oJ7Z96HMRJfzW9bk6DXT4kT6u1lxqbo3DJ5t7tBZPbIYIR3covWr+2rD1lvbtRfGvicM+lGtyZ8ZFvX5Co+bHu8s7P9CUyBOsrjz47H17UWs1PtGV70f8SSNx75+balcTq082q92eUlk8/a7d9NW39x6tvqeS+D7Rv93vBM3eWEVux5plrn6OJd63bfWv+drQTrwyVvNOTJVHT3TEtkY0sjb3bJcbJdSyactPcS+MiAzpqLglmS489PYwLUKvatqnronjvUlDh0/QiOkcvLxc6N0T5auYh22Vo87NKOwo67a7kZN2ZIt9WMHFjdML6mO3jjiPef1pu1xiaTVImKRps4mV2eDV/N5Wgl9whCNed6n1pqmlj8ttj74i4vFEb2Ny2+t0q+sfX605eGcIY0PdO/cur2A8vTOuy+HHsWKM+8Pvta3MmP6iGOfnihKJArPo+ZNLO1saZxhx+XJmjEzdAnGHVu4K/jFUV9udwjYHjeEfZOfkgre3tKvC85U96s+pDaRPYJX+LlvZA3Dk09/oBBP8467WfyQdbc+RrfRrXXyvQlBxRVF57acaJh7o9TzZUeZreX7kycXVx+/bLv6UNCu7YetNJxrjq/WrLs2Oqwh+BSvfti41JhD5d7ss0SL2JlrrCVjJlTqHt66XO+MRaMGa1kL0Qbzf9TOov+8KGf7M0E21urnhNEebYpfFxSyQi/kov7ai+sv0k9U31zc3PLtSn3bb3r+ZfsYacVly20yt9ywzFvINKn2NhpWH3Rb2pa8Cm1hh9QVjzNtKptQV5ZnmZuVqRGRZzHPvAjZ+OCbTcmGK85FM2cqmiyFwo5GV/TOFgPWtW1WWWt/M8tfSqaVFrIVRZW7njS/e1Hzodkqs+EMt/Tc/T3FM2e+yqe5j7oqkPC/S5OKBXYTW1hjyGN1tHRFLpYmH5xpjaVaM8/HB0vf2HIFbZ8/jDlxpNpuW2KO5SLHAq2Vf1Tk9zu+LMsgpaFouEtL6ZSOlA+ySq/muF8tzqFTxjSEa/intLqnRb1caONffd4y+AFhsdkatxMGyzKbh+2qrnc5pX+uwMW37cSQFYW5pOu5y/roIFdneNnVeSyJ+ia+HPREETzzxHEzYcH2u67LstcZ5OYE5NU1GBddvXDr25PXq698+zijLiphYFOKoKEs2C0/yz8i3MSgb/9hZob3pS5WG9FhUwsF37mw/J8zXOdKWLRLjd/24hzC0/zF5kOuFjpEfq8Yc+CJ1f63Na6HbTJN8oWlOgv2Wnld+t0sqGY0Le2PWAX36acU1/FHvayc5RY6A53GmKi/rwjyObTMK1DawE2xa0mL/PGh6cWp6u8/5uTU1E4tuHL1m/PizBNaORuSgysOTuQW7vbJy5xMKsobeulbmU72lWIj/5ktsw2EbbRrrvWipQbVcqoJ6wNxGPr7aanwdDqquOdZ57xOs8lm+bkyZGdWlkXFgMVo/PQTrNi8ZgW1vl7IGuRvYzY7zVlvk43FyEeWyDQTJO9YkF1mwcqo4iU3g8pW9T3X9iisreXsmsLqltrc+k8vK7SKdy+zPCposNsEy//5b1IEl/d5SW9LLGZ+GTeG9uSJc0NMgda5uZzg5uCh3A+M5x9KiTuqiyx4OVm2IwryR5cXn3v2Nb/h+6SrH+7+8r35RoWgKP07q3T9NJ380rkuWXtLrCzHvjPTcrKnjRkkU9hpHX4i9fpYI2A6Z9JSFaUzOX51364YNj25Bcv/r8uy6h57m+Qe0B227MhFacHmReiJLNrMYWpaQpcBZ13N7OYbGIyi5Pnz1IqC06q+pYRkXInyHXzOJWNW27C8zYUGhx7mmu2xQIKfs+z829dHRdXdC0q53s//CT007VvAWpu6X+5YXo3qs3iZTeCJXMec5hODa+oLNF9/SJu1v5rrKc7xEo4vCEpsdjaZVKSlMywu2Erbmutq/CylMHenV8WKGIucHSPHLD75uOLK+63Lal5FNXyvGtbSVH/ySYVpe02hvmvm4hFppTmux6xqpF/MrvAn05ooqYrv08sE3IdtrLRzE3SC3ia5eH0+WKxT3ppvstXhquvihO9WqzH/nxBXTRDfyVKLqKQRH+7W0XpVfUWzdM4y9fl9/TXKmX0MVky+aXh3/jr9Jy0hetFfB+gErb6nTbLZQprqOUNXun2EpeelZwMHKbYNHj0wctA+3fEWG9lvBlQ37jX/WMbvf/G9k0lH3uc+u0YcNM7xlRmZsqaYDbP9YZq6vLwv41lqv8c0hkvd2BUTl6+6NPnYF91JU966u+pmLZgWOKlqCjeAODW83yxnya+bJjifue9EKDJ3zDoVZneodq3t74Lb9o8Ixg4rnwusClcUDf9h1zLy0nSbEb6MqGG/jCwYOnR9E9nsg9WQUYeTxwxuOG5Ni2wfHU+aNqrNMt7mTNKhsRsufhy3f9uE8VXLNkg/vwiPL44YJFvv8DDBcvhqhbU/MynhXT+5x8baxOSHi9L8j/jM7aut/+uQqBtzdsjTU1YPoiRfLlBP/X75/C8l4yqEC93TRA3fXCQvV34TCxeUCaa/VsQRA+1inVw+8LXnHohyNRNxYvZas5mnX0beqdvF/e1kdPRxtaExy/iPeeQrdV79tprR/QYH+aQk5ni3cxr8r+hY+q06yvLd+SiPkbfhLO1Aq6bnJz83j9NW89zd7a+hsnBDtzFNdMrApUuoAbHvgnlEh5BpFYlhOnePhi6t+hJ0Ys+kwKemcwJq0yqZzROfshqDyNMXvImJOLhwR7hjzquZan+MnsHykMwSjd8/O93C6cnR+M+P3944+LR+p+y3GYemNAnqf7yw55U/19BPfebiPaLFkPzsbeiWba/ZbZFvnj0d33w7+83Lxc57X52cwf+9tbzP9wu3brZvEa77UaQZ0jHWaMA3i9R7f1DObvmaWDzji9dL5MPczOr3I6ZmtRqH0d6tQXXatltf+fht7bLP1zr8P0XOTK4Pm3D8nsGi9vsTf5v2oOJjfMOS/EONt4Z8fPTcZ8LDe3qCupaYortH7rXcySi1qVXfEVXjcL3gZpy06fbMAVa3jH7MOjdyzaazaWPun6e7mV+4GhpW/ceUtWe2zbt9em2zcdXuA4wr+edWgKYuXXxnrHtJruV+lSpacG3A7aobNseI191yD+Qo2kUrx1GsV/Uf9XL1pmm7cvcGR697//vQtefSH6/5cqEi/8b+tC25fVw2bU35ttlKUpbXR0Ox3vuE3cZfb37YYOu0aJnWbJ+ls57oL+cvubHi9eb0xQ8+U5bMo6svKht2Pjs7ekPmKYPweU0lg9Jv3n+YEXx1dRZnF3P+JPN+C/UTahcU1DwtWnecvPu6esyer+Ide+ckvyr2MRl9wGSfZP/w8/v3mWe8Kxz/ymFXUkjiDtT16M6zo78UfKBO2rrn+5ztm9dVbquRnS170V/zcGWh25FF1+Yd1XtwrXTyQcNDUYb0gyHcJSWxQ+tOzPYyO675KajcblPOscOLGyoyH1uevD+LdeqNY14l/t0mr6J8yUWdPW8PlJmErYtaYd/aMOHhbMHwYy51m1tGHT632F/71+jHJyWw/K8rP3XX+9G9uPcOisbHoUac8Pyw/gHfbHc2m+YbfYv/so9yseqXxv2FF3hU7YTEEZLr16Rz9681Mze2+ePoDtctNeYLRzbfGEBZIdu2j3y+5JJnqva0A/YF0y8uNy/jTSbVIAtL+S4B4/Vmv173tDVi/sJ1z6feXLiAe/jxpEcs/xU3XEc5KBYYPfv9N0V4oP69N26xp5hftn6xbj2bv85rjO1Qln/YlqefJPS5q7U/Xvi10K3WvKpd5FeT72R9dNiyc+a5+QVzx4zs7/n9+BwySly1os3oc7PX7qnhphvDXXcPO7JyoN3tz1sXxd1IMjXcfXnfE625lVmRZzRvSQtmHe1n2RRRXtQ2pdHMd/6L6puP09P4BuOv/CLwUpzZZqh/5ULsQdnoC+ps34Ly+o95EveVHbfb0wJi3lpMuB/KGHnMYJSbxpHzLSU7tmfrWQ4oeZAxt443MSf5lnXbYPFqtc8jJses3JCZ7To+50346tHBke6H7bS+6DXuZgw6fuPdrteNq34sKh9H4/XbGEyUjnx9wqs1vmG87yXb9K8PQl54cDYVuAq+Xsi83S/W+XSC4eKUKxNmPisOeDLDrSPYb3he8ydZ9pLivi0T+n5xm6e+eeS0CqvBL4egySHrD9Q9XnK5ZIbD5CMRb2bdfB5SkhbgWdv/dYdwyOQoHdrC+y9e2l+ct3y59M6WN6MOfmWunB36bN4ku/BJmTorCpvEDoO1Ty/QF+2aVLbjidvJ0wvaNcaPpvfz9vt44EPSUPtc4y2Lkyut8wbWryP7XGpLHlfsVVVF+H5rZwwaZzkr93PKxDH+63LzZ75/N+y5Rn8BIzKt7uPym1EvXh1pmB24XKd6wsuy1HoaEgvL/5rf7RqzV4RybNlf7w5V3xK3AZb/WvdOE045iw/vD9LRNtvglzCwY/T1ncYLfjkqf3LB8Fq90ardlfvG0Iz7uwxL2pmeGHP+nhEhNfJH8Y2EjZdkl4e+2xHmmWveUjRx//ers4w9NN8ykjiL3l1fNsL9YTP6JZqZMv6r46DV6P2DI4JO3d3w6uX5r4eWbvXQjTJo7a/F991aydw45YHzyCyn9atyA36MG7f+WPCI9obmLSbxM5cq+j5xvD7OuWgvfbGxR5OrGjkj80RSzuPf+nyaMf17xEt63rSQd0M05wnd2dPu7Lm+5PyNoglpLujcuzNHnBEfWpQ05+7by2cldwdGkA5uffJskOmHjJTdPnX8K31KDRR7Z2ytPjbxfBoh/UfRpufrzax0nQucxUzL5YV9YxZUx9+fNrZBMpZ+7Paa1gx1nbVN5Qd/oetvHzTuoIXXtFtUmwjxH2c+PXi5K4fHHFxUcjX1up58DftYzDtNjYbp2MfinK4PkWnKL8LlEjJbKJRwsA/Dx3R9C08eJZVIhFOHCxNVn91i0dFk+ENCetVDYUeTYxK4XDI7Ohr7nBf/djjEz8fPP8wP+dkI/nX2cNk4aFe5n6Hrk23sO2SMCPuAeRI5Ef82VvkxM1nBl8eSo9iJ0QnYh+zDld8lq9rlSMQyiRCaFUeTaTQvIAS25bFcsowt4pJD0MDgseRoLpsj5yex8U+YVQRd35eDCCQJHCV3UAGeP1xG5ovYvK6P7aEI+0xZyMW+o5fLyA7kGH6CTI4zLiPLJJ3fVCdwhXzs237s+2jlx9ZsspirIHfVh32tLREmcXEOpQkSwBWROvkALJUQVN+Qd8kF+jwWePkpMZxXGs7rT9nKuHJyND9BrvyInqTqX1ACZzaPmxDFF2JfayPIbEyl43BVTSE7IMgQZQb2Z/rPsplIAEQ7N0t0p5kKRNKf8setho1vrsCElZjAxTYmKM1n9FgyB//KX4XEVX5xLSNLYqBbCZgU2D12gkCfALyQkAB3NNgjEAn0oPqHegRG4B+tROYnA8wBSAfIVukRCYpNlEeDtLD+MvzdQ3w9ZtOCgmcHeYUEu2O2h9mQ7WyOXDZbjMftVHGlzKk9+RNxga8UfAsKtkkFVwBKoYMFyTgJfKlckoDLPDBRLMbUgG26kak+9cd6L2PHgOFxOf8JT1lMHi4FjK5v+7FPqhOl2NYJvEkljgz7yl/GFeOqRKbHKMhStpjPmQn1UjxxjYLsxg2PdgMJjnQaZwe3LXm6o52ts9PMkcr2sf1DoANwCvLI4bKRoxK4bCEeI48GhkZhjYW5e5InQV70yNHk6QlsRSDY9UjlNpSRM7G9R6GBKAPfAEAOkrPliTKcnoT4SmBAiCYn8dlk72DUE28vgcujSsQxfB50z03lp1gbOG6C0v78JPJYlQnjfY1JkIhw/RKIRDW41FWXRq9L8x8urf/lpf3/8TWJLZ/0N4MyZpmSRLk0UU7GHJGE+Pvo47LtTkNluJNpKN3Xw52EeAQG+gcCSvfy4TLlhp/eea6QafsXuFOndI640/7UFq3bsChO4ibI8RFOjPErxkIuj4s7AEYSxJXj427X+A1YmGl0r0813sEYK8LGcZxRtmpk/jMt7p5oMIysPAmHj489gG8dFBEEY4d+7z7+hSz54p+i7ClDFQPd95chkTt8kcggZwjdkcjgOAinQPophI4QRobIYGCfREbQYOsgGH+wkB4QiODp4AAsVPqAcsPKn5gCj1Q5PY5AQij+/sE0X9Sz03d60HVurOtO0B3vp7h/PgjIQm4SV4jLvNcggcTpNPFO8IrnTxF/SNjp/3ES0yBw7kaN2KWDM+v3Tn98rT9l8fi8+AnuOWVVf9x4WfZg58TQ6ukR74zfrXKdt2Ov5azae1p78kwypxOs7OZXhuj9eiahcqY8veHgj40PNQtPb3wnf7aRf7ql0nOB4enN5630tl61eTlW4i3P2jOjpOhh0tfs+3nTmIG77c6LStRGXjwVMffC+ce6H2oLczw1L3OYpqNH80YeJc0fZzHEhTFW1B7kot+HS5IFJ8WRTmRUXxq3KmHs/G2nmrfuHhDjU+2zRHpByNx+c1Vz/7vrNa3aDJzmG8UVSPvvON130ASh9kgvha9jSUXZREu9Q3739nhP33TZP3ac5IN8xby3yTs5kqyLyaZLLxC1V1Z2fNsRIHxV+XX75/Pxye9t/icX9vwCrWLe0zn2KUdK7Bkth3GXi+kQz0dlMnCazmemaqNV5K4AgEjlcylItceQLJMnsKVgJZxE1SY61XMrMSEBngzQ1tjOzVbk4YljsTkF5iOyTrwgmDtxYpWm9ve48lgpIpNLkWgAbLOSWCLGtggh+BMLm5PJomwRZWiHrx/lscrnqmq+ZqcK7VWhA7ZvLJovUX1uOgEA25Y4EXNN6GMrQBvAF4B2JLIQ+l6oDWAEYA5ABrACGAsAPlYI/lbo7odtTxTiDyLV3Ap7OMqxvbE/H1G4HAvXABQgkRKs7n0BfHxnLMzQ4CkYTWbLyAq2AJ7SkkSY9XVOUKKwbVHYgysR2xeGjS/K8XYcrhkLAEuAgQCDAAYDkAGGAAwFGAZghU0bAUZgagcYNT3AP8wjsGt+7E4l24W6dE4UeRIJpvQ/4ziE2v4jjl2o3X/GCZAouAndUcaSZTBLUo4LqqkSMh1GS1oXCTWQSu6aUYug5/guTg55qmpH6liYD0nxfaE9skm96/Ggugehs+2dnDub/7kpNAXfytwLX7Xbkizj88QwucDmhCnSrjl+T1x8DgJTDNVeZ6z2GEmiGFABa8if8cMSJNBhmHfzOZP+jl8lTuc+5V4y6ZqU4ZP3P9FSe07aVEh0D1/an3npMXCD4ybgG1xdXV0Rj/BgxBeeE0FhMB90D8b9LAw76x0u3wCI4SE1ICQwKFiZ5xWGRyP3xAIIAaQAcgCYH++BufGebICVAJvApyd1W8N0PQDp/hRsDt5jD3YvXGkieFhQYFAAhkzqLA/Bl2XwXFP5D0mV/3NTICPQg9m5RxOMSiX48CFTlSFJid+5mRPHlki5CfhTXIlsH/6THwWuHhk/Gt+xPGSqyjC6lcGiTNxVKIwm9Shjpwhh5tmdDwytkxNVPVKuOBqfkCawxTJ21z5xbA4QxefhdU9TVq3ED6IEYEMnh6vcv9k5guNlNDQY9SUhBII2Mp0FYvq7ZfbfKKEnzd/t3Cb9Dd6fd4ZbqgZgDU1sOz9Bk0AA1jSJBIRI0FQjIGoETXUCoq7clkCA+bdynbXfiKAL45k6MgNBVJsGIvebAjiq/axHXV2XaIXoIaaItnonDjbGI+qaiAb2F8fB2tJW11IjAqYKDfCkqvphDO2ixYdubbz+LloNoNVU00D01DXVNRAjdU0NDcRUXUNTDTHX1NJALDtrVF6E6s6YGvJX1//N8sjiAGK3PukSrJBs6HKWNqGzHPyzGN8zr/Zn+XRVo6vqP/QcytSwMkxD0H8NAtZ/TQL0X1OLgJgTNKHq/gRNHQIygKBJIiAWBE1dAmJJ6M0ZseskB/wKMD0Xs15T4/+98v+7V+SBcoImot4tXYEFBC3ccPF8nDv2MijzgbmF11mAYoAdSKR3FuTB3N47FUMRsKP4yF/oNgrF87V+5hTwuxUTOuBCIkuClb6hjqke25mnro1gDiKEaKwRzkNkCYzPJTOwaJcfaXTZeVeFXfahLEO6lxGUZV3Jv8j76X8anfaljVmSkXbPbvXtlUZMcJ6ImF0SiRidOhHzPW3c9/Rw38MsUAO3QA3EXB2zQEsiZoGDcAscjFsgmaCpR0CGEDT01RArDQM1ZJSGkRoyVtNYA7FV1zTRQBxVrJouNyYU/OxHD8MljEX+ZMiIcjAiIpEH72kofQ3mXG4IMKyDD0Y6eoiejrY6QVfNSscN0daZok7UVbfSiUaMdCLVVWqNLIX5ZCnYQak61NOMy0sD07kGQQ1iRDVEW0NNDXSmarBTlvCnU8JqPceoHv6r01mm8e/GL0yE5J6j2H+v/17/vf57/ff67/Xf67/X/6+uIKZf0P+GHp+L/4fwf3sRVLPSxecGRs2KHx4xfHztBF2h/tMuhMhBBcpw2C5lOHONMpwxTRn2u6cMPUuVoe99ZThruTLU2a0MRycrQ9pVZWgarAyHK9uKtCap6I8rQ4t5KrqzCLaGibTGXsUSOQkpkKefwElIhfku+323roxW4kvlyjC/HOoywqLK9Yxvqqo/y1X4sP7f1wZwVZU2Qdh6WF67Km0Na/h9ADN+yqK4CsBWleoH+DWwPtutSpMR9hKYi5d0lo+HOKzJStao0nYIe2kkpDvxgf+SewDlqrQmzN+jId2qSuvAnB7W2Ac3Kds2vohEGk2Cvkcr01aw1rPKBliKdL8ivb4g2Noj0stLmfb8HcHWej3Tmr3SGvMQXB1/WivOg5XDPIT5Sbne84E2LS9r2meHE7vKsQNXmB97lxO6yrGTaJltvcs76+/9+01ClAj7TZUwDwlEIcSY6n7MA3EeYtmKywbHscT2fhip4pgt8FXxDz/zmV9UcaBlgm7ntijjgUhnPnamnCxebNfpUwQ/gjGRMDQrHRmTnZ6O2C/Y5zYJ0ZlIXLD6ZIPcJC8dW0WRIDDJwqLapE0QLcCiRFIFRDPTsbgaKRNLZJ1NR4w3Ow1Y2DE3flMyz4MWFSZUsEMpIrtYtmhiCpoaTGcmy3wFTn6+Pk4+FH/UM8KJEk6JjQqnMkPtOPZCgYKZ7BfCmyBLkafSSQ6JYe6eFF6EJ5XOcEp090mRidAEuo+tPDXIOYjKiGKnRIiiPKXxLJEU7RVSqX5xPJJ9iodfHNWZkcBJTvKniihohDdFFJoY7RmaIkDFHsGoguGO8hjuHin+VNTeLzgkFRr3cORHYK0KKDwSFmF6JsezRax4RiAgKxgpOJGC7h5h5xfMsPOkiyg8lhdUDahRAoqMLZKK2CK6T0qSe5AzKQgN5vnQgqiJ7gIHmQsvwUMBHZOgEVR/Zqg8yj5QqGAoglDflJAUP/cQHiOOaUd3p9v6BaGqNKl3Bpb24DE8mKGKiDCOQoAG8715SjYpfA7woRRAPJPij3EWQfL0xljjR3moWItTdpvRs9uBPIaY7uPI8PWPSEG9FV4ePCpH2V/oAqB7OHR1GHVhUAQKXgJPIXMS0lExyglCvZM9MKkk0zGKVCbPLy7EjpEaEkF1jKCSBLZKZDGGHOaNdhNwhCcrvku+FA6DEsRRdswWOsbzVXJBwirFuYhw/MlFoicqDVCyzMZYpvE7u6zscSKrUxlxpG5dTu3WZZ9gCmZ9rFQ2kKIy6IbCM0LhyVPQvAVKlpOULJOAZ3dGhDszgiqwVTLo2EPyAuhAN8mjUj+BN8eBlhjo5S2NFoUxSayUiDAnMYvDUkmG9zdhIpPiTBEBa8IIqtwXDXNHw+LQcHeUFCLyRGWeaJKy474ySgSDwlOKLCyCqohw98CFLYhX0HgKT6aCpoBeBPJ8KUom+SQmZp8ibxWXEqXCUkFhyYxUzk+FCURYBV6owosOFXhDBTx/+BfII/kE+aLsODRE4Ina+3q4M1LRUB8FjR2GsSVXssVgYGxxlWyF42yhSrZkChoJ+KIr+QrFqmXE8nxjeT5BcdBLpYI8MAXZMboUpGRdhFk2S5SMklSsQwV+OF/JvAAKVBCMiSk6DuW6A3NhKNiGlM6gyMLoVFRBC5JRwoQYti90AWsv2B1DD8elGiK0x7hXqISKc89Xch+Kc09Xci/BZRKhIHnx/nXjnmwRGh/K87GVKf0ImGEEkyJsf1rwT6G5qMYW3MgxL+pu5KDrn+UkzI5TfyKI3dGgoFQ0HDpGV3gArxjjPapjOPTAJgWBwAHdF+1hAYKfFhAUjNVAifBEk7HKg5XYoDsoCgR3Bkf1hi6GgQ34AwIYKiZghcLDNoIKkvOG/nDBUOPQ4Fiet3c0VuSJKot8sSISKlGVheJlHsoyH6ysm//Gdw2mcWgEhu2HYzMj3EkegO6Nt5Liriyj8XwkCg8BaCdCxmQmM+L+I1+kv2bsP/CFytyVRTgTYMrKMu8eZb2ZSP17Jkh/K51/YKJTEiTgggdO352Jv5KE399piPSfVPRvmCApRfGvmPhLSZD+wU7+UR2kLn38G3X8ha2S/tlY/zMTpG6W+a8k4R/ckwnSv/KY/8AEqbtR/DsmUhjdJUH6t277d0yQelnmv2TCr0sSJG9ed8tkwDDVxQQU0X4ywejOBFZEx4tImCgYeEthgTgTWBnYRIqSiW5FvZlQjhOk/9kA9mdJkP7SR/+lJDDDJP2PR9FeTJD+bqD4d0zY+5G6hsz/h+ogdelDpQ45PGcoXeqQwZO0Sx2dRd2YkMtIvASqg38UTLnCUaaAR5FR3ANRd0cqNQWFeulMHlWE8oU8eqwnyqcFof507FMVX4VHaCzqEIhNx2A8wOaAqEAcREkMTqb3mOHbMn7O8Kkcipfy0RksRX2ceHRJGCrCtBDkjnIoPLqj8kkaTaewGRQaVeGhgBQ8eGC+9ZOOTumi6yIjddJF43Q0R5xOOdv1ABBJZT9nuz+rYrDwqpLwpzPUFdidBWH3qtz5PeiSO+m6yEhddOK/p/Oj/Imui4wk/Q90gX9Pl9BJRvoLutg/i6qTTv6n5kg/6f7cvy66pL/TTEJIkSPBHvtGggjrUoSojhA1YN2KEPUQoj5CNECIxgjRBCH2QYimCLE/QhyAEC0RIqU/3dgznYBoIDqBHu4UD1/fCASSJohmACzWGFi8L6Lv6RFIofva4BhYljqiHcT0c3BwsMVSxogGjR+QgkycRiATBhGrxAixyg7AHsABQPJzaY/oDCCYQ5YI5xE47f57KZ1NmE0cbkKw0+v+8Ttg/sxKUGVpdWbZdWXpdWbZd2UNgix9LGu2LZDyxTwkvk9/Y+Kx5VH2mX4Icd4+wMloRQwJxjqG0ErGJjLUTFIzWVjkZmisScxGSNmIITqKoOyeTlef+/cUyBCVrAwjrAmjSeUFyBh1OzXse0QiYYyGnXbnF//YqwNloXo0R4oYslcZEVZq8TwYHPvQ5OgwWiInlQE3hcmSR4RJkzg8RnKKe2KqjwMjjuvERuOkUl5EvDdbSo+XsT3cPR3kqQxb2QRSvBPDV2Irn8Bh+rD5Yk+WiAULfp7Ih8oSUEXCBG8ZB3NDoVQEC09+lMAdX4o7CaiCCE82MzouimRvBytyZ+ZEWy5tojgiTJgY7kBJ4ogDYxh+Mb7BEY4wUPMgtFWGIU6MYDxM9cNDphMJHidYJBXmOxAyHPxwRIZCSUC3VRLQFUoCDwcGjgcrHpzOI5WkiiiUBbwUGLPkUKkiyCsxTpEkm0C1ladYO/pNEDj7OsXJKOwoL3xgYbPF3nwsJLFFsSI8w9NJFXorsBCWTyJlSBHgoadUogxZicrQG88nRYgpOEGEyImnDGNlyjCQrwyVFXA8Y3n8v+CKJOT5p3glBvfOjxVCA55SWCsHytieFJQlCsQYEnE8AyEvVoLpii2i8CNIHhRsfMTeumAvfjwCOOLYGJY9TRolnKiIcvC2DU/18gHZYbJlxDEwEaXArEpBd2fgcvQjuXtgmbagBMgMScFk6RfMsWfQ6FynAKp1qsw3yMF3As8hMViSJAoLEXlLoGEUJIJ1UBQhImErepaMg3HkiXMpiRIlQ54U5zLKE1alvh60CBEtkUVDxSjFkylUcmbvZBfFoVBJTClmQoKIML+YCNF4NJXi4RPtFRsTZc8SgYnbKkLkYN0COseTpuyZTIBZAfSCxwimOzCoJBRmqwLgnp6CP73iQlJ/dhPSHnS2I8PH2tnPXZCSGMV0kMd5C2LZoDoRW4CrjBSPRaJEuLwTIa7geEqhN7i1SLDHkZQmDnQMdwgURoVThBwRzTY63FsYbj9RxBL7CaMZJA+/OBrfyx/TYzBXkRhMT5KD8yVyqaJAXoR/ICsFOmjLCrNTsMK9Y6O9QlNYoROTwOVSWeHBFHe6E4nJZNmyRLS4aIY3tvRV8k3FXm3QvRS01Ah3CqdXr3t2mtTZa784gZKayqKHibwV2Eu+MFxpWOgUDz0SQE9RjqcTZknYGwO8p6QIdwXTzp8C63h/VA7rfwG0Dg9amFFw7KlCsH9aClPOwPwYmmbEcaAVD0ecnTg6tlwnwRyVgfGDOzrWiV5S7i3kFI6qaZWQSbwAijsnhQarZ0eqLSzmmXRPtifKC+B5p0b4hIFNhaNsgcJT4eIBxhoGUzOeLzOC6oTyWPEwkbJGSWI/nhR80wkXlYMfLioUe0GCresw1rClFfZS0AmXVCqjm6QCeWwwZVxUEbiokmXAowiz9SgQW3dRYZKyZ+CSonp6J8L6CWQlhse0E8mF4oXxAFT/lge0Bw8krL2/5iHwL3hIjqBGeKER2ByTp1zHkwJTHan+UWg41QlTfyJGC4MQpnY+G3/5FsiPwv03NgUzBY4nNnYAjgem3TjUjoQNooxgWIUGYYNphB0+UlBRRzAwnp/yzYwt9rrEi+frKKMwYC6BMij+TDo9AjMXJxeSUgTQAQ4l2htm1C6UGBrMaWEeA3PYFAoFm1xSvAIAV0aJ9+X5wgyK270yElYb9iLMPwYNTIV5cDjKSolwZwag4b36Cw8KVXfZfEeqM/bqxZEq8CWhwaijO4rrBRSU0r2m8KAIqsufasJcSFVTKNWRSlJVZe3bo6ZAMDNUWRMzHA2hRlD5AWhILF5TMFNVEzx/AlNJKh10Y6pnTYp/4InEwxYZcSjTvaum+GA0FGpKxGuisrGamOhUnXlqIUI/mKSjFBQuHwVKZfBQLBGEoo4UCENpzi4obQIvGKX6jmfx0DhqCIr6Bgd6o+7+DCi3J2GE/rD4Q90VGOF4FGWgTHqIGPWgxQmZFCrdA0WDxb5xqIcL1Z1JCXJ3QdEwBeDRFGiAgsIlUagK4N6HjnoGcFJQmKNKXVAOPZCJenkEUv/Pyp0zM/19HUudgbb4ZAPd5mzmCWSWVoL45SC+JdgFZekgAaDRwEByTfR1LDdLBzbZgSHu6BLo6JIN9Ix7lqM/mF8O5AMDOtIU2FGp8A9JruLg4BAuKzQ04IwTAQCfOh2g""")) diff --git a/lib/sqnsupgrade/sqnscodec.py b/lib/sqnsupgrade/sqnscodec.py new file mode 100644 index 0000000..9b4bdf0 --- /dev/null +++ b/lib/sqnsupgrade/sqnscodec.py @@ -0,0 +1,86 @@ +# -*- python -*- +################################################################# +# +# Module : CODEC +# Purpose: Base encoders/decoders +# +################################################################# +# +# Copyright (c) 2011 SEQUANS Communications. +# All rights reserved. +# +# This is confidential and proprietary source code of SEQUANS +# Communications. The use of the present source code and all +# its derived forms is exclusively governed by the restricted +# terms and conditions set forth in the SEQUANS +# Communications' EARLY ADOPTER AGREEMENT and/or LICENCE +# AGREEMENT. The present source code and all its derived +# forms can ONLY and EXCLUSIVELY be used with SEQUANS +# Communications' products. The distribution/sale of the +# present source code and all its derived forms is EXCLUSIVELY +# RESERVED to regular LICENCE holder and otherwise STRICTLY +# PROHIBITED. +# +################################################################# +import struct, array + +LITTLE_ENDIAN = "<" +NATIVE_ENDIAN = "=" +BIG_ENDIAN = ">" + +# -------------------------------------------------// Utility /__________________________________ +class encode: + @staticmethod + def u32 (value, endian = BIG_ENDIAN): + return array.array("c", struct.pack(endian + "I", value)) + + @staticmethod + def s32 (value, endian = BIG_ENDIAN): + if value < 0: + value = 0x100000000 + value + return encode.u32(value, endian) + + @staticmethod + def u16 (value, endian = BIG_ENDIAN): + return array.array("c", struct.pack(endian + "H", value)) + + @staticmethod + def u8 (value, endian = None): + return array.array("c", chr(value)) + + @staticmethod + def string (value, endian = None): + return array.array("c", value + "\x00") + +class decode: + @staticmethod + def u32 (value, endian = BIG_ENDIAN): + return struct.unpack(endian + "I", value)[0] + + @staticmethod + def s32 (value, endian = BIG_ENDIAN): + v = decode.u32(value, endian) + if v & (1 << 31): + return v - 0x100000000 + return v + + @staticmethod + def u16 (value, endian = BIG_ENDIAN): + return struct.unpack(endian + "H", value)[0] + + @staticmethod + def u8 (value, endian = None): + return ord(value) + + @staticmethod + def string (value, endian = None): + offset = 0 + str = "" + c = value[offset] + while c != '\x00': + offset += 1 + str += c + c = value[offset] + + return str + diff --git a/lib/sqnsupgrade/sqnscrc.py b/lib/sqnsupgrade/sqnscrc.py new file mode 100644 index 0000000..772528e --- /dev/null +++ b/lib/sqnsupgrade/sqnscrc.py @@ -0,0 +1,58 @@ +# -*- python -*- +################################################################# +# +# Module : CRC +# Purpose: CRC calculation +# +################################################################# +# +# Copyright (c) 2011 SEQUANS Communications. +# All rights reserved. +# +# This is confidential and proprietary source code of SEQUANS +# Communications. The use of the present source code and all +# its derived forms is exclusively governed by the restricted +# terms and conditions set forth in the SEQUANS +# Communications' EARLY ADOPTER AGREEMENT and/or LICENCE +# AGREEMENT. The present source code and all its derived +# forms can ONLY and EXCLUSIVELY be used with SEQUANS +# Communications' products. The distribution/sale of the +# present source code and all its derived forms is EXCLUSIVELY +# RESERVED to regular LICENCE holder and otherwise STRICTLY +# PROHIBITED. +# +################################################################# +import sqnscodec as codec + +# -------------------------------------------------// Fletcher /_________________________________ +def fletcher32 (data): + l = len(data) + + index = 0 + s1 = s2 = 0xFFFF + while l > 1: + qty = 720 if l > 720 else (l & ~1) + l -= qty + + qty += index + while index < qty: + word = codec.decode.u16(data[index:index+2]) + s1 += word + s2 += s1 + + index += 2 + + s1 = (s1 & 0xFFFF) + (s1 >> 16) + s2 = (s2 & 0xFFFF) + (s2 >> 16) + + if (l & 1): + s1 += ord(data[index]) << 8 + s2 += s1 + + s1 = (s1 & 0xFFFF) + (s1 >> 16) + s2 = (s2 & 0xFFFF) + (s2 >> 16) + + s1 = (s1 & 0xFFFF) + (s1 >> 16) + s2 = (s2 & 0xFFFF) + (s2 >> 16) + + return (s2 << 16) | s1 diff --git a/lib/sqnsupgrade/sqnstp.py b/lib/sqnsupgrade/sqnstp.py new file mode 100755 index 0000000..31280cf --- /dev/null +++ b/lib/sqnsupgrade/sqnstp.py @@ -0,0 +1,414 @@ +#!/usr/bin/env python +################################################################# +# +# Copyright (c) 2011 SEQUANS Communications. +# All rights reserved. +# +# This is confidential and proprietary source code of SEQUANS +# Communications. The use of the present source code and all +# its derived forms is exclusively governed by the restricted +# terms and conditions set forth in the SEQUANS +# Communications' EARLY ADOPTER AGREEMENT and/or LICENCE +# AGREEMENT. The present source code and all its derived +# forms can ONLY and EXCLUSIVELY be used with SEQUANS +# Communications' products. The distribution/sale of the +# present source code and all its derived forms is EXCLUSIVELY +# RESERVED to regular LICENCE holder and otherwise STRICTLY +# PROHIBITED. +# +################################################################# +import struct +import time +import os + +try: + sysname = os.uname().sysname +except: + sysname = 'Windows' + +# CRC-16(CCIT) +def crc16(s): + crc = 0x0000 + table = [0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50a5, 0x60c6, 0x70e7, + 0x8108, 0x9129, 0xa14a, 0xb16b, 0xc18c, 0xd1ad, 0xe1ce, 0xf1ef, + 0x1231, 0x0210, 0x3273, 0x2252, 0x52b5, 0x4294, 0x72f7, 0x62d6, + 0x9339, 0x8318, 0xb37b, 0xa35a, 0xd3bd, 0xc39c, 0xf3ff, 0xe3de, + 0x2462, 0x3443, 0x0420, 0x1401, 0x64e6, 0x74c7, 0x44a4, 0x5485, + 0xa56a, 0xb54b, 0x8528, 0x9509, 0xe5ee, 0xf5cf, 0xc5ac, 0xd58d, + 0x3653, 0x2672, 0x1611, 0x0630, 0x76d7, 0x66f6, 0x5695, 0x46b4, + 0xb75b, 0xa77a, 0x9719, 0x8738, 0xf7df, 0xe7fe, 0xd79d, 0xc7bc, + 0x48c4, 0x58e5, 0x6886, 0x78a7, 0x0840, 0x1861, 0x2802, 0x3823, + 0xc9cc, 0xd9ed, 0xe98e, 0xf9af, 0x8948, 0x9969, 0xa90a, 0xb92b, + 0x5af5, 0x4ad4, 0x7ab7, 0x6a96, 0x1a71, 0x0a50, 0x3a33, 0x2a12, + 0xdbfd, 0xcbdc, 0xfbbf, 0xeb9e, 0x9b79, 0x8b58, 0xbb3b, 0xab1a, + 0x6ca6, 0x7c87, 0x4ce4, 0x5cc5, 0x2c22, 0x3c03, 0x0c60, 0x1c41, + 0xedae, 0xfd8f, 0xcdec, 0xddcd, 0xad2a, 0xbd0b, 0x8d68, 0x9d49, + 0x7e97, 0x6eb6, 0x5ed5, 0x4ef4, 0x3e13, 0x2e32, 0x1e51, 0x0e70, + 0xff9f, 0xefbe, 0xdfdd, 0xcffc, 0xbf1b, 0xaf3a, 0x9f59, 0x8f78, + 0x9188, 0x81a9, 0xb1ca, 0xa1eb, 0xd10c, 0xc12d, 0xf14e, 0xe16f, + 0x1080, 0x00a1, 0x30c2, 0x20e3, 0x5004, 0x4025, 0x7046, 0x6067, + 0x83b9, 0x9398, 0xa3fb, 0xb3da, 0xc33d, 0xd31c, 0xe37f, 0xf35e, + 0x02b1, 0x1290, 0x22f3, 0x32d2, 0x4235, 0x5214, 0x6277, 0x7256, + 0xb5ea, 0xa5cb, 0x95a8, 0x8589, 0xf56e, 0xe54f, 0xd52c, 0xc50d, + 0x34e2, 0x24c3, 0x14a0, 0x0481, 0x7466, 0x6447, 0x5424, 0x4405, + 0xa7db, 0xb7fa, 0x8799, 0x97b8, 0xe75f, 0xf77e, 0xc71d, 0xd73c, + 0x26d3, 0x36f2, 0x0691, 0x16b0, 0x6657, 0x7676, 0x4615, 0x5634, + 0xd94c, 0xc96d, 0xf90e, 0xe92f, 0x99c8, 0x89e9, 0xb98a, 0xa9ab, + 0x5844, 0x4865, 0x7806, 0x6827, 0x18c0, 0x08e1, 0x3882, 0x28a3, + 0xcb7d, 0xdb5c, 0xeb3f, 0xfb1e, 0x8bf9, 0x9bd8, 0xabbb, 0xbb9a, + 0x4a75, 0x5a54, 0x6a37, 0x7a16, 0x0af1, 0x1ad0, 0x2ab3, 0x3a92, + 0xfd2e, 0xed0f, 0xdd6c, 0xcd4d, 0xbdaa, 0xad8b, 0x9de8, 0x8dc9, + 0x7c26, 0x6c07, 0x5c64, 0x4c45, 0x3ca2, 0x2c83, 0x1ce0, 0x0cc1, + 0xef1f, 0xff3e, 0xcf5d, 0xdf7c, 0xaf9b, 0xbfba, 0x8fd9, 0x9ff8, + 0x6e17, 0x7e36, 0x4e55, 0x5e74, 0x2e93, 0x3eb2, 0x0ed1, 0x1ef0] + for ch in s: + crc = ((crc<<8)&0xff00) ^ table[((crc>>8)&0xff)^ch] + return crc + +def usleep(x): + time.sleep(x/1000000.0) + +def hexdump(src, length=32): + if len(src) == 0: + return + src = src[:length] + FILTER = ''.join([(len(repr(chr(x))) == 3) and chr(x) or '.' for x in range(256)]) + lines = [] + for c in range(0, len(src), length): + chars = src[c:c+length] + hex = ' '.join(["%02x" % ord(x) for x in chars]) + printable = ''.join(["%s" % ((ord(x) <= 127 and FILTER[ord(x)]) or '.') for x in chars]) + lines.append("%04x %-*s %s\n" % (c, length*3, hex, printable)) + print(''.join(lines)) + + +class MException(BaseException): + def __init__(self, s): + self.s = s + def __str__(self): + return self.s + +class SerialDev(object): + def __init__(self, serial, baud, timeout=90000): # 90 seconds timeout + self.serial = serial + self.timeout = timeout + + def read(self, n): + global sysname + _n = n + t = self.timeout + r = b'' + while t > 0: + c = self.serial.read(_n) + if c: + r += c + if len(r) == n: + break + _n -= len(c) + if 'FiPy' in sysname or 'GPy' in sysname: + time.sleep_ms(2) + else: + time.sleep(0.002) + t -= 2 + return r + + def write(self, s): + self.serial.write(s) + + def devastate(self): + self.serial.read() + + def close(self): + self.serial.close() + + def set_timeout(self, timeout): + self.timeout = timeout * 1000 + + +class Master: + RESET = 0 + SESSION_OPEN = 1 + TRANSFER_BLOCK_CMD = 2 + TRANSFER_BLOCK = 3 + + MREQH = b">IBBHIHH" + SRSPH = b">IBBHIHH" + SRSP_SESSION_OPEN = b">BBH" + SRSP_TRANSFER_BLOCK = b">H" + + MREQH_SIZE = struct.calcsize(MREQH) + SRSPH_SIZE = struct.calcsize(SRSPH) + SRSP_SESSION_OPEN_SIZE = struct.calcsize(SRSP_SESSION_OPEN) + SRSP_TRANSFER_BLOCK_SIZE = struct.calcsize(SRSP_TRANSFER_BLOCK) + + MREQ_SIGNATURE = 0x66617374 + SRSP_SIGNATURE = 0x74736166 + + def __init__(self, dev, debug=False, pkgdebug=False): + self.sid = 0 + self.tid = 0 + self.dev = dev + self.debug = debug + self.pkgdebug = pkgdebug + self.mreq = [] + self.srsp = [] + self.version = 1 + self.max_transfer = 16 + + @staticmethod + def mreq_ack(op): + return op | 0x80 + + def wipe(self): + self.dev.devastate() + + def read(self, n): + r = self.dev.read(n) + if self.pkgdebug: + print("IN") + hexdump(r) + return r + + + def write(self, s): + self.dev.write(s) + if self.pkgdebug: + print("OUT") + # hexdump(s.decode('ascii')) + + + def make_mreq(self, op, pld): + assert self.MREQH_SIZE + len(pld) <= self.max_transfer + + if len(pld) != 0: + pcrc = crc16(pld) + else: + pcrc = 0 + hcrc = crc16(struct.pack(self.MREQH, + self.MREQ_SIGNATURE, + op, self.sid, len(pld), + self.tid, + 0, pcrc)) + return struct.pack(self.MREQH, + self.MREQ_SIGNATURE, + op, self.sid, len(pld), + self.tid, + hcrc, pcrc) + + + def decode_srsp(self, p, show=False): + if len(p) < self.SRSPH_SIZE: + raise MException("SRSP header too small: %d" % len(p)) + + (magic, op, sid, plen, tid, hcrc, pcrc) = struct.unpack(self.SRSPH, p[:self.SRSPH_SIZE]) + if show and self.debug: + print('magic=0x%08X, op=0x%X, sid=0x%X, plen=0x%X, tid=0x%X, hcrc=0x%X, pcrc=0x%X' % (magic, op, sid, plen, tid, hcrc, pcrc)) + + if magic != self.SRSP_SIGNATURE: + print("Wrong SRSP signature: 0x%08X" % magic) + #raise MException("Wrong SRSP signature: 0x%08X" % magic) + elif show and self.debug: + print("Correct SRSP signature: 0x%08X" % magic) + + if hcrc != 0: + chcrc = crc16(struct.pack(self.SRSPH, self.SRSP_SIGNATURE, op, sid, plen, tid, 0, pcrc)) + if hcrc != chcrc: + raise MException("Wrong header CRC: 0x%04X" % hcrc) + + return dict(op=op, sid=sid, tid=tid, plen=plen, pcrc=pcrc) + + + def verify_srsp_data(self, p, plen, pcrc): + if len(p) != plen: + raise MException("Wrong payload size: %d" % plen) + if plen != 0 and pcrc != 0 and pcrc != crc16(p): + raise MException("Wrong payload CRC: 0x%04X" % pcrc) + + + def verify_session(self, i, op): + if i['op'] != Master.mreq_ack(op): + raise MException("Invalid op: 0x%02x" % i['op']) + if i['sid'] != self.sid: + raise MException("Invalid sid: %d" % i['sid']) + if i['tid'] != self.tid: + raise MException("Invalid sid: %d" % i['tid']) + + + def decode_open_session(self, p): + if len(p) < self.SRSP_SESSION_OPEN_SIZE: + raise MException("OpenSession data too small: %d" % len(p)) + (ok, ver, mts) = struct.unpack(self.SRSP_SESSION_OPEN, p[:self.SRSP_SESSION_OPEN_SIZE]) + if not ok: + raise MException("OpenSession: failed to open") + + self.version = ver + self.max_transfer = mts + print("Session opened: version %d, max transfer %s bytes" % (ver, mts)) + + + def reset(self, closing=False): + self.write(self.make_mreq(self.RESET, [])) + r = self.read(self.SRSPH_SIZE) + if closing: + return + i = self.decode_srsp(r, show=True) + if i['op'] != Master.mreq_ack(self.RESET): + raise MException("Reset: invalid op: 0x%02x" % i['op']) + + self.sid = 0 + self.tid = 0 + + + def open_session(self): + self.sid = 1 + self.tid = 1 + self.write(self.make_mreq(self.SESSION_OPEN, [])) + r = self.read(self.SRSPH_SIZE) + i = self.decode_srsp(r) + self.verify_session(i, self.SESSION_OPEN) + r = self.read(self.SRSP_SESSION_OPEN_SIZE) + self.verify_srsp_data(r, i['plen'], i['pcrc']) + self.decode_open_session(r) + self.tid += 1 + + + def send_data(self, blobfile, filesize, trials=4, bootrom=None): + global sysname + + class Trial: + def __init__(self, trials): + self.trials = trials + def need_retry(self, c, *a, **k): + try: + c(*a, **k) + except MException: + self.trials -= 1 + if self.trials > 0: return True + else: raise + return False + + trial = Trial(trials) + + downloaded = 0 + + while True: + # if 'FiPy' in sysname or 'GPy' in sysname: + # data = blobfile.read(1536) + # else: + # #data = blobfile.read(512) + # data = blobfile.read(768) + data = blobfile.read(2048) + size = len(data) + if size: + while size: + l = min(size, self.max_transfer-self.MREQH_SIZE) + l = min(l, 2048 - 32) # 31x0 mii limitation + + trials = 4 + while True: + pld = struct.pack(">H", l) + self.write(self.make_mreq(self.TRANSFER_BLOCK_CMD, pld)) + self.write(pld) + try: + r = self.read(self.SRSPH_SIZE) + i = self.decode_srsp(r) + except MException: + trials -= 1 + if not trials: raise + continue + break + + if trial.need_retry(self.verify_session, i, self.TRANSFER_BLOCK_CMD): continue + self.tid += 1 + + trials = 4 + while True: + pld = data[:l] + self.write(self.make_mreq(self.TRANSFER_BLOCK, pld)) + self.write(pld) + try: + r = self.read(self.SRSPH_SIZE) + i = self.decode_srsp(r) + except MException: + trials -= 1 + if not trials: raise + continue + if trial.need_retry(self.verify_session, i, self.TRANSFER_BLOCK): continue + r = self.read(self.SRSP_TRANSFER_BLOCK_SIZE) + break + if trial.need_retry(self.verify_srsp_data, r, i['plen'], i['pcrc']): continue + self.tid += 1 + + (residue, ) = struct.unpack(">H", r) + if residue > 0: + print("Slave didn't consume %d bytes" % residue) + l -= residue + + data = data[l:] + size -= l + downloaded += l + self.progress("Sending %d bytes" % filesize, downloaded, filesize) + else: + break + + blobfile.close() + self.progressComplete() + + return True + + + def progress(self, what, downloaded, total, barLen=40): + percent = float(downloaded)/total + hashes = '#' * int(round(percent*barLen)) + spaces = ' ' * (barLen - len(hashes)) + if 'FiPy' in sysname or 'GPy' in sysname: + print('\r%s: [%s%s] %3d%%' % (what, hashes, spaces, int(round(percent*100))), end='') + else: + print('\r%s: [%s%s] %3d%%' % (what, hashes, spaces, int(round(percent*100))), end='', flush=True) + + + def progressComplete(self): + print() + + +class args(object): + pass + +def start(elf, elfsize, serial, baud=3686400, retry=None, debug=None, AT=True, pkgdebug=False): + dev = None + + try: + # The base-two logarithm of the window size, which therefore ranges between 512 and 32768 + # 12 is 4096K + wbits = 12 + dev = SerialDev(serial, baud) + push = lambda m: m.send_data(elf, elfsize) + except: + raise + + time.sleep(0.05) + m = Master(dev, debug=debug, pkgdebug=pkgdebug) + + while True: + try: + if debug: print('running m.wipe') + m.wipe() + if debug: print('running m.reset') + m.reset() + if debug: print('running m.open_session') + m.open_session() + if debug: print('running push(m)') + push(m) + if debug: print('running dev.set_timeout(2)') + dev.set_timeout(2) + if debug: print('running m.reset(True)') + m.reset(True) + return True + except MException as ex: + print(str(ex)) + if retry: + continue + else: + return False + break + return False diff --git a/lib/sqnsupgrade/sqnsupgrade.py b/lib/sqnsupgrade/sqnsupgrade.py new file mode 100644 index 0000000..7c8db01 --- /dev/null +++ b/lib/sqnsupgrade/sqnsupgrade.py @@ -0,0 +1,1080 @@ +#!/usr/bin/env python +VERSION = "1.2.7" + +# Copyright (c) 2021, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + + +import struct +import time +import os +import sys +import sqnscrc as crc +import sqnstp as stp + +release = None + +try: + sysname = os.uname().sysname + if 'FiPy' in sysname or 'GPy' in sysname: + release = os.uname().release +except: + sysname = 'Windows' + +if 'FiPy' in sysname or 'GPy' in sysname: + from machine import UART + from machine import SD + from network import LTE + + def reconnect_uart(): + if hasattr(LTE, 'reconnect_uart'): + LTE.reconnect_uart() +else: # this is a computer + import serial + def reconnect_uart(): + pass + +class sqnsupgrade: + + global sysname + + def __init__(self): + + self.__sysname = sysname + self.__pins = None + self.__connected = False + self.__sdpath = None + self.__resp_921600 = False + self.__serial = None + self.__kill_ppp_ok = False + self.__modem_speed = None + self.__speed_detected = False + + if 'GPy' in self.__sysname: + self.__pins = ('P5', 'P98', 'P7', 'P99') + else: + self.__pins = ('P20', 'P18', 'P19', 'P17') + + + def special_print(self, msg, flush=None, end='\n'): + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + print(msg, end=end) + else: + print(msg, flush=flush, end=end) + + def read_rsp(self, size=None, timeout=-1): + time.sleep(.25) + if timeout < 0: + timeout = 20000 + elif timeout is None: + timeout = 0 + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + while not self.__serial.any() and timeout > 0: + time.sleep_ms(1) + timeout -= 1 + else: + while self.__serial.in_waiting <= 0 and timeout > 0: + time.sleep(0.001) + timeout -= 1 + + if size is not None: + rsp = self.__serial.read(size) + else: + rsp = self.__serial.read() + if rsp is not None: + return rsp + else: + return b'' + + def print_pretty_response(self, rsp, flush=False, prefix=None): + if prefix is not None: self.special_print(prefix, flush=flush, end=' ') + lines = rsp.decode('ascii').split('\r\n') + for line in lines: + if 'OK' not in line and line!='': + self.special_print(line, flush=flush) + + + def return_pretty_response(self, rsp): + ret_str = '' + lines = rsp.decode('ascii').split('\r\n') + for line in lines: + if 'OK' not in line: + ret_str += line + return ret_str + + def return_upgrade_response(self, rsp): + pretty = self.return_pretty_response(rsp) + if "+SMUPGRADE:" in pretty: + try: + return pretty.split(':')[1].strip() + except: + pass + return None + + def return_code(self, rsp, debug=False): + ret_str = b'' + lines = rsp.decode('ascii').split('\r\n') + for line in lines: + if 'OK' not in line and len(line) >0: + try: + if debug: print('Converting response: {} to int...'.format(line)) + return int(line) + except: + pass + raise OSError('Could not decode modem state') + + + def wait_for_modem(self, send=True, expected=b'OK', echo_char=None): + self.__serial.read() + rsp = b'' + start = time.time() + while True: + if send: + self.__serial.write(b"AT\r\n") + r = self.read_rsp(size=(len(expected) + 4), timeout=50) + if r: + rsp += r + if expected in rsp: + if echo_char is not None: + print() + break + else: + if echo_char is not None: + self.special_print(echo_char, end='', flush=True) + time.sleep(0.5) + if time.time() - start >= 300: + raise OSError('Timeout waiting for modem to respond!') + + def __check_file(self, file_path, debug=False): + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + if file_path[0] == '/' and not 'flash' in file_path and not file_path.split('/')[1] in os.listdir('/'): + if self.__sdpath is None: + self.__sdpath = file_path.split('/')[1] + try: + sd = SD() + time.sleep(0.5) + os.mount(sd, '/{}'.format(self.__sdpath)) + except Exception as ex: + print('Unable to mount SD card!') + return False + else: + print('SD card already mounted on {}!'.format(self.__sdpath)) + return False + try: + size = os.stat(file_path)[6] + if debug: print('File {} has size {}'.format(file_path, size)) + return True + except Exception as ex: + print('Exception when checking file {}... wrong file name?'.format(file_path)) + print('{}'.format(ex)) + return False + return False + + + def check_files(self, ffile, mfile=None, debug=False): + if mfile is not None: + if self.__check_file(mfile, debug): + return self.__check_file(ffile, debug) + else: + return False + else: + return self.__check_file(ffile, debug) + + + def __check_resp(self, resp, kill_ppp=False): + if resp is not None: + self.__resp_921600 = b'OK' in resp or b'ERROR' in resp + self.__kill_ppp_ok = self.__kill_ppp_ok or (kill_ppp and b'OK' in resp) + + def __hangup_modem(self, delay, debug): + self.__serial.read() + if not self.__kill_ppp_ok: + self.__serial.write(b"+++") + time.sleep_ms(1150) + resp = self.__serial.read() + if debug: print('Response (+++ #1): {}'.format(resp)) + self.__check_resp(resp, True) + self.__serial.write(b"AT\r\n") + time.sleep_ms(250) + resp = self.__serial.read() + if debug: print('Response (AT #1) {}'.format(resp)) + self.__check_resp(resp) + if resp is not None: + if b'OK' not in resp and not self.__kill_ppp_ok: + self.__serial.write(b"AT\r\n") + time.sleep_ms(250) + resp = self.__serial.read() + if debug: print('Response (AT #2) {}'.format(resp)) + self.__check_resp(resp) + if resp is not None and b'OK' in resp: + return True + self.__serial.write(b"+++") + time.sleep_ms(1150) + resp = self.__serial.read() + if debug: print('Response (+++ #2): {}'.format(resp)) + self.__check_resp(resp, True) + if resp is not None and b'OK' in resp: + self.__serial.write(b"AT\r\n") + time.sleep_ms(250) + resp = self.__serial.read() + if debug: print('Response (AT #2) {}'.format(resp)) + self.__check_resp(resp) + if resp is not None and b'OK' in resp: + return True + return False + + + def detect_modem_state(self, retry=5, initial_delay=1000, hangup=True, debug=False): + count = 0 + self.__serial = UART(1, baudrate=921600, pins=self.__pins, timeout_chars=1) + self.__modem_speed = 921600 + self.__serial.read() + while count < retry: + count += 1 + delay = initial_delay * count + if debug: print("The current delay is {}".format(delay)) + self.__serial = UART(1, baudrate=921600, pins=self.__pins, timeout_chars=10) + self.__modem_speed = 921600 + #if True: + if hangup and self.__hangup_modem(initial_delay, debug): + self.__speed_detected = True + self.__serial.write(b"AT+SMOD?\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + if debug: print('Response (AT+SMOD?) {}'.format(resp)) + try: + return self.return_code(resp, debug) + except: + pass + else: + self.__modem_speed = 921600 + self.__serial = UART(1, baudrate=921600, pins=self.__pins, timeout_chars=1) + self.__serial.read() + self.__serial.write(b"AT\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + self.__check_resp(resp) + if debug: print('Response (AT #3) {}'.format(resp)) + if resp is not None and b'OK' in resp: + self.__speed_detected = True + self.__serial.write(b"AT+SMOD?\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + try: + if debug: print('Response (AT+SMOD?) {}'.format(resp)) + return self.return_code(resp, debug) + except: + pass + self.__serial.write(b"AT\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + self.__check_resp(resp) + if debug: print('Response (AT #4) {}'.format(resp)) + if resp is not None and b'OK' in resp: + self.__speed_detected = True + self.__serial.write(b"AT+SMOD?\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + try: + return self.return_code(resp, debug) + if debug: print('Response (AT+SMOD?) {}'.format(resp)) + except: + pass + else: + if not self.__resp_921600: + self.__modem_speed = 115200 + self.__serial = UART(1, baudrate=115200, pins=self.__pins, timeout_chars=10) + self.__serial.write(b"AT\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + if debug: print('Response (AT #1 @ 115200) {}'.format(resp)) + if resp is not None and b'OK' in resp: + self.__speed_detected = True + self.__serial.write(b"AT+SMOD?\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + try: + if debug: print('Response (AT+SMOD?) {}'.format(resp)) + return self.return_code(resp, debug) + except: + pass + self.__serial.write(b"AT\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + if debug: print('Response (AT #2 @ 115200) {}'.format(resp)) + if resp is not None and b'OK' in resp: + self.__speed_detected = True + self.__serial.write(b"AT+SMOD?\r\n") + time.sleep_ms(delay) + resp = self.__serial.read() + try: + if debug: print('Response (AT+SMOD?) {}'.format(resp)) + return self.return_code(resp, debug) + except: + pass + return None + + def get_imei(self): + self.__serial = UART(1, baudrate=921600, pins=self.__pins, timeout_chars=10) + self.__serial.write(b"AT+CGSN\r\n") + time.sleep(.5) + imei_val = self.read_rsp(2000) + return self.return_pretty_response(imei_val) + + + def __get_power_warning(self): + return "<<<=== DO NOT DISCONNECT POWER ===>>>" + + def __get_wait_msg(self, load_fff=True): + if not self.__wait_msg: + self.__wait_msg = True + if load_fff: + return "Waiting for modem to finish the update...\nThis might take several minutes!\n" + self.__get_power_warning() + else: + return "Waiting for modem to finish the update...\n" + self.__get_power_warning() + return None + + + + def __run(self, file_path=None, baudrate=921600, port=None, resume=False, load_ffh=False, mirror=False, switch_ffh=False, bootrom=False, rgbled=0x050505, debug=False, pkgdebug=False, atneg=True, max_try=10, direct=True, atneg_only=False, info_only=False, expected_smod=None, verbose=False, load_fff=False, mtools=False, fc=False, force_fff=False): + self.__wait_msg = False + mirror = True if atneg_only else mirror + recover = True if atneg_only else load_ffh + resume = True if mirror or recover or atneg_only or info_only else resume + verbose = True if debug else verbose + load_fff = False if bootrom and switch_ffh else load_fff + load_fff = True if force_fff else load_fff + target_baudrate = baudrate + baudrate = self.__modem_speed if self.__speed_detected else baudrate + if debug: print('file_path? {} mirror? {} recover? {} resume? {} direct? {} atneg_only? {} bootrom? {} load_fff? {}'.format(file_path, mirror, recover, resume, direct, atneg_only, bootrom, load_fff)) + if debug: print('baudrate: {} target_baudrate: {}'.format(baudrate, target_baudrate)) + abort = True + external = False + self.__serial = None + + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + + self.__serial = UART(1, baudrate=115200 if recover and not self.__speed_detected else baudrate, pins=self.__pins, timeout_chars=100) + self.__serial.read() + else: + if port is None: + raise ValueError('serial port not specified') + if debug: print('Setting port {}'.format(port)) + external = True + br = 115200 if recover and not direct else baudrate + if debug: print('Setting baudrate to {}'.format(br)) + self.__serial = serial.Serial(port, br, bytesize=serial.EIGHTBITS, timeout=1 if info_only else 0.1, rtscts=fc) + self.__serial.reset_input_buffer() + self.__serial.reset_output_buffer() + + if info_only: + self.__serial.read() + self.__serial.write(b'AT\r\n') + self.__serial.write(b'AT\r\n') + self.__serial.read() + self.__serial.write(b"AT+CGSN\r\n") + time.sleep(.5) + shimei = self.read_rsp(2000) + if verbose: + self.__serial.write(b"AT!=\"showver\"\r\n") + else: + self.__serial.write(b"ATI1\r\n") + time.sleep(.5) + shver = self.read_rsp(2000) + if shver is not None: + self.print_pretty_response(shver) + if shimei is not None: + self.print_pretty_response(shimei, prefix='\nIMEI:') + return True + + if debug: print('Initial prepartion complete...') + + if not mirror: + if bootrom: + if debug: print('Loading built-in recovery bootrom...') + try: + # try compressed bootrom first + from sqnsbrz import bootrom + except: + # fallback to uncompressed + try: + from sqnsbr import bootrom + except: + print('This firmware does not contain a recovery bootrom.') + return False + blob = bootrom() + blobsize = blob.get_size() + else: + if debug: print('Loading {}'.format(file_path)) + blobsize = os.stat(file_path)[6] + if blobsize < 128: + print('Firmware file is too small!') + reconnect_uart() + return False + if blobsize > 4194304: + if load_fff and not force_fff: + print("Firmware file is too big to load via FFF method. Using ON_THE_FLY") + load_fff = False + blob = open(file_path, "rb") + + if not load_ffh: + if not self.wakeup_modem(baudrate, port, 10, 1, debug): + return False + + if (not resume) or mtools: + + # bind to AT channel + self.__serial.write(b"AT+BIND=AT\r\n") + time.sleep(.5) + response = self.read_rsp(size=100) + if debug: print("AT+BIND=AT returned {}".format(response)) + + # disable echo + self.__serial.write(b"ATE0\r\n") + time.sleep(.5) + response = self.read_rsp(size=100) + if debug: print("ATE0 returned {}".format(response)) + + self.__serial.read() + if debug: print('Entering upgrade mode...') + + if verbose: print("Sending AT+SMLOG?") + self.__serial.write(b'AT+SMLOG?\r\n') + response = self.read_rsp(size=100) + if verbose: print("AT+SMLOG? returned {}".format(response)) + + self.__serial.write(b"AT+SMOD?\r\n") + response = self.return_pretty_response(self.read_rsp(size=7)) + if debug: print("AT+SMOD? returned {}".format(response)) + + self.__serial.write(b"AT+SQNSUPGRADENTF=\"started\"\r\n") + response = self.read_rsp(size=100) + if verbose: print('AT+SQNSUPGRADENTF="started" returned {}'.format(response)) + self.wait_for_modem() + + if verbose: print('Sending AT+SQNWL="sqndcc",2') + self.__serial.write(b'AT+SQNWL="sqndcc",2\r\n') + response = self.read_rsp(size=100) + if verbose: print('AT+SQNWL="sqndcc",2 returned {}'.format(response)) + self.__serial.read(100) + + if verbose: print("Sending AT+CFUN=4") + self.__serial.write(b'AT+CFUN=4\r\n') + response = self.read_rsp(size=100) + if verbose: print("AT+CFUN=4 returned {}".format(response)) + self.__serial.read(100) + + if not (load_fff or mtools): + self.__serial.write(b"AT+SMSWBOOT=3,1\r\n") + resp = self.read_rsp(100) + if debug: print('AT+SMSWBOOT=3,1 returned: {}'.format(resp)) + if b'ERROR' in resp: + time.sleep(5) + self.__serial.write(b"AT+SMSWBOOT=3,0\r\n") + resp = self.read_rsp(100) + if debug: print('AT+SMSWBOOT=3,0 returned: {}'.format(resp)) + if b'OK' in resp: + self.__serial.write(b"AT^RESET\r\n") + resp = self.read_rsp(100) + if debug: print('AT^RESET returned: {}'.format(resp)) + else: + print('Received ERROR from AT+SMSWBOOT=3,1! Aborting!') + reconnect_uart() + return False + time.sleep(3) + resp = self.__serial.read() + if debug: print("Response after reset: {}".format(resp)) + self.wait_for_modem() + self.__serial.write(b"AT\r\n") + + if verbose: print("Sending AT+CFUN=4") + self.__serial.write(b'AT+CFUN=4\r\n') + response = self.read_rsp(size=100) + if verbose: print("AT+CFUN=4 returned {}".format(response)) + + if verbose: print("Sending AT+SMLOG?") + self.__serial.write(b'AT+SMLOG?\r\n') + response = self.read_rsp(size=100) + if verbose: print("AT+SMLOG? returned {}".format(response)) + + + else: + self.__serial.read(100) + if debug: print('Entering recovery mode') + + self.__serial.write(b"AT+SMOD?\r\n") + response = self.return_pretty_response(self.read_rsp(size=7)) + self.__serial.read(100) + if debug: print("AT+SMOD? returned {}".format(response)) + + time.sleep(1) + self.__serial.read() + + if (not recover) and (not direct): + if mirror: + time.sleep(.5) + self.__serial.read(100) + print('Going into MIRROR mode... please close this terminal to resume the upgrade via UART') + return self.uart_mirror(rgbled) + + elif bootrom: + if verbose: print('Starting STP [BR]') + else: + if verbose: + if load_fff: + print('Starting STP [FFF]') + else: + print('Starting STP ON_THE_FLY') + + self.__serial.read(100) + + if load_fff: + if debug: print("Sending AT+SMSTPU") + self.__serial.write(b'AT+SMSTPU\r\n') + else: + if debug: print("Sending AT+SMSTPU=\"ON_THE_FLY\"") + self.__serial.write(b'AT+SMSTPU=\"ON_THE_FLY\"\r\n') + + response = self.read_rsp(size=4) + if response != b'OK\r\n' and response != b'\r\nOK' and response != b'\nOK': + raise OSError("Invalid answer '%s' from the device" % response) + try: + blob.close() + except Exception as ex: + if debug: print('Exception: {}'.format(ex)) + pass + self.__serial.read() + elif recover and (not direct): + if atneg: + result = self.at_negotiation(baudrate, port, max_try, mirror, atneg_only, debug, target_baudrate) + if result: + baudrate = target_baudrate + self.__modem_speed = target_baudrate + self.__speed_detected = True + if atneg_only: + return True + if mirror: + time.sleep(.5) + self.__serial.read(100) + print('Going into MIRROR mode... please close this terminal to resume the upgrade via UART') + return self.uart_mirror(rgbled) + else: + self.__serial.write(b"AT+STP\n") + response = self.read_rsp(size=2) + if not b'OK' in response: + print('Failed to start STP mode!') + reconnect_uart() + return False + else: + print('AT auto-negotiation failed! Exiting.') + return False + else: + if debug: print('Starting STP mode...') + self.__serial.write(b"AT+STP\n") + response = self.read_rsp(size=4) + if not b'OK' in response: + print('Failed to start STP mode!') + reconnect_uart() + return False + + try: + if debug: + if verbose: print('Starting STP code upload with pkgdebug={}'.format(pkgdebug)) + time.sleep(.1) + self.__serial.read() + time.sleep(.1) + start = stp.start(blob, blobsize, self.__serial, baudrate, AT=False, debug=debug, pkgdebug=pkgdebug) + if debug: print('start returned {} type {}'.format(start, type(start))) + if start == True: + try: + blob.close() + except Exception as ex: + if debug: print('Exception: {}'.format(ex)) + pass + self.__serial.read() + if switch_ffh: + if verbose: print('Bootrom updated successfully, switching to recovery mode') + abort = False + elif load_ffh: + if not self.wakeup_modem(baudrate, port, 100, 1, debug,'Waiting for updater to load...'): + return False + if verbose: print('Upgrader loaded successfully, modem is in update mode') + return True + else: + if verbose: print('Code download done, returning to user mode') + abort = recover + else: + try: + blob.close() + except Exception as ex: + if debug: print('Exception: {}'.format(ex)) + pass + print('Code download failed[1], aborting!') + return False + except Exception as ex: + try: + blob.close() + except Exception as ex: + if debug: print('Exception: {}'.format(ex)) + pass + + print('Exception: {}'.format(ex)) + print('Code download failed [2], aborting!') + abort = True + + time.sleep(1.5) + + if not abort: + self.__serial.read() + if switch_ffh: + self.__serial.write(b"AT+SMSWBOOT=0,1\r\n") + resp = self.read_rsp(100) + if debug: print("AT+SMSWBOOT=0,1 returned {}".format(resp)) + if b"ERROR" in resp: + time.sleep(5) + self.__serial.write(b"AT+SMSWBOOT=0,0\r\n") + resp = self.read_rsp(100) + if debug: print('AT+SMSWBOOT=0,0 returned: {}'.format(resp)) + if b'OK' in resp: + self.__serial.write(b"AT^RESET\r\n") + resp = self.read_rsp(100) + if debug: print('AT^RESET returned: {}'.format(resp)) + return True + else: + print('Received ERROR from AT+SMSWBOOT=0,0! Aborting!') + return False + return True + else: + if load_fff: + self.__serial.write(b"AT+SMUPGRADE\r\n") + if not self.wakeup_modem(baudrate, port, 100, 1, debug, self.__get_wait_msg(load_fff=load_fff)): + print("Timeout while waiting for modem to finish updating!") + reconnect_uart() + return False + + start = time.time() + while True: + self.__serial.read() + self.__serial.write(b"AT+SMUPGRADE?\r\n") + resp = self.read_rsp(1024) + if debug: print("AT+SMUPGRADE? returned {} [timeout: {}]".format(resp, time.time() - start)) + + if resp == b'\x00' or resp == b'': + time.sleep(2) + + if b'No report' in resp or b'on-going' in resp: + time.sleep(1) + + if b'success' in resp or b'fail' in resp: + break + + if time.time() - start >= 300: + raise OSError('Timeout waiting for modem to respond!') + + self.__serial.write(b"AT+SMSWBOOT?\r\n") + resp = self.read_rsp(100) + if debug: print("AT+SMSWBOOT? returned {}".format(resp)) + start = time.time() + while (b"RECOVERY" not in resp) and (b"FFH" not in resp) and (b"FFF" not in resp): + if debug: print("Timeout: {}".format(time.time() - start)) + if time.time() - start >= 300: + reconnect_uart() + raise OSError('Timeout waiting for modem to respond!') + time.sleep(2) + if not self.wakeup_modem(baudrate, port, 100, 1, debug, self.__get_wait_msg(load_fff=load_fff)): + reconnect_uart() + raise OSError('Timeout while waiting for modem to finish updating!') + self.__serial.read() + self.__serial.write(b"AT+SMSWBOOT?\r\n") + resp = self.read_rsp(100) + if debug: print("AT+SMSWBOOT? returned {}".format(resp)) + self.__serial.read() + self.__serial.write(b"AT+SMUPGRADE?\r\n") + resp = self.read_rsp(1024) + if debug: print("AT+SMUPGRADE? returned {}".format(resp)) + sqnup_result = self.return_upgrade_response(resp) + if debug: print('This is my result: {}'.format(sqnup_result)) + if 'success' in sqnup_result: + if not load_fff: + self.special_print('Resetting.', end='', flush=True) + self.__serial.write(b"AT+SMSWBOOT=1,1\r\n") + if debug: print("AT+SMSWBOOT=1,1 returned {}".format(resp)) + if b"ERROR" in resp: + time.sleep(5) + self.__serial.write(b"AT+SMSWBOOT=1,0\r\n") + resp = self.read_rsp(100) + if debug: print('AT+SMSWBOOT=1,0 returned: {}'.format(resp)) + if b'OK' in resp: + self.__serial.write(b"AT^RESET\r\n") + resp = self.read_rsp(100) + if debug: print('AT^RESET returned: {}'.format(resp)) + return True + else: + print('Received ERROR from AT+SMSWBOOT=1,0! Aborting!') + return False + self.wait_for_modem(send=False, echo_char='.', expected=b'+SYSSTART') + + elif sqnup_result is not None: + print('Upgrade failed with result {}!'.format(sqnup_result)) + print('Please check your firmware file(s)') + else: + print("Invalid response after upgrade... aborting.") + reconnect_uart() + return False + + self.__serial.write(b"AT\r\n") + self.__serial.write(b"AT\r\n") + time.sleep(0.5) + + if 'success' in sqnup_result: + if verbose: print('Sending AT+SQNSUPGRADENTF="success"') + self.__serial.write(b"AT+SQNSUPGRADENTF=\"success\"\r\n") + resonse = self.read_rsp(100) + if verbose: print('AT+SQNSUPGRADENTF="success" returned {}'.format(response)) + time.sleep(.25) + return True + elif sqnup_result is None: + print('Modem upgrade was unsucessfull. Please check your firmware file(s)') + return False + + def __check_br(self, br_only=False, verbose=False, debug=False): + old_br = None + old_sw = None + if debug: print("Checking bootrom & application") + self.__serial.write(b"AT!=\"showver\"\r\n") + time.sleep(.5) + shver = self.read_rsp(2000) + if shver is not None: + for line in shver.decode('ascii').split('\n'): + if debug: print('Checking line {}'.format(line)) + if "Bootloader0" in line: + old_br = "[33080]" in line + if debug: print("old_br: {}".format(old_br)) + + if "Software" in line: + old_sw = "[33080]" in line + if debug: print("old_sw: {}".format(old_sw)) + if old_br is None or old_sw is None: + if debug: print("Returning: None") + return None + if old_br and (br_only or not old_sw): + if debug: print("Returning: True") + return True + if debug: print("Returning: False") + return False + + + + def wakeup_modem(self, baudrate, port, max_try, delay, debug, msg='Attempting AT wakeup...'): + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + self.__serial = UART(1, baudrate=baudrate, pins=self.__pins, timeout_chars=10) + MAX_TRY = max_try + count = 0 + if msg is not None: + if debug: + print(msg + ' [{}]'.format(baudrate)) + else: + print(msg) + + self.__serial.read() + self.__serial.write(b"AT\r\n") + response = self.read_rsp(size=25) + if debug: print('{}'.format(response)) + while (not b'OK' in response) and (count < MAX_TRY): + count = count + 1 + if debug: print('count={}'.format(count)) + time.sleep(delay) + self.__serial.read() + self.__serial.write(b"AT\r\n") + response = self.read_rsp(size=25) + if debug: print('{}'.format(response)) + if 'FiPy' in sysname or 'GPy' in sysname: + self.__serial = UART(1, baudrate=baudrate, pins=self.__pins, timeout_chars=100) + return count < MAX_TRY + + def at_negotiation(self, baudrate, port, max_try, mirror, atneg_only, debug, target_baudrate): + MAX_TRY = max_try + count = 0 + if debug: + print('Attempting AT auto-negotiation... with baudrate {} and target_baudrate {}'.format(baudrate, target_baudrate)) + else: + print('Attempting AT auto-negotiation...') + self.__serial.write(b"AT\r\n") + response = self.read_rsp(size=20) + if debug: print('{}'.format(response)) + while (not b'OK' in response) and (count < MAX_TRY): + count = count + 1 + if debug: print('count={}'.format(count)) + time.sleep(1) + self.__serial.read() + self.__serial.write(b"AT\r\n") + response = self.read_rsp(size=20) + if debug: print('{}'.format(response)) + if b'OK' in response: + self.__serial.read() + cmd = "AT+IPR=%d\n"%target_baudrate + if debug: print('Setting baudrate to {}'.format(target_baudrate)) + self.__serial.write(cmd.encode()) + response = self.read_rsp(size=6) + if debug: print('{}'.format(response)) + if b'OK' in response: + self.__modem_speed = target_baudrate + self.__speed_detected = True + if atneg_only: + return True + if 'FiPy' in self.__sysname or 'GPy' in self.__sysname: + self.__serial = UART(1, baudrate=target_baudrate, pins=self.__pins, timeout_chars=100) + else: + self.__serial = None + self.__serial = serial.Serial(port, target_baudrate, bytesize=serial.EIGHTBITS, timeout=0.1, rtscts=fc) + self.__serial.reset_input_buffer() + self.__serial.reset_output_buffer() + self.__serial.flush() + self.__serial.read() + if debug: print('Checking SMOD') + self.__serial.write(b"AT+SMOD?\r\n") + response = self.read_rsp(size=1) + if b'0' in response: + if debug: print("AT+SMOD? returned {}".format(response)) + self.__serial.read() + return True + else: + print('ERROR in AT+SMOD returned {}'.format(response)) + return False + else: + print('ERROR in AT+IPR={} returned {}'.format(target_baudrate, response)) + return False + else: + print('ERROR sending AT command... no response? {}'.format(response)) + return False + time.sleep(1) + return True + + def uart_mirror(self, color): + import pycom + pycom.heartbeat(False) + time.sleep(.5) + pycom.rgbled(color) + LTE.modem_upgrade_mode() + return True + + def success_message(self, port=None, verbose=False, debug=False): + print("Your modem has been successfully updated.") + print("Here is the current firmware version:\n") + self.show_info(port=port, verbose=verbose, debug=debug) + + def upgrade(self, ffile, mfile=None, baudrate=921600, retry=False, resume=False, debug=False, pkgdebug=False, verbose=False, load_fff=True, load_only=False, mtools=False, force_fff=False): + success = True + if not retry and mfile is not None: + if resume or self.__check_br(br_only=True, verbose=verbose, debug=debug): + success = False + success = self.__run(bootrom=True, resume=resume, switch_ffh=True, direct=False, debug=debug, pkgdebug=pkgdebug, verbose=verbose) + time.sleep(1) + else: + print('{} is not required. Resumining normal upgrade.'.format(mfile)) + mfile=None + success=True + if debug: print('Success1? {}'.format(success)) + if success: + if mfile is not None: + success = False + success = self.__run(file_path=mfile, load_ffh=True, direct=False, baudrate=baudrate, debug=debug, pkgdebug=pkgdebug, verbose=verbose) + time.sleep(1) + if load_only: + return True + else: + success = True + else: + print('Unable to upgrade bootrom.') + if debug: print('Success2? {}'.format(success)) + if success: + if self.__run(file_path=ffile, resume=True if mfile is not None else resume, baudrate=baudrate, direct=False, debug=debug, pkgdebug=pkgdebug, verbose=verbose, load_fff=False if mfile else load_fff, mtools=mtools, force_fff=force_fff): + if self.__check_br(verbose=verbose, debug=debug): + success = self.__run(bootrom=True, debug=debug, direct=False, pkgdebug=pkgdebug, verbose=verbose, load_fff=True) + self.success_message(verbose=verbose, debug=debug) + else: + print('Unable to load updater from {}'.format(mfile)) + return success + + def upgrade_uart(self, ffh_mode=False, mfile=None, retry=False, resume=False, color=0x050505, debug=False, pkgdebug=False, verbose=False, load_fff=True): + success = False + try: + success = hasattr(LTE,'modem_upgrade_mode') + except: + success = False + if not success: + print('Firmware does not support LTE.modem_upgrade_mode()!') + reconnect_uart() + return False + print('Preparing modem for upgrade...') + if not retry and ffh_mode: + success = False + if self.__check_br(br_only=True, verbose=verbose, debug=debug): + success = self.__run(bootrom=True, resume=resume, switch_ffh=True, direct=False, debug=debug, pkgdebug=pkgdebug, verbose=verbose) + time.sleep(1) + else: + print('FFH mode is not necessary... ignoring!') + print('Do not specify updater.elf when updating!') + mfile = None + ffh_mode = False + success = True + if success: + if mfile is not None: + success = False + success = self.__run(file_path=mfile, load_ffh=True, direct=False, debug=debug, pkgdebug=pkgdebug, verbose=verbose) + if debug: print('Success2? {}'.format(success)) + if success: + self.__run(mirror=True, load_ffh=False, direct=False, rgbled=color, debug=debug, verbose=verbose) + else: + print('Unable to load updater from {}'.format(mfile)) + else: + self.__run(mirror=True, load_ffh=ffh_mode, direct=False, rgbled=color, debug=debug, verbose=verbose) + else: + print('Unable to upgrade bootrom.') + + def show_info(self, port=None, debug=False, verbose=False, fc=False): + self.__run(port=port, debug=debug, info_only=True, verbose=verbose, fc=fc) + + def upgrade_ext(self, port, ffile, mfile, resume=False, debug=False, pkgdebug=False, verbose=False, load_fff=True, fc=False, force_fff=False): + success = True + if mfile is not None: + success = False + success = self.__run(file_path=mfile, load_ffh=True, port=port, debug=debug, pkgdebug=pkgdebug, verbose=verbose, fc=fc) + if success: + if self.__run(file_path=ffile, resume=True if mfile is not None else resume, direct=False, port=port, debug=debug, pkgdebug=pkgdebug, verbose=verbose, load_fff=load_fff, fc=fc, force_fff=force_fff): + self.success_message(port=port, verbose=verbose, debug=debug) + else: + print('Unable to load updater from {}'.format(mfile)) + +def detect_error(): + print('Could not detect your modem!') + print('Please try to power off your device and restart in safeboot mode.') + reconnect_uart() + return False + +def print_welcome(): + print('<<< Welcome to the SQN3330 firmware updater [{}] >>>'.format(VERSION)) + if release is not None: + print('>>> {} with firmware version {}'.format(sysname,release)) + + + +if 'FiPy' in sysname or 'GPy' in sysname: + + def load(mfile, baudrate=921600, verbose=False, debug=False, hangup=False): + print_welcome() + sqnup = sqnsupgrade() + if sqnup.check_files(mfile, None, debug): + state = sqnup.detect_modem_state(debug=debug, hangup=hangup) + if debug: print('Modem state: {}'.format(state)) + if state is None: + detect_error() + elif state == 0: + sqnup.upgrade(ffile=None, mfile=mfile, baudrate=baudrate, retry=True, resume=False, debug=debug, pkgdebug=False, verbose=verbose, load_fff=False, load_only=True) + elif state == -1: + detect_error() + else: + print('Modem must be in recovery mode!') + reconnect_uart() + + def run(ffile, mfile=None, baudrate=921600, verbose=False, debug=False, load_fff=True, hangup=True, force_fff=False): + print_welcome() + retry = False + resume = False + mtools = False + success = False + sqnup = sqnsupgrade() + if sqnup.check_files(ffile, mfile, debug): + state = sqnup.detect_modem_state(debug=debug, hangup=hangup) + if debug: print('Modem state: {}'.format(state)) + if state is None: + detect_error() + elif state == 0: + retry = True + if mfile is None: + print('Your modem is in recovery mode. Please specify updater.elf file') + reconnect_uart() + return False + elif state == 4: + resume = True + elif state == 1: + mtools = True + elif state == -1: + detect_error() + success = sqnup.upgrade(ffile=ffile, mfile=mfile, baudrate=baudrate, retry=retry, resume=resume, debug=debug, pkgdebug=False, verbose=verbose, load_fff=load_fff, mtools=mtools, force_fff=force_fff) + reconnect_uart() + return success + + def uart(ffh_mode=False, mfile=None, color=0x050505, verbose=False, debug=False, hangup=True): + print_welcome() + retry = False + resume = False + import pycom + state = None + sqnup = sqnsupgrade() + if verbose: print('Trying to detect modem state...') + state = sqnup.detect_modem_state(debug=debug, hangup=hangup) + if debug: print('Modem state: {}'.format(state)) + + if state is None: + detect_error() + elif state == 0: + print('Your modem is in recovery mode. You will need to use firmware.dup and updater.elf file to upgrade.') + retry = True + ffh_mode = True + elif state == 4: + resume = True + elif state == -1: + detect_error() + sqnup.upgrade_uart(ffh_mode, mfile, retry, resume, color, debug, False, verbose) + + def info(verbose=False, debug=False, hangup=True): + print_welcome() + import pycom + state = None + sqnup = sqnsupgrade() + if verbose: print('Trying to detect modem state...') + state = sqnup.detect_modem_state(debug=debug, hangup=hangup) + if debug: print('Modem state: {}'.format(state)) + + if state is not None: + if state == 2: + print('Your modem is in application mode. Here is the current version:') + sqnup.show_info(verbose=verbose, debug=debug) + elif state == 1: + print('Your modem is in mTools mode.') + elif state == 0: + print('Your modem is in recovery mode! Use firmware.dup and updater.elf to flash new firmware.') + elif state == 4: + print('Your modem is in upgrade mode! Use firmware.dup to flash new firmware.') + elif state == -1: + print('Cannot determine modem state!') + if hasattr(pycom, 'lte_modem_en_on_boot') and verbose: + print('LTE autostart {}.'.format('enabled' if pycom.lte_modem_en_on_boot() else 'disabled')) + else: + print('Cannot determine modem state!') + reconnect_uart() + + def imei(verbose=False, debug=False, retry=5, hangup=False): + sqnup = sqnsupgrade() + state = sqnup.detect_modem_state(debug=debug, hangup=hangup, retry=retry) + return sqnup.get_imei() if state == 2 else None + + def state(verbose=False, debug=False, retry=5, hangup=False): + sqnup = sqnsupgrade() + return sqnup.detect_modem_state(debug=debug, hangup=hangup, retry=retry) + +else: + def run(port, ffile, mfile=None, resume=False, debug=False, verbose=False, load_fff=True, fc=False, force_fff=False): + print_welcome() + sqnup = sqnsupgrade() + if sqnup.check_files(ffile, mfile, debug): + sqnup.upgrade_ext(port=port, ffile=ffile, mfile=mfile, resume=resume, debug=debug, pkgdebug=False, verbose=verbose, load_fff=load_fff, force_fff=force_fff) + + def version(port, verbose=False, debug=False, fc=False): + sqnup = sqnsupgrade() + sqnup.show_info(port=port, debug=debug, verbose=verbose) diff --git a/pycom-docker-fw-build/Dockerfile b/pycom-docker-fw-build/Dockerfile new file mode 100644 index 0000000..226afbf --- /dev/null +++ b/pycom-docker-fw-build/Dockerfile @@ -0,0 +1,11 @@ +FROM ubuntu:bionic + +RUN apt-get update && apt-get -y install wget git build-essential python python-serial && \ + mkdir /opt/frozen/ && cd /opt && \ + wget -q https://dl.espressif.com/dl/xtensa-esp32-elf-linux64-1.22.0-80-g6c4433a-5.2.0.tar.gz && \ + tar -xzvf xtensa-esp32-elf-linux64-1.22.0-80-g6c4433a-5.2.0.tar.gz && \ + git clone --recursive https://github.com/pycom/pycom-esp-idf.git && \ + cd pycom-esp-idf && git submodule update --init && cd .. && \ + git clone --recursive https://github.com/pycom/pycom-micropython-sigfox.git + +ADD assets/build /usr/bin/build diff --git a/pycom-docker-fw-build/README.md b/pycom-docker-fw-build/README.md new file mode 100644 index 0000000..c178072 --- /dev/null +++ b/pycom-docker-fw-build/README.md @@ -0,0 +1,35 @@ +

+ +# Tool for adding your MicroPython code frozen in the flash + +### usage: +``` +sudo docker run -v `pwd`:/opt/frozen -it goinvent/pycom-fw build board-type your-project-version-code [micropython-sigfox-git-tag] [esp-idf-git-tag] +``` +where board in `WIPY LOPY SIPY GPY FIPY LOPY4` and your-project-version-code is a tag for output file: `LOPY4-your-project-version-code.tar.gz` + + +### example: + +If you have your MicroPython project in the current directory `.` just type: + +``` +sudo docker run -v `pwd`:/opt/frozen -it goinvent/pycom-fw build LOPY4 myproject +``` +For building against a specific revision (ex:v1.20.0.rc0 idf_v3.1) you can use: +``` +sudo docker run -v `pwd`:/opt/frozen -it goinvent/pycom-fw build FIPY myproject v1.20.0.rc0 idf_v3.1 +``` + + +### note: + +The Frozen code implementation might not support sub-directories in MicroPython code. + +The Frozen code implementation does not support adding assets (ex: db files, json,) + +### re-generate this goinvent/pycom-fw docker image: + +``` +sudo docker build -t goinvent/pycom-fw . +``` diff --git a/pycom-docker-fw-build/assets/build b/pycom-docker-fw-build/assets/build new file mode 100755 index 0000000..8c50d73 --- /dev/null +++ b/pycom-docker-fw-build/assets/build @@ -0,0 +1,77 @@ +#!/bin/bash + +boards="WIPY LOPY SIPY GPY FIPY LOPY4" + +function usage() { + echo "Usage:" + echo " $0 board-type version-code [micropython-sigfox-git-tag] [esp-idf-git-tag]" + echo "" + echo "where board-type in $boards and version-code is a tag for output file" + echo "" + exit 1 +} + +function prepare_dev() { + echo "Checkout $1 $2 board version-code" + + if [ ! -z "$1" ] + then + cd /opt/pycom-micropython-sigfox + # fetch all branches + git branch -r | grep -v '\->' | while read remote; do git branch --track "${remote#origin/}" "$remote"; done + # try to find branch revision $1 + if git pull ; git rev-list $1.. >/dev/null + + then + echo "Checkout " + else + exit 1 + fi + cd /opt/pycom-micropython-sigfox + git checkout $1 + cd mpy-cross && make clean && make && cd .. + git submodule update --init + fi + + if [ ! -z "$2" ] + then + cd /opt/pycom-esp-idf; + # fetch all branches + git branch -r | grep -v '\->' | while read remote; do git branch --track "${remote#origin/}" "$remote"; done + # try to find branch revision $2 + if git rev-list $2.. >/dev/null + + then + echo "" + # branch exists + else + # branch does not exists + exit + fi + cd /opt/pycom-esp-idf + git checkout $2 + git submodule update --init + fi +} + +function build() { + BOARD=$1 + PROJECT=$2 + export PATH=$PATH:/opt/xtensa-esp32-elf/bin/ + export IDF_PATH=/opt/pycom-esp-idf + board=`echo "$BOARD" | tr '[:upper:]' '[:lower:]'` + cd /opt/pycom-micropython-sigfox/esp32/ && + make clean BOARD=$BOARD && + cd ../mpy-cross && make clean && make && cd ../esp32 && + cp /opt/frozen/*.py ./frozen/Base/ && + mv ./frozen/Base/main.py ./frozen/Base/_main.py + make release BOARD=$BOARD RELEASE_DIR=/tmp + mv /tmp/*.tar.gz /opt/frozen/$board-firmware-$PROJECT.tar.gz + echo "Firmware $board-firmware-$PROJECT.tar.gz ready !" +} + +[[ $boards =~ (^|[[:space:]])$1($|[[:space:]]) ]] && + echo "#goinvent " || + usage +prepare_dev $3 $4 +build $1 $2 diff --git a/pymesh/mobile_app/README.md b/pymesh/mobile_app/README.md new file mode 100644 index 0000000..9d6e426 --- /dev/null +++ b/pymesh/mobile_app/README.md @@ -0,0 +1,18 @@ + +# Pymesh mobile application + +For demonstrating **Pymesh** features, a mobile application was created. + +It is available for both Android and iOS platforms. + +## Android application + +It is available as `.apk` packet in this directory. + +# iOS mobile application + +Pymesh mobile application is released currently (v0.) as Beta testing, on Apple TestFlight (to be installed for free from App Store). + +Pymesh app can be added afterwards, using the link: https://testflight.apple.com/join/PIxwbTKp + +Requirements: iOS minimum version v12.1 diff --git a/pymesh/mobile_app/app-release.apk b/pymesh/mobile_app/app-release.apk new file mode 100644 index 0000000..f48718c Binary files /dev/null and b/pymesh/mobile_app/app-release.apk differ diff --git a/pymesh/pymesh_frozen/README.md b/pymesh/pymesh_frozen/README.md new file mode 100644 index 0000000..a9f0fa6 --- /dev/null +++ b/pymesh/pymesh_frozen/README.md @@ -0,0 +1,8 @@ +# Pymesh micropython code + +This project exemplifies the use of Pycom's proprietary LoRa Mesh network - **Pymesh**. +These scripts were created and tested on Lopy4 and Fipy, using the Pymesh binary release. + +Official Pymesh docs: https://docs.pycom.io/pymesh/ + +Forum Pymesh announcements: https://forum.pycom.io/topic/4449/pymesh-updates diff --git a/pymesh/pymesh_frozen/copy_fw.sh b/pymesh/pymesh_frozen/copy_fw.sh new file mode 100755 index 0000000..33d2e6a --- /dev/null +++ b/pymesh/pymesh_frozen/copy_fw.sh @@ -0,0 +1,35 @@ +#!/bin/bash +set -e +#set -x +SOURCE="$(dirname $0)" +if [ -z $1 ]; then + echo "usage: $0 micropython_firmware_directory" + exit 1 +fi +if [ ! -d $1/esp32/frozen/Common ]; then + echo "Need to specify valid micropython firmware directory!" + exit 1 +fi +if [ ! -d $SOURCE ]; then + echo "Can't find source directory $SOURCE" + exit 1 +fi + +# moving main +# if [ -d $1/esp32/frozen/Pybytes ]; then +# cp $SOURCE/main.py $1/esp32/frozen/Pybytes/_main.py +# cp $1/esp32/frozen/Base/_boot.py $1/esp32/frozen/Pybytes/ +# elif [ -d $1/esp32/frozen/Base ]; then +# cp $SOURCE/main.py $1/esp32/frozen/Base/_main.py +# else +# cp $SOURCE/main.py $1/esp32/frozen/_main.py +# fi + +for i in $SOURCE/lib/*.py; do + SRC=$i + FN=$(basename $i) + cp $SRC $1/esp32/frozen/Common/_$FN +done + +cp -r $SOURCE/lib/msgpack $1/esp32/frozen/Common/ +echo "Done copying Pymesh library to $1/esp32/frozen/Common/" \ No newline at end of file diff --git a/pymesh/pymesh_frozen/lib/ble_rpc.py b/pymesh/pymesh_frozen/lib/ble_rpc.py new file mode 100644 index 0000000..78cffd2 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/ble_rpc.py @@ -0,0 +1,358 @@ +''' +Copyright (c) 2020, Pycom Limited. +This software is licensed under the GNU GPL version 3 or any +later version, with permitted additional terms. For more information +see the Pycom Licence v1.0 document supplied with this file, or +available at https://www.pycom.io/opensource/licensing +''' +import _thread +import time +from machine import Timer +from network import Bluetooth +import sys +import json + +import msgpack + +try: + from ble_services import BleServices +except: + from _ble_services import BleServices + +try: + from pymesh_config import PymeshConfig +except: + from _pymesh_config import PymeshConfig + +try: + from gps import Gps +except: + from _gps import Gps + +class BleRpc: + + def __init__(self, config, mesh): + self.config = config + self.mesh = mesh + self.ble_comm = BleServices(config.get('ble_name_prefix', PymeshConfig.BLE_NAME_PREFIX) + str(config.get('MAC'))) + + self.rx_worker = RXWorker(self.ble_comm) + self.tx_worker = TXWorker(self.ble_comm) + + self.rpc_handler = RPCHandler(self.rx_worker, self.tx_worker, self.mesh, self.ble_comm) + + # setting hooks for triggering when new message was received and ACK + self.mesh.meshaging.on_rcv_message = self.on_rcv_message + self.mesh.meshaging.on_rcv_ack = self.on_rcv_ack + + self.ble_comm.on_disconnect = self.ble_on_disconnect + + + def terminate(self): + ''' kill all, to exit nicely ''' + self.rx_worker.timer_kill() + self.ble_comm.close() + + def on_rcv_message(self, message): + ''' hook triggered when a new message arrived ''' + message_data = { + 'mac' : message.mac, + 'payload' : message.payload, + 'ts' : message.ts, + 'id' : message.id, + } + + msg = msgpack.packb(['notify', 'msg', message_data]) + self.rx_worker.put(msg) + print(message_data['payload']) + print("%d ================= RECEIVED :) :) :) "%time.ticks_ms()) + + + def on_rcv_ack(self, message): + ''' hook triggered when the ACK arrived ''' + message_data = { + 'id' : message.id, + } + + msg = msgpack.packb(['notify', 'msg-ack', message_data]) + self.rx_worker.put(msg) + print("%d ================= ACK RECEIVED :) :) :) "%time.ticks_ms()) + + def ble_on_disconnect(self): + ''' if BLE disconnected, it's better to re-instantiate RPC handler ''' + self.rpc_handler = RPCHandler(self.rx_worker, self.tx_worker, self.mesh, self.ble_comm) + +class RXWorker: + def __init__(self, ble_comm): + self.HEADSIZE = 20 + self.INTERVAL = .1 + self.q = b'' + self.ble_comm = ble_comm + self.chr = ble_comm.chr_rx + self.call_cnt = 0 + + # mutex for self.q usage + self.q_lock = _thread.allocate_lock() + + self._timer = Timer.Alarm(self.interval_cb, self.INTERVAL, periodic=True) + + def put(self, bytes): + with self.q_lock: + self.q = self.q + bytes + + # chunks = [ self.q[i:i+self.HEADSIZE] for i in range(0, len(self.q), self.HEADSIZE) ] + # for chunk in chunks: + # self.chr.value(chunk) + + #self.chr.value('') + #self.chr.value(bytes) + + def interval_cb(self, alarm): + self.call_cnt = self.call_cnt + 1 + if self.call_cnt >= 10: + # print('%d: rx worker interval.... %d'%(time.time(), len(self.q))) + self.call_cnt = 0 + + if len(self.q) == 0: + return + + if not self.ble_comm.status['connected']: + #unpacker._buffer = bytearray([]) + with self.q_lock: + self.q = b'' + return + try: + with self.q_lock: + head = self.q[:self.HEADSIZE] + tail = self.q[self.HEADSIZE:] + self.q = tail + #print('consuming {}, {}', head, tail) + + if self.chr and len(head) > 0: + self.chr.value(head) + #print('sending', list(head)) + except: + pass + + def timer_kill(self): + self._timer.cancel() + +class TXWorker: + def __init__(self, ble_comm): + self.ble_comm = ble_comm + self.chr = ble_comm.chr_tx + self.last_value = b'' + self.on_write = lambda value : 1 + + self.chr.callback(trigger=Bluetooth.CHAR_WRITE_EVENT | Bluetooth.CHAR_READ_EVENT, handler=self.cb_handler) + + def cb_handler(self, chr): + events = chr.events() + if events & Bluetooth.CHAR_WRITE_EVENT: + self.last_value = chr.value() + #print("Write request with value = {}".format(self.last_value)) + + self.on_write(self.last_value) + else: + #print('Read request on char 1') + return self.last_value + +class RPCHandler: + def __init__(self, rx_worker, tx_worker, mesh, ble_comm): + self.rx_worker = rx_worker + self.tx_worker = tx_worker + self.mesh = mesh + self.unpacker = msgpack.Unpacker(raw=False) + self.error = False + ble_comm.unpacker_set(self.unpacker) + + tx_worker.on_write = self.feed + + def feed(self, message): + #print('feeding (rpc)', message) + self.unpacker.feed(message) + try: + [self.resolve(x) for x in self.unpacker] + except Exception as e: + sys.print_exception(e) + print('error in unpacking... reset') + self.unpacker._buffer = bytearray() + self.error = True + + + + def resolve(self, obj): + #print('resolving: ', obj) + obj = list(obj) + type = obj[0] + + if type == 'call': + uuid = obj[1] + fn_name = obj[2] + args = obj[3] + fn = getattr(self, fn_name) + + if not fn: + print('fn {} not defined'.format(fn_name)) + return + + try: + result = fn(*args) + result = json.loads(json.dumps(result)) + print('calling RPC: {} - {}'.format(fn_name, result)) + + message = msgpack.packb(['call_result', uuid, result]) + except Exception as e: + sys.print_exception(e) + print('could not send result: {}'.format(result)) + return + + + #print('result', result) + #print('message', message) + self.rx_worker.put(message) + + # def demo_echo_fn(self, *args): + # return args + + def mesh_is_connected(self): + # True if Node is connected to Mesh; False otherwise + is_connected = self.mesh.is_connected() + return is_connected + + def mesh_ip(self): + # get IP RLOC16 in string + ip = self.mesh.ip() + return ip + + def set_gps(self, latitude, longitude): + print('settings gps!') + Gps.set_location(latitude, longitude) + # with open('/flash/gps', 'w') as fh: + # fh.write('{};{}'.format(lng, lat)) + + def get_mesh_mac_list(self): + """ returns list of distinct MAC address that are in this mesh network + [mac1, mac2, mac 3] """ + last_mesh_mac_list = self.mesh.get_mesh_mac_list() + return last_mesh_mac_list + + def get_mesh_pairs(self, *args): + """ returns list of pairs that is a mesh connection + [ + ('mac1', 'mac2', rssi), + ('mac1', 'mac3', rssi), + #... + ] """ + last_mesh_pairs = self.mesh.get_mesh_pairs() + return last_mesh_pairs + + + def get_node_info(self, mac_id = ' '): + """ Returns the debug info for a specified mac address + takes max 10 sec + { + 'ip': 4c00, # last 2bytes from the ip v6 RLOC16 address + 'r': 3, # not_connected:0 | child:1 | leader:2 | router:3 + 'a': 100, # age[sec], time since last info about this node + 'nn' : 20 # neighbours number + 'nei': { # neighbours enumerated, if any + (mac, ip, role, rssi, age), + (mac, ip, role, rssi, age) + } + 'l': { # location, if available + 'lng': 7, + 'lat': 20, + }, + 'b' : { # BLE infos + 'a': 100 # age, seconds since last ping with that device, None if properly disconnected + 'id': '' # 16byte + 'n': '', # name, max. 16 chars + } + } """ + node_info = self.mesh.get_node_info(mac_id) + return node_info + + def send_message(self, data): + """ sends a message with id, to m(MAC) + return True if there is buffer to store it (to be sent)""" + """ data is dictionary data = { + 'to': 0x5, + 'b': 'text', + 'id': 12345, + 'ts': 123123123, + }""" + print("%d: Send Msg ---------------------->>>>>>>> "%time.ticks_ms()) + return self.mesh.send_message(data) + + def send_message_was_sent(self, mac, msg_id): + """ checks for ACK received for msg_id + returns True if message was delivered to last node connected to BLE mobile """ + error = False + try: + mac_int = int(mac) + msg_id_int = int(msg_id) + except: + error = True + if error: + return False + # mesh.mesage_was_ack(5, 12345) + return self.mesh.mesage_was_ack(mac, msg_id) + + def receive_message(self): + """ + return { + 'b': 'text', + 'from': 'ble_device_id', + 'ts': 123123123, + 'id': '', + } """ + return self.mesh.get_rcv_message() + + # def send_image(self, data): + # """ sends an image + # return True if there is buffer to store it (to be sent)""" + # print("Send Image ---------------------->>>>>>>> ", data) + # start = 0 + # filename = 'dog_2.jpg' + # to = 0 + # # packsize = 500 + # image = list() + + # try: + # filename = data.get('fn', "image.jpg") + # start = int(data['start']) + # image = bytes(data['image']) + # except: + # print('parsing failed') + # return False + + # print("Image chunk size: %d"%len(image)) + # file_handling = "ab" # append, by default + # if start == 0: + # file_handling = "wb" # write/create new + + # with open("/flash/" + filename, file_handling) as file: + # print("file open") + # file.write(image) + # print("file written") + + # print("done") + # return True + + # def stat_start(self, data): + # # do some statistics + # #data = {'mac':6, 'n':3, 't':30} + # res = self.mesh.statistics_start(data) + # print("rpc stat_start? ", res) + # return res + + # def stat_status(self, data): + # print("rpc stat_status ", data) + # try: + # id = int(data) + # except: + # id = 0 + # res = self.mesh.statistics_get(id) + # print("rpc stat_status id:"+ str(id) + ", res: " + str(res)) + # return res diff --git a/pymesh/pymesh_frozen/lib/ble_services.py b/pymesh/pymesh_frozen/lib/ble_services.py new file mode 100644 index 0000000..019bf12 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/ble_services.py @@ -0,0 +1,80 @@ +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +from network import Bluetooth +import time +import msgpack + +VERSION = "1.0.0" + +class BleServices: + + def __init__(self, ble_name): + # self.mesh_mac = mesh_mac + self.ble_name = ble_name + self.on_disconnect = None + self._init() + + def _init(self): + self.status = { + 'connected' : False + } + + bluetooth = Bluetooth(modem_sleep=False) + adv_name = self.ble_name + bluetooth.set_advertisement(name=adv_name, service_uuid=0xec00) + print("BLE name:", adv_name) + + bluetooth.callback(trigger=Bluetooth.CLIENT_CONNECTED | Bluetooth.CLIENT_DISCONNECTED, handler=self.conn_cb) + bluetooth.advertise(True) + + srv_rx = bluetooth.service(uuid=0xec00, isprimary=True) + self.chr_rx = srv_rx.characteristic(uuid=0xec0e, value=0) + + srv_tx = bluetooth.service(uuid=0xed00, isprimary=True) + self.chr_tx = srv_tx.characteristic(uuid=0xed0e, value=0) + + self.unpacker = None + + def conn_cb(self, bt_o): + #global ble_connected + events = bt_o.events() + if events & Bluetooth.CLIENT_CONNECTED: + self.status['connected'] = True + print("Client connected") + elif events & Bluetooth.CLIENT_DISCONNECTED: + self.status['connected'] = False + + if self.on_disconnect: + self.on_disconnect() + + print("Client disconnected") + pass + + def unpacker_set(self, unpacker): + self.unpacker = unpacker + + def close(self): + bluetooth = Bluetooth() + bluetooth.disconnect_client() + bluetooth.deinit() + pass + + def restart(self): + print("BLE disconnnect client") + bluetooth = Bluetooth() + bluetooth.disconnect_client() + time.sleep(2) + self.status['connected'] = False + if self.on_disconnect: + self.on_disconnect() + + # bluetooth.deinit() + # time.sleep(1) + # self._init() + pass + diff --git a/pymesh/pymesh_frozen/lib/cli.py b/pymesh/pymesh_frozen/lib/cli.py new file mode 100644 index 0000000..69242aa --- /dev/null +++ b/pymesh/pymesh_frozen/lib/cli.py @@ -0,0 +1,332 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import time +import json +import sys +import _thread + +try: + from gps import Gps +except: + from _gps import Gps + +__version__ = '3' +""" +__version__ = '3' +* added dynamic start/stop CLI +* h = help, listing all commands +* added tx_pow and s(send) packets with repetitions +* debug can also read the current level + +__version__ = '2' +* added pause/resume and factory reset + +__version__ = '1' +* initial draft +""" + +class Cli: + """ class for CLI commands """ + + def __init__(self, mesh, pymesh): + self.mesh = mesh + self.pymesh = pymesh + # self.rpc_handler = rpc_handler + # self.ble_comm = ble_comm + + # lamda functions + self.sleep = None + return + + def process(self, arg1, arg2): + last_mesh_pairs = [] + last_mesh_mac_list = [] + last_mesh_node_info = {} + + try: + while True: + time.sleep(.1) + cmd = input('>') + + if cmd == 'ip': + print(self.mesh.ip()) + + elif cmd == 'mac': + # read/write LoRa MAC address + try: + id = int(input('(new LoRa MAC (0-64k) [Enter for read])<')) + except: + print(self.mesh.mesh.mesh.MAC) + continue + id = id & 0xFFFF # just 2B value + # it's actually set in main.py (main thread) + print("LoRa MAC set to", id) + self.sleep(1, id) # force restart + + elif cmd == 'mml': + mesh_mac_list = self.mesh.get_mesh_mac_list() + if len(mesh_mac_list) > 0: + last_mesh_mac_list = mesh_mac_list + print('mesh_mac_list ', json.dumps(last_mesh_mac_list)) + + elif cmd == 'self': + node_info = self.mesh.get_node_info() + print("self info:", node_info) + + elif cmd == 'mni': + for mac in last_mesh_mac_list: + node_info = self.mesh.get_node_info(mac) + time.sleep(.5) + if len(node_info) > 0: + last_mesh_node_info[mac] = node_info + print('last_mesh_node_info', json.dumps(last_mesh_node_info)) + + elif cmd == 'mp': + mesh_pairs = self.mesh.get_mesh_pairs() + if len(mesh_pairs) > 0: + last_mesh_pairs = mesh_pairs + print('last_mesh_pairs', json.dumps(last_mesh_pairs)) + + elif cmd == 's': + interval = 0 + repetitions = 0 + try: + to = int(input('(to)<')) + # typ = input('(type, 0=text, 1=file, Enter for text)<') + # if not typ: + # typ = 0 + # else: + # typ = int(typ) + txt = input('(message)<') + repetitions = int(input('(repetitions)')) + if repetitions > 1: + interval = int(input('(interval in seconds)')) + except: + continue + data = { + 'to': to, + # 'ty': 0, + 'b': txt, + 'id': 12345, + 'ts': int(time.time()), + } + while repetitions > 0: + print(self.mesh.send_message(data)) + repetitions = repetitions - 1 + if repetitions > 0: + print("Remaining TX packets:", repetitions) + time.sleep(interval) + + + elif cmd == 'ws': + to = int(input('(to)<')) + try: + id = int(input('(id, default 12345)<')) + except: + id = 12345 + print(self.mesh.mesage_was_ack(to, id)) + + elif cmd == 'rm': + print(self.mesh.get_rcv_message()) + + elif cmd == 'gps': + try: + lat = float(input('(lat [Enter for read])<')) + lon = float(input('(lon)<')) + except: + print("Gps:", (Gps.lat, Gps.lon)) + continue + + Gps.set_location(lat, lon) + print("Gps:", (Gps.lat, Gps.lon)) + + elif cmd == 'sleep': + try: + timeout = int(input('(time[sec])<')) + except: + continue + if self.sleep: + self.sleep(timeout) + + # elif cmd == "ble": + # # reset BLE connection + # self.ble_comm.restart() + + # elif cmd == "stat": + # # do some statistics + # # data = [] + # # data[0] = {'mac':6, 'n':3, 't':30, 's1':0, 's2':0} + # # data[0] = {'mac':6, 'n':3, 't':30, 's1':5, 's2':10} + # # data[2] = {'mac':6, 'n':30, 't':60, 's1':10, 's2':45} + # # for line in data: + # # print() + # # print("1 = {'mac':6, 'n':30, 't':60, 's1':10, 's2':45}<'") + # # print("2 = {'mac':6, 'n':30, 't':60, 's1':10, 's2':45}<'") + # # print("3 = {'mac':6, 'n':30, 't':60, 's1':10, 's2':45}<'") + # # id = int(input('(choice 1-..)<')) + # data = {'mac':6, 'n':3, 't':60, 's1':3, 's2':8} + # res = self.mesh.statistics_start(data) + # print("ok? ", res) + + # elif cmd == "stat?": + # try: + # id = int(input('(id [Enter for all])<')) + # except: + # id = 0 + # res = self.mesh.statistics_get(id) + # print("ok? ", res) + + elif cmd == "rst": + print("Mesh Reset NVM settings ... ") + self.mesh.mesh.mesh.mesh.deinit(reset=True) + if self.sleep: + self.sleep(1) + + # elif cmd == "pyb": + # # print("Pybytes debug menu, Pybytes connection is ", Pybytes_wrap.is_connected()) + # state = 1 + # timeout = 120 + # try: + # state = int(input('(Debug 0=stop, 1=start [Default start])<')) + # except: + # pass + # try: + # timeout = int(input('(Pybytes timeout [Default 120 sec])<')) + # except: + # pass + # self.mesh.pybytes_config((state == 1), timeout) + + elif cmd == "br": + state = 2 # default display BR + try: + state = int(input('(state 0=Disable, 1=Enable, 2=Display [Default Display])<')) + except: + pass + + if state == 2: + print("Border Router state: ", self.mesh.mesh.mesh.mesh.border_router()) + elif state == 1: + # Enable BR + prio = 0 # default normal priority + try: + prio = int(input('(priority -1=Low, 0=Normal or 1=High [Default Normal])<')) + except: + pass + self.mesh.br_set(True, prio, self.new_br_message_cb) + else: + # disable BR function + self.mesh.br_set(False) + + elif cmd == "brs": + """ send data to BR """ + ip_default = "1:2:3::4" + port = 5555 + try: + payload = input("(message<)") + ip = input("(IP destination, Mesh-external [Default: 1:2:3::4])<") + if len(ip) == 0: + ip = ip_default + port = int(input("(port destination [Default: 5555])<")) + except: + pass + data = { + 'ip': ip, + 'port': port, + 'b': payload + } + print("Send BR message:", data) + self.mesh.send_message(data) + + elif cmd == "buf": + print("Buffer info:",self.mesh.mesh.mesh.mesh.cli("bufferinfo")) + + elif cmd == "ot": + cli = input('(openthread cli)<') + print(self.mesh.mesh.mesh.mesh.cli(cli)) + + elif cmd == "debug": + ret = input('(debug level[0-5])<') + try: + level = int(ret) + self.pymesh.debug_level(level) + except: + print(self.pymesh.debug_level()) + + elif cmd == "config": + print(self.mesh.config) + + elif cmd == "pause": + self.pymesh.pause() + + elif cmd == "resume": + self.pymesh.resume() + + elif cmd == "tx_pow": + print("LoRa stats:", self.pymesh.mesh.mesh.mesh.lora.stats()) + tx_str = input('(tx_pow[2-20])<') + try: + tx_pow = int(tx_str) + self.pymesh.pause() + print("Change TX power to", tx_pow) + time.sleep(1) + self.pymesh.resume(tx_pow) + except: + print("Invalid value") + + elif cmd == "stop": + self.pymesh.cli = None + _thread.exit() + + elif cmd == "h": + print("List of available commands") + print("br - enable/disable or display the current Border Router functionality") + print("brs - send packet for Mesh-external, to BR, if any") + print("buf - display buffer info") + print("config - print config file contents") + print("debug - set debug level") + print("gps - get/set location coordinates") + print("h - help, list of commands") + print("ip - display current IPv6 unicast addresses") + print("mac - set or display the current LoRa MAC address") + print("mml - display the Mesh Mac List (MAC of all nodes inside this Mesh), also inquires Leader") + print("mp - display the Mesh Pairs (Pairs of all nodes connections), also inquires Leader") + print("ot - sends command to openthread internal CLI") + print("pause - suspend Pymesh") + print("resume - resume Pymesh") + print("rm - verifies if any message was received") + print("rst - reset NOW, including NVM Pymesh IPv6") + print("s - send message") + print("self - display all info about current node") + print("sleep - deep-sleep") + # print("stat - start statistics") + # print("stat? - display statistics") + print("stop - close this CLI") + print("tx_pow - set LoRa TX power in dBm (2-20)") + print("ws - verifies if message sent was acknowledged") + + except KeyboardInterrupt: + print('CLI Ctrl-C') + except Exception as e: + sys.print_exception(e) + finally: + print('CLI stopped') + if self.pymesh.cli is not None: + self.sleep(0) + + def new_br_message_cb(self, rcv_ip, rcv_port, rcv_data, dest_ip, dest_port): + ''' callback triggered when a new packet arrived for the current Border Router, + having destination an IP which is external from Mesh ''' + print('CLI BR default handler') + print('Incoming %d bytes from %s (port %d), to external IPv6 %s (port %d)' % + (len(rcv_data), rcv_ip, rcv_port, dest_ip, dest_port)) + print(rcv_data) + + # user code to be inserted, to send packet to the designated Mesh-external interface + # ... + return diff --git a/pymesh/pymesh_frozen/lib/gps.py b/pymesh/pymesh_frozen/lib/gps.py new file mode 100644 index 0000000..cd30b9c --- /dev/null +++ b/pymesh/pymesh_frozen/lib/gps.py @@ -0,0 +1,88 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +try: + from pymesh_debug import print_debug +except: + from _pymesh_debug import print_debug + +import time +# from pytrack import Pytrack +# from L76GNSS import L76GNSS +from machine import Timer + +__version__ = '1' +""" +* initial version +""" + +class Gps: + # Pycom office GPS coordinates + lat = 51.45 + lon = 5.45313 + + l76 = None + _timer = None + #is_set = False + + @staticmethod + def set_location(latitude, longitude): + dlat = str(type(latitude)) + dlon = str(type(longitude)) + if dlat == dlon == "": + Gps.lat = latitude + Gps.lon = longitude + is_set = True + else: + print_debug(3, "Error parsing ", latitude, longitude) + + @staticmethod + def get_location(): + return (Gps.lat, Gps.lon) + + # @staticmethod + # def init_static(): + # is_pytrack = True + # try: + # py = Pytrack() + # Gps.l76 = L76GNSS(py, timeout=30) + # #l76.coordinates() + # Gps._timer = Timer.Alarm(Gps.gps_periodic, 30, periodic=True) + # print_debug(3, "Pytrack detected") + # except: + # is_pytrack = False + # print_debug(3, "Pytrack NOT detected") + # #TODO: how to check if GPS is conencted + # return is_pytrack + + # @staticmethod + # def gps_periodic(alarm): + # t0 = time.ticks_ms() + # coord = Gps.l76.coordinates() + # if coord[0] != None: + # Gps.lat, Gps.lon = coord + # print_debug(3, "New coord ", coord) + # dt = time.ticks_ms() - t0 + # print_debug(3, " =====>>>> gps_periodic ", dt) + + # @staticmethod + # def terminate(): + # if Gps._timer is not None: + # Gps._timer.cancel() + # pass + +""" +from pytrack import Pytrack +from L76GNSS import L76GNSS +py = Pytrack() +l76 = L76GNSS(py, timeout=30) +t0 = time.ticks_ms() +l76.coordinates() +y = time.ticks_ms() - t0 +y +""" diff --git a/pymesh/pymesh_frozen/lib/loramesh.py b/pymesh/pymesh_frozen/lib/loramesh.py new file mode 100644 index 0000000..5f21c09 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/loramesh.py @@ -0,0 +1,810 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +from network import LoRa +import socket +import time +import utime +import ubinascii +import pycom + +from struct import * + +try: + from pymesh_debug import print_debug +except: + from _pymesh_debug import print_debug + +try: + from gps import Gps +except: + from _gps import Gps + +__version__ = '7' +""" +__version__ = '7' +* added pause/resume + +__version__ = '6' +* added IPv6 unicast addresses as fdde:ad00:beef:0:: +""" + +class Loramesh: + """ Class for using Lora Mesh - openThread """ + + STATE_DISABLED = const(0) + STATE_DETACHED = const(1) + STATE_CHILD = const(2) + STATE_ROUTER = const(3) + STATE_LEADER = const(4) + STATE_LEADER_SINGLE = const(5) + + # rgb LED color for each state: disabled, detached, child, router, leader and single leader + #RGBLED = [0x0A0000, 0x0A0000, 0x0A0A0A, 0x000A00, 0x00000A, 0x0A000A] + RGBLED = [0x0A0000, 0x0A0000, 0x0A0A0A, 0x000A00, 0x0A000A, 0x000A0A] + + # TTN conf mode + #RGBLED = [0x200505, 0x200505, 0x202020, 0x052005, 0x200020, 0x001818] + + # for outside/bright sun + #RGBLED = [0xFF0000, 0xFF0000, 0x808080, 0x00FF00, 0x0000FF, 0xFF00FF] + + # mesh node state string + STATE_STRING_LIST = ['Disabled','Detached', 'Child', 'Router', 'Leader'] + + # address to be used for multicasting + MULTICAST_MESH_ALL = 'ff03::1' + MULTICAST_MESH_FTD = 'ff03::2' + + MULTICAST_LINK_ALL = 'ff02::1' + MULTICAST_LINK_FTD = 'ff02::2' + + # Leader has an unicast IPv6: fdde:ad00:beef:0:0:ff:fe00:fc00 + LEADER_DEFAULT_RLOC = 'fc00' + + def __init__(self, config): + """ Constructor """ + self.config_lora = config.get('LoRa') + self._lora_init() + + # get Lora MAC address + #self.MAC = str(ubinascii.hexlify(lora.mac()))[2:-1] + self.MAC = int(str(ubinascii.hexlify(self.lora.mac()))[2:-1], 16) + + #last 2 letters from MAC, as integer + self.mac_short = self.MAC & 0xFFFF #int(self.MAC[-4:], 16) + print_debug(5, "LoRa MAC: %s, short: %s"%(hex(self.MAC), self.mac_short)) + + self.rloc16 = 0 + self.rloc = '' + self.net_addr = '' + self.ip_eid = '' + self.ip_link = '' + self.state = STATE_DISABLED + + # a dictionary with all direct neighbors + # key is MAC for each neighbor + # value is pair (age, mac, rloc16, role, rssi) + #self.neigh_dict = {} + self.router_data = RouterData() + self.router_data.mac = self.MAC + + # a dictionary with all routers direct neighbors + # key is MAC for each router + # value is pair (age, rloc, neigh_num, (age, mac, rloc16, role, rssi)) + #self.leader_dict = {} + self.leader_data = LeaderData() + self.leader_data.mac = self.MAC + + # set of all MACS from whole current Mesh Network + self.macs = set() + self.macs_ts = -65535 # very old + + # list of all pairs (direct radio connections) inside Mesh + self.connections = list() + self.connections_ts = -65535 # very old + + # set a new unicast address + self._add_ipv6_unicast() + + def _lora_init(self, tx_dBm = 14): + self.lora = LoRa(mode=LoRa.LORA, + region = self.config_lora.get("region"), + frequency = self.config_lora.get("freq"), + bandwidth = self.config_lora.get("bandwidth"), + sf = self.config_lora.get("sf"), + tx_power = tx_dBm) + self.mesh = self.lora.Mesh() #start Mesh + + def pause(self): + self.mesh.deinit() + + def resume(self, tx_dBm = 14): + self._lora_init(tx_dBm) + self._add_ipv6_unicast() + + def ip_mac_unique(self, mac): + ip = self.unique_ip_prefix + hex(mac & 0xFFFF)[2:] + return ip + + def _add_ipv6_unicast(self): + self.unique_ip_prefix = "fdde:ad00:beef:0::" + command = "ipaddr add " + self.ip_mac_unique(self.mac_short) + self.mesh.cli(command) + + def update_internals(self): + self._state_update() + self._rloc16_update() + self._update_ips() + self._rloc_ip_net_addr() + + def _rloc16_update(self): + self.rloc16 = self.mesh.rloc() + return self.rloc16 + + def _update_ips(self): + """ Updates all the unicast IPv6 of the Thread interface """ + ips = self.mesh.ipaddr() + for line in ips: + if line.startswith('fd'): + # Mesh-Local unicast IPv6 + try: + addr = int(line.split(':')[-1], 16) + except Exception: + continue + if addr == self.rloc16: + # found RLOC + # RLOC IPv6 has x:x:x:x:0:ff:fe00:RLOC16 + self.rloc = line + elif ':0:ff:fe00:' not in line: + # found Mesh-Local EID + self.ip_eid = line + elif line.startswith('fe80'): + # Link-Local + self.ip_link = line + + def is_connected(self): + """ Returns true if it is connected if its Child, Router or Leader """ + connected = False + if self.state in (STATE_CHILD, STATE_ROUTER, STATE_LEADER, STATE_LEADER_SINGLE): + connected = True + return connected + + def _state_update(self): + """ Returns the Thread role """ + self.state = self.mesh.state() + if self.state < 0: + self.state = self.STATE_DISABLED + return self.state + + def _rloc_ip_net_addr(self): + """ returns the family part of RLOC IPv6, without last word (2B) """ + self.net_addr = ':'.join(self.rloc.split(':')[:-1]) + ':' + return self.net_addr + + def state_string(self): + if self.state >= len(self.STATE_STRING_LIST): + return 'none' + return self.STATE_STRING_LIST[self.state] + + def led_state(self): + """ Sets the LED according to the Thread role """ + if self.state == STATE_LEADER and self.mesh.single(): + pycom.rgbled(self.RGBLED[self.STATE_LEADER_SINGLE]) + else: + pycom.rgbled(self.RGBLED[self.state]) + + def ip(self): + """ Returns the IPv6 RLOC """ + return self.rloc + + # def parent_ip(self): + # # DEPRECATED, unused + # """ Returns the IP of the parent, if it's child node """ + # ip = None + # state = self.state + # if state == STATE_CHILD or state == STATE_ROUTER: + # try: + # ip_words = self.rloc.split(':') + # parent_rloc = int(self.lora.cli('parent').split('\r\n')[1].split(' ')[1], 16) + # ip_words[-1] = hex(parent_rloc)[2:] + # ip = ':'.join(ip_words) + # except Exception: + # pass + # return ip + + + # def neighbors_ip(self): + # # DEPRECATED, unused + # """ Returns a list with IP of the neighbors (children, parent, other routers) """ + # state = self.state + # neigh = [] + # if state == STATE_ROUTER or state == STATE_LEADER: + # ip_words = self.rloc.split(':') + # # obtain RLOC16 neighbors + # neighbors = self.lora.cli('neighbor list').split(' ') + # for rloc in neighbors: + # if len(rloc) == 0: + # continue + # try: + # ip_words[-1] = str(rloc[2:]) + # nei_ip = ':'.join(ip_words) + # neigh.append(nei_ip) + # except Exception: + # pass + # elif state == STATE_CHILD: + # neigh.append(self.parent_ip()) + # return neigh + + # def cli(self, command): + # """ Simple wrapper for OpenThread CLI """ + # return self.mesh.cli(command) + + def ipaddr(self): + """ returns all unicast IPv6 addr """ + return self.mesh.ipaddr() + + # def ping(self, ip): + # """ Returns ping return time, to an IP """ + # res = self.cli('ping ' + str(ip)) + # """ + # '8 bytes from fdde:ad00:beef:0:0:ff:fe00:e000: icmp_seq=2 hlim=64 time=236ms\r\n' + # 'Error 6: Parse\r\n' + # no answer + # """ + # ret_time = -1 + # try: + # ret_time = int(res.split('time=')[1].split('ms')[0]) + # except Exception: + # pass + # return ret_time + + def blink(self, num = 3, period = .5, color = None): + """ LED blink """ + if color is None: + color = self.RGBLED[self.state] + for _ in range(num): + pycom.rgbled(0) + time.sleep(period) + pycom.rgbled(color) + time.sleep(period) + self.led_state() + + def neighbors_update(self): + """ update neigh_dict from cli:'neighbor table' """ + """ >>> print_debug(3, lora.cli("neighbor table")) + | Role | RLOC16 | Age | Avg RSSI | Last RSSI |R|S|D|N| Extended MAC | + +------+--------+-----+----------+-----------+-+-+-+-+------------------+ + | C | 0x2801 | 219 | 0 | 0 |1|1|1|1| 0000000000000005 | + | R | 0x7400 | 9 | 0 | 0 |1|0|1|1| 0000000000000002 | + + """ + x = self.mesh.neighbors() + print_debug(3,"Neighbors Table: %s"%x) + + if x is None: + # bad read, just keep previous neigbors + return + + # clear all pre-existing neigbors + self.router_data = RouterData() + self.router_data.mac = self.MAC + self.router_data.rloc16 = self.rloc16 + self.router_data.role = self.state + self.router_data.ts = time.time() + self.router_data.coord = Gps.get_location() + + for nei_rec in x: + # nei_rec = (role=3, rloc16=10240, rssi=0, age=28, mac=5) + age = nei_rec.age + if age > 300: + continue # shouln't add neighbors too old + role = nei_rec.role + rloc16 = nei_rec.rloc16 + # maybe we shouldn't add Leader (because this info is already available at Leader) + # if rloc16 == self.leader_rloc(): + # continue + rssi = nei_rec.rssi + mac = nei_rec.mac + neighbor = NeighborData((mac, age, rloc16, role, rssi,)) + self.router_data.add_neighbor(neighbor) + #print_debug(3, "new Neighbor: %s"%(neighbor.to_string())) + #except: + # pass + # add own info in dict + #self.neigh_dict[self.MAC] = (0, self.rloc16, self.state, 0) + print_debug(3, "Neighbors: %s"%(self.router_data.to_string())) + return + + def leader_add_own_neigh(self): + """ leader adds its own neighbors in leader_dict """ + self.leader_data.add_router(self.router_data) + return + + def neighbors_pack(self): + """ packs in a struct all neighbors as (MAC, RLOC16, Role, rssi, age) """ + data = self.router_data.pack() + + return data + + def routers_neigh_update(self, data_pack): + """ unpacks the PACK_ROUTER_NEIGHBORS, adding them in leader_dict """ + # key is MAC for each router + # value is pair (age, rloc, neigh_num, (age, mac, rloc16, role, rssi)) + router = RouterData(data_pack) + self.leader_data.add_router(router) + return + + def leader_dict_cleanup(self): + """ cleanup the leader_dict for old entries """ + #print_debug(3, "Leader Data before cleanup: %s"%self.leader_data.to_string()) + self.leader_data.cleanup() + print_debug(3, "Leader Data : %s"%self.leader_data.to_string()) + + def routers_rloc_list(self, age_min, resolve_mac = None): + """ return list of all routers IPv6 RLOC16 + if mac parameter is present, then returns just the RLOC16 of that mac, if found + """ + mac_ip = None + data = self.mesh.routers() + print_debug(3, "Routers Table: "+ str(data)) + '''>>> print_debug(3, lora.cli('router table')) + | ID | RLOC16 | Next Hop | Path Cost | LQ In | LQ Out | Age | Extended MAC | + +----+--------+----------+-----------+-------+--------+-----+------------------+ + | 12 | 0x3000 | 63 | 0 | 0 | 0 | 0 | 0000000000000002 |''' + + if data is None: + # bad read + return () + + net_addr = self.net_addr + routers_list = [] + for line in data: + # line = (mac=123456, rloc16=20480, id=20, path_cost=0, age=7) + age = line.age + if age > 300: + continue # shouldn't add/resolve very old Routers + rloc16 = line.rloc16 + + # check if it's own rloc16 + if rloc16 == self.rloc16: + continue + + if resolve_mac is not None: + if resolve_mac == line.mac: + mac_ip = rloc16 + break + + # look for this router in Leader Data + # if doesn't exist, add it to routers_list with max ts + # if it exists, just add it with its ts + last_ts = self.leader_data.get_mac_ts(line.mac) + if time.time() - last_ts < age_min: + continue # shouldn't add/resolve very "recent" Routers + + ipv6 = net_addr + hex(rloc16)[2:] + routers_list.append((last_ts, ipv6)) + + if resolve_mac is not None: + print_debug(3, "Mac found in Router %s"%str(mac_ip)) + return mac_ip + + # sort the list in the ascending values of timestamp + routers_list.sort() + + print_debug(3, "Routers list %s"%str(routers_list)) + return routers_list + + def leader_data_pack(self): + """ creates packet with all Leader data, leader_dict """ + self.leader_data.rloc16 = self.rloc16 + data = self.leader_data.pack() + return data + + def leader_data_unpack(self, data): + self.leader_data = LeaderData(data) + print_debug(3, "Leader Data : %s"%self.leader_data.to_string()) + return self.leader_data.ok + + def neighbor_resolve_mac(self, mac): + mac_ip = self.router_data.resolve_mac(mac) + return mac_ip + + def resolve_mac_from_leader_data(self, mac): + mac_ip = self.leader_data.resolve_mac(mac) + print_debug(3, "Mac %x found as IP %s"%(mac, str(mac_ip))) + return mac_ip + + def macs_get(self): + """ returns the set of the macs, hopefully it was received from Leader """ + #print_debug(3, "Macs: %s"%(str(self.macs))) + return (self.macs, self.macs_ts) + + def macs_set(self, data): + MACS_FMT = '!H' + field_size = calcsize(MACS_FMT) + #print_debug(3, "Macs pack: %s"%(str(data))) + n, = unpack(MACS_FMT, data) + #print_debug(3, "Macs pack(%d): %s"%(n, str(data))) + index = field_size + self.macs = set() + + for _ in range(n): + mac, = unpack(MACS_FMT, data[index:]) + self.macs.add(mac) + #print_debug(3, "Macs %d, %d: %s"%(index, mac, str(self.macs))) + index = index + field_size + + self.macs_ts = time.time() + pass + + def connections_get(self): + """ returns the list of all connections inside Mesh, hopefully it was received from Leader """ + return (self.connections, self.connections_ts) + + def connections_set(self, data): + CONNECTIONS_FMT = '!HHb' + field_size = calcsize(CONNECTIONS_FMT) + n, = unpack('!H', data) + index = calcsize('!H') + self.connections = list() + for _ in range(n): + #(mac1, mac2, rssi) + record = unpack(CONNECTIONS_FMT, data[index:]) + self.connections.append(record) + index = index + field_size + self.connections_ts = time.time() + pass + + def node_info_get(self, mac): + """ returns the RouterData or NeighborData for the specified mac """ + #try to find it as router or a neighbor of a router + node, role = self.leader_data.node_info_mac(mac) + if node is None: + return {} + # try to create dict for RPC answer + data = {} + data['ip'] = node.rloc16 + data['r'] = node.role + if role is self.STATE_CHILD: + data['a'] = node.age + elif role is self.STATE_ROUTER: + data['a'] = time.time() - node.ts + data['l'] = {'lat':node.coord[0], 'lng':node.coord[1]} + data['nn'] = node.neigh_num() + nei_macs = node.get_macs_set() + data['nei'] = list() + for nei_mac in nei_macs: + nei = node.dict[nei_mac] + data['nei'].append((nei.mac, nei.rloc16, nei.role, nei.rssi, nei.age)) + return data + + def node_info_set(self, data): + (role, ) = unpack('!B', data) + + if role is self.STATE_ROUTER: + router = RouterData(data[1:]) + self.leader_data.add_router(router) + print_debug(3, "Added as router %s"%router.to_string()) + elif role is self.STATE_CHILD: + node = NeighborData(data[1:]) + router = RouterData(node) + self.leader_data.add_router(router) + print_debug(3, "Added as Router-Neigh %s"%router.to_string()) + pass + +class NeighborData: + """ class for storing info about a Neighbor """ + #self.neigh_dict[mac] = (mac(2B), rloc16(2B), role(1B), rssi(signed char), age (1B)) + PACKING_FMT = '!HHBbB' + + def __init__(self, data = None): + self.mac = 0 + self.age = 0xFFFF + self.rloc16 = 0 + self.role = 0 + self.rssi = -150 + + if data is None: + return + + datatype = str(type(data)) + #print_debug(3, 'NeighborData __init__ %s'%str(data)) + if datatype == "": + self._init_tuple(data) + elif datatype == "": + self._init_bytes(data) + #print_debug(3, 'NeighborData done __init__') + + def _init_tuple(self, data): + #print_debug(3, '_init_tuple %s'%str(data)) + (self.mac, self.age, self.rloc16, self.role, self.rssi) = data + return + + def _init_bytes(self, data): + #print_debug(3, 'NeighborData._init_bytes %s'%str(data)) + self.mac, self.rloc16, self.role, self.rssi, self.age = unpack(self.PACKING_FMT, + data[:self.pack_fmt_size()]) + return + + def pack(self): + data = pack(self.PACKING_FMT, self.mac & 0xFFFF, self.rloc16, self.role, self.rssi, self.age) + return data + + def to_string(self): + x = 'MAC 0x%X, rloc16 0x%x, role %d, rssi %i, age %d'%(self.mac, + self.rloc16, self.role, self.rssi, self.age) + return x + + def pack_fmt_size(self): + return calcsize(self.PACKING_FMT) + +class RouterData: + #self.neigh_dict[mac] = (age, rloc16, role, rssi) + + # MAC, rloc16, lat, lon, neighbors number + PACK_HEADER_FMT = '!HHffB' + + def __init__(self, data = None): + self.mac = 0 + self.rloc16 = 0 + self.role = 0 + self.ts = 0 + self.dict = {} + self.pack_index_last = 0 + self.coord = Gps.get_location() + + if data is None: + return + + datatype = str(type(data)) + if datatype == "": + self._init_bytes(data) + elif datatype == "": + self._init_neighbordata(data) + + + def _init_bytes(self, data_pack): + + #print_debug(3, 'RouterData._init_bytes %s'%str(data_pack)) + index = calcsize(self.PACK_HEADER_FMT) + (self.mac, self.rloc16, lat, lon, neigh_num) = \ + unpack(self.PACK_HEADER_FMT, data_pack[: index]) + + self.coord = (lat, lon) + + self.role = Loramesh.STATE_ROUTER # forcer role as Router + + self.ts = time.time() + + for _ in range(neigh_num): + neighbor = NeighborData(data_pack[index:]) + + index = index + neighbor.pack_fmt_size() + + #(mac, rloc16, role, rssi, age) = unpack('!QHBBB', data_pack[index : index_new]) + + # don't add connection from Router to Leader (Leader already knows this) + if neighbor.rloc16 == self.rloc16: + continue + #record = record + (age, mac, rloc16, role, rssi) + self.dict[neighbor.mac] = neighbor + + self.pack_index_last = index + return + + def _init_neighbordata(self, data): + """ data is NeighborData """ + self.mac = data.mac + self.rloc16 = data.rloc16 + self.role = data.role + self.ts = time.time() + return + + def add_neighbor(self, neighbor): + self.dict[neighbor.mac] = neighbor + #print_debug(3, "add_neighbor type: %s"%str(type(neighbor))) + return + + def neigh_num(self): + return len(self.dict) + + def pack(self): + data = pack(self.PACK_HEADER_FMT, self.mac & 0xFFFF, \ + self.rloc16, self.coord[0], self.coord[1], len(self.dict)) + for mac, nei in self.dict.items(): + data = data + nei.pack() + return data + + def clear(self): + self.dict.clear() + return + + def to_string(self): + x = 'Router MAC 0x%X, rloc16 0x%x, coord %s, neigh_num %d, ts %d\n'\ + %(self.mac, self.rloc16, str(self.coord), len(self.dict), self.ts) + for mac, nei in self.dict.items(): + #print_debug(3, "type: %s, %s"%(str(type(nei)),str(nei))) + x = x + nei.to_string() + '\n' + return x + + def resolve_mac(self, mac): + """ returns the NeighborData for a specified mac """ + try: + nei = self.dict[mac] + except: + nei = None + return nei + + def as_dict(self): + dict = {} + dict['mac'] = self.mac + dict['ip'] = self.rloc16 + dict['role'] = self.role + dict['age'] = time.time() - self.ts + dict['loc'] = {"lat":self.coord[0], "lng":self.coord[1]} + dict['ble'] = "ble" + return dict + + def get_all_pairs(self): + lst = list() + for mac, nei in self.dict.items(): + # consider all neighbors with mac smaller than parent's + # so to not have pair (x, y) and (y, x) + if mac < self.mac: + pair = (mac, self.mac, nei.rssi) + lst = lst + [pair] + # else: + # break + return lst + + def get_macs_set(self): + """ returns set of all MACs neighbors """ + macs = set() + for mac, _ in self.dict.items(): + macs.add(mac) + return macs + +class LeaderData: + PACK_HEADER_FMT = '!QHB' + + def __init__(self, data = None): + self.routers_num = 0 + self.mac = 0 + self.rloc16 = 0 + self.ts = 0 + self.dict = {} + self.ok = False + + if data is None: + return + + datatype = str(type(data)) + if datatype == "": + self._init_bytes(data) + pass + + def _init_bytes(self, data_pack): + + #print_debug(3, 'LeaderData._init_bytes %s'%str(data_pack)) + index = calcsize(self.PACK_HEADER_FMT) + (self.mac, self.rloc16, routers_num) = unpack(self.PACK_HEADER_FMT, data_pack[: index]) + self.ts = time.time() + + for _ in range(routers_num): + router = RouterData(data_pack[index:]) + + index = index + router.pack_index_last + + self.dict[router.mac] = router + self.ok = True + pass + + def add_router(self, router_data): + self.dict[router_data.mac] = router_data + return + + def cleanup(self): + for mac, router in self.dict.items(): + if time.time() - router.ts > 300: + print_debug(3, "Deleted old Router %d"%mac) + del self.dict[mac] + return + + def pack(self): + data = pack(self.PACK_HEADER_FMT, self.mac, self.rloc16, len(self.dict)) + for mac, value in self.dict.items(): + data = data + value.pack() + return data + + def to_string(self): + x = 'Leader data: MAC %X, rloc16 %x, routers_num %d\n'%(self.mac, self.rloc16, len(self.dict)) + for mac, router in self.dict.items(): + x = x + router.to_string() + return x + + def node_info_mac(self, mac): + # first check if the MAC is a known router + router = self.dict.get(mac, None) + if router is not None: + return (router, Loramesh.STATE_ROUTER) + + # next check if MAC is a neighbor of a router + child = None + for _, router in self.dict.items(): + child = router.resolve_mac(mac) + if child is not None: + return (child, Loramesh.STATE_CHILD) + return (None, None) + + def node_info_mac_pack(self, mac): + node, role = self.node_info_mac(mac) + if node is None: + print_debug(3, "Node is None %d"%mac) + return bytes() + # pack type: RouterData or Child (basically NeighborData) + data = pack('!B', role) + data = data + node.pack() + return data + + def resolve_mac(self, mac): + """ returns the RLOC of the mac, if found """ + mac_ip = None + data, _ = self.node_info_mac(mac) + if data is not None: + mac_ip = data.rloc16 + return mac_ip + + def records_num(self): + return len(self.dict) + + def as_list(self): + lst = list() + for mac, router in self.dict.items(): + record = router.as_dict() + lst = lst + [record] + return lst + + def get_mesh_connections(self): + lst = list() + for mac, router in self.dict.items(): + record = router.get_all_pairs() + lst = lst + record + return lst + + def get_connections_pack(self): + connections = self.get_mesh_connections() + print_debug(3, "Connections "+ str(connections)) + data = pack('!H', len(connections)) + for record in connections: + (mac1, mac2, rssi) = record + data = data + pack('!HHb', mac1, mac2, rssi) + return data + + def get_macs_set(self): + macs = set() + for mac, router in self.dict.items(): + macs.add(mac) + macs = macs.union(router.get_macs_set()) + return macs + + def get_macs_pack(self): + macs = self.get_macs_set() + data = pack('!H', len(macs)) + for mac in macs: + data = data + pack('!H', mac) + #print_debug(3, "Macs pack:%s"%(str(data))) + return data + + def get_mac_ts(self, mac): + # return the ts (last time Leader received pack) of a Router + router = self.dict.get(mac, None) + if router is None: + # if this mac is not a router, just return ts as the oldest + return 0 + return router.ts diff --git a/pymesh/pymesh_frozen/lib/mesh_interface.py b/pymesh/pymesh_frozen/lib/mesh_interface.py new file mode 100644 index 0000000..0287b5e --- /dev/null +++ b/pymesh/pymesh_frozen/lib/mesh_interface.py @@ -0,0 +1,292 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import time +from machine import Timer +import _thread + +try: + from mesh_internal import MeshInternal +except: + from _mesh_internal import MeshInternal + +try: + from meshaging import Meshaging +except: + from _meshaging import Meshaging + +try: + from pymesh_debug import * +except: + from _pymesh_debug import * + +__version__ = '6' +""" +__version__ = '6' +* added loRa_mac field to get_node_info + +__version__ = '5' +* added pause/resume + +__version__ = '3' +* added file send/receive debug + +__version__ = '2' +* add sending messages + +__version__ = '1' +* initial version, lock and get_mesh_members(), get_mesh_pairs +""" + +class MeshInterface: + """ Class for Mesh interface, + all modules that uses Mesh should call only this class methods """ + + INTERVAL = const(10) + + def __init__(self, config, message_cb): + self.lock = _thread.allocate_lock() + self.meshaging = Meshaging(self.lock) + self.config = config + self.mesh = MeshInternal(self.meshaging, config, message_cb) + self.sleep_function = None + self.single_leader_ts = 0 + + self.end_device_m = False + self.is_paused = False + self._start_timer() + # self.statistics = Statistics(self.meshaging) + + pass + + def _start_timer(self): + self._timer = Timer.Alarm(self.periodic_cb, self.INTERVAL, periodic=True) + + # just run this ASAP + self.periodic_cb(None) + + def periodic_cb(self, alarm): + # wait lock forever + if self.lock.acquire(): + print_debug(2, "============ MESH THREAD >>>>>>>>>>> ") + t0 = time.ticks_ms() + + self.mesh.process() + if self.mesh.is_connected(): + # self.statistics.process() + self.mesh.process_messages() + + # if Single Leader for 3 mins should reset + # if self.mesh.mesh.state == self.mesh.mesh.STATE_LEADER and self.mesh.mesh.mesh.single(): + # if self.single_leader_ts == 0: + # # first time Single Leader, record time + # self.single_leader_ts = time.time() + # print_debug(3, "Single Leader", self.mesh.mesh.state, self.mesh.mesh.mesh.single(), + # time.time() - self.single_leader_ts) + + # if time.time() - self.single_leader_ts > 180: + # print_debug(3, "Single Leader, just reset") + # if self.sleep_function: + # self.sleep_function(1) + # else: + # # print_debug(3, "Not Single Leader", self.mesh.mesh.state, self.mesh.mesh.mesh.single()) + # self.single_leader_ts = 0 + + self.lock.release() + + print_debug(2, ">>>>>>>>>>> DONE MESH THREAD ============ %d\n"%(time.ticks_ms() - t0)) + + pass + + def pause(self): + # with self.lock: + self._timer.cancel() + self.mesh.pause() + + def resume(self, tx_dBm = 14): + self.mesh.resume(tx_dBm) + self._start_timer() + + def get_mesh_mac_list(self): + mac_list = list() + if self.lock.acquire(): + # mac_list = list(self.mesh.get_all_macs_set()) + # mac_list.sort() + mac_list = {0:list(self.mesh.get_all_macs_set())} + self.lock.release() + print_debug(3, "get_mesh_mac_list:" + str(mac_list)) + return mac_list + + def get_mesh_pairs(self): + mesh_pairs = [] + if self.lock.acquire(): + mesh_pairs = self.mesh.get_mesh_pairs() + self.lock.release() + #print_debug(3, "get_mesh_pairs: %s"%str(mesh_pairs)) + return mesh_pairs + + def set_gps(self, lng, lat): + with open('/flash/gps', 'w') as fh: + fh.write('%d;%d'.format(lng, lat)) + + def is_connected(self): + is_connected = None + if self.lock.acquire(): + is_connected = self.mesh.is_connected() + self.lock.release() + return is_connected + + def ip(self): + ip = None + if self.lock.acquire(): + ip = self.mesh.mesh.mesh.ipaddr() + self.lock.release() + return ip + + def get_node_info(self, mac_id = ""): + data = {} + try: + mac = int(mac_id) + except: + mac = self.mesh.MAC + print_debug(3, "get_node_info own mac") + if self.lock.acquire(): + data = self.mesh.node_info(mac) + data['loRa_mac'] = mac + self.lock.release() + return data + + def send_message(self, data): + ## WARNING: is locking required for just adding + ret = False + + # check if message is for BR + if len(data.get('ip','')) > 0: + with self.lock: + self.mesh.br_send(data) + return + # check input parameters + try: + mac = int(data['to']) + msg_type = data.get('ty', 0) # text type, by default + payload = data['b'] + id = int(data['id']) + ts = int(data['ts']) + except: + print_debug(1, 'send_message: wrong input params') + return False + + if self.lock.acquire(): + print_debug(3, "Send message to %d, typ %d, load %s"%(mac, msg_type, payload)) + ret = self.meshaging.send_message(mac, msg_type, payload, id, ts) + # send messages ASAP + self.mesh.process_messages() + self.lock.release() + + return ret + + def mesage_was_ack(self, mac, id): + ret = False + if self.lock.acquire(): + ret = self.meshaging.mesage_was_ack(mac, id) + self.lock.release() + #print_debug(3, "mesage_was_ack (%X, %d): %d"%(mac, id, ret)) + return ret + + def get_rcv_message(self): + """ returns a message that was received, {} if none is received """ + message = None + if self.lock.acquire(): + message = self.meshaging.get_rcv_message() + self.lock.release() + if message is not None: + (mac, id, ts, payload) = message + return {'from':mac, + 'b':payload, + 'id':id, + 'ts':ts} + return {} + + # def statistics_start(self, data): + # """ starts to do statistics based on message send/ack """ + # # data = {'mac':6, 'n':3, 't':30, 's1':10, 's2':30} + # try: + # # validate input params + # mac = int(data['mac']) + # num_mess = int(data['n']) + # timeout = int(data['t']) + # except: + # print_debug(3, "statistics_start failed") + # print_debug(3, data) + # return 0 + # if mac == self.mesh.MAC: + # data['mac'] = 2 + # res = self.statistics.add_stat_mess(data) + # return res + + # def statistics_get(self, id): + # res = self.statistics.status(id) + # print_debug(3, res) + # return res + + def br_set(self, enable, prio = 0, br_mess_cb = None): + with self.lock: + self.mesh.border_router(enable, prio, br_mess_cb) + + def ot_cli(self, command): + """ Executes commands in Openthread CLI, + see https://github.com/openthread/openthread/tree/master/src/cli """ + return self.mesh.mesh.mesh.cli(command) + + def end_device(self, state = None): + if state is None: + # read status of end_device + state = self.ot_cli('routerrole') + return state == 'Disabled' + self.end_device_m = False + state_str = 'enable' + if state == True: + self.end_device_m = True + state_str = 'disable' + ret = self.ot_cli('routerrole '+ state_str) + return ret == '' + + def leader_priority(self, weight = None): + if weight is None: + # read status of end_device + ret = self.ot_cli('leaderweight') + try: + weight = int(ret) + except: + weight = -1 + return weight + try: + x = int(weight) + except: + return False + # weight should be uint8, positive and <256 + if weight > 0xFF: + weight = 0xFF + elif weight < 0: + weight = 0 + ret = self.ot_cli('leaderweight '+ str(weight)) + return ret == '' + + # def parent(self): + # """ Returns the Parent MAC for the current Child node + # Returns 0 if node is not Child """ + + # if self.mesh.mesh.mesh.state() != self.mesh.mesh.STATE_CHILD: + # print_debug(3, "Not Child, no Parent") + # return 0 + # # try: + # parent_mac = int(self.mesh.mesh.mesh.cli('parent').split('\r\n')[0].split('Ext Addr: ')[1], 16) + # # except: + # # parent_mac = 0 + # print_debug(3, 'Parent mac is: %s'%parent_mac) + # return parent_mac diff --git a/pymesh/pymesh_frozen/lib/mesh_internal.py b/pymesh/pymesh_frozen/lib/mesh_internal.py new file mode 100644 index 0000000..4457239 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/mesh_internal.py @@ -0,0 +1,609 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import socket +import time +import utime +import ubinascii +import pycom +from machine import Timer +from struct import * + +try: + from loramesh import Loramesh +except: + from _loramesh import Loramesh + +try: + from meshaging import * +except: + from _meshaging import * + +try: + from pymesh_debug import print_debug +except: + from _pymesh_debug import print_debug + +__version__ = '7' +""" +__version__ = '7' +* added pause/resume + +__version__ = '6' +* refactorized file send/receive in state machines + +__version__ = '5' +* added file send/receive debug + +__version__ = '4' +* add sending messages + +__version__ = '3' +* initial version, in ble project +""" + + +class MeshInternal: + """ Class for internal protocol inside Mesh network """ + +################################################################################ + # Border router constants + + BR_NET_ADDRESS = '2001:cafe:cafe:cafe::/64' + + EXTERNAL_NET = '1:2:3:4::' + BR_HEADER_FMT = '!BHHHHHHHHH' + BR_MAGIC_BYTE = const(0xBB) + PACK_BR = const(0x90) + +################################################################################ + + # port number opened by all nodes for communicating neighbors + PORT_MESH_INTERNAL = const(1234) + + # each packet starts with Type(1B) and Length(2B) + PACK_HEADER_FMT = '!BH' +################################################################################ + # packs sent by Leader (received by Routers) + + # packet type for Leader to inquire Routers for their neighbors + PACK_LEADER_ASK_NEIGH = const(0x80) + + # packet type for Leader to respond to with all its data + # answer of PACK_ROUTER_ASK_LEADER_DATA + PACK_LEADER_DATA = const(0x81) + + PACK_LEADER_MACS = const(0x82) + PACK_LEADER_CONNECTIONS = const(0x83) + PACK_LEADER_MAC_DETAILS = const(0x84) +################################################################################ + # packs sent by Routers (received by Leader) + + # packet type for Routers (containing their neighbors) sent to Leader + # answer of PACK_LEADER_ASK_NEIGH + PACK_ROUTER_NEIGHBORS = const(0xF0) + + # packet type for Router to interrogate Leader data + PACK_ROUTER_ASK_LEADER_DATA = const(0xF1) + + PACK_ROUTER_ASK_MACS = const(0xF2) + PACK_ROUTER_ASK_CONNECTIONS = const(0xF3) + PACK_ROUTER_ASK_MAC_DETAILS = const(0xF4) + +################################################################################ + + # packet holding a message + PACK_MESSAGE = const(0x10) + + # packet holding a message ACK + PACK_MESSAGE_ACK = const(0x11) + +################################################################################ + + # constants for file sending + #FILE_SEND_PACKSIZE = const(750) + # PACK_FILE_SEND = const(0x20) + # PACK_FILE_SEND_ACK = const(0x21) + +################################################################################ + + # timeout for Leader to interrogate Routers + LEADER_INTERVAL = const(30) # seconds + +################################################################################ + + def __init__(self, meshaging, config, message_cb): + """ Constructor """ + # enable Thread interface + self.mesh = Loramesh(config) + + self.MAC = self.mesh.MAC + self.sock = None + self.leader_ts = -self.LEADER_INTERVAL + self.router_ts = 0 + self.leader_data_ok = False + self.interrogate_leader_ts = -self.LEADER_INTERVAL + self.messages = meshaging + self.send_table = {} + self.rx_cb_registered = False + self.file_packsize = 0 + self.file_size = 0 + self.send_f = None + self.br_handler = None + self.EXTERNAL_IP = self.EXTERNAL_NET + hex(self.MAC & 0xFFFF)[2:] + self.ext_mesh_ts = -30 + self.message_cb = message_cb + self.br_message_cb = None + pass + + def pause(self): + self.rx_cb_registered = False + self.sock = None + self.mesh.pause() + + def resume(self, tx_dBm = 14): + self.mesh.resume(tx_dBm) + + def create_socket(self): + """ create UDP socket """ + self.sock = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + self.sock.bind(self.PORT_MESH_INTERNAL) + print_debug(5, "Socket created on port %d" % self.PORT_MESH_INTERNAL) + + def process_messages(self): + """ consuming message queue """ + for mac, mess in self.messages.dict.items(): + if mess.state == Message.MESS_STATE_IP_PENDING: + mess.ip = self.mesh.ip_mac_unique(mac) + mess.mac = self.mesh.MAC + mess.last_tx_ts = time.time() + self.send_message(mess) + mess.state = Message.MESS_STATE_SENT + # elif mess.state == Message.MESS_STATE_SENT: + # # try to resend + # if time.time() - mess.last_tx_ts > 15: + # print_debug(3, "Re-transmit %x %s" % (mac, mess.ip)) + # mess.last_tx_ts = time.time() + # self.send_message(mess) + pass + + def send_message(self, message, answer = None): + """ actual sending of a message on socket """ + payload = message.pack(self.MAC, answer) + pack_type = self.PACK_MESSAGE + # if message.type == message.TYPE_IMAGE: + # pack_type = self.PACK_FILE_SEND + if payload: + print_debug(4, "Send message " + str(payload)) + self.send_pack(pack_type, payload, message.ip) + pass + + def is_connected(self): + # if detached erase all leader_data + # return true if either child, router or leader + return self.mesh.is_connected() + + def led_state(self): + self.mesh.led_state() + + def ip(self): + return self.mesh.ip() + + # def _process_router(self): + # print_debug(3, "Process Router") + # # just update internal neighbor table + # self.mesh.neighbors_update() + # # add itself and neighbors in the leader data + # self.mesh.leader_add_own_neigh() + + def _process_leader(self): + print_debug(3, "Process Leader") + + """ + if state == self.mesh.STATE_LEADER_SINGLE: + # no neighbors routers, so nothing left to do + return + """ + # cleanup old entries + self.mesh.leader_dict_cleanup() + + if time.time() - self.leader_ts < self.LEADER_INTERVAL: + return + + # ask each router + self.leader_ts = time.time() + router_list = self.mesh.routers_rloc_list(60) + router_num = min(len(router_list), 5) + idx = 0 + for router_pair in router_list[:router_num]: + (age, router) = router_pair + self.send_pack(self.PACK_LEADER_ASK_NEIGH, '', router) + print_debug(3, "Leader inquire Router %s" % router) + idx = idx + 1 + if idx < router_num: + time.sleep(.5) + + def br_send(self, data): + """ if BR is available in whole Mesh, send some data """ + ret = False + # first, make sure this node is not BR (BR data is sent directly) + if len(self.mesh.mesh.border_router()) > 0: + print_debug(3, "Node is BR, so shouldn't send data to another BR") + return False + + # check if we have a BR network prefix in ipaddr + for ip in self.mesh.ipaddr(): + if ip.startswith(self.BR_NET_ADDRESS[0:-4]): + print_debug(3, "found BR address: %s"%ip) + if time.time() - self.ext_mesh_ts >= 0: + ret = True + try: + ip = data['ip'] + port = int(data['port']) + payload = data['b'] + except: + print_debug(3, "Error parsing packet for Mesh-external") + ret = False + if ret: + self.send_pack(self.PACK_BR, payload, ip, port) + # self.send_pack(self.PACK_BR, self.debug_data(False), self.EXTERNAL_IP) + self.ext_mesh_ts = time.time() + else: + print_debug(3, "BR sending too fast") + ret = False + if not ret: + print_debug(3, "no BR (mesh-external IPv6) found") + return ret + + def process(self): + self.mesh.update_internals() + self.mesh.led_state() + print_debug(3, "%d: MAC %s(%d), State %s, Single %s" % (time.time(), + hex(self.MAC), self.MAC, self.mesh.state_string(), str(self.mesh.mesh.single()))) + print_debug(3, self.mesh.ipaddr()) + leader = self.mesh.mesh.leader() + if leader is not None: + print_debug(3,"Leader: mac %s, rloc %s, net: %s" % + (hex(leader.mac), hex(leader.rloc16), hex(leader.part_id))) + if not self.mesh.is_connected(): + return # nothing to do + + # create socket + if self.sock is None: + self.create_socket() + + if not self.rx_cb_registered: + self.rx_cb_registered = True + self.mesh.mesh.rx_cb(self.receive_all_data, None) + + # update internal neighbor table + self.mesh.neighbors_update() + + self.mesh.leader_add_own_neigh() + + # # if file to be sent + # if self.send_f is not None: + # data, ip = self.send_f.process(None) + # if len(data) > 0: + # self.send_pack(self.PACK_FILE_SEND, data, ip) + + # if self.mesh.state == self.mesh.STATE_LEADER: + # self._process_leader() + return + + def debug_data(self, br = True): + """ Creating a debug string """ + if br: + # BR can send more data + data = "%d: MAC %s(%d), State %s, Single %s" % (time.time(), + hex(self.MAC), self.MAC, self.mesh.state_string(), str(self.mesh.mesh.single())) + data = data + "\n" + str(self.mesh.ipaddr()) + data = data + "\n" + str(self.mesh.mesh.routers()) + data = data + "\n" + str(self.mesh.mesh.neighbors()) + else: + # normal node sends data over Mesh to BR, so less/compressed data + data = "%d: M=%d, %s," % (time.time(), self.MAC, self.mesh.state_string()) + data = data +" nei:" + str(self.mesh.mesh.neighbors()) + return data + + def border_router(self, enable, prio = 0, br_mess_cb = None): + """ Disables/Enables the Border Router functionality, with priority and callback """ + net_list = self.mesh.mesh.border_router() + print_debug(3, "State:" + str(enable) + "BR: "+ str(net_list)) + + if not enable: + # disable all BR network registrations (possible multiple) + self.br_handler = None + for net in net_list: + self.mesh.mesh.border_router_del(net.net) + print_debug(3, "Done remove BR") + else: + self.br_handler = br_mess_cb + # check if BR already added + try: + # print_debug(3, net[0].net) + # print_debug(3, self.BR_NET_ADDRESS) + # if net[0].net != self.BR_NET_ADDRESS: + if not net_list[0].net.startswith(self.BR_NET_ADDRESS[0:-3]): + # enable BR + self.mesh.mesh.border_router(self.BR_NET_ADDRESS, prio) + print_debug(3, "Done add BR") + except: + # enable BR + self.mesh.mesh.border_router(self.BR_NET_ADDRESS, prio) + print_debug(3, "Force add BR") + + # print again the BR, to confirm + net_list = self.mesh.mesh.border_router() + print_debug(3, "BR: " + str(net_list)) + pass + + def _check_to_send(self, pack_type, ip): + send_it = True + try: + # invent some small hash, to uniquely identify packet + key = (100 * pack_type) + int(ip[-4:], 16) + except: + # just send it + #print_debug(3, "just send it, ? ", ip) + send_it = False + if not send_it: + return True + + now = time.time() + try: + timestamp = self.send_table[key] + if now - timestamp < 35: + send_it = False + else: + self.send_table[key] = now + except: + #print_debug(3, "%s not in send_table"%str(key)) + send_it = True + + if send_it: + # mark packet as sent now + self.send_table[key] = now + #print_debug(3, "Packet sent now") + return send_it # packet already send + + def send_pack(self, pack_type, data, ip, port=PORT_MESH_INTERNAL): + if self.sock is None: + return False + + print_debug(3, "Send pack: 0x%X to IP %s" % (pack_type, ip)) + + # check not to send same (packet, destination) too often + # if not self._check_to_send(pack_type, ip): + # print_debug(3, "NO send") + # return False + + sent_ok = True + header = pack('!BH', pack_type, len(data)) + + try: + self.sock.sendto(header + data, (ip, port)) + #self.mesh.blink(2, .1) + except Exception as ex: + print_debug(3, "Socket.sendto exception: {}".format(ex)) + sent_ok = False + return sent_ok + + def get_type(self, data): + + (pack_type, len1) = unpack(self.PACK_HEADER_FMT, + data[:calcsize(self.PACK_HEADER_FMT)]) + data = data[calcsize(self.PACK_HEADER_FMT):] + + len2 = len(data) + if len1 != len2: + print_debug(3, "PACK_HEADER length not ok %d %d" % (len1, len2)) + print_debug(3, str(data)) + return + + return (pack_type, data) + + def get_mesh_pairs(self): + """ Returns the list of all pairs of nodes directly connected inside mesh """ + # try to obtain if we already have them + (pairs, pairs_ts) = self.mesh.connections_get() + + if len(pairs) or time.time() - pairs_ts > 30: + # if there's none or too old, require new one from Leader + leader_ip = self.mesh._rloc_ip_net_addr() + self.mesh.LEADER_DEFAULT_RLOC + self.send_pack(self.PACK_ROUTER_ASK_CONNECTIONS, '', leader_ip) + + return pairs + + def get_all_macs_set(self): + """ Returns the set of all distinct MACs of all nodes inside mesh """ + # try to obtain if we already have them + (macs, macs_ts) = self.mesh.macs_get() + + if len(macs) == 0 or time.time() - macs_ts > 30: + # if there's none or too old, require new one from Leader + leader_ip = self.mesh._rloc_ip_net_addr() + self.mesh.LEADER_DEFAULT_RLOC + self.send_pack(self.PACK_ROUTER_ASK_MACS, '', leader_ip) + + return macs + + def node_info(self, mac): + """ Returns the info about a specified Node inside mesh """ + # try to obtain if we already have them + node_data = self.mesh.node_info_get(mac) + + if len(node_data) == 0 or node_data['a'] > 120 or \ + node_data.get('nn', None) is None: + # if there's none or too old, require new one from the node + node_ip = self.mesh.ip_mac_unique(mac) + payload = pack('!H', mac) + self.send_pack(self.PACK_ROUTER_ASK_MAC_DETAILS, + payload, node_ip) + + return node_data + + def receive_all_data(self, arg): + """ receives all packages on socket """ + + while True: + rcv_data, rcv_addr = self.sock.recvfrom(1024) + if len(rcv_data) == 0: + break # out of while, no packet + rcv_ip = rcv_addr[0] + rcv_port = rcv_addr[1] + print_debug(4, 'Incoming %d bytes from %s (port %d):' % + (len(rcv_data), rcv_ip, rcv_port)) + # print_debug(3, rcv_data) + print_debug(5, str(self.mesh.lora.stats())) + + # check if Node is BR + if self.br_handler: + #check if data is for the external of the Pymesh (for Pybytes) + if rcv_data[0] == self.BR_MAGIC_BYTE and len(rcv_data) >= calcsize(self.BR_HEADER_FMT): + br_header = unpack(self.BR_HEADER_FMT, rcv_data) + print_debug(3, "BR pack, IP dest: %x:%x:%x:%x:%x:%x:%x:%x (port %d)"%( + br_header[1],br_header[2],br_header[3],br_header[4], + br_header[5],br_header[6],br_header[7],br_header[8], br_header[9])) + rcv_data = rcv_data[calcsize(self.BR_HEADER_FMT):] + + dest_ip = "%x:%x:%x:%x:%x:%x:%x:%x"%( + br_header[1],br_header[2],br_header[3],br_header[4], + br_header[5],br_header[6],br_header[7],br_header[8]) + + dest_port = br_header[9] + + print_debug(3, rcv_data) + (type, rcv_data) = self.get_type(rcv_data) + print_debug(3, rcv_data) + + self.br_handler(rcv_ip, rcv_port, rcv_data, dest_ip, dest_port) + return # done, no more parsing as this pack was for BR + + # check packet type + (type, rcv_data) = self.get_type(rcv_data) + # LEADER + if type == self.PACK_ROUTER_NEIGHBORS: + print_debug(3, "PACK_ROUTER_NEIGHBORS received") + self.mesh.routers_neigh_update(rcv_data) + # no answer + # elif type == self.PACK_ROUTER_ASK_LEADER_DATA: + # print_debug(3, "PACK_ROUTER_ASK_LEADER_DATA received") + # # send answer with Leader data + # pack = self.mesh.leader_data_pack() + # self.send_pack(self.PACK_LEADER_DATA, pack, rcv_ip) + + # ROUTER + elif type == self.PACK_LEADER_ASK_NEIGH: + print_debug(3, "PACK_LEADER_ASK_NEIGH received") + payload = self.mesh.neighbors_pack() + #time.sleep(.2) + self.send_pack(self.PACK_ROUTER_NEIGHBORS, payload, rcv_ip) + # elif type == self.PACK_LEADER_DATA: + # print_debug(3, "PACK_LEADER_DATA received") + # if self.mesh.leader_data_unpack(rcv_data): + # self.interrogate_leader_ts = time.time() + + # ALL NODES + elif type == self.PACK_MESSAGE: + print_debug(3, "PACK_MESSAGE received") + # add new pack received + message = Message(rcv_data) + # print_debug(3, message.payload) + message.ip = rcv_ip + self.messages.add_rcv_message(message) + + # send back ACK + self.send_pack(self.PACK_MESSAGE_ACK, message.pack_ack(self.MAC), rcv_ip) + + # forward message to user-application layer + if self.message_cb: + self.message_cb(rcv_ip, rcv_port, message.payload) + + + elif type == self.PACK_MESSAGE_ACK: + print_debug(3, "PACK_MESSAGE_ACK received") + # mark message as received + self.messages.rcv_ack(rcv_data) + + elif type == self.PACK_ROUTER_ASK_MACS: + print_debug(3, "PACK_ROUTER_ASK_MACS received") + payload = self.mesh.leader_data.get_macs_pack() + self.send_pack(self.PACK_LEADER_MACS, payload, rcv_ip) + + elif type == self.PACK_LEADER_MACS: + print_debug(3, "PACK_LEADER_MACS received") + self.mesh.macs_set(rcv_data) + + elif type == self.PACK_ROUTER_ASK_CONNECTIONS: + print_debug(3, "PACK_ROUTER_ASK_CONNECTIONS received") + payload = self.mesh.leader_data.get_connections_pack() + self.send_pack(self.PACK_LEADER_CONNECTIONS, payload, rcv_ip) + + elif type == self.PACK_LEADER_CONNECTIONS: + print_debug(3, "PACK_LEADER_CONNECTIONS received") + self.mesh.connections_set(rcv_data) + + elif type == self.PACK_ROUTER_ASK_MAC_DETAILS: + print_debug(3, "PACK_ROUTER_ASK_MAC_DETAILS received") + (mac_req, ) = unpack('!H', rcv_data) + print_debug(3, str(mac_req)) + payload = self.mesh.leader_data.node_info_mac_pack(mac_req) + if len(payload) > 0: + self.send_pack(self.PACK_LEADER_MAC_DETAILS, + payload, rcv_ip) + else: + print_debug(3, "No info found about MAC %d"%mac_req) + + elif type == self.PACK_LEADER_MAC_DETAILS: + print_debug(3, "PACK_LEADER_MAC_DETAILS received") + self.mesh.node_info_set(rcv_data) + + # elif type == self.PACK_FILE_SEND: + # print_debug(3, "PACK_FILE_SEND received") + # payload = pack("!Q", self.MAC) + # self.send_pack(self.PACK_FILE_SEND_ACK, payload, rcv_ip) + # # rcv data contains '!QHH' as header + # chunk = len(rcv_data) -12 + # self.file_size += chunk + # print_debug(3, "size: %d, chunk %d" % (self.file_size, chunk)) + # file_handler = "ab" # append, by default + # if chunk > self.file_packsize: + # # started receiving a new file + # print_debug(3, "started receiving a new image") + # file_handler = "wb" # write/create new file + # self.file_packsize = chunk + # elif chunk < self.file_packsize: + # print_debug(3, "DONE receiving the image") + # # done receiving the file + # self.file_packsize = 0 + # self.file_size = 0 + # self.messages.file_transfer_done(rcv_data[:12]) + # # else: + # # #middle of the file, just write data + # # self.file.write(rcv_data) + # with open('/flash/dog_rcv.jpg', file_handler) as file: + # file.write(rcv_data[12:]) + # print_debug(3, "writing the image") + + + # elif type == self.PACK_FILE_SEND_ACK: + # mac_rcv = unpack("!Q", rcv_data) + # print_debug(3, "PACK_FILE_SEND_ACK received from MAC %d"%mac_rcv) + # mac_rcv = 6 + # message = self.messages.dict.get(mac_rcv, None) + # if message: + # print_debug(3, "message found") + # self.send_message(message, rcv_data) + # else: + # print_debug(3, "message NOT found ", mac_rcv, self.messages.dict) + + else: + print_debug(3, "Unknown packet, type: 0x%X" % (type)) + print_debug(3, str(rcv_data)) + + pass diff --git a/pymesh/pymesh_frozen/lib/meshaging.py b/pymesh/pymesh_frozen/lib/meshaging.py new file mode 100644 index 0000000..7eef06d --- /dev/null +++ b/pymesh/pymesh_frozen/lib/meshaging.py @@ -0,0 +1,263 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import time +from struct import * + +try: + from pymesh_debug import print_debug +except: + from _pymesh_debug import print_debug + +__version__ = '1' +""" +* initial version +""" + +class Meshaging: + """ class that manages sending/receiving messages inside Mesh network """ + + PROCESS_INTERVAL = const(5) + + on_rcv_message = None + on_rcv_ack = None + + def __init__(self, lock): + #self.mesh = mesh + self.lock = lock + self.dict = {} + self.rcv_dict = {} + self.rcv_mess_new = None + + def send_message(self, mac, msg_type, payload, id, ts): + """ send a new message """ + already = self.dict.get(mac, None) + if already: + print_debug(3, 'old message deleted for %X' % mac) + message = Message((mac, msg_type, payload, id, ts)) + self.dict[mac] = message + print_debug(3, "Added new message for %X: %s" % (mac, str(payload))) + + return True + + def add_rcv_message(self, message): + """ received a new message """ + message.ts = time.time() + self.rcv_dict[message.mac] = message + self.rcv_mess_new = message + + if message.payload == b'dog':#πŸ•': + message.payload = 'Picture started receiving' + print_debug(3, 'Rcv mess about dog, so we start receiving picture') + # else: + # print_debug(3, 'payload is not dog') + + if self.on_rcv_message: + self.on_rcv_message(message) + + return True + + def rcv_ack(self, data): + """ just received an ACK for a previously sent message """ + message = Message() + message.unpack_ack(data) + + # check if message was really in send buffer + if message.mac in self.dict: + self.dict[message.mac].state = Message.MESS_STATE_ACK + + if self.on_rcv_ack: + self.on_rcv_ack(message) + + # check if message was about picture sending, to start actual file sending + mess = self.dict[message.mac] + if mess.payload == 'dog': + print_debug(3, 'ACK from dog message, start picture sending') + del self.dict[message.mac] + self.send_message(message.mac, message.TYPE_IMAGE, 'dog.jpg', message.id, time.time()) + + if self.on_rcv_message: + mess = Message((message.mac, message.TYPE_TEXT, 'Receiving the picture', message.id+1, time.time())) + self.on_rcv_message(mess) + else: + print_debug(3, str(message.mac) + str(self.dict)) + pass + + def mesage_was_ack(self, mac, id): + """ return True/False if a message was ACK """ + done = False + try: + message = self.dict[mac] + if id == message.id: + if message.state == Message.MESS_STATE_ACK: + done = True + except: + pass + print_debug(3, "ACK? mac %x, id %d => %d" % (mac, id, done)) + return done + + def get_rcv_message(self): + """ returns first message that was received, None if none received """ + if len(self.rcv_dict) == 0: + return None + + # get first message + (mac, mess) = list(self.rcv_dict.items())[0] + return (mac, mess.id, mess.ts, mess.payload) + + def file_transfer_done(self, rcv_data): + message = Message(rcv_data) + message.payload = 'Picture was received' + print_debug(3, 'Picture done receiving from %d', message.mac) + message.id = message.id + 1 + if self.on_rcv_message: + self.on_rcv_message(message) + + self.send_message(message.mac, message.TYPE_TEXT, 'Picture was received', message.id+1, time.time()) + + pass + +class Message: + + PACK_MESSAGE = '!QHH' # mac, id, payload size, and payload(char[]) + PACK_MESSAGE_ACK = '!QH' # mac, id + + #MESS_STATE_INIT = const(1) + MESS_STATE_IP_PENDING = const(2) + MESS_STATE_SENT = const(3) + MESS_STATE_ACK = const(4) + + # type of message: TEXT or IMAGE + TYPE_TEXT = const(0) + TYPE_IMAGE = const(1) + + """ class that holds a message and its properties """ + + def __init__(self, data=None): + self.local_ts = time.time() + self.ip = "0" + self.last_tx_ts = 0 + self.type = TYPE_TEXT + self.send_f = None + self.ts = time.time() + + if data is None: + return + + datatype = str(type(data)) + if datatype == "": + self._init_tuple(data) + elif datatype == "": + self._init_bytes(data) + # limit id to 2B + self.id = self.id & 0xFFFF + + def _init_tuple(self, data): + # (mac, payload, id, ts) + (self.mac, self.type, self.payload, self.id, self.ts) = data + self.state = self.MESS_STATE_IP_PENDING + if self.type == TYPE_IMAGE: + self.send_f = Send_File(self.payload) + return + + def _init_bytes(self, data): + #print_debug(3, 'NeighborData._init_bytes %s'%str(data)) + self.mac, self.id, n = unpack(self.PACK_MESSAGE, data) + self.payload = data[calcsize(self.PACK_MESSAGE):] + #self.payload = unpack('!' + str(n) + 's', data[calcsize(self.PACK_MESSAGE):]) + return + + def pack(self, sender_mac, answer): + n = len(self.payload) + data = pack(self.PACK_MESSAGE, sender_mac, self.id, n) + if self.type == TYPE_IMAGE: + if self.send_f: + file_chunk = self.send_f.process(answer) + if len(file_chunk) == 0: + self.state = MESS_STATE_ACK + data = None + else: + data = data + file_chunk + else: + #data = data + pack('!' + str(n) + 's', self.payload) + data = data + self.payload + return data + + def pack_ack(self, sender_mac): + data = pack(self.PACK_MESSAGE_ACK, sender_mac, self.id) + return data + + def unpack_ack(self, data): + (self.mac, self.id) = unpack(self.PACK_MESSAGE_ACK, data) + pass + +class Send_File: + INIT = const(1) + WAIT_ACK = const(2) + DONE = const(3) + + RETRIES_MAX = const(3) + + def __init__(self, filename): + self.packsize = 400 # packsize + self.buffer = bytearray(self.packsize) + self.mv = memoryview(self.buffer) + #self.ip = 0 # ip + self.chunk = 0 + self.filename = filename + try: + self.file = open(filename, "rb") + except: + print_debug(3, "File %s can't be opened !!!!"%filename) + self.state = DONE + return + self.size = 0 + + + self.start = time.time() + self.state = INIT + + def process(self, last_response): + if self.state == INIT: + self.chunk = self.file.readinto(self.buffer) + self.state = WAIT_ACK + self.retries = 0 + self.size = self.chunk + self.start = time.time() + + elif self.state == WAIT_ACK: + if last_response is not None: + # got ACK, send next chunk + self.chunk = self.file.readinto(self.buffer) + self.size = self.size + self.chunk + print_debug(3, "%d Bytes sent, time: %4d sec" % (self.size, time.time() - self.start)) + if self.chunk == 0: + self._end_transfer() + + self.retries = 0 + else: + print_debug(3, "No answer, so retry?") + if time.time() - self.last_ts < 5: + #if we just sent the retry, don't resend anything, still wait for answer + print_debug(3, "No retry, too soon") + return '' + self.retries = self.retries + 1 + + if self.retries > RETRIES_MAX: + self._end_transfer() + + elif self.state == DONE: + self.chunk = 0 + + self.last_ts = time.time() + return self.mv[:self.chunk] + + def _end_transfer(self): + self.state = DONE + print_debug(3, "Done sending %d B in %s sec"%(self.size, time.time() - self.start)) + self.file.close() diff --git a/pymesh/pymesh_frozen/lib/msgpack/__init__.py b/pymesh/pymesh_frozen/lib/msgpack/__init__.py new file mode 100644 index 0000000..96a710a --- /dev/null +++ b/pymesh/pymesh_frozen/lib/msgpack/__init__.py @@ -0,0 +1,59 @@ +# coding: utf-8 +from msgpack._version import version +from msgpack.exceptions import * + +from ucollections import namedtuple + + +class ExtType(namedtuple('ExtType', 'code data')): + """ExtType represents ext type in msgpack.""" + def __new__(cls, code, data): + if not isinstance(code, int): + raise TypeError("code must be int") + if not isinstance(data, bytes): + raise TypeError("data must be bytes") + if not 0 <= code <= 127: + raise ValueError("code must be 0~127") + return super(ExtType, cls).__new__(cls, code, data) + + +import os +from msgpack.fallback import Packer, unpackb, Unpacker + + +def pack(o, stream, **kwargs): + """ + Pack object `o` and write it to `stream` + + See :class:`Packer` for options. + """ + packer = Packer(**kwargs) + stream.write(packer.pack(o)) + + +def packb(o, **kwargs): + """ + Pack object `o` and return packed bytes + + See :class:`Packer` for options. + """ + return Packer(**kwargs).pack(o) + + +def unpack(stream, **kwargs): + """ + Unpack an object from `stream`. + + Raises `ExtraData` when `stream` contains extra bytes. + See :class:`Unpacker` for options. + """ + data = stream.read() + return unpackb(data, **kwargs) + + +# alias for compatibility to simplejson/marshal/pickle. +load = unpack +loads = unpackb + +dump = pack +dumps = packb diff --git a/pymesh/pymesh_frozen/lib/msgpack/_version.py b/pymesh/pymesh_frozen/lib/msgpack/_version.py new file mode 100644 index 0000000..d28f0de --- /dev/null +++ b/pymesh/pymesh_frozen/lib/msgpack/_version.py @@ -0,0 +1 @@ +version = (0, 5, 6) diff --git a/pymesh/pymesh_frozen/lib/msgpack/exceptions.py b/pymesh/pymesh_frozen/lib/msgpack/exceptions.py new file mode 100644 index 0000000..11c1598 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/msgpack/exceptions.py @@ -0,0 +1,41 @@ +class UnpackException(Exception): + """Deprecated. Use Exception instead to catch all exception during unpacking.""" + + +class BufferFull(UnpackException): + pass + + +class OutOfData(UnpackException): + pass + + +class UnpackValueError(UnpackException): + """Deprecated. Use ValueError instead.""" + + +class ExtraData(UnpackValueError): + def __init__(self, unpacked, extra): + self.unpacked = unpacked + self.extra = extra + + def __str__(self): + return "unpack(b) received extra data." + + +class PackException(Exception): + """Deprecated. Use Exception instead to catch all exception during packing.""" + + +class PackValueError(PackException): + """PackValueError is raised when type of input data is supported but it's value is unsupported. + + Deprecated. Use ValueError instead. + """ + + +class PackOverflowError(PackValueError): + """PackOverflowError is raised when integer value is out of range of msgpack support [-2**31, 2**32). + + Deprecated. Use ValueError instead. + """ diff --git a/pymesh/pymesh_frozen/lib/msgpack/fallback.py b/pymesh/pymesh_frozen/lib/msgpack/fallback.py new file mode 100644 index 0000000..c4f0543 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/msgpack/fallback.py @@ -0,0 +1,951 @@ +"""Fallback pure Python implementation of msgpack""" + +import sys +import struct + +if sys.version_info[0] == 3: + PY3 = True + int_types = int + Unicode = str + xrange = range + def dict_iteritems(d): + return d.items() +else: + PY3 = False + int_types = (int, long) + Unicode = unicode + def dict_iteritems(d): + return d.iteritems() + +from uio import BytesIO as StringIO +newlist_hint = lambda size: [] + + +from msgpack.exceptions import ( + BufferFull, + OutOfData, + UnpackValueError, + PackValueError, + PackOverflowError, + ExtraData) + +from msgpack import ExtType + + +EX_SKIP = 0 +EX_CONSTRUCT = 1 +EX_READ_ARRAY_HEADER = 2 +EX_READ_MAP_HEADER = 3 + +TYPE_IMMEDIATE = 0 +TYPE_ARRAY = 1 +TYPE_MAP = 2 +TYPE_RAW = 3 +TYPE_BIN = 4 +TYPE_EXT = 5 + +DEFAULT_RECURSE_LIMIT = 511 + + +def _check_type_strict(obj, t, type=type, tuple=tuple): + if type(t) is tuple: + return type(obj) in t + else: + return type(obj) is t + + +def _get_data_from_buffer(obj): + try: + view = memoryview(obj) + except TypeError: + # try to use legacy buffer protocol if 2.7, otherwise re-raise + if not PY3: + view = memoryview(buffer(obj)) + print("using old buffer interface to unpack %s; " + "this leads to unpacking errors if slicing is used and " + "will be removed in a future version" % type(obj), + RuntimeWarning) + else: + view = memoryview(bytes([obj])) + # raise + + # @TODO: not sure what is is for... + # if view.itemsize != 1: + # raise ValueError("cannot unpack from multi-byte object") + return view + + +def unpack(stream, **kwargs): + print( + "Direct calling implementation's unpack() is deprecated, Use msgpack.unpack() or unpackb() instead.", + PendingDeprecationWarning) + data = stream.read() + return unpackb(data, **kwargs) + + +def unpackb(packed, **kwargs): + """ + Unpack an object from `packed`. + + Raises `ExtraData` when `packed` contains extra bytes. + See :class:`Unpacker` for options. + """ + unpacker = Unpacker(None, **kwargs) + unpacker.feed(packed) + try: + ret = unpacker._unpack() + except OutOfData: + raise UnpackValueError("Data is not enough.") + if unpacker._got_extradata(): + raise ExtraData(ret, unpacker._get_extradata()) + return ret + + +class Unpacker(object): + """Streaming unpacker. + + arguments: + + :param file_like: + File-like object having `.read(n)` method. + If specified, unpacker reads serialized data from it and :meth:`feed()` is not usable. + + :param int read_size: + Used as `file_like.read(read_size)`. (default: `min(16*1024, max_buffer_size)`) + + :param bool use_list: + If true, unpack msgpack array to Python list. + Otherwise, unpack to Python tuple. (default: True) + + :param bool raw: + If true, unpack msgpack raw to Python bytes (default). + Otherwise, unpack to Python str (or unicode on Python 2) by decoding + with UTF-8 encoding (recommended). + Currently, the default is true, but it will be changed to false in + near future. So you must specify it explicitly for keeping backward + compatibility. + + *encoding* option which is deprecated overrides this option. + + :param callable object_hook: + When specified, it should be callable. + Unpacker calls it with a dict argument after unpacking msgpack map. + (See also simplejson) + + :param callable object_pairs_hook: + When specified, it should be callable. + Unpacker calls it with a list of key-value pairs after unpacking msgpack map. + (See also simplejson) + + :param str encoding: + Encoding used for decoding msgpack raw. + If it is None (default), msgpack raw is deserialized to Python bytes. + + :param str unicode_errors: + (deprecated) Used for decoding msgpack raw with *encoding*. + (default: `'strict'`) + + :param int max_buffer_size: + Limits size of data waiting unpacked. 0 means system's INT_MAX (default). + Raises `BufferFull` exception when it is insufficient. + You should set this parameter when unpacking data from untrusted source. + + :param int max_str_len: + Limits max length of str. (default: 2**31-1) + + :param int max_bin_len: + Limits max length of bin. (default: 2**31-1) + + :param int max_array_len: + Limits max length of array. (default: 2**31-1) + + :param int max_map_len: + Limits max length of map. (default: 2**31-1) + + + example of streaming deserialize from file-like object:: + + unpacker = Unpacker(file_like, raw=False) + for o in unpacker: + process(o) + + example of streaming deserialize from socket:: + + unpacker = Unpacker(raw=False) + while True: + buf = sock.recv(1024**2) + if not buf: + break + unpacker.feed(buf) + for o in unpacker: + process(o) + """ + + def __init__(self, file_like=None, read_size=0, use_list=True, raw=True, + object_hook=None, object_pairs_hook=None, list_hook=None, + encoding=None, unicode_errors=None, max_buffer_size=0, + ext_hook=ExtType, + max_str_len=2147483647, # 2**32-1 + max_bin_len=2147483647, + max_array_len=2147483647, + max_map_len=2147483647, + max_ext_len=2147483647): + + if encoding is not None: + print( + "encoding is deprecated, Use raw=False instead.", + PendingDeprecationWarning) + + if unicode_errors is None: + unicode_errors = 'strict' + + if file_like is None: + self._feeding = True + else: + if not callable(file_like.read): + raise TypeError("`file_like.read` must be callable") + self.file_like = file_like + self._feeding = False + + #: array of bytes fed. + self._buffer = bytearray() + #: Which position we currently reads + self._buff_i = 0 + + # When Unpacker is used as an iterable, between the calls to next(), + # the buffer is not "consumed" completely, for efficiency sake. + # Instead, it is done sloppily. To make sure we raise BufferFull at + # the correct moments, we have to keep track of how sloppy we were. + # Furthermore, when the buffer is incomplete (that is: in the case + # we raise an OutOfData) we need to rollback the buffer to the correct + # state, which _buf_checkpoint records. + self._buf_checkpoint = 0 + + self._max_buffer_size = max_buffer_size or 2**31-1 + if read_size > self._max_buffer_size: + raise ValueError("read_size must be smaller than max_buffer_size") + self._read_size = read_size or min(self._max_buffer_size, 16*1024) + self._raw = bool(raw) + self._encoding = encoding + self._unicode_errors = unicode_errors + self._use_list = use_list + self._list_hook = list_hook + self._object_hook = object_hook + self._object_pairs_hook = object_pairs_hook + self._ext_hook = ext_hook + self._max_str_len = max_str_len + self._max_bin_len = max_bin_len + self._max_array_len = max_array_len + self._max_map_len = max_map_len + self._max_ext_len = max_ext_len + self._stream_offset = 0 + + if list_hook is not None and not callable(list_hook): + raise TypeError('`list_hook` is not callable') + if object_hook is not None and not callable(object_hook): + raise TypeError('`object_hook` is not callable') + if object_pairs_hook is not None and not callable(object_pairs_hook): + raise TypeError('`object_pairs_hook` is not callable') + if object_hook is not None and object_pairs_hook is not None: + raise TypeError("object_pairs_hook and object_hook are mutually " + "exclusive") + if not callable(ext_hook): + raise TypeError("`ext_hook` is not callable") + + def feed(self, next_bytes): + assert self._feeding + view = _get_data_from_buffer(next_bytes) + if (len(self._buffer) - self._buff_i + len(view) > self._max_buffer_size): + raise BufferFull + + # Strip buffer before checkpoint before reading file. + if self._buf_checkpoint > 0: + # del self._buffer[:self._buf_checkpoint] + self._buffer = self._buffer[self._buf_checkpoint:] + + + self._buff_i -= self._buf_checkpoint + self._buf_checkpoint = 0 + + self._buffer += view + + def _consume(self): + """ Gets rid of the used parts of the buffer. """ + self._stream_offset += self._buff_i - self._buf_checkpoint + self._buf_checkpoint = self._buff_i + + def _got_extradata(self): + return self._buff_i < len(self._buffer) + + def _get_extradata(self): + return self._buffer[self._buff_i:] + + def read_bytes(self, n): + return self._read(n) + + def _read(self, n): + # (int) -> bytearray + self._reserve(n) + i = self._buff_i + self._buff_i = i+n + return self._buffer[i:i+n] + + def _reserve(self, n): + remain_bytes = len(self._buffer) - self._buff_i - n + + # Fast path: buffer has n bytes already + if remain_bytes >= 0: + return + + if self._feeding: + self._buff_i = self._buf_checkpoint + raise OutOfData + + # Strip buffer before checkpoint before reading file. + if self._buf_checkpoint > 0: + del self._buffer[:self._buf_checkpoint] + self._buff_i -= self._buf_checkpoint + self._buf_checkpoint = 0 + + # Read from file + remain_bytes = -remain_bytes + while remain_bytes > 0: + to_read_bytes = max(self._read_size, remain_bytes) + read_data = self.file_like.read(to_read_bytes) + if not read_data: + break + assert isinstance(read_data, bytes) + self._buffer += read_data + remain_bytes -= len(read_data) + + if len(self._buffer) < n + self._buff_i: + self._buff_i = 0 # rollback + raise OutOfData + + def _read_header(self, execute=EX_CONSTRUCT): + typ = TYPE_IMMEDIATE + n = 0 + obj = None + self._reserve(1) + b = self._buffer[self._buff_i] + self._buff_i += 1 + if b & 0b10000000 == 0: + obj = b + elif b & 0b11100000 == 0b11100000: + obj = -1 - (b ^ 0xff) + elif b & 0b11100000 == 0b10100000: + n = b & 0b00011111 + typ = TYPE_RAW + if n > self._max_str_len: + raise UnpackValueError("%s exceeds max_str_len(%s)", n, self._max_str_len) + obj = self._read(n) + elif b & 0b11110000 == 0b10010000: + n = b & 0b00001111 + typ = TYPE_ARRAY + if n > self._max_array_len: + raise UnpackValueError("%s exceeds max_array_len(%s)", n, self._max_array_len) + elif b & 0b11110000 == 0b10000000: + n = b & 0b00001111 + typ = TYPE_MAP + if n > self._max_map_len: + raise UnpackValueError("%s exceeds max_map_len(%s)", n, self._max_map_len) + elif b == 0xc0: + obj = None + elif b == 0xc2: + obj = False + elif b == 0xc3: + obj = True + elif b == 0xc4: + typ = TYPE_BIN + self._reserve(1) + n = self._buffer[self._buff_i] + self._buff_i += 1 + if n > self._max_bin_len: + raise UnpackValueError("%s exceeds max_bin_len(%s)" % (n, self._max_bin_len)) + obj = self._read(n) + elif b == 0xc5: + typ = TYPE_BIN + self._reserve(2) + n = struct.unpack_from(">H", self._buffer, self._buff_i)[0] + self._buff_i += 2 + if n > self._max_bin_len: + raise UnpackValueError("%s exceeds max_bin_len(%s)" % (n, self._max_bin_len)) + obj = self._read(n) + elif b == 0xc6: + typ = TYPE_BIN + self._reserve(4) + n = struct.unpack_from(">I", self._buffer, self._buff_i)[0] + self._buff_i += 4 + if n > self._max_bin_len: + raise UnpackValueError("%s exceeds max_bin_len(%s)" % (n, self._max_bin_len)) + obj = self._read(n) + elif b == 0xc7: # ext 8 + typ = TYPE_EXT + self._reserve(2) + L, n = struct.unpack_from('Bb', self._buffer, self._buff_i) + self._buff_i += 2 + if L > self._max_ext_len: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (L, self._max_ext_len)) + obj = self._read(L) + elif b == 0xc8: # ext 16 + typ = TYPE_EXT + self._reserve(3) + L, n = struct.unpack_from('>Hb', self._buffer, self._buff_i) + self._buff_i += 3 + if L > self._max_ext_len: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (L, self._max_ext_len)) + obj = self._read(L) + elif b == 0xc9: # ext 32 + typ = TYPE_EXT + self._reserve(5) + L, n = struct.unpack_from('>Ib', self._buffer, self._buff_i) + self._buff_i += 5 + if L > self._max_ext_len: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (L, self._max_ext_len)) + obj = self._read(L) + elif b == 0xca: + self._reserve(4) + obj = struct.unpack_from(">f", self._buffer, self._buff_i)[0] + self._buff_i += 4 + elif b == 0xcb: + self._reserve(8) + obj = struct.unpack_from(">d", self._buffer, self._buff_i)[0] + self._buff_i += 8 + elif b == 0xcc: + self._reserve(1) + obj = self._buffer[self._buff_i] + self._buff_i += 1 + elif b == 0xcd: + self._reserve(2) + obj = struct.unpack_from(">H", self._buffer, self._buff_i)[0] + self._buff_i += 2 + elif b == 0xce: + self._reserve(4) + obj = struct.unpack_from(">I", self._buffer, self._buff_i)[0] + self._buff_i += 4 + elif b == 0xcf: + self._reserve(8) + obj = struct.unpack_from(">Q", self._buffer, self._buff_i)[0] + self._buff_i += 8 + elif b == 0xd0: + self._reserve(1) + obj = struct.unpack_from("b", self._buffer, self._buff_i)[0] + self._buff_i += 1 + elif b == 0xd1: + self._reserve(2) + obj = struct.unpack_from(">h", self._buffer, self._buff_i)[0] + self._buff_i += 2 + elif b == 0xd2: + self._reserve(4) + obj = struct.unpack_from(">i", self._buffer, self._buff_i)[0] + self._buff_i += 4 + elif b == 0xd3: + self._reserve(8) + obj = struct.unpack_from(">q", self._buffer, self._buff_i)[0] + self._buff_i += 8 + elif b == 0xd4: # fixext 1 + typ = TYPE_EXT + if self._max_ext_len < 1: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (1, self._max_ext_len)) + self._reserve(2) + n, obj = struct.unpack_from("b1s", self._buffer, self._buff_i) + self._buff_i += 2 + elif b == 0xd5: # fixext 2 + typ = TYPE_EXT + if self._max_ext_len < 2: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (2, self._max_ext_len)) + self._reserve(3) + n, obj = struct.unpack_from("b2s", self._buffer, self._buff_i) + self._buff_i += 3 + elif b == 0xd6: # fixext 4 + typ = TYPE_EXT + if self._max_ext_len < 4: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (4, self._max_ext_len)) + self._reserve(5) + n, obj = struct.unpack_from("b4s", self._buffer, self._buff_i) + self._buff_i += 5 + elif b == 0xd7: # fixext 8 + typ = TYPE_EXT + if self._max_ext_len < 8: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (8, self._max_ext_len)) + self._reserve(9) + n, obj = struct.unpack_from("b8s", self._buffer, self._buff_i) + self._buff_i += 9 + elif b == 0xd8: # fixext 16 + typ = TYPE_EXT + if self._max_ext_len < 16: + raise UnpackValueError("%s exceeds max_ext_len(%s)" % (16, self._max_ext_len)) + self._reserve(17) + n, obj = struct.unpack_from("b16s", self._buffer, self._buff_i) + self._buff_i += 17 + elif b == 0xd9: + typ = TYPE_RAW + self._reserve(1) + n = self._buffer[self._buff_i] + self._buff_i += 1 + if n > self._max_str_len: + raise UnpackValueError("%s exceeds max_str_len(%s)", n, self._max_str_len) + obj = self._read(n) + elif b == 0xda: + typ = TYPE_RAW + self._reserve(2) + n, = struct.unpack_from(">H", self._buffer, self._buff_i) + self._buff_i += 2 + if n > self._max_str_len: + raise UnpackValueError("%s exceeds max_str_len(%s)", n, self._max_str_len) + obj = self._read(n) + elif b == 0xdb: + typ = TYPE_RAW + self._reserve(4) + n, = struct.unpack_from(">I", self._buffer, self._buff_i) + self._buff_i += 4 + if n > self._max_str_len: + raise UnpackValueError("%s exceeds max_str_len(%s)", n, self._max_str_len) + obj = self._read(n) + elif b == 0xdc: + typ = TYPE_ARRAY + self._reserve(2) + n, = struct.unpack_from(">H", self._buffer, self._buff_i) + self._buff_i += 2 + if n > self._max_array_len: + raise UnpackValueError("%s exceeds max_array_len(%s)", n, self._max_array_len) + elif b == 0xdd: + typ = TYPE_ARRAY + self._reserve(4) + n, = struct.unpack_from(">I", self._buffer, self._buff_i) + self._buff_i += 4 + if n > self._max_array_len: + raise UnpackValueError("%s exceeds max_array_len(%s)", n, self._max_array_len) + elif b == 0xde: + self._reserve(2) + n, = struct.unpack_from(">H", self._buffer, self._buff_i) + self._buff_i += 2 + if n > self._max_map_len: + raise UnpackValueError("%s exceeds max_map_len(%s)", n, self._max_map_len) + typ = TYPE_MAP + elif b == 0xdf: + self._reserve(4) + n, = struct.unpack_from(">I", self._buffer, self._buff_i) + self._buff_i += 4 + if n > self._max_map_len: + raise UnpackValueError("%s exceeds max_map_len(%s)", n, self._max_map_len) + typ = TYPE_MAP + else: + raise UnpackValueError("Unknown header: 0x%x" % b) + return typ, n, obj + + def _unpack(self, execute=EX_CONSTRUCT): + typ, n, obj = self._read_header(execute) + + if execute == EX_READ_ARRAY_HEADER: + if typ != TYPE_ARRAY: + raise UnpackValueError("Expected array") + return n + if execute == EX_READ_MAP_HEADER: + if typ != TYPE_MAP: + raise UnpackValueError("Expected map") + return n + # TODO should we eliminate the recursion? + if typ == TYPE_ARRAY: + if execute == EX_SKIP: + for i in xrange(n): + # TODO check whether we need to call `list_hook` + self._unpack(EX_SKIP) + return + ret = newlist_hint(n) + for i in xrange(n): + ret.append(self._unpack(EX_CONSTRUCT)) + if self._list_hook is not None: + ret = self._list_hook(ret) + # TODO is the interaction between `list_hook` and `use_list` ok? + return ret if self._use_list else tuple(ret) + if typ == TYPE_MAP: + if execute == EX_SKIP: + for i in xrange(n): + # TODO check whether we need to call hooks + self._unpack(EX_SKIP) + self._unpack(EX_SKIP) + return + if self._object_pairs_hook is not None: + ret = self._object_pairs_hook( + (self._unpack(EX_CONSTRUCT), + self._unpack(EX_CONSTRUCT)) + for _ in xrange(n)) + else: + ret = {} + for _ in xrange(n): + key = self._unpack(EX_CONSTRUCT) + ret[key] = self._unpack(EX_CONSTRUCT) + if self._object_hook is not None: + ret = self._object_hook(ret) + return ret + if execute == EX_SKIP: + return + if typ == TYPE_RAW: + if self._encoding is not None: + obj = obj.decode(self._encoding, self._unicode_errors) + elif self._raw: + obj = bytes(obj) + else: + obj = struct.pack("b"*len(obj),*obj).decode('utf8') + # obj = obj.decode('utf_8') + # obj = obj + return obj + if typ == TYPE_EXT: + return self._ext_hook(n, bytes(obj)) + if typ == TYPE_BIN: + return bytes(obj) + assert typ == TYPE_IMMEDIATE + return obj + + def __iter__(self): + return self + + def __next__(self): + try: + ret = self._unpack(EX_CONSTRUCT) + self._consume() + return ret + except OutOfData: + self._consume() + raise StopIteration + + next = __next__ + + def skip(self, write_bytes=None): + self._unpack(EX_SKIP) + if write_bytes is not None: + print("`write_bytes` option is deprecated. Use `.tell()` instead.", DeprecationWarning) + write_bytes(self._buffer[self._buf_checkpoint:self._buff_i]) + self._consume() + + def unpack(self, write_bytes=None): + ret = self._unpack(EX_CONSTRUCT) + if write_bytes is not None: + print("`write_bytes` option is deprecated. Use `.tell()` instead.", DeprecationWarning) + write_bytes(self._buffer[self._buf_checkpoint:self._buff_i]) + self._consume() + return ret + + def read_array_header(self, write_bytes=None): + ret = self._unpack(EX_READ_ARRAY_HEADER) + if write_bytes is not None: + print("`write_bytes` option is deprecated. Use `.tell()` instead.", DeprecationWarning) + write_bytes(self._buffer[self._buf_checkpoint:self._buff_i]) + self._consume() + return ret + + def read_map_header(self, write_bytes=None): + ret = self._unpack(EX_READ_MAP_HEADER) + if write_bytes is not None: + print("`write_bytes` option is deprecated. Use `.tell()` instead.", DeprecationWarning) + write_bytes(self._buffer[self._buf_checkpoint:self._buff_i]) + self._consume() + return ret + + def tell(self): + return self._stream_offset + + +class Packer(object): + """ + MessagePack Packer + + usage: + + packer = Packer() + astream.write(packer.pack(a)) + astream.write(packer.pack(b)) + + Packer's constructor has some keyword arguments: + + :param callable default: + Convert user type to builtin type that Packer supports. + See also simplejson's document. + + :param bool use_single_float: + Use single precision float type for float. (default: False) + + :param bool autoreset: + Reset buffer after each pack and return its content as `bytes`. (default: True). + If set this to false, use `bytes()` to get content and `.reset()` to clear buffer. + + :param bool use_bin_type: + Use bin type introduced in msgpack spec 2.0 for bytes. + It also enables str8 type for unicode. + + :param bool strict_types: + If set to true, types will be checked to be exact. Derived classes + from serializeable types will not be serialized and will be + treated as unsupported type and forwarded to default. + Additionally tuples will not be serialized as lists. + This is useful when trying to implement accurate serialization + for python types. + + :param str encoding: + (deprecated) Convert unicode to bytes with this encoding. (default: 'utf-8') + + :param str unicode_errors: + Error handler for encoding unicode. (default: 'strict') + """ + def __init__(self, default=None, encoding=None, unicode_errors=None, + use_single_float=False, autoreset=True, use_bin_type=False, + strict_types=False): + if encoding is None: + encoding = 'utf_8' + else: + print( + "encoding is deprecated, Use raw=False instead.", + PendingDeprecationWarning) + + if unicode_errors is None: + unicode_errors = 'strict' + + self._strict_types = strict_types + self._use_float = use_single_float + self._autoreset = autoreset + self._use_bin_type = use_bin_type + self._encoding = encoding + self._unicode_errors = unicode_errors + self._buffer = StringIO() + if default is not None: + if not callable(default): + raise TypeError("default must be callable") + self._default = default + + def _pack(self, obj, nest_limit=DEFAULT_RECURSE_LIMIT, + check=isinstance, check_type_strict=_check_type_strict): + default_used = False + if self._strict_types: + check = check_type_strict + list_types = list + else: + list_types = (list, tuple) + while True: + if nest_limit < 0: + raise PackValueError("recursion limit exceeded") + if obj is None: + return self._buffer.write(b"\xc0") + if check(obj, bool): + if obj: + return self._buffer.write(b"\xc3") + return self._buffer.write(b"\xc2") + if check(obj, int_types): + if 0 <= obj < 0x80: + return self._buffer.write(struct.pack("B", obj)) + if -0x20 <= obj < 0: + return self._buffer.write(struct.pack("b", obj)) + if 0x80 <= obj <= 0xff: + return self._buffer.write(struct.pack("BB", 0xcc, obj)) + if -0x80 <= obj < 0: + return self._buffer.write(struct.pack(">Bb", 0xd0, obj)) + if 0xff < obj <= 0xffff: + return self._buffer.write(struct.pack(">BH", 0xcd, obj)) + if -0x8000 <= obj < -0x80: + return self._buffer.write(struct.pack(">Bh", 0xd1, obj)) + if 0xffff < obj <= 0xffffffff: + return self._buffer.write(struct.pack(">BI", 0xce, obj)) + if -0x80000000 <= obj < -0x8000: + return self._buffer.write(struct.pack(">Bi", 0xd2, obj)) + if 0xffffffff < obj <= 0xffffffffffffffff: + return self._buffer.write(struct.pack(">BQ", 0xcf, obj)) + if -0x8000000000000000 <= obj < -0x80000000: + return self._buffer.write(struct.pack(">Bq", 0xd3, obj)) + if not default_used and self._default is not None: + obj = self._default(obj) + default_used = True + continue + raise PackOverflowError("Integer value out of range") + if check(obj, (bytes, bytearray)): + n = len(obj) + if n >= 2**32: + raise PackValueError("%s is too large" % type(obj).__name__) + self._pack_bin_header(n) + return self._buffer.write(obj) + if check(obj, Unicode): + if self._encoding is None: + raise TypeError( + "Can't encode unicode string: " + "no encoding is specified") + obj = obj.encode(self._encoding, self._unicode_errors) + n = len(obj) + if n >= 2**32: + raise PackValueError("String is too large") + self._pack_raw_header(n) + return self._buffer.write(obj) + if check(obj, memoryview): + n = len(obj) * obj.itemsize + if n >= 2**32: + raise PackValueError("Memoryview is too large") + self._pack_bin_header(n) + return self._buffer.write(obj) + if check(obj, float): + if self._use_float: + return self._buffer.write(struct.pack(">Bf", 0xca, obj)) + return self._buffer.write(struct.pack(">Bd", 0xcb, obj)) + if check(obj, ExtType): + code = obj.code + data = obj.data + assert isinstance(code, int) + assert isinstance(data, bytes) + L = len(data) + if L == 1: + self._buffer.write(b'\xd4') + elif L == 2: + self._buffer.write(b'\xd5') + elif L == 4: + self._buffer.write(b'\xd6') + elif L == 8: + self._buffer.write(b'\xd7') + elif L == 16: + self._buffer.write(b'\xd8') + elif L <= 0xff: + self._buffer.write(struct.pack(">BB", 0xc7, L)) + elif L <= 0xffff: + self._buffer.write(struct.pack(">BH", 0xc8, L)) + else: + self._buffer.write(struct.pack(">BI", 0xc9, L)) + self._buffer.write(struct.pack("b", code)) + self._buffer.write(data) + return + if check(obj, list_types): + n = len(obj) + self._pack_array_header(n) + for i in xrange(n): + self._pack(obj[i], nest_limit - 1) + return + if check(obj, dict): + return self._pack_map_pairs(len(obj), dict_iteritems(obj), + nest_limit - 1) + if not default_used and self._default is not None: + obj = self._default(obj) + default_used = 1 + continue + raise TypeError("Cannot serialize %r" % (obj, )) + + def pack(self, obj): + try: + self._pack(obj) + except: + self._buffer = StringIO() # force reset + raise + ret = self._buffer.getvalue() + if self._autoreset: + self._buffer = StringIO() + elif USING_STRINGBUILDER: + self._buffer = StringIO(ret) + return ret + + def pack_map_pairs(self, pairs): + self._pack_map_pairs(len(pairs), pairs) + ret = self._buffer.getvalue() + if self._autoreset: + self._buffer = StringIO() + elif USING_STRINGBUILDER: + self._buffer = StringIO(ret) + return ret + + def pack_array_header(self, n): + if n >= 2**32: + raise PackValueError + self._pack_array_header(n) + ret = self._buffer.getvalue() + if self._autoreset: + self._buffer = StringIO() + elif USING_STRINGBUILDER: + self._buffer = StringIO(ret) + return ret + + def pack_map_header(self, n): + if n >= 2**32: + raise PackValueError + self._pack_map_header(n) + ret = self._buffer.getvalue() + if self._autoreset: + self._buffer = StringIO() + elif USING_STRINGBUILDER: + self._buffer = StringIO(ret) + return ret + + def pack_ext_type(self, typecode, data): + if not isinstance(typecode, int): + raise TypeError("typecode must have int type.") + if not 0 <= typecode <= 127: + raise ValueError("typecode should be 0-127") + if not isinstance(data, bytes): + raise TypeError("data must have bytes type") + L = len(data) + if L > 0xffffffff: + raise PackValueError("Too large data") + if L == 1: + self._buffer.write(b'\xd4') + elif L == 2: + self._buffer.write(b'\xd5') + elif L == 4: + self._buffer.write(b'\xd6') + elif L == 8: + self._buffer.write(b'\xd7') + elif L == 16: + self._buffer.write(b'\xd8') + elif L <= 0xff: + self._buffer.write(b'\xc7' + struct.pack('B', L)) + elif L <= 0xffff: + self._buffer.write(b'\xc8' + struct.pack('>H', L)) + else: + self._buffer.write(b'\xc9' + struct.pack('>I', L)) + self._buffer.write(struct.pack('B', typecode)) + self._buffer.write(data) + + def _pack_array_header(self, n): + if n <= 0x0f: + return self._buffer.write(struct.pack('B', 0x90 + n)) + if n <= 0xffff: + return self._buffer.write(struct.pack(">BH", 0xdc, n)) + if n <= 0xffffffff: + return self._buffer.write(struct.pack(">BI", 0xdd, n)) + raise PackValueError("Array is too large") + + def _pack_map_header(self, n): + if n <= 0x0f: + return self._buffer.write(struct.pack('B', 0x80 + n)) + if n <= 0xffff: + return self._buffer.write(struct.pack(">BH", 0xde, n)) + if n <= 0xffffffff: + return self._buffer.write(struct.pack(">BI", 0xdf, n)) + raise PackValueError("Dict is too large") + + def _pack_map_pairs(self, n, pairs, nest_limit=DEFAULT_RECURSE_LIMIT): + self._pack_map_header(n) + for (k, v) in pairs: + self._pack(k, nest_limit - 1) + self._pack(v, nest_limit - 1) + + def _pack_raw_header(self, n): + if n <= 0x1f: + self._buffer.write(struct.pack('B', 0xa0 + n)) + elif self._use_bin_type and n <= 0xff: + self._buffer.write(struct.pack('>BB', 0xd9, n)) + elif n <= 0xffff: + self._buffer.write(struct.pack(">BH", 0xda, n)) + elif n <= 0xffffffff: + self._buffer.write(struct.pack(">BI", 0xdb, n)) + else: + raise PackValueError('Raw is too large') + + def _pack_bin_header(self, n): + if not self._use_bin_type: + return self._pack_raw_header(n) + elif n <= 0xff: + return self._buffer.write(struct.pack('>BB', 0xc4, n)) + elif n <= 0xffff: + return self._buffer.write(struct.pack(">BH", 0xc5, n)) + elif n <= 0xffffffff: + return self._buffer.write(struct.pack(">BI", 0xc6, n)) + else: + raise PackValueError('Bin is too large') + + def bytes(self): + return self._buffer.getvalue() + + def reset(self): + self._buffer = StringIO() diff --git a/pymesh/pymesh_frozen/lib/pymesh.py b/pymesh/pymesh_frozen/lib/pymesh.py new file mode 100644 index 0000000..cf7ddd2 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/pymesh.py @@ -0,0 +1,247 @@ +''' +Copyright (c) 2020, Pycom Limited. +This software is licensed under the GNU GPL version 3 or any +later version, with permitted additional terms. For more information +see the Pycom Licence v1.0 document supplied with this file, or +available at https://www.pycom.io/opensource/licensing +''' + +import os +import machine +from machine import Timer +import _thread +import sys +import time + +try: + from mesh_interface import MeshInterface +except: + from _mesh_interface import MeshInterface + +try: + from cli import Cli +except: + from _cli import Cli + +try: + from pymesh_debug import * +except: + from _pymesh_debug import * + +try: + from pymesh_config import PymeshConfig +except: + from _pymesh_config import PymeshConfig + + +__version__ = '3' +""" +__version__ = '3' +* CLI can start/stop dynamically +* replaced all print with print_debug + +__version__ = '2' +* added pause/resume + +__version__ = '1' +* not-versioned prior 5th of Febr 2020 + +""" + +class Pymesh: + + def __init__(self, config, message_cb): + # print MAC, set MAC is given and restart + + self.config = config + self.mesh = MeshInterface(self.config, message_cb) + + self.kill_all = False + self.deepsleep_timeout = 0 + self.new_lora_mac = None + + # self.mesh.statistics.sleep_function = self.deepsleep_init + self.mesh.sleep_function = self.deepsleep_init + + self.is_paused = False + self._threads_start() + + self.cli = None + + self.ble_rpc = None + if config.get("ble_api", False): + try: + from ble_rpc import BleRpc + except: + from _ble_rpc import BleRpc + + self.ble_rpc = BleRpc(self.config, self.mesh) + + def cli_start(self): + if self.cli is None: + self.cli = Cli(self.mesh, self) + self.cli.sleep = self.deepsleep_init + # self.cli_thread = _thread.start_new_thread(self.cli.process, (1, 2)) + self.cli.process(None, None) + + def deepsleep_now(self): + """ prepare scripts for graceful exit, deepsleeps if case """ + print_debug(1, "deepsleep_now") + self.mesh.pause() + if self.ble_rpc: + self.ble_rpc.terminate() + # watchdog.timer_kill() + # Gps.terminate() + # self.mesh.statistics.save_all() + print_debug(1, 'Cleanup code, all Alarms cb should be stopped') + if self.new_lora_mac: + fo = open("/flash/sys/lpwan.mac", "wb") + mac_write=bytes([0, 0, 0, 0, 0, 0, (self.new_lora_mac >> 8) & 0xFF, self.new_lora_mac & 0xFF]) + fo.write(mac_write) + fo.close() + print_debug(1, "Really LoRa MAC set to " + str(self.new_lora_mac)) + if self.deepsleep_timeout > 0: + print_debug(1, 'Going to deepsleep for %d seconds'%self.deepsleep_timeout) + time.sleep(1) + machine.deepsleep(self.deepsleep_timeout * 1000) + else: + raise Exception("Pymesh done") + sys.exit() + + def deepsleep_init(self, timeout, new_MAC = None): + """ initializes an deep-sleep sequence, that will be performed later """ + print_debug(3, "deepsleep_init") + self.deepsleep_timeout = timeout + self.kill_all = True + if new_MAC: + self.new_lora_mac = new_MAC + return + + def process(self, arg1, arg2): + try: + while True: + if self.kill_all: + self.deepsleep_now() + if self.is_paused: + # break + _thread.exit() + time.sleep(.5) + pass + + except KeyboardInterrupt: + print_debug(1, 'Got Ctrl-C') + except Exception as e: + sys.print_exception(e) + + def _threads_start(self): + _thread.start_new_thread(self.process, (1,2)) + + def pause(self): + if self.is_paused: + # print_debug(5, "Pymesh already paused") + return + + print_debug(3, "Pymesh pausing") + + self.mesh.pause() + if self.ble_rpc: + self.ble_rpc.terminate() + + self.is_paused = True + return + + def resume(self, tx_dBm = 14): + if not self.is_paused: + # print_debug(5, "Pymesh can't be resumed, not paused") + return + + print_debug(3, "Pymesh resuming") + self.is_paused = False + self._threads_start() + self.mesh.resume(tx_dBm) + + return + + def send_mess(self, mac, mess): + """ send mess to specified MAC address + data is dictionary data = { + 'to': 0x5, + 'b': 'text', + 'id': 12345, + 'ts': 123123123, + } """ + data = { + 'to': mac, + 'b': mess, + 'id': 12345, + 'ts': time.time(), + } + return self.mesh.send_message(data) + + def br_set(self, prio, br_mess_cb): + """ Enable BR functionality on this Node, with priority and callback """ + return self.mesh.br_set(True, prio, br_mess_cb) + + def br_remove(self): + """ Disable BR functionality on this Node """ + return self.mesh.br_set(False) + + def status_str(self): + message = "Role " + str(self.mesh.mesh.mesh.mesh.state()) + \ + ", Single " + str(self.mesh.mesh.mesh.mesh.single()) + \ + ", IPv6: " + str(self.mesh.mesh.mesh.mesh.ipaddr()) + return message + + def is_connected(self): + return self.mesh.is_connected() + + def send_mess_external(self, ip, port, payload): + """ send mess to specified IP+port address + data is dictionary data = { + 'ip': '1:2:3::4', + 'port': 12345, + 'to': 0x5, + 'b': 'text', + 'id': 12345, + 'ts': 123123123, + } """ + data = { + 'ip': ip, + 'port': port, + 'b': payload + } + return self.mesh.send_message(data) + + def config_get(self): + return self.config + + def config_set(self, config_json_dict): + PymeshConfig.write_config(config_json_dict) + return self.config + + def mac(self): + return self.mesh.mesh.MAC + + def ot_cli(self, command): + """ Call OpenThread internal CLI """ + return self.mesh.ot_cli(command) + + def end_device(self, state = None): + """ Set current node and End (Sleepy) Device, always a Child """ + return self.mesh.end_device(state) + + def leader_priority(self, weight = None): + """ Set for the current node the Leader Weight; + it's a 0 to 255 value, which increases/decreases probability to become Leader; + by default any node has weight of 64 """ + return self.mesh.leader_priority(weight) + + def debug_level(self, level = None): + """ Set the debug level, 0 - off; recommended levels are: + DEBUG_DEBG = const(5) + DEBUG_INFO = const(4) + DEBUG_NOTE = const(3) + DEBUG_WARN = const(2) + DEBUG_CRIT = const(1) + DEBUG_NONE = const(0) """ + return debug_level(level) diff --git a/pymesh/pymesh_frozen/lib/pymesh_config.py b/pymesh/pymesh_frozen/lib/pymesh_config.py new file mode 100644 index 0000000..74a0292 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/pymesh_config.py @@ -0,0 +1,143 @@ +''' +Copyright (c) 2020, Pycom Limited. +This software is licensed under the GNU GPL version 3 or any +later version, with permitted additional terms. For more information +see the Pycom Licence v1.0 document supplied with this file, or +available at https://www.pycom.io/opensource/licensing +''' +import json + +from network import LoRa +import machine +import time + +try: + from pymesh_debug import print_debug +except: + from _pymesh_debug import print_debug + +__version__ = '2' +""" +__version__ = '2' +* added BR_ena, BR_prio + +__version__ = '' +* first release + +""" + +class PymeshConfig: + + CONFIG_FILENAME = "/flash/pymesh_config.json" + + ############################################################ + # DEFAULT SETTINGS + + # LoRa region is one of LoRa.US915, LoRa.EU868, LoRa.AS923, LoRa.AU915 + LORA_REGION = LoRa.EU868 + + # frequency expressed in Hz, for EU868 863000000 Hz, for US915 904600000 Hz + LORA_FREQ = const(869000000) + + # bandwidth options are: LoRa.BW_125KHZ, LoRa.BW_250KHZ or LoRa.BW_500KHZ + LORA_BW = LoRa.BW_500KHZ + + # spreading factor options are 7 to 12 + LORA_SF = const(7) + + # Pymesh 128b key, used for auth. and encryption + KEY = "112233" + + # if true, Pymesh is auto-started + AUTOSTART = True + DEBUG_LEVEL = 5 + + # if true, it will start as BLE Server, to be connected with mobile app + BLE_API = False + BLE_NAME_PREFIX = "PyGo " + + ############################################################ + # Border router preference priority + BR_PRIORITY_NORM = const(0) + BR_PRIORITY_LOW = const(-1) + BR_PRIORITY_HIGH = const(1) + + # if true then this node is Border Router + BR_ENABLE = False + # the default BR priority + BR_PRIORITY = BR_PRIORITY_NORM + + def write_config(pymesh_config, force_restart = False): + cf = open(PymeshConfig.CONFIG_FILENAME, 'w') + cf.write(json.dumps(pymesh_config)) + cf.close() + + if force_restart: + print_debug(3, "write_config force restart") + time.sleep(1) + machine.deepsleep(1000) + + def check_mac(pymesh_config, MAC): + if pymesh_config.get('MAC') is None: + # if MAC config unspecified, set it to LoRa MAC + print_debug(3, "Set MAC in config file as " + str(MAC)) + pymesh_config['MAC'] = MAC + PymeshConfig.write_config(pymesh_config, False) + else: + mac_from_config = pymesh_config.get('MAC') + if mac_from_config != MAC: + print_debug(3, "MAC different"+ str(mac_from_config) + str(MAC)) + pymesh_config['MAC'] = MAC + # if MAC in config different than LoRa MAC, set LoRa MAC as in config file + fo = open("/flash/sys/lpwan.mac", "wb") + mac_write=bytes([(MAC >> 56) & 0xFF, (MAC >> 48) & 0xFF, (MAC >> 40) & 0xFF, (MAC >> 32) & 0xFF, (MAC >> 24) & 0xFF, (MAC >> 16) & 0xFF, (MAC >> 8) & 0xFF, MAC & 0xFF]) + fo.write(mac_write) + fo.close() + # print_debug(3, "reset") + PymeshConfig.write_config(pymesh_config, False) + + print_debug(3, "MAC ok" + str(MAC)) + + def read_config(MAC): + file = PymeshConfig.CONFIG_FILENAME + pymesh_config = {} + error_file = True + + try: + import json + f = open(file, 'r') + jfile = f.read() + f.close() + try: + pymesh_config = json.loads(jfile.strip()) + # pymesh_config['cfg_msg'] = "Pymesh configuration read from {}".format(file) + error_file = False + except Exception as ex: + print_debug(1, "Error reading {} file!\n Exception: {}".format(file, ex)) + except Exception as ex: + print_debug(1, "Final error reading {} file!\n Exception: {}".format(file, ex)) + + if error_file: + # config file can't be read, so it needs to be created and saved + pymesh_config = {} + print_debug(3, "Can't find" +str(file) + ", or can't be parsed as json; Set default settings and reset") + # don't write MAC, just to use the hardware one + pymesh_config['LoRa'] = {"region": PymeshConfig.LORA_REGION, + "freq": PymeshConfig.LORA_FREQ, + "bandwidth": PymeshConfig.LORA_BW, + "sf": PymeshConfig.LORA_SF} + pymesh_config['Pymesh'] = {"key": PymeshConfig.KEY} + pymesh_config['autostart'] = PymeshConfig.AUTOSTART + pymesh_config['debug'] = PymeshConfig.DEBUG_LEVEL + pymesh_config['ble_api'] = PymeshConfig.BLE_API + pymesh_config['ble_name_prefix'] = PymeshConfig.BLE_NAME_PREFIX + pymesh_config['br_ena'] = PymeshConfig.BR_ENABLE + pymesh_config['br_prio'] = PymeshConfig.BR_PRIORITY + + PymeshConfig.check_mac(pymesh_config, MAC) + print_debug(3, "Default settings:" + str(pymesh_config)) + PymeshConfig.write_config(pymesh_config, True) + + PymeshConfig.check_mac(pymesh_config, MAC) + print_debug(3, "Settings:" + str(pymesh_config)) + return pymesh_config diff --git a/pymesh/pymesh_frozen/lib/pymesh_debug.py b/pymesh/pymesh_frozen/lib/pymesh_debug.py new file mode 100644 index 0000000..ee1e515 --- /dev/null +++ b/pymesh/pymesh_frozen/lib/pymesh_debug.py @@ -0,0 +1,45 @@ + +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import pycom + +# recommended debug levels, from the most verbose to off +DEBUG_DEBG = const(5) +DEBUG_INFO = const(4) +DEBUG_NOTE = const(3) +DEBUG_WARN = const(2) +DEBUG_CRIT = const(1) +DEBUG_NONE = const(0) + +try: + DEBUG = pycom.nvs_get('pymesh_debug') +except: + DEBUG = None + + +def print_debug(level, msg): + """Print log messages into console.""" + if DEBUG is not None and level <= DEBUG: + print(msg) + +def debug_level(level): + global DEBUG + if level is None: + try: + ret = pycom.nvs_get('pymesh_debug') + except: + ret = None + return ret + try: + ret = int(level) + except: + return + + DEBUG = ret + pycom.nvs_set('pymesh_debug', DEBUG) + \ No newline at end of file diff --git a/pymesh/pymesh_frozen/lib/statistics.py b/pymesh/pymesh_frozen/lib/statistics.py new file mode 100644 index 0000000..640694e --- /dev/null +++ b/pymesh/pymesh_frozen/lib/statistics.py @@ -0,0 +1,241 @@ +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing + +import uos +import time +import ujson + +__version__ = '1' +""" +first draft +""" + +class Statistics: + """ Class for keeping profiling statistics inside Pymesh """ + + TYPE_MESSAGE = const(0) + FILENAME = '/flash/statistics.json' + + def __init__(self, meshaging): + self.meshaging = meshaging + + # dictionary with all ongoing statistics, having key a unique 32bit number + self.dict = {} + self._restore_data() + self.sleep_function = None + + def _restore_data(self): + """ restore all data from file """ + try: + f = open(self.FILENAME, 'r') + except: + print('no ', self.FILENAME) + return + # with open(self.FILENAME, 'r+') as f: + for line in f: + try: + stat = StatJob(ujson.loads(line.strip())) + if stat.valid: + self.dict[stat.id] = stat + print('Stat added: ', stat.to_string()) + except: + print("parsing failed ", line) + continue + f.close() + print("Statistics file ok?!") + pass + + def save_all(self): + """ save all data as json into a file """ + # if len(self.dict) > 0: + with open(self.FILENAME, 'w') as f: + for _, job in self.dict.items(): + f.write(ujson.dumps(job.to_dict()) + '\n') + pass + + def _get_new_id(self): + id = 1 + while (1): + x = self.dict.get(id, None) + if x is None: + break + id = id + 1 + return id + + def num(self): + return len(self.dict) + + def add_stat_mess(self, data): + ret = False + id = self._get_new_id() + print("id:", id) + stat = StatJob([id, TYPE_MESSAGE, data]) + if stat.valid: + self.dict[id] = stat + ret = stat.status() + print("Stat: added ", ret) + return ret + + def process(self): + for id, job in self.dict.items(): + # print(job.to_string()) + if job.state == job.STATE_STARTED: + + # send new message + job.last_ack = False + job.last_send = time.time() + job.last_mess_num = job.last_mess_num + 1 + payload = "Test %d, pack from node %d, #%d/%d"%(job.id, job.mac, job.last_mess_num, job.repetitions) + print("Stat: " + payload) + self.meshaging.send_message(job.mac, 0, payload, id*1000+job.last_mess_num, job.last_send) + + job.state = job.STATE_WAIT_ANS + + elif job.state == job.STATE_WAIT_ANS: + # check if last message was ack + if job.last_ack == False: + job.last_ack = self.meshaging.mesage_was_ack(job.mac, id*1000+job.last_mess_num) + # job.last_ack could be False (0 int value) or True (1 int value) + job.ack_num = job.ack_num + job.last_ack + print("Stat: last mess ack? ", job.last_ack) + + # check if it's ACK or time for a new message + if job.last_ack or time.time() - job.last_send > job.period: + print("Timeout or ACK", job.to_string()) + # check if job is done + + if job.repetitions == job.last_mess_num: + job.state = job.STATE_DONE + print("Stat: done") + self.save_all() # just to be safe + continue # no need to send another message + + job.state = job.STATE_STARTED + self.sleep(job.s1, job.s2) + + def status(self, id): + data = list() + if id == 0: # print all jobs + data = list() + for _, job in self.dict.items(): + data.append(job.status()) + elif id == 1234: # print all Active jobs + for _, job in self.dict.items(): + if job.state != job.STATE_DONE: + data.append(job.status()) + elif id == 123456: # DELETE all done jobs + for idx, job in self.dict.items(): + if job.state == job.STATE_DONE: + del self.dict[idx] + elif id == 123456789: # DELETE ALL jobs + for idx, job in self.dict.items(): + del self.dict[idx] + else: # print a specific job + stat = self.dict.get(id, None) + data.append(stat.status()) + # print(stat.to_string()) + return data + + def sleep(self, s1, s2): + # nothing to do if s2 is 0 + if s2 == 0 or s2 < s1 or not self.sleep_function: + print("no sleep") + return + # random seconds between s1 and s2 (including them) + t = s1 + (uos.urandom(1)[0] % (s2 - s1 + 1)) + self.sleep_function(t) + +class StatJob: + """ Class for keeping a message statistics """ + STATE_STARTED = const(0) + STATE_WAIT_ANS = const(1) + STATE_DONE = const(2) + + def __init__(self, data): + self.type = 0 # by default message TEXT type + self.state = STATE_STARTED + self.valid = False + self.last_send = 0 + self.last_ack = False + self.last_mess_num = 0 + self.ack_num = 0 # num of ack messages + self.s1 = 0 + self.s2 = 0 + + if type(data) is dict: + self._init_dict(data) + elif type(data) is list: + self._init_list(data) + self.last_send = time.time() - self.period + + def _init_list(self, data_list): + id, jobtype, data = data_list + self.id = id + self.type = jobtype + + try: + self.mac = data['mac'] + self.repetitions = data['n'] + self.period = data['t'] + self.s1 = data.get('s1', 0) + self.s2 = data.get('s2', 0) + except: + print("StatJob init failed") + print(data) + return + self.valid = True + self.last_send = -self.period + + def to_string(self): + text = "%d: %s Send %d mess, to %d, every %d sec\n\ + Sent %d,ack %d"%(self.id, + ('Done' if self.state == STATE_DONE else 'Ongoing'), + self.repetitions, self.mac, + self.period, self.last_mess_num, self.ack_num) + return text + + def status(self): + data = {'id':self.id, 'm': self.mac, 'left': (self.repetitions - self.last_mess_num), + 'sc': str(self.ack_num)+':'+str(self.last_mess_num)} + #'done': self.state, + return data + + def to_dict(self): + d = {'id':self.id, + 'mac':self.mac, + 'period': self.period, + 'repetitions': self.repetitions, + 'ack_num': self.ack_num, + 'state': self.state, + 'last_send': self.last_send, + 'last_ack': self.last_ack, + 'last_mess_num': self.last_mess_num, + 'type': self.type, + 's1': self.s1, + 's2': self.s2, + } + return d + + def _init_dict(self, d): + self.valid = True + try: + self.id = d['id'] + self.mac = d['mac'] + self.period = d['period'] + self.repetitions = d['repetitions'] + self.ack_num = d['ack_num'] + self.state = d['state'] + self.last_send = d['last_send'] + self.last_ack = d['last_ack'] + self.last_mess_num = d['last_mess_num'] + self.type = d['type'] + self.s1 = d['s1'] + self.s2 = d['s2'] + except: + print("error parsing ", d) + self.valid = False + return diff --git a/pymesh/pymesh_frozen/lorawan/lorawan.py b/pymesh/pymesh_frozen/lorawan/lorawan.py new file mode 100644 index 0000000..4199f20 --- /dev/null +++ b/pymesh/pymesh_frozen/lorawan/lorawan.py @@ -0,0 +1,59 @@ +""" OTAA Node example compatible with the LoPy Nano Gateway """ + +from network import LoRa +from network import WLAN +import socket +import binascii +import struct +import time +from machine import RTC + +class Lorawan: + + def __init__(self): + # create an OTA authentication params + self.dev_eui = binascii.unhexlify('007926C9EAE4C922') + self.app_eui = binascii.unhexlify('70B3D57ED001D8C8') + self.app_key = binascii.unhexlify('2C4D6AE9CEBA8B0EB4430C33C17750CB') + + def send(self): + t0 = time.time() + print("LoRaWAN start") + lora = LoRa(mode=LoRa.LORAWAN, region=LoRa.EU868) + + # set the 3 default channels to the same frequency (must be before sending the OTAA join request) + lora.add_channel(0, frequency=config.LORA_FREQUENCY, dr_min=0, dr_max=5) + lora.add_channel(1, frequency=config.LORA_FREQUENCY, dr_min=0, dr_max=5) + lora.add_channel(2, frequency=config.LORA_FREQUENCY, dr_min=0, dr_max=5) + + # join a network using OTAA + lora.join(activation=LoRa.OTAA, auth=(self.dev_eui, self.app_eui, self.app_key), timeout=0, dr=config.LORA_NODE_DR) + + # wait until the module has joined the network + while not lora.has_joined(): + time.sleep(2.5) + print('Not joined yet...', time.localtime()) + + # remove all the non-default channels + for i in range(3, 16): + lora.remove_channel(i) + + # create a LoRa socket + s = socket.socket(socket.AF_LORA, socket.SOCK_RAW) + + # set the LoRaWAN data rate + s.setsockopt(socket.SOL_LORA, socket.SO_DR, config.LORA_NODE_DR) + + # make the socket non-blocking + s.setblocking(False) + + pkt = b'PKT #' + bytes([i]) + print('Sending:', pkt) + s.send(pkt) + print("LoRaWAN done in ", time.time() - t0, " seconds") + +class config: + # for EU868 + LORA_FREQUENCY = 867500000 + LORA_GW_DR = "SF7BW125" # DR_5 + LORA_NODE_DR = 5 diff --git a/pymesh/pymesh_frozen/lorawan/main.py b/pymesh/pymesh_frozen/lorawan/main.py new file mode 100644 index 0000000..33f33da --- /dev/null +++ b/pymesh/pymesh_frozen/lorawan/main.py @@ -0,0 +1,66 @@ +import time +import ubinascii +import pycom +from network import LoRa + +try: + from pymesh_config import PymeshConfig +except: + from _pymesh_config import PymeshConfig + +try: + from pymesh import Pymesh +except: + from _pymesh import Pymesh + +def new_message_cb(rcv_ip, rcv_port, rcv_data): + ''' callback triggered when a new packet arrived ''' + print('Incoming %d bytes from %s (port %d):' % + (len(rcv_data), rcv_ip, rcv_port)) + print(rcv_data) + + # user code to be inserted, to send packet to the designated Mesh-external interface + for _ in range(3): + pycom.rgbled(0x888888) + time.sleep(.2) + pycom.rgbled(0) + time.sleep(.1) + return + + +pycom.heartbeat(False) + +lora = LoRa(mode=LoRa.LORA, region= LoRa.EU868) +lora_mac = int(str(ubinascii.hexlify(lora.mac()))[2:-1], 16) + +# read config file, or set default values +pymesh_config = PymeshConfig.read_config(lora_mac) + +#initialize Pymesh +pymesh = Pymesh(pymesh_config, new_message_cb) + +while not pymesh.is_connected(): + print(pymesh.status_str()) + time.sleep(3) + +# send message to the Node having MAC address 5 +pymesh.send_mess(20, "Hello World") + +print("done Pymesh init, forever loop, exit/stop with Ctrl+C multiple times") + +from lorawan import Lorawan +lorawan = Lorawan() +t0 = time.time() + +while True: + if time.time() - t0 > 60: + pymesh.pause() + + lorawan.send() + + pymesh.resume() + t0 = time.time() + if time.time() - t0 > 35: + pymesh.send_mess(20, "heloo again, #22 here, " + str(time.time())) + + time.sleep(5) diff --git a/pymesh/pymesh_frozen/main-pybytes.py b/pymesh/pymesh_frozen/main-pybytes.py new file mode 100644 index 0000000..f38d127 --- /dev/null +++ b/pymesh/pymesh_frozen/main-pybytes.py @@ -0,0 +1,22 @@ +import time +import pycom + +# todo: add try/except for checking pybytes object exists +pymesh = pybytes.__pymesh.__pymesh + +print("Set maximum debug level, disable debug using pymesh.debug_level(0)") +pymesh.debug_level(5) + +while not pymesh.is_connected(): + print(pymesh.status_str()) + time.sleep(3) + +print(pymesh.status_str()) +# send message to the Node having MAC address 5 +pymesh.send_mess(18, "Hello World") + +print("done Pymesh init, CLI is started, h - help/command list, stop - CLI will be stopped") +pymesh.cli_start() + +# send a packet to Pybytes, thru the BR (if any enabled) +# pybytes.send_signal(100, "Hello from device" + str(pymesh.mac())) diff --git a/pymesh/pymesh_frozen/main.py b/pymesh/pymesh_frozen/main.py new file mode 100644 index 0000000..5d7c17a --- /dev/null +++ b/pymesh/pymesh_frozen/main.py @@ -0,0 +1,81 @@ +import time +import ubinascii +import pycom +from network import LoRa + +try: + from pymesh_config import PymeshConfig +except: + from _pymesh_config import PymeshConfig + +try: + from pymesh import Pymesh +except: + from _pymesh import Pymesh + +def new_message_cb(rcv_ip, rcv_port, rcv_data): + ''' callback triggered when a new packet arrived ''' + print('Incoming %d bytes from %s (port %d):' % + (len(rcv_data), rcv_ip, rcv_port)) + print(rcv_data) + + # user code to be inserted, to send packet to the designated Mesh-external interface + for _ in range(3): + pycom.rgbled(0x888888) + time.sleep(.2) + pycom.rgbled(0) + time.sleep(.1) + return + + +pycom.heartbeat(False) + +lora = LoRa(mode=LoRa.LORA, region= LoRa.EU868) +lora_mac = int(str(ubinascii.hexlify(lora.mac()))[2:-1], 16) + +# read config file, or set default values +pymesh_config = PymeshConfig.read_config(lora_mac) + +#initialize Pymesh +pymesh = Pymesh(pymesh_config, new_message_cb) + +# mac = pymesh.mac() +# if mac > 10: +# pymesh.end_device(True) +# elif mac == 5: +# pymesh.leader_priority(255) + +while not pymesh.is_connected(): + print(pymesh.status_str()) + time.sleep(3) + +# send message to the Node having MAC address 5 +pymesh.send_mess(5, "Hello World") + +# def new_br_message_cb(rcv_ip, rcv_port, rcv_data, dest_ip, dest_port): +# ''' callback triggered when a new packet arrived for the current Border Router, +# having destination an IP which is external from Mesh ''' +# print('Incoming %d bytes from %s (port %d), to external IPv6 %s (port %d)' % +# (len(rcv_data), rcv_ip, rcv_port, dest_ip, dest_port)) +# print(rcv_data) + +# # user code to be inserted, to send packet to the designated Mesh-external interface +# # ... +# return + +# add current node as Border Router, with a priority and a message handler callback +# pymesh.br_set(PymeshConfig.BR_PRIORITY_NORM, new_br_message_cb) + +# remove Border Router function from current node +# pymesh.br_remove() + +# send data for Mesh-external, basically to the BR +# ip = "1:2:3::4" +# port = 5555 +# pymesh.send_mess_external(ip, port, "Hello World") + +print("done Pymesh init, CLI is started, h - help/command list, stop - CLI will be stopped") +pymesh.cli_start() + +# while True: +# time.sleep(3) diff --git a/pymesh/pymesh_frozen/main_BR.py b/pymesh/pymesh_frozen/main_BR.py new file mode 100644 index 0000000..fdb9780 --- /dev/null +++ b/pymesh/pymesh_frozen/main_BR.py @@ -0,0 +1,135 @@ +import time +import ubinascii +import pycom +from network import LoRa + +# 2 = test pybytes OTA feature +# 4 = added device_id (pybytes token) in the packets to BR +__VERSION__ = 4 + +try: + from pymesh_config import PymeshConfig +except: + from _pymesh_config import PymeshConfig + +try: + from pymesh import Pymesh +except: + from _pymesh import Pymesh + +PACK_TOCKEN_PREFIX = "tkn" +PACK_TOCKEN_SEP = "#" + +print("Scripts version ", __VERSION__) + +if 'pybytes' not in globals(): + pybytes = None + +def new_message_cb(rcv_ip, rcv_port, rcv_data): + ''' callback triggered when a new packet arrived ''' + print('Incoming %d bytes from %s (port %d):' % + (len(rcv_data), rcv_ip, rcv_port)) + print(rcv_data) + + # user code to be inserted, to send packet to the designated Mesh-external interface + for _ in range(3): + pycom.rgbled(0x888888) + time.sleep(.2) + pycom.rgbled(0) + time.sleep(.1) + return + +def new_br_message_cb(rcv_ip, rcv_port, rcv_data, dest_ip, dest_port): + ''' callback triggered when a new packet arrived for the current Border Router, + having destination an IP which is external from Mesh ''' + print('Incoming %d bytes from %s (port %d), to external IPv6 %s (port %d)' % + (len(rcv_data), rcv_ip, rcv_port, dest_ip, dest_port)) + print(rcv_data) + + for _ in range(2): + pycom.rgbled(0x0) + time.sleep(.1) + # pycom.rgbled(0x001010) + pycom.rgbled(0x663300) + # time.sleep(.2) + + if pybytes is not None and pybytes.isconnected(): + # try to find Pybytes Token if include in rcv_data + token = "" + if rcv_data.startswith(PACK_TOCKEN_PREFIX): + x = rcv_data.split(PACK_TOCKEN_SEP.encode()) + if len(x)>2: + token = x[1] + rcv_data = rcv_data[len(PACK_TOCKEN_PREFIX) + len(token) + len(PACK_TOCKEN_SEP):] + pkt = 'BR %d B from %s (%s), to %s ( %d): %s'%(len(rcv_data), token, rcv_ip, dest_ip, dest_port, str(rcv_data)) + pybytes.send_signal(1, pkt) + + return + +pycom.heartbeat(False) + +lora = LoRa(mode=LoRa.LORA, region= LoRa.EU868) +lora_mac = int(str(ubinascii.hexlify(lora.mac()))[2:-1], 16) + +# read config file, or set default values +pymesh_config = PymeshConfig.read_config(lora_mac) + +#initialize Pymesh +pymesh = Pymesh(pymesh_config, new_message_cb) + +# mac = pymesh.mac() +# if mac > 10: +# pymesh.end_device(True) +# elif mac == 5: +# pymesh.leader_priority(255) + +while not pymesh.is_connected(): + print(pymesh.status_str()) + time.sleep(3) + +# send message to the Node having MAC address 5 +pymesh.send_mess(2, "Hello World") + + +print("done Pymesh init, forever loop, exit/stop with Ctrl+C multiple times") +# set BR with callback +if pybytes is not None and pybytes.isconnected(): + pybytes.send_signal(1, "RESTART") + +pyb_port = pymesh.mac() & 0xFFFF +pyb_ip = '1:2:3::' + hex(pyb_port)[2:] +pkt_start = "" +# add pybytes token +if pybytes is not None: + pyb_dev_id = pybytes.get_config().get("device_id", "None") + pkt_start = PACK_TOCKEN_PREFIX + PACK_TOCKEN_SEP + pyb_dev_id + PACK_TOCKEN_SEP +pkt_start = pkt_start + "Hello, from " + str(pymesh.mac()) + ", time " + + +br_enabled = False + +while True: + # add current node as Border Router, with a priority and a message handler callback + + free_mem = pycom.get_free_heap() + + if pymesh_config.get("br_ena", False): + if pybytes is not None and pybytes.isconnected(): + if not br_enabled: + br_enabled = True + print("Set as BR") + pymesh.br_set(PymeshConfig.BR_PRIORITY_NORM, new_br_message_cb) + + pybytes.send_signal(1, str(pymesh.mac()) +" : " + str(time.time()) + "s, "+ str(free_mem)) + print("Send to Pyb,", free_mem) + else: # not connected anymore to pybytes + if br_enabled: + br_enabled = False + print("Remove as BR") + pymesh.br_remove() + else: # not MAC_BR + pkt = pkt_start + str(time.time()) + ", mem " + str(free_mem) + pymesh.send_mess_external(pyb_ip, pyb_port, pkt) + print("Sending to BR: ", pkt) + + time.sleep(20) diff --git a/pymesh/readme.md b/pymesh/readme.md new file mode 100644 index 0000000..0be1a7a --- /dev/null +++ b/pymesh/readme.md @@ -0,0 +1,3 @@ +# For reference only + +This library is the open-source part of the Pymesh LoRa Firmware which can be provisioned through Pybytes and cannot be used as stand-alone on a Pybytes or Pygate type firmware. Please check the [Pycom Documentation](https://docs.pycom.io/pybytes/pymeshintegration/provisioning/) on how to get Pymesh type firmware provisioned to your device. You will not have to use this library in your project or flash it to your device separately, as it is already included in the firmware. diff --git a/pysense/lib/pysense.py b/pysense/lib/pysense.py deleted file mode 100644 index 394f812..0000000 --- a/pysense/lib/pysense.py +++ /dev/null @@ -1,8 +0,0 @@ -from pycoproc import Pycoproc - -__version__ = '1.4.0' - -class Pysense(Pycoproc): - - def __init__(self, i2c=None, sda='P22', scl='P21'): - Pycoproc.__init__(self, i2c, sda, scl) diff --git a/pysense/main.py b/pysense/main.py deleted file mode 100644 index fa08ccd..0000000 --- a/pysense/main.py +++ /dev/null @@ -1,31 +0,0 @@ -# See https://docs.pycom.io for more information regarding library specifics - -from pysense import Pysense -from LIS2HH12 import LIS2HH12 -from SI7006A20 import SI7006A20 -from LTR329ALS01 import LTR329ALS01 -from MPL3115A2 import MPL3115A2,ALTITUDE,PRESSURE - -py = Pysense() -mp = MPL3115A2(py,mode=ALTITUDE) # Returns height in meters. Mode may also be set to PRESSURE, returning a value in Pascals -si = SI7006A20(py) -lt = LTR329ALS01(py) -li = LIS2HH12(py) - -print("MPL3115A2 temperature: " + str(mp.temperature())) -print("Altitude: " + str(mp.altitude())) -mpp = MPL3115A2(py,mode=PRESSURE) # Returns pressure in Pa. Mode may also be set to ALTITUDE, returning a value in meters -print("Pressure: " + str(mpp.pressure())) - -print("Temperature: " + str(si.temperature())+ " deg C and Relative Humidity: " + str(si.humidity()) + " %RH") -print("Dew point: "+ str(si.dew_point()) + " deg C") -t_ambient = 24.4 -print("Humidity Ambient for " + str(t_ambient) + " deg C is " + str(si.humid_ambient(t_ambient)) + "%RH") - -print("Light (channel Blue lux, channel Red lux): " + str(lt.light())) - -print("Acceleration: " + str(li.acceleration())) -print("Roll: " + str(li.roll())) -print("Pitch: " + str(li.pitch())) - -print("Battery voltage: " + str(py.read_battery_voltage())) diff --git a/pytrack/.DS_Store b/pytrack/.DS_Store deleted file mode 100644 index a41a491..0000000 Binary files a/pytrack/.DS_Store and /dev/null differ diff --git a/pytrack/lib/LIS2HH12.py b/pytrack/lib/LIS2HH12.py deleted file mode 100644 index cc421f3..0000000 --- a/pytrack/lib/LIS2HH12.py +++ /dev/null @@ -1,146 +0,0 @@ -import math -import time -import struct -from machine import Pin - - -FULL_SCALE_2G = const(0) -FULL_SCALE_4G = const(2) -FULL_SCALE_8G = const(3) - -ODR_POWER_DOWN = const(0) -ODR_10_HZ = const(1) -ODR_50_HZ = const(2) -ODR_100_HZ = const(3) -ODR_200_HZ = const(4) -ODR_400_HZ = const(5) -ODR_800_HZ = const(6) - -ACC_G_DIV = 1000 * 65536 - -class LIS2HH12: - - ACC_I2CADDR = const(30) - - PRODUCTID_REG = const(0x0F) - CTRL1_REG = const(0x20) - CTRL2_REG = const(0x21) - CTRL3_REG = const(0x22) - CTRL4_REG = const(0x23) - CTRL5_REG = const(0x24) - ACC_X_L_REG = const(0x28) - ACC_X_H_REG = const(0x29) - ACC_Y_L_REG = const(0x2A) - ACC_Y_H_REG = const(0x2B) - ACC_Z_L_REG = const(0x2C) - ACC_Z_H_REG = const(0x2D) - ACT_THS = const(0x1E) - ACT_DUR = const(0x1F) - - def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): - if pysense is not None: - self.i2c = pysense.i2c - else: - from machine import I2C - self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl)) - - self.reg = bytearray(1) - self.odr = 0 - self.full_scale = 0 - self.x = 0 - self.y = 0 - self.z = 0 - self.int_pin = None - self.act_dur = 0 - self.debounced = False - - self.scales = {FULL_SCALE_2G: 4000, FULL_SCALE_4G: 8000, FULL_SCALE_8G: 16000} - self.odrs = [0, 10, 50, 100, 200, 400, 800] - - whoami = self.i2c.readfrom_mem(ACC_I2CADDR , PRODUCTID_REG, 1) - if (whoami[0] != 0x41): - raise ValueError("LIS2HH12 not found") - - # enable acceleration readings at 50Hz - self.set_odr(ODR_50_HZ) - - # change the full-scale to 4g - self.set_full_scale(FULL_SCALE_4G) - - # set the interrupt pin as active low and open drain - self.i2c.readfrom_mem_into(ACC_I2CADDR , CTRL5_REG, self.reg) - self.reg[0] |= 0b00000011 - self.i2c.writeto_mem(ACC_I2CADDR , CTRL5_REG, self.reg) - - # make a first read - self.acceleration() - - def acceleration(self): - x = self.i2c.readfrom_mem(ACC_I2CADDR , ACC_X_L_REG, 2) - self.x = struct.unpack('

+ +# Pytrack, Pysense and Pyscan libraries + +## Introduction +This directory contains libraries and out of the box examples for Pysense, Pytrack, and Pyscan, both versions 1 and 2.0x of these shields. + +1. Upload all the files to your module +2. Open Pymakr +3. Run an example that matches to your shield + +Note: For using Pyscan, you need to upload either MFRC630.mpy or MFRC630.py. +It is always recommended to use MFRC630.mpy as it takes less space on your module. diff --git a/pytrack/lib/L76GNSS.py b/shields/lib/L76GNSS.py similarity index 57% rename from pytrack/lib/L76GNSS.py rename to shields/lib/L76GNSS.py index 5c81170..c493eb1 100644 --- a/pytrack/lib/L76GNSS.py +++ b/shields/lib/L76GNSS.py @@ -1,13 +1,24 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + from machine import Timer import time import gc import binascii + class L76GNSS: GPS_I2CADDR = const(0x10) - def __init__(self, pytrack=None, sda='P22', scl='P21', timeout=None): + def __init__(self, pytrack=None, sda='P22', scl='P21', timeout=None, buffer=64): if pytrack is not None: self.i2c = pytrack.i2c else: @@ -18,12 +29,13 @@ def __init__(self, pytrack=None, sda='P22', scl='P21', timeout=None): self.timeout = timeout self.timeout_status = True + self.buffer = buffer self.reg = bytearray(1) self.i2c.writeto(GPS_I2CADDR, self.reg) def _read(self): - self.reg = self.i2c.readfrom(GPS_I2CADDR, 64) + self.reg = self.i2c.readfrom(GPS_I2CADDR, self.buffer) return self.reg def _convert_coords(self, gngll_s): @@ -39,22 +51,25 @@ def _convert_coords(self, gngll_s): def coordinates(self, debug=False): lat_d, lon_d, debug_timeout = None, None, False - if self.timeout != None: + if self.timeout is not None: self.chrono.reset() self.chrono.start() nmea = b'' while True: - if self.timeout != None and self.chrono.read() >= self.timeout: + if self.timeout is not None and self.chrono.read() >= self.timeout: self.chrono.stop() chrono_timeout = self.chrono.read() self.chrono.reset() self.timeout_status = False debug_timeout = True - if self.timeout_status != True: + if not self.timeout_status: gc.collect() break nmea += self._read().lstrip(b'\n\n').rstrip(b'\n\n') gngll_idx = nmea.find(b'GNGLL') + gpgll_idx = nmea.find(b'GPGLL') + if gngll_idx < 0 and gpgll_idx >= 0: + gngll_idx = gpgll_idx if gngll_idx >= 0: gngll = nmea[gngll_idx:] e_idx = gngll.find(b'\r\n') @@ -71,8 +86,8 @@ def coordinates(self, debug=False): break else: gc.collect() - if len(nmea) > 4096: - nmea = b'' + if len(nmea) > 410: # i suppose it can be safely changed to 82, which is longest NMEA frame + nmea = nmea[-5:] # $GNGL without last L time.sleep(0.1) self.timeout_status = True if debug and debug_timeout: @@ -80,3 +95,25 @@ def coordinates(self, debug=False): return(None, None) else: return(lat_d, lon_d) + + def dump_nmea(self): + nmea = b'' + while True: + nmea = self._read().lstrip(b'\n\n').rstrip(b'\n\n') + start_idx = nmea.find(b'$') + #print('raw[{}]: {}'.format(start_idx, nmea)) + if nmea is not None and len(nmea) > 0: + if start_idx != 0: + if len(nmea[:start_idx]) > 1: + print('{}'.format(nmea[:start_idx].decode('ASCII')), end='') + if len(nmea[start_idx:]) > 1: + print('{}'.format(nmea[start_idx:].decode('ASCII')), end='') + + def _checksum(self, nmeadata): + calc_cksum = 0 + for s in nmeadata: + calc_cksum ^= ord(s) + return('{:02X}'.format(calc_cksum)) + + def write(self, data): + self.i2c.writeto(GPS_I2CADDR, '${}*{}\r\n'.format(data, self._checksum(data)) ) diff --git a/pysense/lib/LIS2HH12.py b/shields/lib/LIS2HH12.py similarity index 58% rename from pysense/lib/LIS2HH12.py rename to shields/lib/LIS2HH12.py index cc421f3..fffa757 100644 --- a/pysense/lib/LIS2HH12.py +++ b/shields/lib/LIS2HH12.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import math import time import struct @@ -18,6 +28,7 @@ ACC_G_DIV = 1000 * 65536 + class LIS2HH12: ACC_I2CADDR = const(30) @@ -37,6 +48,9 @@ class LIS2HH12: ACT_THS = const(0x1E) ACT_DUR = const(0x1F) + SCALES = {FULL_SCALE_2G: 4000, FULL_SCALE_4G: 8000, FULL_SCALE_8G: 16000} + ODRS = [0, 10, 50, 100, 200, 400, 800] + def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): if pysense is not None: self.i2c = pysense.i2c @@ -44,7 +58,6 @@ def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): from machine import I2C self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl)) - self.reg = bytearray(1) self.odr = 0 self.full_scale = 0 self.x = 0 @@ -54,9 +67,6 @@ def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): self.act_dur = 0 self.debounced = False - self.scales = {FULL_SCALE_2G: 4000, FULL_SCALE_4G: 8000, FULL_SCALE_8G: 16000} - self.odrs = [0, 10, 50, 100, 200, 400, 800] - whoami = self.i2c.readfrom_mem(ACC_I2CADDR , PRODUCTID_REG, 1) if (whoami[0] != 0x41): raise ValueError("LIS2HH12 not found") @@ -68,9 +78,7 @@ def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): self.set_full_scale(FULL_SCALE_4G) # set the interrupt pin as active low and open drain - self.i2c.readfrom_mem_into(ACC_I2CADDR , CTRL5_REG, self.reg) - self.reg[0] |= 0b00000011 - self.i2c.writeto_mem(ACC_I2CADDR , CTRL5_REG, self.reg) + self.set_register(CTRL5_REG, 3, 0, 3) # make a first read self.acceleration() @@ -82,7 +90,7 @@ def acceleration(self): self.y = struct.unpack(' self.SCALES[self.full_scale]: + error = "threshold %d exceeds full scale %d" % (threshold, self.SCALES[self.full_scale]) + print(error) + raise ValueError(error) - self.i2c.writeto_mem(ACC_I2CADDR , ACT_THS, _ths) - self.i2c.writeto_mem(ACC_I2CADDR , ACT_DUR, _dur) + if threshold < self.SCALES[self.full_scale] / 128: + error = "threshold %d below resolution %d" % (threshold, self.SCALES[self.full_scale]/128) + print(error) + raise ValueError(error) + + if duration > 255 * 1000 * 8 / self.ODRS[self.odr]: + error = "duration %d exceeds max possible value %d" % (duration, 255 * 1000 * 8 / self.ODRS[self.odr]) + print(error) + raise ValueError(error) + + if duration < 1000 * 8 / self.ODRS[self.odr]: + error = "duration %d below resolution %d" % (duration, 1000 * 8 / self.ODRS[self.odr]) + print(error) + raise ValueError(error) + + _ths = int(127 * threshold / self.SCALES[self.full_scale]) & 0x7F + _dur = int((duration * self.ODRS[self.odr]) / 1000 / 8) + + self.i2c.writeto_mem(ACC_I2CADDR, ACT_THS, _ths) + self.i2c.writeto_mem(ACC_I2CADDR, ACT_DUR, _dur) # enable the activity/inactivity interrupt - self.i2c.readfrom_mem_into(ACC_I2CADDR , CTRL3_REG, self.reg) - self.reg[0] |= 0b00100000 - self.i2c.writeto_mem(ACC_I2CADDR , CTRL3_REG, self.reg) + self.set_register(CTRL3_REG, 1, 5, 1) self._user_handler = handler self.int_pin = Pin('P13', mode=Pin.IN) self.int_pin.callback(trigger=Pin.IRQ_FALLING | Pin.IRQ_RISING, handler=self._int_handler) + # return actual used threshold and duration + return (_ths * self.SCALES[self.full_scale] / 128, _dur * 8 * 1000 / self.ODRS[self.odr]) + def activity(self): if not self.debounced: time.sleep_ms(self.act_dur) diff --git a/pysense/lib/LTR329ALS01.py b/shields/lib/LTR329ALS01.py similarity index 57% rename from pysense/lib/LTR329ALS01.py rename to shields/lib/LTR329ALS01.py index 748aa54..8915ac4 100644 --- a/pysense/lib/LTR329ALS01.py +++ b/shields/lib/LTR329ALS01.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time from machine import I2C @@ -18,6 +28,14 @@ class LTR329ALS01: ALS_GAIN_8X = const(0x03) ALS_GAIN_48X = const(0x06) ALS_GAIN_96X = const(0x07) + ALS_GAIN_VALUES = { + ALS_GAIN_1X: 1, + ALS_GAIN_2X: 2, + ALS_GAIN_4X: 4, + ALS_GAIN_8X: 8, + ALS_GAIN_48X: 48, + ALS_GAIN_96X: 96 + } ALS_INT_50 = const(0x01) ALS_INT_100 = const(0x00) @@ -27,6 +45,16 @@ class LTR329ALS01: ALS_INT_300 = const(0x06) ALS_INT_350 = const(0x07) ALS_INT_400 = const(0x03) + ALS_INT_VALUES = { + ALS_INT_50: 0.5, + ALS_INT_100: 1, + ALS_INT_150: 1.5, + ALS_INT_200: 2, + ALS_INT_250: 2.5, + ALS_INT_300: 3, + ALS_INT_350: 3.5, + ALS_INT_400: 4 + } ALS_RATE_50 = const(0x00) ALS_RATE_100 = const(0x01) @@ -41,6 +69,9 @@ def __init__(self, pysense = None, sda = 'P22', scl = 'P21', gain = ALS_GAIN_1X, else: self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl)) + self.gain = gain + self.integration = integration + contr = self._getContr(gain) self.i2c.writeto_mem(ALS_I2CADDR, ALS_CONTR_REG, bytearray([contr])) @@ -68,3 +99,19 @@ def light(self): data0 = int(self._getWord(ch0high[0], ch0low[0])) return (data0, data1) + + def lux(self): + # Calculate Lux value from formular in Appendix A of the datasheet + light_level = self.light() + if light_level[0]+light_level[1] > 0: + ratio = light_level[1]/(light_level[0]+light_level[1]) + if ratio < 0.45: + return (1.7743 * light_level[0] + 1.1059 * light_level[1]) / self.ALS_GAIN_VALUES[self.gain] / self.ALS_INT_VALUES[self.integration] + elif ratio < 0.64 and ratio >= 0.45: + return (4.2785 * light_level[0] - 1.9548 * light_level[1]) / self.ALS_GAIN_VALUES[self.gain] / self.ALS_INT_VALUES[self.integration] + elif ratio < 0.85 and ratio >= 0.64: + return (0.5926 * light_level[0] + 0.1185 * light_level[1]) / self.ALS_GAIN_VALUES[self.gain] / self.ALS_INT_VALUES[self.integration] + else: + return 0 + else: + return 0 diff --git a/shields/lib/MFRC630.mpy b/shields/lib/MFRC630.mpy new file mode 100644 index 0000000..f6af944 Binary files /dev/null and b/shields/lib/MFRC630.mpy differ diff --git a/shields/lib/MFRC630.mpy-1.18 b/shields/lib/MFRC630.mpy-1.18 new file mode 100644 index 0000000..7baa005 Binary files /dev/null and b/shields/lib/MFRC630.mpy-1.18 differ diff --git a/shields/lib/MFRC630.py b/shields/lib/MFRC630.py new file mode 100644 index 0000000..0d6c2f2 --- /dev/null +++ b/shields/lib/MFRC630.py @@ -0,0 +1,763 @@ +''' +Pyscan NFC library +Copyright (c) 2019, Pycom Limited. + +Based on a library for NXP's MFRC630 NFC IC https://github.com/iwanders/MFRC630 + +The MIT License (MIT) + +Copyright (c) 2016 Ivor Wanders + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +''' + +import time, binascii + +class MFRC630: + + NFC_I2CADDR = const(0x28) + # commands + MFRC630_CMD_IDLE = const(0x00) # (no arguments) ; no action, cancels current command execution. */ + MFRC630_CMD_LPCD = const(0x01) # (no arguments) ; low-power card detection. */ + MFRC630_CMD_LOADKEY = const(0x02) # (keybyte1), (keybyte2), (keybyte3), (keybyte4), (keybyte5), + MFRC630_CMD_MFAUTHENT = const(0x03) # 60h or 61h, (block address), (card serial number byte0), (card + MFRC630_CMD_RECEIVE = const(0x05) # (no arguments) ; activates the receive circuit. */ + MFRC630_CMD_TRANSMIT = const(0x06) # bytes to send: byte1, byte2, ...; transmits data from the FIFO + MFRC630_CMD_TRANSCEIVE = const(0x07) # bytes to send: byte1, byte2, ....; transmits data from the FIFO + MFRC630_CMD_WRITEE2 = const(0x08) # addressH, addressL, data; gets one byte from FIFO buffer and + MFRC630_CMD_WRITEE2PAGE = const(0x09) # (page Address), data0, [data1..data63]; gets up to 64 bytes (one + MFRC630_CMD_READE2 = const(0x0A) # addressH, address L, length; reads data from the EEPROM and copies + MFRC630_CMD_LOADREG = const(0x0C) # (EEPROM addressH), (EEPROM addressL), RegAdr, (number of Register + MFRC630_CMD_LOADPROTOCOL = const(0x0D) # (Protocol number RX), (Protocol number TX) reads data from the + MFRC630_CMD_LOADKEYE2 = const(0x0E) # KeyNr; copies a key from the EEPROM into the key buffer. */ + MFRC630_CMD_STOREKEYE2 = const(0x0F) # KeyNr, byte1, byte2, byte3, byte4, byte5, byte6; stores a MIFARE + MFRC630_CMD_READRNR = const(0x1C) # (no arguments) ; Copies bytes from the Random Number generator + MFRC630_CMD_SOFTRESET = const(0x1F) # (no arguments) ; resets the MFRC630. */ + + MFRC630_STATUS_STATE_IDLE = const(0b000) # Status register; Idle + MFRC630_STATUS_STATE_TXWAIT = const(0b001) # Status register; Tx wait + MFRC630_STATUS_STATE_TRANSMITTING = const(0b011) # Status register; Transmitting. + MFRC630_STATUS_STATE_RXWAIT = const(0b101) # Status register; Rx wait. + MFRC630_STATUS_STATE_WAIT_FOR_DATA = const(0b110) # Status register; Waiting for data. + MFRC630_STATUS_STATE_RECEIVING = const(0b111) # Status register; Receiving data. + MFRC630_STATUS_STATE_NOT_USED = const(0b100) # Status register; Not used. + MFRC630_STATUS_CRYPTO1_ON = const(1 << 5) # Status register; Crypto1 (MIFARE authentication) is on. + + MFRC630_PROTO_ISO14443A_106_MILLER_MANCHESTER = const(0) + MFRC630_PROTO_ISO14443A_212_MILLER_BPSK = const(1) + MFRC630_PROTO_ISO14443A_424_MILLER_BPSK = const(2) + MFRC630_PROTO_ISO14443A_848_MILLER_BPSK = const(3) + MFRC630_PROTO_ISO14443B_106_NRZ_BPSK = const(4) + MFRC630_PROTO_ISO14443B_212_NRZ_BPSK = const(5) + MFRC630_PROTO_ISO14443B_424_NRZ_BPSK = const(6) + MFRC630_PROTO_ISO14443B_848_NRZ_BPSK = const(7) + MFRC630_PROTO_FELICA_212_MANCHESTER_MANCHESTER = const(8) + MFRC630_PROTO_FELICA_424_MANCHESTER_MANCHESTER = const(9) + MFRC630_PROTO_ISO15693_1_OF_4_SSC = const(10) + MFRC630_PROTO_ISO15693_1_OF_4_DSC = const(11) + MFRC630_PROTO_ISO15693_1_OF_256_SSC = const(12) + MFRC630_PROTO_EPC_UID_UNITRAY_SSC = const(13) + MFRC630_PROTO_ISO18000_MODE_3 = const(14) + MFRC630_RECOM_14443A_ID1_106 = [ 0x8A, 0x08, 0x21, 0x1A, 0x18, 0x18, 0x0F, 0x27, 0x00, 0xC0, 0x12, 0xCF, 0x00, 0x04, 0x90, 0x32, 0x12, 0x0A ] + MFRC630_RECOM_14443A_ID1_212 = [ 0x8E, 0x12, 0x11, 0x06, 0x18, 0x18, 0x0F, 0x10, 0x00, 0xC0, 0x12, 0xCF, 0x00, 0x05, 0x90, 0x3F, 0x12, 0x02 ] + MFRC630_RECOM_14443A_ID1_424 = [ 0x8E, 0x12, 0x11, 0x06, 0x18, 0x18, 0x0F, 0x08, 0x00, 0xC0, 0x12, 0xCF, 0x00, 0x06, 0x90, 0x3F, 0x12, 0x0A ] + MFRC630_RECOM_14443A_ID1_848 = [ 0x8F, 0xDB, 0x11, 0x06, 0x18, 0x18, 0x0F, 0x02, 0x00, 0xC0, 0x12, 0xCF, 0x00, 0x07, 0x90, 0x3F, 0x12, 0x02 ] + MFRC630_ISO14443_CMD_REQA = const(0x26) # request (idle -> ready) + MFRC630_ISO14443_CMD_WUPA = const(0x52) # wake up type a (idle / halt -> ready) + MFRC630_ISO14443_CAS_LEVEL_1 = const(0x93) # Cascade level 1 for select. + MFRC630_ISO14443_CAS_LEVEL_2 = const(0x95) # Cascade level 2 for select. + MFRC630_ISO14443_CAS_LEVEL_3 = const(0x97) # Cascade level 3 for select. + MFRC630_MF_AUTH_KEY_A = const(0x60) # A key_type for mifare auth. + MFRC630_MF_AUTH_KEY_B = const(0x61) # A key_type for mifare auth. + MFRC630_MF_CMD_READ = const(0x30) # To read a block from mifare card. + MFRC630_MF_CMD_WRITE = const(0xA0) # To write a block to a mifare card. + MFRC630_MF_ACK = const(0x0A) # Sent by cards to acknowledge an operation. + + # registers + MFRC630_REG_COMMAND = const(0x00) # Starts and stops command execution + MFRC630_REG_HOSTCTRL = const(0x01) # Host control register + MFRC630_REG_FIFOCONTROL = const(0x02) # Control register of the FIFO + MFRC630_REG_WATERLEVEL = const(0x03) # Level of the FIFO underflow and overflow warning + MFRC630_REG_FIFOLENGTH = const(0x04) # Length of the FIFO + MFRC630_REG_FIFODATA = const(0x05) # Data In/Out exchange register of FIFO buffer + MFRC630_REG_IRQ0 = const(0x06) # Interrupt register 0 + MFRC630_REG_IRQ1 = const(0x07) # Interrupt register 1 + MFRC630_REG_IRQ0EN = const(0x08) # Interrupt enable register 0 + MFRC630_REG_IRQ1EN = const(0x09) # Interrupt enable register 1 + MFRC630_REG_ERROR = const(0x0A) # Error bits showing the error status of the last command execution + MFRC630_REG_STATUS = const(0x0B) # Contains status of the communication + MFRC630_REG_RXBITCTRL = const(0x0C) # Control for anticoll. adjustments for bit oriented protocols + MFRC630_REG_RXCOLL = const(0x0D) # Collision position register + MFRC630_REG_TCONTROL = const(0x0E) # Control of Timer 0..3 + MFRC630_REG_T0CONTROL = const(0x0F) # Control of Timer0 + MFRC630_REG_T0RELOADHI = const(0x10) # High register of the reload value of Timer0 + MFRC630_REG_T0RELOADLO = const(0x11) # Low register of the reload value of Timer0 + MFRC630_REG_T0COUNTERVALHI = const(0x12) # Counter value high register of Timer0 + MFRC630_REG_T0COUNTERVALLO = const(0x13) # Counter value low register of Timer0 + MFRC630_REG_T1CONTROL = const(0x14) # Control of Timer1 + MFRC630_REG_T1RELOADHI = const(0x15) # High register of the reload value of Timer1 + MFRC630_REG_T1COUNTERVALHI = const(0x17) # Counter value high register of Timer1 + MFRC630_REG_T1COUNTERVALLO = const(0x18) # Counter value low register of Timer1 + MFRC630_REG_T2CONTROL = const(0x19) # Control of Timer2 + MFRC630_REG_T2RELOADHI = const(0x1A) # High byte of the reload value of Timer2 + MFRC630_REG_T2RELOADLO = const(0x1B) # Low byte of the reload value of Timer2 + MFRC630_REG_T2COUNTERVALHI = const(0x1C) # Counter value high byte of Timer2 + MFRC630_REG_T2COUNTERVALLO = const(0x1D) # Counter value low byte of Timer2 + MFRC630_REG_T3CONTROL = const(0x1E) # Control of Timer3 + MFRC630_REG_T3RELOADHI = const(0x1F) # High byte of the reload value of Timer3 + MFRC630_REG_T3RELOADLO = const(0x20) # Low byte of the reload value of Timer3 + MFRC630_REG_T3COUNTERVALHI = const(0x21) # Counter value high byte of Timer3 + MFRC630_REG_T3COUNTERVALLO = const(0x22) # Counter value low byte of Timer3 + MFRC630_REG_T4CONTROL = const(0x23) # Control of Timer4 + MFRC630_REG_T4RELOADHI = const(0x24) # High byte of the reload value of Timer4 + MFRC630_REG_T4RELOADLO = const(0x25) # Low byte of the reload value of Timer4 + MFRC630_REG_T4COUNTERVALHI = const(0x26) # Counter value high byte of Timer4 + MFRC630_REG_T4COUNTERVALLO = const(0x27) # Counter value low byte of Timer4 + MFRC630_REG_DRVMOD = const(0x28) # Driver mode register + MFRC630_REG_TXAMP = const(0x29) # Transmitter amplifier register + MFRC630_REG_DRVCON = const(0x2A) # Driver configuration register + MFRC630_REG_TXL = const(0x2B) # Transmitter register + MFRC630_REG_TXCRCPRESET = const(0x2C) # Transmitter CRC control register, preset value + MFRC630_REG_RXCRCCON = const(0x2D) # Receiver CRC control register, preset value + MFRC630_REG_TXDATANUM = const(0x2E) # Transmitter data number register + MFRC630_REG_TXMODWIDTH = const(0x2F) # Transmitter modulation width register + MFRC630_REG_TXSYM10BURSTLEN = const(0x30) # Transmitter symbol 1 + symbol 0 burst length register + MFRC630_REG_TXWAITCTRL = const(0x31) # Transmitter wait control + MFRC630_REG_TXWAITLO = const(0x32) # Transmitter wait low + MFRC630_REG_FRAMECON = const(0x33) # Transmitter frame control + MFRC630_REG_RXSOFD = const(0x34) # Receiver start of frame detection + MFRC630_REG_RXCTRL = const(0x35) # Receiver control register + MFRC630_REG_RXWAIT = const(0x36) # Receiver wait register + MFRC630_REG_RXTHRESHOLD = const(0x37) # Receiver threshold register + MFRC630_REG_RCV = const(0x38) # Receiver register + MFRC630_REG_RXANA = const(0x39) # Receiver analog register + MFRC630_REG_RFU = const(0x3A) # (Reserved for future use) + MFRC630_REG_SERIALSPEED = const(0x3B) # Serial speed register + MFRC630_REG_LFO_TRIMM = const(0x3C) # Low-power oscillator trimming register + MFRC630_REG_PLL_CTRL = const(0x3D) # IntegerN PLL control register, for mcu clock output adjustment + MFRC630_REG_PLL_DIVOUT = const(0x3E) # IntegerN PLL control register, for mcu clock output adjustment + MFRC630_REG_LPCD_QMIN = const(0x3F) # Low-power card detection Q channel minimum threshold + MFRC630_REG_LPCD_QMAX = const(0x40) # Low-power card detection Q channel maximum threshold + MFRC630_REG_LPCD_IMIN = const(0x41) # Low-power card detection I channel minimum threshold + MFRC630_REG_LPCD_I_RESULT = const(0x42) # Low-power card detection I channel result register + MFRC630_REG_LPCD_Q_RESULT = const(0x43) # Low-power card detection Q channel result register + MFRC630_REG_PADEN = const(0x44) # PIN enable register + MFRC630_REG_PADOUT = const(0x45) # PIN out register + MFRC630_REG_PADIN = const(0x46) # PIN in register + MFRC630_REG_SIGOUT = const(0x47) # Enables and controls the SIGOUT Pin + MFRC630_REG_VERSION = const(0x7F) # Version and subversion register + + MFRC630_TXDATANUM_DATAEN = const(1 << 3) + MFRC630_RECOM_14443A_CRC = const(0x18) + + MFRC630_ERROR_EE_ERR = const(1 << 7) + MFRC630_ERROR_FIFOWRERR = const(1 << 6) + MFRC630_ERROR_FIFOOVL = const(1 << 5) + MFRC630_ERROR_MINFRAMEERR = const(1 << 4) + MFRC630_ERROR_NODATAERR = const(1 << 3) + MFRC630_ERROR_COLLDET = const(1 << 2) + MFRC630_ERROR_PROTERR = const(1 << 1) + MFRC630_ERROR_INTEGERR = const(1 << 0) + + MFRC630_CRC_ON = const(1) + MFRC630_CRC_OFF = const(0) + + MFRC630_IRQ0EN_IRQ_INV = const(1 << 7) + MFRC630_IRQ0EN_HIALERT_IRQEN = const(1 << 6) + MFRC630_IRQ0EN_LOALERT_IRQEN = const(1 << 5) + MFRC630_IRQ0EN_IDLE_IRQEN = const(1 << 4) + MFRC630_IRQ0EN_TX_IRQEN = const(1 << 3) + MFRC630_IRQ0EN_RX_IRQEN = const(1 << 2) + MFRC630_IRQ0EN_ERR_IRQEN = const(1 << 1) + MFRC630_IRQ0EN_RXSOF_IRQEN = const(1 << 0) + + MFRC630_IRQ1EN_TIMER0_IRQEN = const(1 << 0) + + MFRC630_TCONTROL_CLK_211KHZ = const(0b01) + MFRC630_TCONTROL_START_TX_END = const(0b01 << 4) + + MFRC630_IRQ1_GLOBAL_IRQ = const(1 << 6) + + MFRC630_IRQ0_ERR_IRQ = const(1 << 1) + MFRC630_IRQ0_RX_IRQ = const(1 << 2) + + def __init__(self, pyscan=None, sda='P22', scl='P21', timeout=None, debug=False): + if pyscan is not None: + self.i2c = pyscan.i2c + else: + from machine import I2C + self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl)) + self._DEBUG = debug + self.mfrc630_cmd_reset() + + # ToDo: Timeout not yet implemented! + # self.chrono = Timer.Chrono() + # self.timeout = timeout + # self.timeout_status = True + + def print_debug(self, msg): + if self._DEBUG: + print(msg) + + def mfrc630_read_reg(self, reg): + return self.i2c.readfrom_mem(NFC_I2CADDR, reg, 1)[0] + + def mfrc630_write_reg(self, reg, data): + self.i2c.writeto_mem(NFC_I2CADDR, reg, bytes([data & 0xFF])) + + def mfrc630_write_regs(self, reg, data): + self.i2c.writeto_mem(NFC_I2CADDR, reg, bytes(data)) + + def mfrc630_read_fifo(self, len): + if len > 0: + return self.i2c.readfrom_mem(NFC_I2CADDR, MFRC630_REG_FIFODATA, len) + else: + return None + + def mfrc630_cmd_idle(self): + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_IDLE) + + def mfrc630_flush_fifo(self): + self.mfrc630_write_reg(MFRC630_REG_FIFOCONTROL, 1 << 4) + + def mfrc630_setup_fifo(self): + self.mfrc630_write_reg(MFRC630_REG_FIFOCONTROL, 0x90) + self.mfrc630_write_reg(MFRC630_REG_WATERLEVEL, 0xFE) + + def mfrc630_write_fifo(self, data): + self.mfrc630_write_regs(MFRC630_REG_FIFODATA, data) + + def mfrc630_cmd_load_protocol(self, rx, tx): + self.mfrc630_flush_fifo() + self.mfrc630_write_fifo([rx, tx]) + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_LOADPROTOCOL) + + def mfrc630_cmd_transceive(self, data): + self.mfrc630_cmd_idle() + self.mfrc630_flush_fifo() + self.mfrc630_setup_fifo() + self.mfrc630_write_fifo(data) + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_TRANSCEIVE) + + def mfrc630_cmd_init(self): + self.mfrc630_write_regs(MFRC630_REG_DRVMOD, self.MFRC630_RECOM_14443A_ID1_106) + self.mfrc630_write_reg(0x28, 0x8E) + self.mfrc630_write_reg(0x29, 0x15) + self.mfrc630_write_reg(0x2A, 0x11) + self.mfrc630_write_reg(0x2B, 0x06) + + def mfrc630_cmd_reset(self): + self.mfrc630_cmd_idle() + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_SOFTRESET) + + def mfrc630_clear_irq0(self): + self.mfrc630_write_reg(MFRC630_REG_IRQ0, ~(1 << 7)) + + def mfrc630_clear_irq1(self): + self.mfrc630_write_reg(MFRC630_REG_IRQ1, ~(1 << 7)) + + def mfrc630_irq0(self): + return self.mfrc630_read_reg(MFRC630_REG_IRQ0) + + def mfrc630_irq1(self): + return self.mfrc630_read_reg(MFRC630_REG_IRQ1) + + def mfrc630_timer_set_control(self, timer, value): + self.mfrc630_write_reg(MFRC630_REG_T0CONTROL + (5 * timer), value) + + def mfrc630_timer_set_reload(self, timer, value): + self.mfrc630_write_reg(MFRC630_REG_T0RELOADHI + (5 * timer), value >> 8) + self.mfrc630_write_reg(MFRC630_REG_T0RELOADLO + (5 * timer), 0xFF) + + def mfrc630_timer_set_value(self, timer, value): + self.mfrc630_write_reg(MFRC630_REG_T0COUNTERVALHI + (5 * timer), value >> 8) + self.mfrc630_write_reg(MFRC630_REG_T0COUNTERVALLO + (5 * timer), 0xFF) + + def mfrc630_fifo_length(self): + # should do 512 byte fifo handling here + return self.mfrc630_read_reg(MFRC630_REG_FIFOLENGTH) + + def mfrc630_status(self): + return self.mfrc630_read_reg(MFRC630_REG_STATUS) + + def mfrc630_error(self): + return self.mfrc630_read_reg(MFRC630_REG_ERROR) + + def mfrc630_cmd_load_key(self, key): + self.mfrc630_cmd_idle() + self.mfrc630_flush_fifo() + self.mfrc630_write_fifo(key) + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_LOADKEY) + + def mfrc630_cmd_auth(self, key_type, block_address, card_uid): + self.mfrc630_cmd_idle() + parameters = [ key_type, block_address, card_uid[0], card_uid[1], card_uid[2], card_uid[3] ] + self.mfrc630_flush_fifo() + self.mfrc630_write_fifo(parameters) + self.mfrc630_write_reg(MFRC630_REG_COMMAND, MFRC630_CMD_MFAUTHENT) + + def mfrc630_MF_read_block(self, block_address, dest): + self.mfrc630_flush_fifo() + + self.mfrc630_write_reg(MFRC630_REG_TXCRCPRESET, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_ON) + self.mfrc630_write_reg(MFRC630_REG_RXCRCCON, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_ON) + + send_req = [ MFRC630_MF_CMD_READ, block_address ] + + # configure a timeout timer. + timer_for_timeout = 0 # should match the enabled interupt. + + # enable the global IRQ for idle, errors and timer. + self.mfrc630_write_reg(MFRC630_REG_IRQ0EN, MFRC630_IRQ0EN_IDLE_IRQEN | MFRC630_IRQ0EN_ERR_IRQEN) + self.mfrc630_write_reg(MFRC630_REG_IRQ1EN, MFRC630_IRQ1EN_TIMER0_IRQEN) + + + # Set timer to 221 kHz clock, start at the end of Tx. + self.mfrc630_timer_set_control(timer_for_timeout, MFRC630_TCONTROL_CLK_211KHZ | MFRC630_TCONTROL_START_TX_END) + # Frame waiting time: FWT = (256 x 16/fc) x 2 FWI + # FWI defaults to four... so that would mean wait for a maximum of ~ 5ms + self.mfrc630_timer_set_reload(timer_for_timeout, 2000) # 2000 ticks of 5 usec is 10 ms. + self.mfrc630_timer_set_value(timer_for_timeout, 2000) + + irq1_value = 0 + irq0_value = 0 + + self.mfrc630_clear_irq0() # clear irq0 + self.mfrc630_clear_irq1() # clear irq1 + + # Go into send, then straight after in receive. + self.mfrc630_cmd_transceive(send_req) + + # block until we are done + while not (irq1_value & (1 << timer_for_timeout)): + irq1_value = self.mfrc630_irq1() + if (irq1_value & MFRC630_IRQ1_GLOBAL_IRQ): + self.print_debug("irq1: %x" % irq1_value) + break # stop polling irq1 and quit the timeout loop. + + self.mfrc630_cmd_idle() + + if irq1_value & (1 << timer_for_timeout): + self.print_debug("this indicates a timeout") + # this indicates a timeout + return 0 + + irq0_value = self.mfrc630_irq0() + if (irq0_value & MFRC630_IRQ0_ERR_IRQ): + self.print_debug("some error") + # some error + return 0 + + self.print_debug("all seems to be well...") + # all seems to be well... + buffer_length = self.mfrc630_fifo_length() + rx_len = buffer_length if (buffer_length <= 16) else 16 + dest = self.mfrc630_read_fifo(rx_len) + return dest + + + def mfrc630_iso14443a_WUPA_REQA(self, instruction): + self.mfrc630_cmd_idle() + + self.mfrc630_flush_fifo() + + #Set register such that we sent 7 bits, set DataEn such that we can send data + self.mfrc630_write_reg(MFRC630_REG_TXDATANUM, 7 | MFRC630_TXDATANUM_DATAEN) + + # disable the CRC registers + self.mfrc630_write_reg(MFRC630_REG_TXCRCPRESET, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_OFF) + self.mfrc630_write_reg(MFRC630_REG_RXCRCCON, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_OFF) + self.mfrc630_write_reg(MFRC630_REG_RXBITCTRL, 0) + + # clear interrupts + self.mfrc630_clear_irq0() + self.mfrc630_clear_irq1() + + # enable the global IRQ for Rx done and Errors. + self.mfrc630_write_reg(MFRC630_REG_IRQ0EN, MFRC630_IRQ0EN_RX_IRQEN | MFRC630_IRQ0EN_ERR_IRQEN) + self.mfrc630_write_reg(MFRC630_REG_IRQ1EN, MFRC630_IRQ1EN_TIMER0_IRQEN) + + # configure timer + timer_for_timeout = 0 + # Set timer to 221 kHz clock, start at the end of Tx. + self.mfrc630_timer_set_control(timer_for_timeout, MFRC630_TCONTROL_CLK_211KHZ | MFRC630_TCONTROL_START_TX_END) + + # Frame waiting time: FWT = (256 x 16/fc) x 2 FWI + # FWI defaults to four... so that would mean wait for a maximum of ~ 5ms + self.mfrc630_timer_set_reload(timer_for_timeout, 1000) # 1000 ticks of 5 usec is 5 ms. + self.mfrc630_timer_set_value(timer_for_timeout, 1000) + + # Go into send, then straight after in receive. + self.mfrc630_cmd_transceive([instruction]) + self.print_debug('Sending REQA') + + # block until we are done + irq1_value = 0 + while not (irq1_value & (1 << timer_for_timeout)): + irq1_value = self.mfrc630_irq1() + if irq1_value & MFRC630_IRQ1_GLOBAL_IRQ: # either ERR_IRQ or RX_IRQ + break # stop polling irq1 and quit the timeout loop + + self.print_debug('After waiting for answer') + self.mfrc630_cmd_idle() + + # if no Rx IRQ, or if there's an error somehow, return 0 + irq0 = self.mfrc630_irq0() + if (not (irq0 & MFRC630_IRQ0_RX_IRQ)) or (irq0 & MFRC630_IRQ0_ERR_IRQ): + self.print_debug('No RX, irq1: %x irq0: %x' % (irq1_value, irq0)) + return 0 + + return self.mfrc630_fifo_length() + self.print_debug("rx_len:", rx_len) + if rx_len == 2: # ATQA should answer with 2 bytes + res = self.mfrc630_read_fifo(rx_len) + self.print_debug('ATQA answer:', res) + return res + return 0 + + def mfrc630_print_block(self, data, len): + if self._DEBUG: + print(self.mfrc630_format_block(data, len)) + + def mfrc630_format_block(self, data, len): + if type(data) == bytearray: + len_i = 0 + try: + len_i = int(len) + except: + pass + if (len_i > 0): + return ' '.join('{:02x}'.format(x) for x in data[:len_i]).upper() + else: + return ' '.join('{:02x}'.format(x) for x in data).upper() + else: + self.print_debug("DATA has type: " + str(type(data))) + try: + return "Length: %d Data: %s" % (len,binascii.hexlify(data,' ')) + except: + return "Data: %s with Length: %s" % (str(data), len) + + + def mfrc630_iso14443a_select(self, uid): + + self.print_debug("Starting select") + + self.mfrc630_cmd_idle() + self.mfrc630_flush_fifo() + + # enable the global IRQ for Rx done and Errors. + self.mfrc630_write_reg(MFRC630_REG_IRQ0EN, MFRC630_IRQ0EN_RX_IRQEN | MFRC630_IRQ0EN_ERR_IRQEN) + self.mfrc630_write_reg(MFRC630_REG_IRQ1EN, MFRC630_IRQ1EN_TIMER0_IRQEN) # only trigger on timer for irq1 + + # configure a timeout timer, use timer 0. + timer_for_timeout = 0 + + # Set timer to 221 kHz clock, start at the end of Tx. + self.mfrc630_timer_set_control(timer_for_timeout, MFRC630_TCONTROL_CLK_211KHZ | MFRC630_TCONTROL_START_TX_END) + # Frame waiting time: FWT = (256 x 16/fc) x 2 FWI + # FWI defaults to four... so that would mean wait for a maximum of ~ 5ms + + self.mfrc630_timer_set_reload(timer_for_timeout, 1000) # 1000 ticks of 5 usec is 5 ms. + self.mfrc630_timer_set_value(timer_for_timeout, 1000) + + for cascade_level in range(1, 4): + self.print_debug("Starting cascade level: %d" % cascade_level) + cmd = 0 + known_bits = 0 # known bits of the UID at this level so far. + send_req = bytearray(7) # used as Tx buffer. + uid_this_level = send_req[2:] + message_length = 0 + if cascade_level == 1: + cmd = MFRC630_ISO14443_CAS_LEVEL_1; + elif cascade_level == 2: + cmd = MFRC630_ISO14443_CAS_LEVEL_2; + elif cascade_level == 3: + cmd = MFRC630_ISO14443_CAS_LEVEL_3; + + # disable CRC in anticipation of the anti collision protocol + self.mfrc630_write_reg(MFRC630_REG_TXCRCPRESET, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_OFF) + self.mfrc630_write_reg(MFRC630_REG_RXCRCCON, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_OFF) + + # max 32 loops of the collision loop. + for collision_n in range(0, 33): + self.print_debug("CL: %d, coll loop: %d, kb %d long" % (cascade_level, collision_n, known_bits)) + self.mfrc630_print_block(uid_this_level, (known_bits + 8 - 1) / 8) + # clear interrupts + self.mfrc630_clear_irq0() + self.mfrc630_clear_irq1() + + send_req[0] = cmd; + send_req[1] = 0x20 + known_bits + send_req[2:5] = uid_this_level[0:3] + + # Only transmit the last 'x' bits of the current byte we are discovering + # First limit the txdatanum, such that it limits the correct number of bits. + self.mfrc630_write_reg(MFRC630_REG_TXDATANUM, (known_bits % 8) | MFRC630_TXDATANUM_DATAEN) + + # ValuesAfterColl: If cleared, every received bit after a collision is + # replaced by a zero. This function is needed for ISO/IEC14443 anticollision (0<<7). + # We want to shift the bits with RxAlign + rxalign = known_bits % 8; + self.print_debug("Setting rx align to: %d" % rxalign) + self.mfrc630_write_reg(MFRC630_REG_RXBITCTRL, (0 << 7) | (rxalign << 4)) + + # then sent the send_req to the hardware, + # (known_bits / 8) + 1): The ceiled number of bytes by known bits. + # +2 for cmd and NVB. + if ((known_bits % 8) == 0): + message_length = ((known_bits / 8)) + 2; + else: + message_length = ((known_bits / 8) + 1) + 2; + + # Send message + self.mfrc630_cmd_transceive(send_req[:int(message_length)]) + + # block until we are done + irq1_value = 0 + while not (irq1_value & (1 << timer_for_timeout)): + irq1_value = self.mfrc630_irq1() + # either ERR_IRQ or RX_IRQ or Timer + if (irq1_value & MFRC630_IRQ1_GLOBAL_IRQ): + break # stop polling irq1 and quit the timeout loop. + + self.mfrc630_cmd_idle() + + # next up, we have to check what happened. + irq0 = self.mfrc630_irq0() + error = self.mfrc630_read_reg(MFRC630_REG_ERROR) + coll = self.mfrc630_read_reg(MFRC630_REG_RXCOLL) + self.print_debug("irq0: %x coll: %x error: %x " % (irq0, coll, error)) + collision_pos = 0 + if irq0 and MFRC630_IRQ0_ERR_IRQ: # some error occured. + self.print_debug("some error occured.") + # Check what kind of error. + if (error & MFRC630_ERROR_COLLDET): + # A collision was detected... + if (coll & (1 << 7)): + collision_pos = coll & (~(1 << 7)) + self.print_debug("Collision at %x", collision_pos) + # This be a true collision... we have to select either the address + # with 1 at this position or with zero + # ISO spec says typically a 1 is added, that would mean: + # uint8_t selection = 1; + + # However, it makes sense to allow some kind of user input for this, so we use the + # current value of uid at this position, first index right byte, then shift such + # that it is in the rightmost position, ten select the last bit only. + # We cannot compensate for the addition of the cascade tag, so this really + # only works for the first cascade level, since we only know whether we had + # a cascade level at the end when the SAK was received. + choice_pos = known_bits + collision_pos + selection = (uid[((choice_pos + (cascade_level - 1) * 3) / 8)] >> ((choice_pos) % 8)) & 1 + + # We just OR this into the UID at the right position, later we + # OR the UID up to this point into uid_this_level. + uid_this_level[((choice_pos) / 8)] |= selection << ((choice_pos) % 8) + known_bits = known_bits + 1 # add the bit we just decided. + self.print_debug("Known Bits: %d" % known_bits) + + self.print_debug("uid_this_level now kb %d long: " % known_bits) + self.mfrc630_print_block(uid_this_level, 10) + else: + # Datasheet of mfrc630: + # bit 7 (CollPosValid) not set: + # Otherwise no collision is detected or + # the position of the collision is out of the range of bits CollPos. + self.print_debug("Collision but no valid collpos.") + collision_pos = 0x20 - known_bits + else: + # we got data despite an error, and no collisions, that means we can still continue. + collision_pos = 0x20 - known_bits + self.print_debug("Got data despite error: %x, setting collision_pos to: %x" % (error, collision_pos)) + elif (irq0 & MFRC630_IRQ0_RX_IRQ): + # we got data, and no collisions, that means all is well. + self.print_debug("we got data, and no collisions, that means all is well.") + collision_pos = 0x20 - known_bits + self.print_debug("Got data, no collision, setting to: %x" % collision_pos) + else: + # We have no error, nor received an RX. No response, no card? + self.print_debug("We have no error, nor received an RX. No response, no card?") + return 0 + + self.print_debug("collision_pos: %x" % collision_pos) + # read the UID Cln so far from the buffer. + rx_len = self.mfrc630_fifo_length() + buf = self.mfrc630_read_fifo(rx_len if rx_len < 5 else 5) + + self.print_debug("Fifo %d long" % rx_len) + self.mfrc630_print_block(buf, rx_len) + + self.print_debug("uid_this_level kb %d long: " % known_bits) + self.mfrc630_print_block(uid_this_level, (known_bits + 8 - 1) / 8) + + # move the buffer into the uid at this level, but OR the result such that + # we do not lose the bit we just set if we have a collision. + for rbx in range(0, rx_len): + uid_this_level[int(known_bits / 8) + rbx] = uid_this_level[int(known_bits / 8) + rbx] | buf[rbx] + self.print_debug("uid_this_level after reading buffer (known_bits=%d):" % known_bits) + self.mfrc630_print_block(uid_this_level, 0) + self.print_debug("known_bits: %x + collision_pos: %x = %x" % (known_bits, collision_pos, known_bits + collision_pos)) + known_bits = known_bits + collision_pos + self.print_debug("known_bits: %x" % known_bits) + + if known_bits >= 32: + self.print_debug("exit collision loop: uid_this_level kb %d long: " % known_bits); + self.mfrc630_print_block(uid_this_level, 10) + break; # done with collision loop + # end collission loop + + # check if the BCC matches + bcc_val = uid_this_level[4] # always at position 4, either with CT UID[0-2] or UID[0-3] in front. + bcc_calc = uid_this_level[0] ^ uid_this_level[1] ^ uid_this_level[2] ^ uid_this_level[3] + self.print_debug("BCC calc: %x" % bcc_calc) + if (bcc_val != bcc_calc): + self.print_debug("Something went wrong, BCC does not match.") + return 0 + + # clear interrupts + self.mfrc630_clear_irq0() + self.mfrc630_clear_irq1() + + send_req[0] = cmd + send_req[1] = 0x70 + send_req[2] = uid_this_level[0] + send_req[3] = uid_this_level[1] + send_req[4] = uid_this_level[2] + send_req[5] = uid_this_level[3] + send_req[6] = bcc_calc + message_length = 7 + + # Ok, almost done now, we re-enable the CRC's + self.mfrc630_write_reg(MFRC630_REG_TXCRCPRESET, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_ON) + self.mfrc630_write_reg(MFRC630_REG_RXCRCCON, MFRC630_RECOM_14443A_CRC | MFRC630_CRC_ON) + + # reset the Tx and Rx registers (disable alignment, transmit full bytes) + self.mfrc630_write_reg(MFRC630_REG_TXDATANUM, (known_bits % 8) | MFRC630_TXDATANUM_DATAEN) + rxalign = 0 + self.mfrc630_write_reg(MFRC630_REG_RXBITCTRL, (0 << 7) | (rxalign << 4)) + + # actually send it! + self.mfrc630_cmd_transceive(send_req) + self.print_debug("send_req %d long: " % message_length) + self.mfrc630_print_block(send_req, message_length) + + # Block until we are done... + irq1_value = 0 + while not (irq1_value & (1 << timer_for_timeout)): + irq1_value = self.mfrc630_irq1() + if (irq1_value & MFRC630_IRQ1_GLOBAL_IRQ): # either ERR_IRQ or RX_IRQ + break # stop polling irq1 and quit the timeout loop. + self.mfrc630_cmd_idle() + + # Check the source of exiting the loop. + irq0_value = self.mfrc630_irq0() + self.print_debug("irq0: %x" % irq0_value) + if irq0_value & MFRC630_IRQ0_ERR_IRQ: + # Check what kind of error. + error = self.mfrc630_read_reg(MFRC630_REG_ERROR) + self.print_debug("error: %x" % error) + if error & MFRC630_ERROR_COLLDET: + # a collision was detected with NVB=0x70, should never happen. + self.print_debug("a collision was detected with NVB=0x70, should never happen.") + return 0 + # Read the sak answer from the fifo. + sak_len = self.mfrc630_fifo_length() + self.print_debug("sak_len: %x" % sak_len) + if sak_len != 1: + return 0 + + sak_value = self.mfrc630_read_fifo(sak_len) + + self.print_debug("SAK answer: ") + self.mfrc630_print_block(sak_value, 1) + + if (sak_value[0] & (1 << 2)): + # UID not yet complete, continue with next cascade. + # This also means the 0'th byte of the UID in this level was CT, so we + # have to shift all bytes when moving to uid from uid_this_level. + for UIDn in range(0, 3): + # uid_this_level[UIDn] = uid_this_level[UIDn + 1]; + uid[(cascade_level - 1) * 3 + UIDn] = uid_this_level[UIDn + 1] + else: + # Done according so SAK! + # Add the bytes at this level to the UID. + for UIDn in range(0, 4): + uid[(cascade_level - 1) * 3 + UIDn] = uid_this_level[UIDn]; + + # Finally, return the length of the UID that's now at the uid "pointer". + return cascade_level * 3 + 1 + + self.print_debug("Exit cascade loop nr. %d: " % cascade_level) + self.mfrc630_print_block(uid, 10) + + return 0 # getting a UID failed. + + def mfrc630_MF_auth(self, uid, key_type, block): + # Enable the right interrupts. + + # configure a timeout timer. + timer_for_timeout = 0 # should match the enabled interrupt. + + # According to datasheet Interrupt on idle and timer with MFAUTHENT, but lets + # include ERROR as well. + self.mfrc630_write_reg(MFRC630_REG_IRQ0EN, MFRC630_IRQ0EN_IDLE_IRQEN | MFRC630_IRQ0EN_ERR_IRQEN) + self.mfrc630_write_reg(MFRC630_REG_IRQ1EN, MFRC630_IRQ1EN_TIMER0_IRQEN) # only trigger on timer for irq1 + + # Set timer to 221 kHz clock, start at the end of Tx. + self.mfrc630_timer_set_control(timer_for_timeout, MFRC630_TCONTROL_CLK_211KHZ | MFRC630_TCONTROL_START_TX_END) + # Frame waiting time: FWT = (256 x 16/fc) x 2 FWI + # FWI defaults to four... so that would mean wait for a maximum of ~ 5ms + + self.mfrc630_timer_set_reload(timer_for_timeout, 2000) # 2000 ticks of 5 usec is 10 ms. + self.mfrc630_timer_set_value(timer_for_timeout, 2000) + + irq1_value = 0 + + self.mfrc630_clear_irq0() # clear irq0 + self.mfrc630_clear_irq1() # clear irq1 + + # start the authentication procedure. + self.mfrc630_cmd_auth(key_type, block, uid) + + # block until we are done + while not (irq1_value & (1 << timer_for_timeout)): + irq1_value = self.mfrc630_irq1() + if (irq1_value & MFRC630_IRQ1_GLOBAL_IRQ): + break # stop polling irq1 and quit the timeout loop. + + if (irq1_value & (1 << timer_for_timeout)): + # this indicates a timeout + return 0 # we have no authentication + + # status is always valid, it is set to 0 in case of authentication failure. + status = self.mfrc630_read_reg(MFRC630_REG_STATUS) + return (status & MFRC630_STATUS_CRYPTO1_ON) + + def mfrc630_MF_deauth(self): + self.mfrc630_write_reg(MFRC630_REG_STATUS, 0) + + def format_block(self, block, length): + ret_val = "" + for i in range(0, length): + if (block[i] < 16): + ret_val += ("0%x " % block[i]) + else: + ret_val += ("%x " % block[i]) + return ret_val.upper() diff --git a/pysense/lib/MPL3115A2.py b/shields/lib/MPL3115A2.py similarity index 86% rename from pysense/lib/MPL3115A2.py rename to shields/lib/MPL3115A2.py index ecea6e0..ce90631 100644 --- a/pysense/lib/MPL3115A2.py +++ b/shields/lib/MPL3115A2.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time from machine import I2C @@ -69,8 +79,10 @@ def __init__(self, pysense = None, sda = 'P22', scl = 'P21', mode = PRESSURE): raise MPL3115A2exception("Error with MPL3115A2") def _read_status(self): - while True: + read_attempts = 0 + while read_attempts < 500: self.i2c.readfrom_mem_into(MPL3115_I2CADDR, MPL3115_STATUS, self.STA_reg) + read_attempts += 1 if(self.STA_reg[0] == 0): time.sleep(0.01) @@ -80,6 +92,11 @@ def _read_status(self): else: return False + # If we get here the sensor isn't responding. Reset it so next time in it should work + self.i2c.writeto_mem(MPL3115_I2CADDR, MPL3115_CTRL_REG1, bytes([0x00])) # put into standby + self.i2c.writeto_mem(MPL3115_I2CADDR, MPL3115_CTRL_REG1, bytes([0x04])) # reset + return False + def pressure(self): if self.mode == ALTITUDE: raise MPL3115A2exception("Incorrect Measurement Mode MPL3115A2") diff --git a/pysense/lib/SI7006A20.py b/shields/lib/SI7006A20.py similarity index 74% rename from pysense/lib/SI7006A20.py rename to shields/lib/SI7006A20.py index 0246f86..f49a302 100644 --- a/pysense/lib/SI7006A20.py +++ b/shields/lib/SI7006A20.py @@ -1,3 +1,13 @@ +#!/usr/bin/env python +# +# Copyright (c) 2019, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + import time from machine import I2C import math @@ -12,8 +22,18 @@ class SI7006A20: SI7006A20_I2C_ADDR = const(0x40) + # I2C commands TEMP_NOHOLDMASTER = const(0xF3) HUMD_NOHOLDMASTER = const(0xF5) + WRITE_USER_REG1 = const(0xE6) + READ_USER_REG1 = const(0xE7) + WRITE_HEATER_CTRL_REG = const(0x51) + READ_HEATER_CTRL_REG = const(0x11) + + # Register masks and offsets + USER_REG1_HTR_ENABLE_MASK = const(0b00000100) + USER_REG1_HTR_ENABLE_OFFSET = const(0x02) + HTR_CTRL_REG_MASK = const(0b00001111) def __init__(self, pysense = None, sda = 'P22', scl = 'P21'): if pysense is not None: @@ -45,18 +65,32 @@ def humidity(self): def read_user_reg(self): """ reading the user configuration register """ - self.i2c.writeto(SI7006A20_I2C_ADDR, bytearray([0xE7])) + self.i2c.writeto(SI7006A20_I2C_ADDR, bytearray([READ_USER_REG1])) time.sleep(0.5) data = self.i2c.readfrom(SI7006A20_I2C_ADDR, 1) return data[0] def read_heater_reg(self): """ reading the heater configuration register """ - self.i2c.writeto(SI7006A20_I2C_ADDR, bytearray([0x11])) + self.i2c.writeto(SI7006A20_I2C_ADDR, bytearray([READ_HEATER_CTRL_REG])) time.sleep(0.5) data = self.i2c.readfrom(SI7006A20_I2C_ADDR, 1) return data[0] + def write_heater_reg(self, heater_value): + """ writing the heater configuration register """ + # We should only set the bottom four bits of this register + heater_setting = heater_value & HTR_CTRL_REG_MASK + self.write_reg(WRITE_HEATER_CTRL_REG, heater_setting) + + def heater_control(self, on_off): + """ turn the heater on or off """ + # Get current settings for everything else + user_reg = self.read_user_reg() + # Set the heater bit + user_reg = (user_reg & ~USER_REG1_HTR_ENABLE_MASK) | (on_off << USER_REG1_HTR_ENABLE_OFFSET) + self.write_reg(WRITE_USER_REG1, user_reg) + def read_electronic_id(self): """ reading electronic identifier """ self.i2c.writeto(SI7006A20_I2C_ADDR, bytearray([0xFA]) + bytearray([0x0F])) diff --git a/lib/pycoproc/pycoproc.py b/shields/lib/pycoproc_1.py similarity index 77% rename from lib/pycoproc/pycoproc.py rename to shields/lib/pycoproc_1.py index 4533718..a8f7e82 100644 --- a/lib/pycoproc/pycoproc.py +++ b/shields/lib/pycoproc_1.py @@ -1,9 +1,21 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +# See https://docs.pycom.io for more information regarding library specifics + from machine import Pin from machine import I2C import time import pycom -__version__ = '0.0.1' +__version__ = '0.0.2' """ PIC MCU wakeup reason types """ WAKE_REASON_ACCELEROMETER = 1 @@ -12,10 +24,16 @@ WAKE_REASON_INT_PIN = 8 class Pycoproc: - """ class for handling interraction with PIC MCU """ + """ class for handling the interaction with PIC MCU """ I2C_SLAVE_ADDR = const(8) + PYSENSE = const(1) + PYTRACK = const(2) + PYSCAN = const(3) + + BOARD_TYPE_SET = (PYSENSE, PYTRACK, PYSCAN) + CMD_PEEK = const(0x0) CMD_POKE = const(0x01) CMD_MAGIC = const(0x02) @@ -71,42 +89,46 @@ class Pycoproc: EXP_RTC_PERIOD = const(7000) - def __init__(self, i2c=None, sda='P22', scl='P21'): + def __init__(self, board_type, i2c=None, sda='P22', scl='P21'): if i2c is not None: self.i2c = i2c else: self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl)) + if board_type not in self.BOARD_TYPE_SET: + raise Exception('Board type not in the set {}'.format(self.BOARD_TYPE_SET)) + self.sda = sda self.scl = scl + self.board_type = board_type self.clk_cal_factor = 1 self.reg = bytearray(6) self.wake_int = False self.wake_int_pin = False self.wake_int_pin_rising_edge = True + # Make sure we are inserted into the + # correct board and can talk to the PIC try: self.read_fw_version() - except Exception: - time.sleep_ms(2) - try: - # init the ADC for the battery measurements - self.poke_memory(ANSELC_ADDR, 1 << 2) - self.poke_memory(ADCON0_ADDR, (0x06 << _ADCON0_CHS_POSN) | _ADCON0_ADON_MASK) - self.poke_memory(ADCON1_ADDR, (0x06 << _ADCON1_ADCS_POSN)) - # enable the pull-up on RA3 - self.poke_memory(WPUA_ADDR, (1 << 3)) - # make RC5 an input - self.set_bits_in_memory(TRISC_ADDR, 1 << 5) - # set RC6 and RC7 as outputs and enable power to the sensors and the GPS - self.mask_bits_in_memory(TRISC_ADDR, ~(1 << 6)) - self.mask_bits_in_memory(TRISC_ADDR, ~(1 << 7)) - - if self.read_fw_version() < 6: - raise ValueError('Firmware out of date') + except Exception as e: + raise Exception('Board not detected: {}'.format(e)) + + # init the ADC for the battery measurements + self.poke_memory(ANSELC_ADDR, 1 << 2) + self.poke_memory(ADCON0_ADDR, (0x06 << _ADCON0_CHS_POSN) | _ADCON0_ADON_MASK) + self.poke_memory(ADCON1_ADDR, (0x06 << _ADCON1_ADCS_POSN)) + # enable the pull-up on RA3 + self.poke_memory(WPUA_ADDR, (1 << 3)) + # make RC5 an input + self.set_bits_in_memory(TRISC_ADDR, 1 << 5) + # set RC6 and RC7 as outputs and enable power to the sensors and the GPS + self.mask_bits_in_memory(TRISC_ADDR, ~(1 << 6)) + self.mask_bits_in_memory(TRISC_ADDR, ~(1 << 7)) + + if self.read_fw_version() < 6: + raise ValueError('Firmware for Shield1 out of date') - except Exception: - raise Exception('Board not detected') def _write(self, data, wait=True): self.i2c.writeto(I2C_SLAVE_ADDR, data) @@ -187,14 +209,19 @@ def setup_sleep(self, time_s): except Exception: pass time_s = int((time_s * self.clk_cal_factor) + 0.5) # round to the nearest integer + if time_s >= 2**(8*3): + time_s = 2**(8*3)-1 self._write(bytes([CMD_SETUP_SLEEP, time_s & 0xFF, (time_s >> 8) & 0xFF, (time_s >> 16) & 0xFF])) def go_to_sleep(self, gps=True): - # enable or disable back-up power to the GPS receiver - if gps: + # if we have a Pytrack then enable or disable back-up power to the GPS receiver + if self.board_type == self.PYTRACK and gps: + # disable GPS only if Pytrack self.set_bits_in_memory(PORTC_ADDR, 1 << 7) else: + # Pysense or Pyscan or no GPS self.mask_bits_in_memory(PORTC_ADDR, ~(1 << 7)) + # disable the ADC self.poke_memory(ADCON0_ADDR, 0) @@ -232,14 +259,21 @@ def calibrate_rtc(self): self._write(bytes([CMD_CALIBRATE]), wait=False) self.i2c.deinit() Pin('P21', mode=Pin.IN) - pulses = pycom.pulses_get('P21', 50) + pulses = pycom.pulses_get('P21', 100) self.i2c.init(mode=I2C.MASTER, pins=(self.sda, self.scl)) + idx = 0 + for i in range(len(pulses)): + if pulses[i][1] > EXP_RTC_PERIOD: + idx = i + break try: - period = pulses[2][1] - pulses[0][1] + period = pulses[idx][1] - pulses[(idx - 1)][1] except: - pass + period = 0 if period > 0: self.clk_cal_factor = (EXP_RTC_PERIOD / period) * (1000 / 1024) + if self.clk_cal_factor > 1.25 or self.clk_cal_factor < 0.75: + self.clk_cal_factor = 1 def button_pressed(self): button = self.peek_memory(PORTA_ADDR) & (1 << 3) diff --git a/shields/lib/pycoproc_2.py b/shields/lib/pycoproc_2.py new file mode 100644 index 0000000..a1ead51 --- /dev/null +++ b/shields/lib/pycoproc_2.py @@ -0,0 +1,381 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +# See https://docs.pycom.io for more information regarding library specifics + +from machine import Pin +from machine import I2C +import time +import pycom + +__version__ = '0.0.5' + +""" PIC MCU wakeup reason types """ +WAKE_REASON_ACCELEROMETER = 1 +WAKE_REASON_PUSH_BUTTON = 2 +WAKE_REASON_TIMER = 4 +WAKE_REASON_INT_PIN = 8 + +class Pycoproc: + """ class for handling the interaction with PIC MCU """ + + I2C_SLAVE_ADDR = const(8) + + CMD_PEEK = const(0x0) + CMD_POKE = const(0x01) + CMD_MAGIC = const(0x02) + CMD_HW_VER = const(0x10) + CMD_FW_VER = const(0x11) #define SW_VERSION (15) + CMD_PROD_ID = const(0x12) #USB product ID, e.g. PYSENSE (0xF012) + CMD_SETUP_SLEEP = const(0x20) + CMD_GO_SLEEP = const(0x21) + CMD_CALIBRATE = const(0x22) + CMD_GO_NAP = const(0x23) + CMD_BAUD_CHANGE = const(0x30) + CMD_DFU = const(0x31) + CMD_RESET = const(0x40) + # CMD_GO_NAP options + SD_CARD_OFF = const(0x1) + SENSORS_OFF = const(0x2) + ACCELEROMETER_OFF = const(0x4) + FIPY_OFF = const(0x8) + + + REG_CMD = const(0) + REG_ADDRL = const(1) + REG_ADDRH = const(2) + REG_AND = const(3) + REG_OR = const(4) + REG_XOR = const(5) + + ANSELA_ADDR = const(0x18C) + ANSELB_ADDR = const(0x18D) + ANSELC_ADDR = const(0x18E) + + ADCON0_ADDR = const(0x9D) + ADCON1_ADDR = const(0x9E) + + IOCAP_ADDR = const(0x391) + IOCAN_ADDR = const(0x392) + + INTCON_ADDR = const(0x0B) + OPTION_REG_ADDR = const(0x95) + + _ADCON0_CHS_POSN = const(0x02) + _ADCON0_CHS_AN5 = const(0x5) # AN5 / RC1 + _ADCON0_CHS_AN6 = const(0x6) # AN6 / RC2 + _ADCON0_ADON_MASK = const(0x01) + _ADCON0_ADCS_F_OSC_64 = const(0x6) # A/D Conversion Clock + _ADCON0_GO_nDONE_MASK = const(0x02) + _ADCON1_ADCS_POSN = const(0x04) + + ADRESL_ADDR = const(0x09B) + ADRESH_ADDR = const(0x09C) + + TRISA_ADDR = const(0x08C) + TRISB_ADDR = const(0x08D) + TRISC_ADDR = const(0x08E) + + LATA_ADDR = const(0x10C) + LATB_ADDR = const(0x10D) + LATC_ADDR = const(0x10E) + + PORTA_ADDR = const(0x00C) + PORTB_ADDR = const(0x00C) + PORTC_ADDR = const(0x00E) + + WPUA_ADDR = const(0x20C) + + WAKE_REASON_ADDR = const(0x064C) + MEMORY_BANK_ADDR = const(0x0620) + + PCON_ADDR = const(0x096) + STATUS_ADDR = const(0x083) + + EXP_RTC_PERIOD = const(7000) + + USB_PID_PYSENSE = const(0xf012) + USB_PID_PYTRACK = const(0xf013) + + @staticmethod + def wake_up(): + # P9 is connected to RC1, make P9 an output + p9 = Pin("P9", mode=Pin.OUT) + # toggle rc1 to trigger wake up + p9(1) + time.sleep(0.1) + p9(0) + time.sleep(0.1) + + def __init__(self, i2c=None, sda='P22', scl='P21'): + if i2c is not None: + self.i2c = i2c + else: + self.i2c = I2C(0, mode=I2C.MASTER, pins=(sda, scl), baudrate=100000) + + self.sda = sda + self.scl = scl + self.clk_cal_factor = 1 + self.reg = bytearray(6) + + # Make sure we are inserted into the + # correct board and can talk to the PIC + retry = 0 + while True: + try: + self.read_fw_version() + break + except Exception as e: + if retry > 10: + raise Exception('Board not detected: {}'.format(e)) + print("Couldn't init Pycoproc. Maybe the PIC is still napping. Try to wake it. ({}, {})".format(retry, e)) + Pycoproc.wake_up() + # # p9 is connected to RC1, toggle it to wake PIC + # p9 = Pin("P9", mode=Pin.OUT) + # p9(1) + # time.sleep(0.1) + # p9(0) + # time.sleep(0.1) + # Pin("P9", mode=Pin.IN) + retry += 1 + + usb_pid=self.read_product_id() + if usb_pid != USB_PID_PYSENSE and usb_pid != USB_PID_PYTRACK: + raise ValueError('Not a Pysense2/Pytrack2 ({})'.format(hex(usb_pid))) + # for Pysense/Pytrack 2.0, the minimum firmware version is 15 + fw = self.read_fw_version() + if fw < 16: + raise ValueError('Firmware for Shield2 out of date', fw) + + # init the ADC for the battery measurements + self.write_byte(ANSELC_ADDR, 1 << 2) # RC2 analog input + self.write_byte(ADCON0_ADDR, (_ADCON0_CHS_AN6 << _ADCON0_CHS_POSN) | _ADCON0_ADON_MASK) # select analog channel and enable ADC + self.write_byte(ADCON1_ADDR, (_ADCON0_ADCS_F_OSC_64 << _ADCON1_ADCS_POSN)) # ADC conversion clock + + # enable the pull-up on RA3 + self.write_byte(WPUA_ADDR, (1 << 3)) + + # set RC6 and RC7 as outputs + self.write_bit(TRISC_ADDR, 6, 0) # 3V3SENSOR_A, power to Accelerometer + self.write_bit(TRISC_ADDR, 7, 0) # PWR_CTRL power to other sensors + + # enable power to the sensors and the GPS + self.gps_standby(False) # GPS, RC4 + self.sensor_power() # PWR_CTRL, RC7 + self.sd_power() # LP_CTRL, RA5 + + + def _write(self, data, wait=True): + self.i2c.writeto(I2C_SLAVE_ADDR, data) + if wait: + self._wait() + + def _read(self, size): + return self.i2c.readfrom(I2C_SLAVE_ADDR, size + 1)[1:(size + 1)] + + def _wait(self): + count = 0 + time.sleep_us(10) + while self.i2c.readfrom(I2C_SLAVE_ADDR, 1)[0] != 0xFF: + time.sleep_us(100) + count += 1 + if (count > 500): # timeout after 50ms + raise Exception('Board timeout') + + def _send_cmd(self, cmd): + self._write(bytes([cmd])) + + def read_hw_version(self): + self._send_cmd(CMD_HW_VER) + d = self._read(2) + return (d[1] << 8) + d[0] + + def read_fw_version(self): + self._send_cmd(CMD_FW_VER) + d = self._read(2) + return (d[1] << 8) + d[0] + + def read_product_id(self): + self._send_cmd(CMD_PROD_ID) + d = self._read(2) + return (d[1] << 8) + d[0] + + def read_byte(self, addr): + self._write(bytes([CMD_PEEK, addr & 0xFF, (addr >> 8) & 0xFF])) + return self._read(1)[0] + + def write_byte(self, addr, value): + self._write(bytes([CMD_POKE, addr & 0xFF, (addr >> 8) & 0xFF, value & 0xFF])) + + def magic_write_read(self, addr, _and=0xFF, _or=0, _xor=0): + self._write(bytes([CMD_MAGIC, addr & 0xFF, (addr >> 8) & 0xFF, _and & 0xFF, _or & 0xFF, _xor & 0xFF])) + return self._read(1)[0] + + def toggle_bits_in_memory(self, addr, bits): + self.magic_write_read(addr, _xor=bits) + + def mask_bits_in_memory(self, addr, mask): + self.magic_write_read(addr, _and=mask) + + def set_bits_in_memory(self, addr, bits): + self.magic_write_read(addr, _or=bits) + + def read_bit(self, address, bit): + b = self.read_byte(address) + # print("{0:08b}".format(b)) + mask = (1<= 2**(8*3): + time_s = 2**(8*3)-1 + self._write(bytes([CMD_SETUP_SLEEP, time_s & 0xFF, (time_s >> 8) & 0xFF, (time_s >> 16) & 0xFF])) + + def go_to_sleep(self, gps=True, pycom_module_off=True, accelerometer_off=True, wake_interrupt=False): + # enable or disable back-up power to the GPS receiver + self.gps_standby(gps) + + # disable the ADC + self.write_byte(ADCON0_ADDR, 0) + + # RC0, RC1, RC2, analog input + self.set_bits_in_memory(TRISC_ADDR, (1<<2) | (1<<1) | (1<<0) ) + self.set_bits_in_memory(ANSELC_ADDR, (1<<2) | (1<<1) | (1<<0) ) + + # RA4 analog input + self.set_bits_in_memory(TRISA_ADDR, (1<<4) ) + self.set_bits_in_memory(ANSELA_ADDR, (1<<4) ) + + # RB4, RB5 analog input + self.set_bits_in_memory(TRISB_ADDR, (1<<5) | (1<<4) ) + self.set_bits_in_memory(ANSELB_ADDR, (1<<5) | (1<<4) ) + + if wake_interrupt: + # print("enable wake up PIC from RC1") + self.set_bits_in_memory(OPTION_REG_ADDR, 1 << 6) # rising edge of INT pin + self.mask_bits_in_memory(ANSELC_ADDR, ~(1 << 1)) # disable analog function for RC1 pin + self.set_bits_in_memory(TRISC_ADDR, 1 << 1) # make RC1 input pin + self.mask_bits_in_memory(INTCON_ADDR, ~(1 << 1)) # clear INTF + self.set_bits_in_memory(INTCON_ADDR, 1 << 4) # enable interrupt; set INTE) + + nap_options = SD_CARD_OFF | SENSORS_OFF + if pycom_module_off: + nap_options |= FIPY_OFF + if accelerometer_off: + nap_options |= ACCELEROMETER_OFF + + # print("CMD_GO_NAP {0:08b}".format(nap_options)) + self._write(bytes([CMD_GO_NAP, nap_options]), wait=False) + + def calibrate_rtc(self): + # the 1.024 factor is because the PIC LF operates at 31 KHz + # WDT has a frequency divider to generate 1 ms + # and then there is a binary prescaler, e.g., 1, 2, 4 ... 512, 1024 ms + # hence the need for the constant + self._write(bytes([CMD_CALIBRATE]), wait=False) + self.i2c.deinit() + Pin('P21', mode=Pin.IN) + pulses = pycom.pulses_get('P21', 100) + self.i2c.init(mode=I2C.MASTER, pins=(self.sda, self.scl), baudrate=100000) + idx = 0 + for i in range(len(pulses)): + if pulses[i][1] > EXP_RTC_PERIOD: + idx = i + break + try: + period = pulses[idx][1] - pulses[(idx - 1)][1] + except: + period = 0 + if period > 0: + self.clk_cal_factor = (EXP_RTC_PERIOD / period) * (1000 / 1024) + if self.clk_cal_factor > 1.25 or self.clk_cal_factor < 0.75: + self.clk_cal_factor = 1 + time.sleep(0.5) + + def button_pressed(self): + retry = 0 + while True: + try: + button = self.read_bit(PORTA_ADDR, 3) + return not button + except Exception as e: + if retry > 10: + raise Exception('Failed to read button state: {}'.format(e)) + print("Failed to read button state, retry ... ({}, {})".format(retry, e)) + retry += 1 + + def read_battery_voltage(self): + self.set_bits_in_memory(ADCON0_ADDR, _ADCON0_GO_nDONE_MASK) + time.sleep_us(50) + while self.read_byte(ADCON0_ADDR) & _ADCON0_GO_nDONE_MASK: + time.sleep_us(100) + adc_val = (self.read_byte(ADRESH_ADDR) << 2) + (self.read_byte(ADRESL_ADDR) >> 6) + return (((adc_val * 3.3 * 280) / 1023) / 180) + 0.01 # add 10mV to compensate for the drop in the FET + + def gps_standby(self, enabled=True): + if enabled: + # make RC4 input + self.set_bits_in_memory(TRISC_ADDR, 1 << 4) + else: + # make RC4 an output + self.mask_bits_in_memory(TRISC_ADDR, ~(1 << 4)) + # drive RC4 high + self.set_bits_in_memory(PORTC_ADDR, 1 << 4) + time.sleep(0.2) + # drive RC4 low + self.mask_bits_in_memory(PORTC_ADDR, ~(1 << 4)) + time.sleep(0.2) + # drive RC4 high + self.set_bits_in_memory(PORTC_ADDR, 1 << 4) + time.sleep(0.2) + + def sensor_power(self, enabled=True): + # make RC7 an output + self.write_bit(TRISC_ADDR, 7, 0) + if enabled: + # drive RC7 high + self.write_bit(LATC_ADDR, 7, 1) + else: + # drive RC7 low + self.write_bit(LATC_ADDR, 7, 0) + + def sd_power(self, enabled=True): + # make RA5 an output + self.write_bit(TRISA_ADDR, 5, 0) + if enabled: + # drive RA5 high + self.write_bit(LATA_ADDR, 5, 1) + else: + # drive RA5 low + self.write_bit(LATA_ADDR, 5, 0) + + def reset_cmd(self): + self._send_cmd(CMD_RESET) + return diff --git a/shields/pyscan_1.py b/shields/pyscan_1.py new file mode 100644 index 0000000..9e9c263 --- /dev/null +++ b/shields/pyscan_1.py @@ -0,0 +1,114 @@ +''' +Simple Pyscan NFC / MiFare Classic Example +Copyright (c) 2019, Pycom Limited. + +This example continuously sends a REQA for ISO14443A card type +If a card is discovered, it will read the UID +If DECODE_CARD = True, will attempt to authenticate with CARDkey +If authentication succeeds will attempt to read sectors from the card +''' + +from pycoproc_1 import Pycoproc +from MFRC630 import MFRC630 +from LIS2HH12 import LIS2HH12 +from LTR329ALS01 import LTR329ALS01 +import time +import pycom + +#add your card UID here +VALID_CARDS = [[0x43, 0x95, 0xDD, 0xF8], + [0x43, 0x95, 0xDD, 0xF9], + [0x46, 0x5A, 0xEB, 0x7D, 0x8A, 0x08, 0x04]] + + +# This is the default key for an unencrypted MiFare card +CARDkey = [ 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF ] +DECODE_CARD = False + +py = Pycoproc(Pycoproc.PYSCAN) +nfc = MFRC630(py) +lt = LTR329ALS01(py) +li = LIS2HH12(py) + +pybytes_enabled = False +if 'pybytes' in globals(): + if(pybytes.isconnected()): + print('Pybytes is connected, sending signals to Pybytes') + pybytes_enabled = True + +RGB_BRIGHTNESS = 0x8 + +RGB_RED = (RGB_BRIGHTNESS << 16) +RGB_GREEN = (RGB_BRIGHTNESS << 8) +RGB_BLUE = (RGB_BRIGHTNESS) + +counter = 0 + +def check_uid(uid, len): + return VALID_CARDS.count(uid[:len]) + +def send_sensor_data(name, timeout): + if(pybytes_enabled): + while(True): + pybytes.send_signal(2, lt.light()) + pybytes.send_signal(3, li.acceleration()) + time.sleep(timeout) + +# Make sure heartbeat is disabled before setting RGB LED +pycom.heartbeat(False) + +# Initialise the MFRC630 with some settings +nfc.mfrc630_cmd_init() + +print('Scanning for cards') +while True: + # Send REQA for ISO14443A card type + atqa = nfc.mfrc630_iso14443a_WUPA_REQA(nfc.MFRC630_ISO14443_CMD_REQA) + if (atqa != 0): + # A card has been detected, read UID + print('A card has been detected, reading its UID ...') + uid = bytearray(10) + uid_len = nfc.mfrc630_iso14443a_select(uid) + print('UID has length {}'.format(uid_len)) + if (uid_len > 0): + # A valid UID has been detected, print details + counter += 1 + print("%d\tUID [%d]: %s" % (counter, uid_len, nfc.format_block(uid, uid_len))) + if DECODE_CARD: + # Try to authenticate with CARD key + nfc.mfrc630_cmd_load_key(CARDkey) + for sector in range(0, 16): + if (nfc.mfrc630_MF_auth(uid, nfc.MFRC630_MF_AUTH_KEY_A, sector * 4)): + pycom.rgbled(RGB_GREEN) + # Authentication was sucessful, read card data + readbuf = bytearray(16) + for b in range(0, 4): + f_sect = sector * 4 + b + len = nfc.mfrc630_MF_read_block(f_sect, readbuf) + print("\t\tSector %s: Block: %s: %s" % (nfc.format_block([sector], 1), nfc.format_block([b], 1), nfc.format_block(readbuf, len))) + else: + print("Authentication denied for sector %s!" % nfc.format_block([sector], 1)) + pycom.rgbled(RGB_RED) + # It is necessary to call mfrc630_MF_deauth after authentication + # Although this is also handled by the reset / init cycle + nfc.mfrc630_MF_deauth() + else: + #check if card uid is listed in VALID_CARDS + if (check_uid(list(uid), uid_len)) > 0: + print('Card is listed, turn LED green') + pycom.rgbled(RGB_GREEN) + if(pybytes_enabled): + pybytes.send_signal(1, ('Card is listed', uid)) + else: + print('Card is not listed, turn LED red') + pycom.rgbled(RGB_RED) + if(pybytes_enabled): + pybytes.send_signal(1, ('Unauthorized card detected', uid)) + + else: + pycom.rgbled(RGB_BLUE) + # We could go into power saving mode here... to be investigated + nfc.mfrc630_cmd_reset() + time.sleep(.5) + # Re-Initialise the MFRC630 with settings as these got wiped during reset + nfc.mfrc630_cmd_init() diff --git a/shields/pysense_1.py b/shields/pysense_1.py new file mode 100644 index 0000000..2a4e126 --- /dev/null +++ b/shields/pysense_1.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +# See https://docs.pycom.io for more information regarding library specifics + +import time +import pycom +from pycoproc_1 import Pycoproc +import machine + +from LIS2HH12 import LIS2HH12 +from SI7006A20 import SI7006A20 +from LTR329ALS01 import LTR329ALS01 +from MPL3115A2 import MPL3115A2,ALTITUDE,PRESSURE + +pycom.heartbeat(False) +pycom.rgbled(0x0A0A08) # white + +py = Pycoproc(Pycoproc.PYSENSE) + +pybytes_enabled = False +if 'pybytes' in globals(): + if(pybytes.isconnected()): + print('Pybytes is connected, sending signals to Pybytes') + pybytes_enabled = True + +mp = MPL3115A2(py,mode=ALTITUDE) # Returns height in meters. Mode may also be set to PRESSURE, returning a value in Pascals +print("MPL3115A2 temperature: " + str(mp.temperature())) +print("Altitude: " + str(mp.altitude())) +mpp = MPL3115A2(py,mode=PRESSURE) # Returns pressure in Pa. Mode may also be set to ALTITUDE, returning a value in meters +print("Pressure: " + str(mpp.pressure())) + +si = SI7006A20(py) +print("Temperature: " + str(si.temperature())+ " deg C and Relative Humidity: " + str(si.humidity()) + " %RH") +print("Dew point: "+ str(si.dew_point()) + " deg C") +t_ambient = 24.4 +print("Humidity Ambient for " + str(t_ambient) + " deg C is " + str(si.humid_ambient(t_ambient)) + "%RH") + + +lt = LTR329ALS01(py) +print("Light (channel Blue lux, channel Red lux): " + str(lt.light())) + +li = LIS2HH12(py) +print("Acceleration: " + str(li.acceleration())) +print("Roll: " + str(li.roll())) +print("Pitch: " + str(li.pitch())) + +# set your battery voltage limits here +vmax = 4.2 +vmin = 3.3 +battery_voltage = py.read_battery_voltage() +battery_percentage = (battery_voltage - vmin / (vmax - vmin))*100 +print("Battery voltage: " + str(py.read_battery_voltage()), " percentage: ", battery_percentage) +if(pybytes_enabled): + pybytes.send_signal(1, mpp.pressure()) + pybytes.send_signal(2, si.temperature()) + pybytes.send_signal(3, lt.light()) + pybytes.send_signal(4, li.acceleration()) + pybytes.send_battery_level(int(battery_percentage)) + print("Sent data to pybytes") + +time.sleep(5) +py.setup_sleep(10) +py.go_to_sleep() diff --git a/shields/pysense_2.py b/shields/pysense_2.py new file mode 100644 index 0000000..ff793e7 --- /dev/null +++ b/shields/pysense_2.py @@ -0,0 +1,72 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +# See https://docs.pycom.io for more information regarding library specifics + +import time +import pycom +from pycoproc_2 import Pycoproc +import machine + +from LIS2HH12 import LIS2HH12 +from SI7006A20 import SI7006A20 +from LTR329ALS01 import LTR329ALS01 +from MPL3115A2 import MPL3115A2,ALTITUDE,PRESSURE + +pycom.heartbeat(False) +pycom.rgbled(0x0A0A08) # white + +py = Pycoproc() +if py.read_product_id() != Pycoproc.USB_PID_PYSENSE: + raise Exception('Not a Pysense') + +pybytes_enabled = False +if 'pybytes' in globals(): + if(pybytes.isconnected()): + print('Pybytes is connected, sending signals to Pybytes') + pybytes_enabled = True + +mp = MPL3115A2(py,mode=ALTITUDE) # Returns height in meters. Mode may also be set to PRESSURE, returning a value in Pascals +print("MPL3115A2 temperature: " + str(mp.temperature())) +print("Altitude: " + str(mp.altitude())) +mpp = MPL3115A2(py,mode=PRESSURE) # Returns pressure in Pa. Mode may also be set to ALTITUDE, returning a value in meters +print("Pressure: " + str(mpp.pressure())) + + +si = SI7006A20(py) +print("Temperature: " + str(si.temperature())+ " deg C and Relative Humidity: " + str(si.humidity()) + " %RH") +print("Dew point: "+ str(si.dew_point()) + " deg C") +t_ambient = 24.4 +print("Humidity Ambient for " + str(t_ambient) + " deg C is " + str(si.humid_ambient(t_ambient)) + "%RH") + + +lt = LTR329ALS01(py) +print("Light (channel Blue, channel Red): " + str(lt.light())," Lux: ", str(lt.lux()), "lx") + +li = LIS2HH12(py) +print("Acceleration: " + str(li.acceleration())) +print("Roll: " + str(li.roll())) +print("Pitch: " + str(li.pitch())) + +print("Battery voltage: " + str(py.read_battery_voltage())) + +# set your battery voltage limits here +vmax = 4.2 +vmin = 3.3 +battery_voltage = py.read_battery_voltage() +battery_percentage = (battery_voltage - vmin / (vmax - vmin))*100 +print("Battery voltage: " + str(py.read_battery_voltage()), " percentage: ", battery_percentage) +if(pybytes_enabled): + pybytes.send_signal(1, mpp.pressure()) + pybytes.send_signal(2, si.temperature()) + pybytes.send_signal(3, lt.light()) + pybytes.send_signal(4, li.acceleration()) + pybytes.send_battery_level(int(battery_percentage)) + print("Sent data to pybytes") diff --git a/shields/pytrack_1.py b/shields/pytrack_1.py new file mode 100644 index 0000000..cab1cb9 --- /dev/null +++ b/shields/pytrack_1.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import machine +import math +import network +import os +import time +import utime +import gc +from machine import RTC +from machine import SD +from L76GNSS import L76GNSS +from pycoproc_1 import Pycoproc + +time.sleep(2) +gc.enable() + +# setup rtc +rtc = machine.RTC() +rtc.ntp_sync("pool.ntp.org") +utime.sleep_ms(750) +print('\nRTC Set from NTP to UTC:', rtc.now()) +utime.timezone(7200) +print('Adjusted from UTC to EST timezone', utime.localtime(), '\n') + +py = Pycoproc(Pycoproc.PYTRACK) +l76 = L76GNSS(py, timeout=30) + +pybytes_enabled = False +if 'pybytes' in globals(): + if(pybytes.isconnected()): + print('Pybytes is connected, sending signals to Pybytes') + pybytes_enabled = True + +# sd = SD() +# os.mount(sd, '/sd') +# f = open('/sd/gps-record.txt', 'w') + +while (True): + coord = l76.coordinates() + #f.write("{} - {}\n".format(coord, rtc.now())) + print("{} - {} - {}".format(coord, rtc.now(), gc.mem_free())) + if(pybytes_enabled): + pybytes.send_signal(1, coord) + time.sleep(10) + +""" +# sleep procedure +time.sleep(3) +py.setup_sleep(10) +py.go_to_sleep() +""" diff --git a/shields/pytrack_2.py b/shields/pytrack_2.py new file mode 100644 index 0000000..52fe547 --- /dev/null +++ b/shields/pytrack_2.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +import machine +import math +import network +import os +import time +import utime +import gc +import pycom +from machine import RTC +from machine import SD +from L76GNSS import L76GNSS +from pycoproc_2 import Pycoproc + +pycom.heartbeat(False) +pycom.rgbled(0x0A0A08) # white + +time.sleep(2) +gc.enable() + +# setup rtc +rtc = machine.RTC() +rtc.ntp_sync("pool.ntp.org") +utime.sleep_ms(750) +print('\nRTC Set from NTP to UTC:', rtc.now()) +utime.timezone(7200) +print('Adjusted from UTC to EST timezone', utime.localtime(), '\n') + +py = Pycoproc() +if py.read_product_id() != Pycoproc.USB_PID_PYTRACK: + raise Exception('Not a Pytrack') + +time.sleep(1) +l76 = L76GNSS(py, timeout=30, buffer=512) + +pybytes_enabled = False +if 'pybytes' in globals(): + if(pybytes.isconnected()): + print('Pybytes is connected, sending signals to Pybytes') + pybytes_enabled = True + +# sd = SD() +# os.mount(sd, '/sd') +# f = open('/sd/gps-record.txt', 'w') + +while (True): + coord = l76.coordinates() + #f.write("{} - {}\n".format(coord, rtc.now())) + print("{} - {} - {}".format(coord, rtc.now(), gc.mem_free())) + if(pybytes_enabled): + pybytes.send_signal(1, coord) + time.sleep(10) + +""" +# sleep procedure +time.sleep(3) +py.setup_sleep(10) +py.go_to_sleep() +""" diff --git a/shields/shield_2.py b/shields/shield_2.py new file mode 100644 index 0000000..8fb74e0 --- /dev/null +++ b/shields/shield_2.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python +# +# Copyright (c) 2020, Pycom Limited. +# +# This software is licensed under the GNU GPL version 3 or any +# later version, with permitted additional terms. For more information +# see the Pycom Licence v1.0 document supplied with this file, or +# available at https://www.pycom.io/opensource/licensing +# + +# This script support Pysense 2 and Pytrack 2 +# It demonstrates two examples: +# * go to ultra low power mode (~10uA @3.75V) with all sensors, incl accelerometer and also pycom module (Fipy, Gpy, etc) off - tap the MCLR button for this +# * go to low power mode (~165uA @3.75V) with accelerometer on, pycom module in deepsleep and wake from accelerometer interrupt - hold the MCLR button down for this + +# See https://docs.pycom.io for more information regarding library specifics + +import time +import pycom +import struct +from machine import Pin +from pycoproc_2 import Pycoproc +import machine +from LIS2HH12 import LIS2HH12 + +def accelerometer(): + print("ACCELEROMETER:", "accel:", accelerometer_sensor.acceleration(), "roll:", accelerometer_sensor.roll(), "pitch:", accelerometer_sensor.pitch(), "x/y/z:", accelerometer_sensor.x, accelerometer_sensor.y, accelerometer_sensor.z ) + +def activity_int_handler(pin_o): + if pin_o(): + print('[Activity]') + pycom.rgbled(0x00000A) # blue + else: + print('[Inactivity]') + pycom.rgbled(0x0A0A00) # yellow + +def activity_int_handler_none(pin_o): + pass + +def blink(color=0x0a0a0a, ct=5, on_ms=100, off_ms=100 ): + while ct >= 0 : + ct -= 1 + pycom.rgbled(color) + time.sleep_ms(on_ms) + pycom.rgbled(0x000000) + time.sleep_ms(off_ms) + +def wait(color=0x0a0a0a, hold_timeout_ms=3000): + print(" - tap MCLR button to go to ultra low power mode (everything off)") + print(" - hold MCLR button down for", round(hold_timeout_ms/1000,1), "sec to go to low power mode and wake from accelerometer") + print("wait for button ...") + ct = 0 + pressed_time_ms = 0 + dot = '.' + while True: + if pycoproc.button_pressed(): + if pressed_time_ms == 0: + # the button just started to be pressed + pressed_time_ms = time.ticks_ms() + print("button pressed") + pycom.rgbled(color) + dot = '*' + else: + # the button is still being held down + if time.ticks_ms() - pressed_time_ms > hold_timeout_ms: + pycom.rgbled(0) + dot = '_' + else: + if pressed_time_ms != 0: + # the button was released + print("button released") + if time.ticks_ms() - pressed_time_ms > hold_timeout_ms: + return True + else: + return False + time.sleep(0.1) + ct += 1 + if ct % 10 == 0: + print(dot, end='') + +def pretty_reset_cause(): + mrc = machine.reset_cause() + print('reset_cause', mrc, end=' ') + if mrc == machine.PWRON_RESET: + print("PWRON_RESET") + # plug in + # press reset button on module + # reset button on JTAG board + # core dump + elif mrc == machine.HARD_RESET: + print("HARD_RESET") + elif mrc == machine.WDT_RESET: + print("WDT_RESET") + # machine.reset() + elif mrc == machine.DEEPSLEEP_RESET: + print("DEEPSLEEP_RESET") + # machine.deepsleep() + elif mrc == machine.SOFT_RESET: + print("SOFT_RESET") + # Ctrl-D + elif mrc == machine.BROWN_OUT_RESET: + print("BROWN_OUT_RESET") + +def pretty_wake_reason(): + mwr = machine.wake_reason() + print("wake_reason", mwr, end=' ') + if mwr[0] == machine.PWRON_WAKE: + print("PWRON_WAKE") + # reset button + elif mwr[0] == machine.PIN_WAKE: + print("PIN_WAKE") + elif mwr[0] == machine.RTC_WAKE: + print("RTC_WAKE") + # from deepsleep + elif mwr[0] == machine.ULP_WAKE: + print("ULP_WAKE") + + +############################################################### +sleep_time_s = 300 # 5 min +pycom.heartbeat(False) +pycom.rgbled(0x0a0a0a) # white +import binascii +import machine +print(os.uname().sysname.lower() + '-' + binascii.hexlify(machine.unique_id()).decode("utf-8")[-4:], "pysense2") + +pretty_wake_reason() +pretty_reset_cause() +print("pycoproc init") +pycoproc = Pycoproc() +print("battery {:.2f} V".format(pycoproc.read_battery_voltage())) + +# init accelerometer +accelerometer_sensor = LIS2HH12() +# read accelerometer sensor values +accelerometer() +print("enable accelerometer interrupt") + +# enable_activity_interrupt( [mG], [ms], callback) +# accelerometer_sensor.enable_activity_interrupt(8000, 200, activity_int_handler) # low sensitivty +# accelerometer_sensor.enable_activity_interrupt(2000, 200, activity_int_handler) # medium sensitivity +accelerometer_sensor.enable_activity_interrupt( 100, 200, activity_int_handler) # high sensitivity +# accelerometer_sensor.enable_activity_interrupt(63, 160, activity_int_handler) # ultra sensitivty + +if wait(0x0A000A): # purple + print("button was held") + blink(0x000a00) # green + print("enable pycom module to wake up from accelerometer interrupt") + wake_pins = [Pin('P13', mode=Pin.IN, pull=Pin.PULL_DOWN)] + machine.pin_sleep_wakeup(wake_pins, machine.WAKEUP_ANY_HIGH, True) + + print("put pycoproc to sleep and pycom module to deepsleep for", round(sleep_time_s/60,1), "minutes") + pycoproc.setup_sleep(sleep_time_s) + pycoproc.go_to_sleep(pycom_module_off=False, accelerometer_off=False, wake_interrupt=True) + machine.deepsleep(sleep_time_s * 1000) +else: + print("button was tapped") + blink(0x100600) # orange + print("put pycoproc to sleep and turn pycom module off for", round(sleep_time_s/60,1), "minutes") + pycoproc.setup_sleep(sleep_time_s) + pycoproc.go_to_sleep() + +print("we never reach here!")