diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index c44a140faf0ff..1ec6c7067f9cd 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -11,19 +11,10 @@ body: * If you have a question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. * For missing or incorrect documentation, or feature requests, then please [choose a different issue type](https://github.com/micropython/micropython/issues/new/choose). - - type: checkboxes - id: terms - attributes: - label: Checks - description: | - Before submitting your bug report, please go over these check points: - options: - - label: | - I agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone. - required: true - - label: | - I've searched for [existing issues](https://github.com/micropython/micropython/issues) matching this bug, and didn't find any. - required: true + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) matching this bug before reporting. - type: input id: port-board-hw attributes: @@ -33,7 +24,7 @@ body: placeholder: | esp32 port, ESP32-Fantastic board. validations: - required: true + required: true - type: textarea id: version attributes: @@ -101,6 +92,17 @@ body: description: | Is there anything else that might help to resolve this issue? value: No, I've provided everything above. + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true - type: markdown attributes: value: | diff --git a/.github/ISSUE_TEMPLATE/documentation.yml b/.github/ISSUE_TEMPLATE/documentation.yml index 5be0241701747..93051e51c80c7 100644 --- a/.github/ISSUE_TEMPLATE/documentation.yml +++ b/.github/ISSUE_TEMPLATE/documentation.yml @@ -9,19 +9,10 @@ body: This form is for reporting issues with the documentation or examples provided with MicroPython. If you have a general question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. - - type: checkboxes - id: terms - attributes: - label: Checks - description: | - Before submitting your bug report, please go over these check points: - options: - - label: | - I agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone. - required: true - - label: | - I've searched for [existing issues](https://github.com/micropython/micropython/issues) and didn't find any that matched. - required: true + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. - type: input id: page attributes: @@ -38,6 +29,17 @@ body: Please describe what was missing from the documentation and/or what was incorrect/incomplete. validations: required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true - type: markdown attributes: value: | diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml index 845fbed810ef9..7d5162a32a4af 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yml +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -15,19 +15,10 @@ body: * If you have a question \"How Do I ...?\", please post it on GitHub Discussions or Discord instead of here. * Could this feature be implemented as a pure Python library? If so, please open the request on the [micropython-lib repository](https://github.com/micropython/micropython-lib/issues) instead. - - type: checkboxes - id: terms - attributes: - label: Checks - description: | - Before submitting your feature request, please go over these check points: - options: - - label: | - I agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone. - required: true - - label: | - I've searched for [existing issues](https://github.com/micropython/micropython/issues) regarding this feature, and didn't find any. - required: true + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before opening a new one. - type: textarea id: feature attributes: @@ -51,14 +42,32 @@ body: MicroPython aims to strike a balance between functionality and code size. Can this feature be optionally enabled? If you believe the usefulness of this feature would outweigh the additional code size, please explain. (It's OK to say you're unsure here, we're happy to discuss this with you.) - - type: checkboxes + - type: dropdown id: implementation attributes: label: Implementation + description: | + What is your suggestion for implementing this feature? + + (See also: [How to sponsor](https://github.com/sponsors/micropython#sponsors), [How to submit a Pull Request](https://github.com/micropython/micropython/wiki/ContributorGuidelines).) options: - - label: I intend to implement this feature and would submit a Pull Request if desirable. - - label: I hope the MicroPython maintainers or community will implement this feature. - - label: I would like to [Sponsor](https://github.com/sponsors/micropython#sponsors) development of this feature. + - I hope the MicroPython maintainers or community will implement this feature + - I intend to implement this feature and would submit a Pull Request if desirable + - I would like to sponsor development of this feature + multiple: true + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true - type: markdown attributes: value: | diff --git a/.github/ISSUE_TEMPLATE/security.yml b/.github/ISSUE_TEMPLATE/security.yml index 7d66a72f64120..57c2a5885ed85 100644 --- a/.github/ISSUE_TEMPLATE/security.yml +++ b/.github/ISSUE_TEMPLATE/security.yml @@ -9,21 +9,11 @@ body: 1. For issues that are readily exploitable or have high impact, please email contact@micropython.org instead. 1. If this is a question about security, please ask it in [Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead. - - type: checkboxes - id: terms - attributes: - label: Checks - description: | - Before submitting your bug report, please go over these check points: - options: - - label: | - I agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone. - required: true - - label: I wish to report a specific security issue that is **not readily exploitable and does not have high impact** for MicroPython developers or users. - required: true - - label: | - I've searched for [existing issues](https://github.com/micropython/micropython/issues) and didn't find any that matched. - required: true + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. + - type: input id: port-board-hw attributes: @@ -57,3 +47,14 @@ body: * How does the attacker exploit this issue? validations: required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000000000..e11cebddb3710 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,33 @@ + + +### Summary + + + + +### Testing + + + + +### Trade-offs and Alternatives + + + diff --git a/.github/workflows/biome.yml b/.github/workflows/biome.yml index 88744f16ca7d6..fea9c9c8bd7bc 100644 --- a/.github/workflows/biome.yml +++ b/.github/workflows/biome.yml @@ -7,7 +7,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Setup Biome uses: biomejs/setup-biome@v2 with: diff --git a/.github/workflows/code_formatting.yml b/.github/workflows/code_formatting.yml index 9f30f048cfdbe..8a18ca034540c 100644 --- a/.github/workflows/code_formatting.yml +++ b/.github/workflows/code_formatting.yml @@ -10,11 +10,11 @@ jobs: code-formatting: runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 - name: Install packages - run: source tools/ci.sh && ci_c_code_formatting_setup + run: tools/ci.sh c_code_formatting_setup - name: Run code formatting - run: source tools/ci.sh && ci_c_code_formatting_run + run: tools/ci.sh c_code_formatting_run - name: Check code formatting run: git diff --exit-code diff --git a/.github/workflows/code_size.yml b/.github/workflows/code_size.yml index 1ef43d1fd396c..f82d1d0c73567 100644 --- a/.github/workflows/code_size.yml +++ b/.github/workflows/code_size.yml @@ -1,7 +1,6 @@ name: Check code size on: - push: pull_request: paths: - '.github/workflows/*.yml' @@ -24,17 +23,17 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: fetch-depth: 100 - name: Install packages - run: source tools/ci.sh && ci_code_size_setup + run: tools/ci.sh code_size_setup - name: Build - run: source tools/ci.sh && ci_code_size_build + run: tools/ci.sh code_size_build - name: Compute code size difference - run: tools/metrics.py diff ~/size0 ~/size1 | tee diff + run: source tools/ci.sh && ci_code_size_report - name: Save PR number if: github.event_name == 'pull_request' env: diff --git a/.github/workflows/code_size_comment.yml b/.github/workflows/code_size_comment.yml index a394f7a14b054..2eed0b06b8ee5 100644 --- a/.github/workflows/code_size_comment.yml +++ b/.github/workflows/code_size_comment.yml @@ -11,11 +11,11 @@ concurrency: jobs: comment: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - name: 'Download artifact' id: download-artifact - uses: actions/github-script@v7 + uses: actions/github-script@v8 with: result-encoding: string script: | @@ -56,7 +56,7 @@ jobs: run: unzip code-size-report.zip - name: Post comment to pull request if: steps.download-artifact.outputs.result == 'ok' - uses: actions/github-script@v7 + uses: actions/github-script@v8 with: github-token: ${{secrets.GITHUB_TOKEN}} script: | diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml index 2d8b4627aac2d..688134b42515c 100644 --- a/.github/workflows/codespell.yml +++ b/.github/workflows/codespell.yml @@ -6,8 +6,8 @@ jobs: codespell: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 # codespell version should be kept in sync with .pre-commit-config.yml - - run: pip install --user codespell==2.2.6 tomli + - run: pip install --user codespell==2.4.1 tomli - run: codespell diff --git a/.github/workflows/commit_formatting.yml b/.github/workflows/commit_formatting.yml index 3fdcabc4ca73f..ca63fc796b0cc 100644 --- a/.github/workflows/commit_formatting.yml +++ b/.github/workflows/commit_formatting.yml @@ -1,6 +1,6 @@ name: Check commit message formatting -on: [push, pull_request] +on: [pull_request] concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -10,9 +10,9 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: - fetch-depth: '100' - - uses: actions/setup-python@v5 + fetch-depth: 100 + - uses: actions/setup-python@v6 - name: Check commit message formatting - run: source tools/ci.sh && ci_commit_formatting_run + run: tools/ci.sh commit_formatting_run diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index d01a4b50c9810..aa05cfb13873b 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -5,6 +5,8 @@ on: pull_request: paths: - docs/** + - py/** + - tests/cpydiff/** concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -15,9 +17,11 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 - name: Install Python packages run: pip install -r docs/requirements.txt + - name: Build unix port + run: tools/ci.sh unix_build_helper - name: Build docs run: make -C docs/ html diff --git a/.github/workflows/examples.yml b/.github/workflows/examples.yml index 6613f106625a2..d16122b720b6c 100644 --- a/.github/workflows/examples.yml +++ b/.github/workflows/examples.yml @@ -18,7 +18,7 @@ jobs: embedding: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build run: make -C examples/embedding -f micropython_embed.mk && make -C examples/embedding - name: Run diff --git a/.github/workflows/mpremote.yml b/.github/workflows/mpremote.yml index ee91b6360b9b4..36904764c991b 100644 --- a/.github/workflows/mpremote.yml +++ b/.github/workflows/mpremote.yml @@ -11,12 +11,12 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: # Setting this to zero means fetch all history and tags, # which hatch-vcs can use to discover the version tag. fetch-depth: 0 - - uses: actions/setup-python@v5 + - uses: actions/setup-python@v6 - name: Install build tools run: pip install build - name: Build mpremote wheel diff --git a/.github/workflows/mpy_format.yml b/.github/workflows/mpy_format.yml index baa02ce08d507..e692a853e7b2c 100644 --- a/.github/workflows/mpy_format.yml +++ b/.github/workflows/mpy_format.yml @@ -15,10 +15,10 @@ concurrency: jobs: test: - runs-on: ubuntu-20.04 # use 20.04 to get python2 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_mpy_format_setup + run: tools/ci.sh mpy_format_setup - name: Test mpy-tool.py - run: source tools/ci.sh && ci_mpy_format_test + run: tools/ci.sh mpy_format_test diff --git a/.github/workflows/ports.yml b/.github/workflows/ports.yml index 1f262b0ba4bee..d4e89bd1aa924 100644 --- a/.github/workflows/ports.yml +++ b/.github/workflows/ports.yml @@ -17,6 +17,6 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build ports download metadata run: mkdir boards && ./tools/autobuild/build-downloads.py . ./boards diff --git a/.github/workflows/ports_alif.yml b/.github/workflows/ports_alif.yml new file mode 100644 index 0000000000000..eea8f53900ebc --- /dev/null +++ b/.github/workflows/ports_alif.yml @@ -0,0 +1,33 @@ +name: alif port + +on: + push: + pull_request: + paths: + - '.github/workflows/*.yml' + - 'tools/**' + - 'py/**' + - 'extmod/**' + - 'shared/**' + - 'lib/**' + - 'drivers/**' + - 'ports/alif/**' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + build_alif: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - alif_ae3_build + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh alif_setup + - name: Build ci_${{matrix.ci_func }} + run: tools/ci.sh ${{ matrix.ci_func }} diff --git a/.github/workflows/ports_cc3200.yml b/.github/workflows/ports_cc3200.yml index f178a140587db..c57309c23a6d0 100644 --- a/.github/workflows/ports_cc3200.yml +++ b/.github/workflows/ports_cc3200.yml @@ -21,8 +21,8 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_cc3200_setup + run: tools/ci.sh cc3200_setup - name: Build - run: source tools/ci.sh && ci_cc3200_build + run: tools/ci.sh cc3200_build diff --git a/.github/workflows/ports_esp32.yml b/.github/workflows/ports_esp32.yml index 45808b659add7..8ee3f44d6a2e1 100644 --- a/.github/workflows/ports_esp32.yml +++ b/.github/workflows/ports_esp32.yml @@ -12,6 +12,10 @@ on: - 'lib/**' - 'drivers/**' - 'ports/esp32/**' + schedule: + # Scheduled run exists to keep master branch ESP-IDF cache entry hot + # and prevent creating many redundant per-branch cache entries instead. + - cron: "20 0 * * *" concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -25,13 +29,14 @@ jobs: ci_func: # names are functions in ci.sh - esp32_build_cmod_spiram_s2 - esp32_build_s3_c3 - runs-on: ubuntu-20.04 + - esp32_build_c2_c6 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - id: idf_ver - name: Read the ESP-IDF version - run: source tools/ci.sh && echo "IDF_VER=$IDF_VER" | tee "$GITHUB_OUTPUT" + name: Read the ESP-IDF version (including Python version) + run: source tools/ci.sh && echo "IDF_VER=${IDF_VER}-py${PYTHON_VER}" | tee "$GITHUB_OUTPUT" - name: Cached ESP-IDF install id: cache_esp_idf @@ -46,7 +51,7 @@ jobs: - name: Install ESP-IDF packages if: steps.cache_esp_idf.outputs.cache-hit != 'true' - run: source tools/ci.sh && ci_esp32_idf_setup + run: tools/ci.sh esp32_idf_setup - name: ccache uses: hendrikmuhs/ccache-action@v1.2 @@ -54,4 +59,4 @@ jobs: key: esp32-${{ matrix.ci_func }} - name: Build ci_${{matrix.ci_func }} - run: source tools/ci.sh && ci_${{ matrix.ci_func }} + run: tools/ci.sh ${{ matrix.ci_func }} diff --git a/.github/workflows/ports_esp8266.yml b/.github/workflows/ports_esp8266.yml index 5236edf40b959..96cf0c5a5cd3f 100644 --- a/.github/workflows/ports_esp8266.yml +++ b/.github/workflows/ports_esp8266.yml @@ -21,8 +21,8 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_esp8266_setup && ci_esp8266_path >> $GITHUB_PATH + run: tools/ci.sh esp8266_setup && tools/ci.sh esp8266_path >> $GITHUB_PATH - name: Build - run: source tools/ci.sh && ci_esp8266_build + run: tools/ci.sh esp8266_build diff --git a/.github/workflows/ports_mimxrt.yml b/.github/workflows/ports_mimxrt.yml index 9e782bf63b31c..bcbaf3de066c9 100644 --- a/.github/workflows/ports_mimxrt.yml +++ b/.github/workflows/ports_mimxrt.yml @@ -19,15 +19,15 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest defaults: run: working-directory: 'micropython repo' # test build with space in path steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: path: 'micropython repo' - name: Install packages - run: source tools/ci.sh && ci_mimxrt_setup + run: tools/ci.sh mimxrt_setup - name: Build - run: source tools/ci.sh && ci_mimxrt_build + run: tools/ci.sh mimxrt_build diff --git a/.github/workflows/ports_nrf.yml b/.github/workflows/ports_nrf.yml index d9cffb9778cba..995f65933e0b0 100644 --- a/.github/workflows/ports_nrf.yml +++ b/.github/workflows/ports_nrf.yml @@ -19,10 +19,10 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_nrf_setup + run: tools/ci.sh nrf_setup - name: Build - run: source tools/ci.sh && ci_nrf_build + run: tools/ci.sh nrf_build diff --git a/.github/workflows/ports_powerpc.yml b/.github/workflows/ports_powerpc.yml index c41b13e5ddffe..27417fb3c3cf2 100644 --- a/.github/workflows/ports_powerpc.yml +++ b/.github/workflows/ports_powerpc.yml @@ -21,8 +21,8 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_powerpc_setup + run: tools/ci.sh powerpc_setup - name: Build - run: source tools/ci.sh && ci_powerpc_build + run: tools/ci.sh powerpc_build diff --git a/.github/workflows/ports_qemu-arm.yml b/.github/workflows/ports_qemu-arm.yml deleted file mode 100644 index db3cd7871d591..0000000000000 --- a/.github/workflows/ports_qemu-arm.yml +++ /dev/null @@ -1,32 +0,0 @@ -name: qemu-arm port - -on: - push: - pull_request: - paths: - - '.github/workflows/*.yml' - - 'tools/**' - - 'py/**' - - 'extmod/**' - - 'shared/**' - - 'lib/**' - - 'drivers/**' - - 'ports/qemu-arm/**' - - 'tests/**' - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - build_and_test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - name: Install packages - run: source tools/ci.sh && ci_qemu_arm_setup - - name: Build and run test suite - run: source tools/ci.sh && ci_qemu_arm_build - - name: Print failures - if: failure() - run: grep --before-context=100 --text "FAIL" ports/qemu-arm/build/console.out diff --git a/.github/workflows/ports_qemu.yml b/.github/workflows/ports_qemu.yml new file mode 100644 index 0000000000000..b038b03e71fed --- /dev/null +++ b/.github/workflows/ports_qemu.yml @@ -0,0 +1,64 @@ +name: qemu port + +on: + push: + pull_request: + paths: + - '.github/workflows/*.yml' + - 'tools/**' + - 'py/**' + - 'extmod/**' + - 'shared/**' + - 'lib/**' + - 'drivers/**' + - 'ports/qemu/**' + - 'tests/**' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + build_and_test_arm: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - bigendian + - sabrelite + - thumb_softfp + - thumb_hardfp + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh qemu_setup_arm + - name: Build and run test suite ci_qemu_build_arm_${{ matrix.ci_func }} + run: tools/ci.sh qemu_build_arm_${{ matrix.ci_func }} + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + build_and_test_rv32: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh qemu_setup_rv32 + - name: Build and run test suite + run: tools/ci.sh qemu_build_rv32 + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + build_and_test_rv64: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh qemu_setup_rv64 + - name: Build and run test suite + run: tools/ci.sh qemu_build_rv64 + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_renesas-ra.yml b/.github/workflows/ports_renesas-ra.yml index b1a30c2f11798..600b8ea8046bd 100644 --- a/.github/workflows/ports_renesas-ra.yml +++ b/.github/workflows/ports_renesas-ra.yml @@ -19,11 +19,11 @@ concurrency: jobs: build_renesas_ra_board: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_renesas_ra_setup + run: tools/ci.sh renesas_ra_setup - name: Build - run: source tools/ci.sh && ci_renesas_ra_board_build + run: tools/ci.sh renesas_ra_board_build diff --git a/.github/workflows/ports_rp2.yml b/.github/workflows/ports_rp2.yml index 748f38e143893..0837c06c97bd4 100644 --- a/.github/workflows/ports_rp2.yml +++ b/.github/workflows/ports_rp2.yml @@ -24,10 +24,10 @@ jobs: run: working-directory: 'micropython repo' # test build with space in path steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 with: path: 'micropython repo' - name: Install packages - run: source tools/ci.sh && ci_rp2_setup + run: tools/ci.sh rp2_setup - name: Build - run: source tools/ci.sh && ci_rp2_build + run: tools/ci.sh rp2_build diff --git a/.github/workflows/ports_samd.yml b/.github/workflows/ports_samd.yml index 5bf1826cd17bb..d159fde175d54 100644 --- a/.github/workflows/ports_samd.yml +++ b/.github/workflows/ports_samd.yml @@ -21,8 +21,8 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_samd_setup + run: tools/ci.sh samd_setup - name: Build - run: source tools/ci.sh && ci_samd_build + run: tools/ci.sh samd_build diff --git a/.github/workflows/ports_stm32.yml b/.github/workflows/ports_stm32.yml index f5e01dc1f620e..eae3bae871f45 100644 --- a/.github/workflows/ports_stm32.yml +++ b/.github/workflows/ports_stm32.yml @@ -26,11 +26,11 @@ jobs: - stm32_pyb_build - stm32_nucleo_build - stm32_misc_build - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_stm32_setup + run: tools/ci.sh stm32_setup - name: Build ci_${{matrix.ci_func }} - run: source tools/ci.sh && ci_${{ matrix.ci_func }} + run: tools/ci.sh ${{ matrix.ci_func }} diff --git a/.github/workflows/ports_unix.yml b/.github/workflows/ports_unix.yml index 4473847db61ef..deee7b265fcb7 100644 --- a/.github/workflows/ports_unix.yml +++ b/.github/workflows/ports_unix.yml @@ -23,11 +23,11 @@ jobs: minimal: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build - run: source tools/ci.sh && ci_unix_minimal_build + run: tools/ci.sh unix_minimal_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_minimal_run_tests + run: tools/ci.sh unix_minimal_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures @@ -35,9 +35,9 @@ jobs: reproducible: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build with reproducible date - run: source tools/ci.sh && ci_unix_minimal_build + run: tools/ci.sh unix_minimal_build env: SOURCE_DATE_EPOCH: 1234567890 - name: Check reproducible build date @@ -46,11 +46,11 @@ jobs: standard: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build - run: source tools/ci.sh && ci_unix_standard_build + run: tools/ci.sh unix_standard_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_standard_run_tests + run: tools/ci.sh unix_standard_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures @@ -58,11 +58,11 @@ jobs: standard_v2: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build - run: source tools/ci.sh && ci_unix_standard_v2_build + run: tools/ci.sh unix_standard_v2_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_standard_v2_run_tests + run: tools/ci.sh unix_standard_v2_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures @@ -70,25 +70,30 @@ jobs: coverage: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Install packages - run: source tools/ci.sh && ci_unix_coverage_setup + run: tools/ci.sh unix_coverage_setup - name: Build - run: source tools/ci.sh && ci_unix_coverage_build + run: tools/ci.sh unix_coverage_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_coverage_run_tests + run: tools/ci.sh unix_coverage_run_tests - name: Test merging .mpy files - run: source tools/ci.sh && ci_unix_coverage_run_mpy_merge_tests + run: tools/ci.sh unix_coverage_run_mpy_merge_tests - name: Build native mpy modules - run: source tools/ci.sh && ci_native_mpy_modules_build + run: tools/ci.sh native_mpy_modules_build - name: Test importing .mpy generated by mpy_ld.py - run: source tools/ci.sh && ci_unix_coverage_run_native_mpy_tests + run: tools/ci.sh unix_coverage_run_native_mpy_tests - name: Run gcov coverage analysis run: | (cd ports/unix && gcov -o build-coverage/py ../../py/*.c || true) (cd ports/unix && gcov -o build-coverage/extmod ../../extmod/*.c || true) - name: Upload coverage to Codecov - uses: codecov/codecov-action@v4 + uses: codecov/codecov-action@v5 with: fail_ci_if_error: true verbose: true @@ -98,33 +103,47 @@ jobs: run: tests/run-tests.py --print-failures coverage_32bit: - runs-on: ubuntu-20.04 # use 20.04 to get libffi-dev:i386 + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_unix_32bit_setup + run: tools/ci.sh unix_32bit_setup - name: Build - run: source tools/ci.sh && ci_unix_coverage_32bit_build + run: tools/ci.sh unix_coverage_32bit_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_coverage_32bit_run_tests + run: tools/ci.sh unix_coverage_32bit_run_tests - name: Build native mpy modules - run: source tools/ci.sh && ci_native_mpy_modules_32bit_build + run: tools/ci.sh native_mpy_modules_32bit_build - name: Test importing .mpy generated by mpy_ld.py - run: source tools/ci.sh && ci_unix_coverage_32bit_run_native_mpy_tests + run: tools/ci.sh unix_coverage_32bit_run_native_mpy_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures nanbox: - runs-on: ubuntu-20.04 # use 20.04 to get python2, and libffi-dev:i386 + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_unix_32bit_setup + run: tools/ci.sh unix_32bit_setup - name: Build - run: source tools/ci.sh && ci_unix_nanbox_build + run: tools/ci.sh unix_nanbox_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_nanbox_run_tests + run: tools/ci.sh unix_nanbox_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + longlong: + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh unix_32bit_setup + - name: Build + run: tools/ci.sh unix_longlong_build + - name: Run main test suite + run: tools/ci.sh unix_longlong_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures @@ -132,106 +151,192 @@ jobs: float: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build - run: source tools/ci.sh && ci_unix_float_build + run: tools/ci.sh unix_float_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_float_run_tests + run: tools/ci.sh unix_float_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + gil_enabled: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Build + run: tools/ci.sh unix_gil_enabled_build + - name: Run main test suite + run: tools/ci.sh unix_gil_enabled_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures stackless_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_unix_clang_setup + run: tools/ci.sh unix_clang_setup - name: Build - run: source tools/ci.sh && ci_unix_stackless_clang_build + run: tools/ci.sh unix_stackless_clang_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_stackless_clang_run_tests + run: tools/ci.sh unix_stackless_clang_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures float_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_unix_clang_setup + run: tools/ci.sh unix_clang_setup - name: Build - run: source tools/ci.sh && ci_unix_float_clang_build + run: tools/ci.sh unix_float_clang_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_float_clang_run_tests + run: tools/ci.sh unix_float_clang_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures - settrace: + settrace_stackless: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Build - run: source tools/ci.sh && ci_unix_settrace_build + run: tools/ci.sh unix_settrace_stackless_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_settrace_run_tests + run: tools/ci.sh unix_settrace_stackless_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures - settrace_stackless: - runs-on: ubuntu-latest + repr_b: + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh unix_32bit_setup - name: Build - run: source tools/ci.sh && ci_unix_settrace_stackless_build + run: tools/ci.sh unix_repr_b_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_settrace_stackless_run_tests + run: tools/ci.sh unix_repr_b_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures macos: - runs-on: macos-11.0 + runs-on: macos-26 steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v5 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 with: python-version: '3.8' - name: Build - run: source tools/ci.sh && ci_unix_macos_build + run: tools/ci.sh unix_macos_build - name: Run tests - run: source tools/ci.sh && ci_unix_macos_run_tests + run: tools/ci.sh unix_macos_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures qemu_mips: - runs-on: ubuntu-latest + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_unix_qemu_mips_setup + run: tools/ci.sh unix_qemu_mips_setup - name: Build - run: source tools/ci.sh && ci_unix_qemu_mips_build + run: tools/ci.sh unix_qemu_mips_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_qemu_mips_run_tests + run: tools/ci.sh unix_qemu_mips_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures qemu_arm: + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh unix_qemu_arm_setup + - name: Build + run: tools/ci.sh unix_qemu_arm_build + - name: Run main test suite + run: tools/ci.sh unix_qemu_arm_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + qemu_riscv64: + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v5 + - name: Install packages + run: tools/ci.sh unix_qemu_riscv64_setup + - name: Build + run: tools/ci.sh unix_qemu_riscv64_build + - name: Run main test suite + run: tools/ci.sh unix_qemu_riscv64_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + sanitize_address: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Install packages - run: source tools/ci.sh && ci_unix_qemu_arm_setup + run: tools/ci.sh unix_coverage_setup - name: Build - run: source tools/ci.sh && ci_unix_qemu_arm_build + run: tools/ci.sh unix_sanitize_address_build - name: Run main test suite - run: source tools/ci.sh && ci_unix_qemu_arm_run_tests + run: tools/ci.sh unix_sanitize_address_run_tests + - name: Test merging .mpy files + run: tools/ci.sh unix_coverage_run_mpy_merge_tests + - name: Build native mpy modules + run: tools/ci.sh native_mpy_modules_build + - name: Test importing .mpy generated by mpy_ld.py + run: tools/ci.sh unix_coverage_run_native_mpy_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + sanitize_undefined: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - uses: actions/setup-python@v6 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' + - name: Install packages + run: tools/ci.sh unix_coverage_setup + - name: Build + run: tools/ci.sh unix_sanitize_undefined_build + - name: Run main test suite + run: tools/ci.sh unix_sanitize_undefined_run_tests + - name: Test merging .mpy files + run: tools/ci.sh unix_coverage_run_mpy_merge_tests + - name: Build native mpy modules + run: tools/ci.sh native_mpy_modules_build + - name: Test importing .mpy generated by mpy_ld.py + run: tools/ci.sh unix_coverage_run_native_mpy_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_webassembly.yml b/.github/workflows/ports_webassembly.yml index 880f15ab34469..6bfbb5aec6c8e 100644 --- a/.github/workflows/ports_webassembly.yml +++ b/.github/workflows/ports_webassembly.yml @@ -11,6 +11,7 @@ on: - 'shared/**' - 'lib/**' - 'ports/webassembly/**' + - 'tests/**' concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -20,13 +21,13 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_webassembly_setup + run: tools/ci.sh webassembly_setup - name: Build - run: source tools/ci.sh && ci_webassembly_build + run: tools/ci.sh webassembly_build - name: Run tests - run: source tools/ci.sh && ci_webassembly_run_tests + run: tools/ci.sh webassembly_run_tests - name: Print failures if: failure() run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_windows.yml b/.github/workflows/ports_windows.yml index be2a2a8dac332..e4e0152d3e9da 100644 --- a/.github/workflows/ports_windows.yml +++ b/.github/workflows/ports_windows.yml @@ -28,13 +28,12 @@ jobs: visualstudio: ['2017', '2019', '2022'] include: - visualstudio: '2017' - runner: windows-latest vs_version: '[15, 16)' + custom_vs_install: true - visualstudio: '2019' - runner: windows-2019 vs_version: '[16, 17)' + custom_vs_install: true - visualstudio: '2022' - runner: windows-2022 vs_version: '[17, 18)' # trim down the number of jobs in the matrix exclude: @@ -42,22 +41,25 @@ jobs: configuration: Debug - visualstudio: '2019' configuration: Debug - runs-on: ${{ matrix.runner }} + runs-on: windows-latest + env: + CI_BUILD_CONFIGURATION: ${{ matrix.configuration }} steps: - - name: Install Visual Studio 2017 - if: matrix.visualstudio == '2017' + - name: Install Visual Studio ${{ matrix.visualstudio }} + if: matrix.custom_vs_install + shell: bash + # Shell functions in this block are to retry intermittent corrupt + # downloads (with a clean download dir) before failing the job outright run: | - choco install visualstudio2017buildtools - choco install visualstudio2017-workload-vctools - choco install windows-sdk-8.1 + try () { ($@) || ($@) || ($@) || ($@) } + clean_install () ( rm -rf $TEMP/chocolatey; choco install $1 ) + try clean_install visualstudio${{ matrix.visualstudio }}buildtools + try clean_install visualstudio${{ matrix.visualstudio }}-workload-vctools + try clean_install windows-sdk-8.1 - uses: microsoft/setup-msbuild@v2 with: vs-version: ${{ matrix.vs_version }} - - uses: actions/setup-python@v5 - if: matrix.runner == 'windows-2019' - with: - python-version: '3.9' - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Build mpy-cross.exe run: msbuild mpy-cross\mpy-cross.vcxproj -maxcpucount -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} - name: Update submodules @@ -101,23 +103,18 @@ jobs: env: i686 - sys: mingw64 env: x86_64 - runs-on: windows-2022 + runs-on: windows-latest env: CHERE_INVOKING: enabled_from_arguments defaults: run: shell: msys2 {0} steps: - - name: Get Python path - id: python_path - shell: python - run: | - import os - import sys - output = f"python={os.fspath(sys.executable)}" - print(output) - with open(os.environ["GITHUB_OUTPUT"], "w") as f: - f.write(output) + - uses: actions/setup-python@v6 + # note: can go back to installing mingw-w64-${{ matrix.env }}-python after + # MSYS2 updates to Python >3.12 (due to settrace compatibility issue) + with: + python-version: '3.11' - uses: msys2/setup-msys2@v2 with: msystem: ${{ matrix.sys }} @@ -126,10 +123,10 @@ jobs: make mingw-w64-${{ matrix.env }}-gcc pkg-config - python3 git diffutils - - uses: actions/checkout@v4 + path-type: inherit # Remove when setup-python is removed + - uses: actions/checkout@v5 - name: Build mpy-cross.exe run: make -C mpy-cross -j2 - name: Update submodules @@ -138,8 +135,7 @@ jobs: run: make -C ports/windows -j2 VARIANT=${{ matrix.variant }} - name: Run tests id: test - # msys python breaks tests so we need to use "real" windows python - run: MICROPY_CPYTHON3=$(cygpath "${{ steps.python_path.outputs.python }}") make -C ports/windows test_full VARIANT=${{ matrix.variant }} + run: make -C ports/windows test_full VARIANT=${{ matrix.variant }} - name: Print failures if: failure() && steps.test.conclusion == 'failure' working-directory: tests @@ -148,8 +144,8 @@ jobs: cross-build-on-linux: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Install packages - run: source tools/ci.sh && ci_windows_setup + run: tools/ci.sh windows_setup - name: Build - run: source tools/ci.sh && ci_windows_build + run: tools/ci.sh windows_build diff --git a/.github/workflows/ports_zephyr.yml b/.github/workflows/ports_zephyr.yml index f6f328c9277e4..3579f4e1bc90c 100644 --- a/.github/workflows/ports_zephyr.yml +++ b/.github/workflows/ports_zephyr.yml @@ -11,6 +11,11 @@ on: - 'shared/**' - 'lib/**' - 'ports/zephyr/**' + - 'tests/**' + schedule: + # Scheduled run exists to keep master branch Zephyr cache entry hot + # and prevent creating many redundant per-branch cache entries instead. + - cron: "40 4 * * *" concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -20,10 +25,44 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: jlumbroso/free-disk-space@main + with: + # Only free up a few things so this step runs quickly. + # (android would save 9.6GiB, but takes about 13m) + # (large-packages would save 4.6GiB, but takes about 3m) + android: false + dotnet: true + haskell: true + large-packages: false + docker-images: false + tool-cache: true + swap-storage: false + - uses: actions/checkout@v5 + - id: versions + name: Read Zephyr version + run: source tools/ci.sh && echo "ZEPHYR=$ZEPHYR_VERSION" | tee "$GITHUB_OUTPUT" + - name: Cached Zephyr Workspace + id: cache_workspace + uses: actions/cache@v4 + with: + # note that the Zephyr CI docker image is 15GB. At time of writing + # GitHub caches are limited to 10GB total for a project. So we only + # cache the "workspace" + path: ./zephyrproject + key: zephyr-workspace-${{ steps.versions.outputs.ZEPHYR }} + - name: ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: zephyr - name: Install packages - run: source tools/ci.sh && ci_zephyr_setup + run: tools/ci.sh zephyr_setup - name: Install Zephyr - run: source tools/ci.sh && ci_zephyr_install + if: steps.cache_workspace.outputs.cache-hit != 'true' + run: tools/ci.sh zephyr_install - name: Build - run: source tools/ci.sh && ci_zephyr_build + run: tools/ci.sh zephyr_build + - name: Run main test suite + run: tools/ci.sh zephyr_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ruff.yml b/.github/workflows/ruff.yml index 29e9ddbf8b61a..633b0cdf82ef4 100644 --- a/.github/workflows/ruff.yml +++ b/.github/workflows/ruff.yml @@ -6,8 +6,8 @@ jobs: ruff: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - # ruff version should be kept in sync with .pre-commit-config.yaml - - run: pip install --user ruff==0.1.3 + - uses: actions/checkout@v5 + # ruff version should be kept in sync with .pre-commit-config.yaml & also micropython-lib + - run: pipx install ruff==0.11.6 - run: ruff check --output-format=github . - run: ruff format --diff . diff --git a/.gitignore b/.gitignore index 2d20cb18970e8..56616426fb27a 100644 --- a/.gitignore +++ b/.gitignore @@ -11,8 +11,9 @@ build/ build-*/ docs/genrst/ -# Test failure outputs +# Test failure outputs and intermediate artefacts tests/results/* +tests/ports/unix/ffi_lib.so # Python cache files __pycache__/ @@ -23,3 +24,11 @@ user.props # MacOS desktop metadata files .DS_Store + +# Created by ci.sh zephyr targets +/.ccache +/zephyrproject + +# Created by ci.sh esp8266 targets +/xtensa-lx106-elf-standalone.tar.gz +/xtensa-lx106-elf/ diff --git a/.gitmodules b/.gitmodules index e6712c1f9f40e..d2c229dd6d751 100644 --- a/.gitmodules +++ b/.gitmodules @@ -3,7 +3,7 @@ url = https://github.com/micropython/axtls.git [submodule "lib/libffi"] path = lib/libffi - url = https://github.com/atgreen/libffi + url = https://github.com/libffi/libffi [submodule "lib/lwip"] path = lib/lwip url = https://github.com/lwip-tcpip/lwip.git @@ -35,7 +35,7 @@ url = https://github.com/bluekitchen/btstack.git [submodule "lib/nxp_driver"] path = lib/nxp_driver - url = https://github.com/hathach/nxp_driver.git + url = https://github.com/micropython/nxp_driver.git [submodule "lib/libhydrogen"] path = lib/libhydrogen url = https://github.com/jedisct1/libhydrogen.git @@ -68,3 +68,9 @@ [submodule "lib/arduino-lib"] path = lib/arduino-lib url = https://github.com/arduino/arduino-lib-mpy.git +[submodule "lib/alif_ensemble-cmsis-dfp"] + path = lib/alif_ensemble-cmsis-dfp + url = https://github.com/alifsemi/alif_ensemble-cmsis-dfp.git +[submodule "lib/alif-security-toolkit"] + path = lib/alif-security-toolkit + url = https://github.com/micropython/alif-security-toolkit.git diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index a1c811339adcc..ac9785bb59232 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -12,14 +12,14 @@ repos: verbose: true stages: [commit-msg] - repo: https://github.com/charliermarsh/ruff-pre-commit - # Version should be kept in sync with .github/workflows/ruff.yml - rev: v0.1.3 + # Version should be kept in sync with .github/workflows/ruff.yml & also micropython-lib + rev: v0.11.6 hooks: - id: ruff - id: ruff-format - repo: https://github.com/codespell-project/codespell # Version should be kept in sync with .github/workflows/codespell.yml - rev: v2.2.6 + rev: v2.4.1 hooks: - id: codespell name: Spellcheck for changed files (codespell) diff --git a/CODECONVENTIONS.md b/CODECONVENTIONS.md index d6af0418e42ef..d3f71cb083deb 100644 --- a/CODECONVENTIONS.md +++ b/CODECONVENTIONS.md @@ -206,14 +206,21 @@ adhere to the existing style and use `tools/codeformat.py` to check any changes. The main conventions, and things not enforceable via the auto-formatter, are described below. -White space: +As the MicroPython code base is over ten years old, not every source file +conforms fully to these conventions. If making small changes to existing code, +then it's usually acceptable to follow the existing code's style. New code or +major changes should follow the conventions described here. + +## White space + - Expand tabs to 4 spaces. - Don't leave trailing whitespace at the end of a line. - For control blocks (if, for, while), put 1 space between the keyword and the opening parenthesis. - Put 1 space after a comma, and 1 space around operators. -Braces: +## Braces + - Use braces for all blocks, even no-line and single-line pieces of code. - Put opening braces on the end of the line it belongs to, not on @@ -221,18 +228,43 @@ Braces: - For else-statements, put the else on the same line as the previous closing brace. -Header files: +## Header files + - Header files should be protected from multiple inclusion with #if directives. See an existing header for naming convention. -Names: +## Names + - Use underscore_case, not camelCase for all names. - Use CAPS_WITH_UNDERSCORE for enums and macros. - When defining a type use underscore_case and put '_t' after it. -Integer types: MicroPython runs on 16, 32, and 64 bit machines, so it's -important to use the correctly-sized (and signed) integer types. The -general guidelines are: +### Public names (declared in headers) + +- MicroPython-specific names (especially any declared in `py/` and `extmod/` + directories) should generally start with `mp_` or `MP_`. +- Functions and variables declared in a header should generally share a longer + common prefix. Usually the prefix matches the file name (i.e. items defined in + `py/obj.c` are declared in `py/obj.h` and should be prefixed `mp_obj_`). There + are exceptions, for example where one header file contains declarations + implemented in multiple source files for expediency. + +### Private names (specific to a single .c file) + +- For static functions and variables exposed to Python (i.e. a static C function + that is wrapped in `MP_DEFINE_CONST_FUN_...` and attached to a module), use + the file-level shared common prefix, i.e. name them as if the function or + variable was not static. +- Other static definitions in source files (i.e. functions or variables defined + in a .c file that are only used within that .c file) don't need any prefix + (specifically: no `s_` or `_` prefix, and generally avoid adding the + file-level common prefix). + +## Integer types + +MicroPython runs on 16, 32, and 64 bit machines, so it's important to use the +correctly-sized (and signed) integer types. The general guidelines are: + - For most cases use mp_int_t for signed and mp_uint_t for unsigned integer values. These are guaranteed to be machine-word sized and therefore big enough to hold the value from a MicroPython small-int @@ -241,11 +273,13 @@ general guidelines are: - You can use int/uint, but remember that they may be 16-bits wide. - If in doubt, use mp_int_t/mp_uint_t. -Comments: +## Comments + - Be concise and only write comments for things that are not obvious. - Use `// ` prefix, NOT `/* ... */`. No extra fluff. -Memory allocation: +## Memory allocation + - Use m_new, m_renew, m_del (and friends) to allocate and free heap memory. These macros are defined in py/misc.h. diff --git a/LICENSE b/LICENSE index 9142c46045846..929a2e97de7bf 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2013-2023 Damien P. George +Copyright (c) 2013-2025 Damien P. George Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -59,7 +59,6 @@ used during the build process and is not part of the compiled source code. /pico-sdk (BSD-3-clause) /re15 (BSD-3-clause) /stm32lib (BSD-3-clause) - /tinytest (BSD-3-clause) /tinyusb (MIT) /uzlib (Zlib) /wiznet5k (MIT) @@ -73,6 +72,7 @@ used during the build process and is not part of the compiled source code. /ppp_set_auth.* (Apache-2.0) /rp2 /mutex_extra.c (BSD-3-clause) + /clocks_extra.c (BSD-3-clause) /stm32 /usbd*.c (MCD-ST Liberty SW License Agreement V2) /stm32_it.* (MIT + BSD-3-clause) diff --git a/README.md b/README.md index f5bc6d78f0a40..375ff17d92481 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ Python 3.5 and some select features from later versions). The following core datatypes are provided: `str`(including basic Unicode support), `bytes`, `bytearray`, `tuple`, `list`, `dict`, `set`, `frozenset`, `array.array`, `collections.namedtuple`, classes and instances. Builtin modules include -`os`, `sys`, `time`, `re`, and `struct`, etc. Select ports have support for +`os`, `sys`, `time`, `re`, and `struct`, etc. Some ports have support for `_thread` module (multithreading), `socket` and `ssl` for networking, and `asyncio`. Note that only a subset of Python 3 functionality is implemented for the data types and modules. @@ -35,8 +35,8 @@ DAC, PWM, SPI, I2C, CAN, Bluetooth, and USB. Getting started --------------- -See the [online documentation](https://docs.micropython.org/) for API -references and information about using MicroPython and information about how +See the [online documentation](https://docs.micropython.org/) for the API +reference and information about using MicroPython and information about how it is implemented. We use [GitHub Discussions](https://github.com/micropython/micropython/discussions) @@ -80,9 +80,8 @@ This repository contains the following components: - [examples/](examples/) -- a few example Python scripts. "make" is used to build the components, or "gmake" on BSD-based systems. -You will also need bash, gcc, and Python 3.3+ available as the command `python3` -(if your system only has Python 2.7 then invoke make with the additional option -`PYTHON=python2`). Some ports (rp2 and esp32) additionally use CMake. +You will also need bash, gcc, and Python 3.3+ available as the command `python3`. +Some ports (rp2 and esp32) additionally use CMake. Supported platforms & architectures ----------------------------------- @@ -99,28 +98,74 @@ development and testing of MicroPython itself, as well as providing lightweight alternative to CPython on these platforms (in particular on embedded Linux systems). -The ["minimal"](ports/minimal) port provides an example of a very basic -MicroPython port and can be compiled as both a standalone Linux binary as -well as for ARM Cortex M4. Start with this if you want to port MicroPython to -another microcontroller. Additionally the ["bare-arm"](ports/bare-arm) port -is an example of the absolute minimum configuration, and is used to keep -track of the code size of the core runtime and VM. - -In addition, the following ports are provided in this repository: - - [cc3200](ports/cc3200) -- Texas Instruments CC3200 (including PyCom WiPy). - - [esp32](ports/esp32) -- Espressif ESP32 SoC (including ESP32S2, ESP32S3, ESP32C3). - - [esp8266](ports/esp8266) -- Espressif ESP8266 SoC. - - [mimxrt](ports/mimxrt) -- NXP m.iMX RT (including Teensy 4.x). - - [nrf](ports/nrf) -- Nordic Semiconductor nRF51 and nRF52. - - [pic16bit](ports/pic16bit) -- Microchip PIC 16-bit. - - [powerpc](ports/powerpc) -- IBM PowerPC (including Microwatt) - - [qemu-arm](ports/qemu-arm) -- QEMU-based emulated target, for testing) - - [renesas-ra](ports/renesas-ra) -- Renesas RA family. - - [rp2](ports/rp2) -- Raspberry Pi RP2040 (including Pico and Pico W). - - [samd](ports/samd) -- Microchip (formerly Atmel) SAMD21 and SAMD51. - - [stm32](ports/stm32) -- STMicroelectronics STM32 family (including F0, F4, F7, G0, G4, H7, L0, L4, WB) - - [webassembly](ports/webassembly) -- Emscripten port targeting browsers and NodeJS. - - [zephyr](ports/zephyr) -- Zephyr RTOS. +Over twenty different MicroPython ports are provided in this repository, +split across three +[MicroPython Support Tiers](https://docs.micropython.org/en/latest/develop/support_tiers.html). + +Tier 1 Ports +============ + +👑 Ports in [Tier 1](https://docs.micropython.org/en/latest/develop/support_tiers.html) +are mature and have the most active development, support and testing: + +| Port | Target | Quick Reference | +|--------------------------|----------------------------------------------------------------------------------------|----------------------------------------------------------------------| +| [esp32](ports/esp32)* | Espressif ESP32 SoCs (ESP32, ESP32S2, ESP32S3, ESP32C3, ESP32C6) | [here](https://docs.micropython.org/en/latest/esp32/quickref.html) | +| [mimxrt](ports/mimxrt) | NXP m.iMX RT | [here](https://docs.micropython.org/en/latest/mimxrt/quickref.html) | +| [rp2](ports/rp2) | Raspberry Pi RP2040 and RP2350 | [here](https://docs.micropython.org/en/latest/rp2/quickref.html) | +| [samd](ports/samd) | Microchip (formerly Atmel) SAMD21 and SAMD51 | [here](https://docs.micropython.org/en/latest/samd/quickref.html) | +| [stm32](ports/stm32) | STMicroelectronics STM32 MCUs (F0, F4, F7, G0, G4, H5, H7, L0, L1, L4, N6, WB, WL) | [here](https://docs.micropython.org/en/latest/pyboard/quickref.html) | +| [unix](ports/unix) | Linux, BSD, macOS, WSL | [here](https://docs.micropython.org/en/latest/unix/quickref.html) | +| [windows](ports/windows) | Microsoft Windows | [here](https://docs.micropython.org/en/latest/unix/quickref.html) | + +An asterisk indicates that the port has ongoing financial support from the vendor. + +Tier 2 Ports +============ + +✔ Ports in [Tier 2](https://docs.micropython.org/en/latest/develop/support_tiers.html) +are less mature and less actively developed and tested than Tier 1, but +still fully supported: + +| Port | Target | Quick Reference | +|----------------------------------|-------------------------------------------------------------|-------------------------------------------------------------------------| +| [alif](ports/alif) | Alif Semiconductor Ensemble MCUs (E3, E7) | | +| [embed](ports/embed) | Generates a set of .c/.h files for embedding into a project | | +| [nrf](ports/nrf) | Nordic Semiconductor nRF51 and nRF52 | | +| [renesas-ra](ports/renesas-ra) | Renesas RA family | [here](https://docs.micropython.org/en/latest/renesas-ra/quickref.html) | +| [webassembly](ports/webassembly) | Emscripten port targeting browsers and NodeJS | | +| [zephyr](ports/zephyr) | Zephyr RTOS | [here](https://docs.micropython.org/en/latest/zephyr/quickref.html) | + +Tier 3 Ports +============ + +Ports in [Tier 3](https://docs.micropython.org/en/latest/develop/support_tiers.html) +are built in CI but not regularly tested by the MicroPython maintainers: + +| Port | Target | Quick Reference | +|----------------------------|-------------------------------------------------------------------|-------------------------------------------------------------------------| +| [cc3200](ports/cc3200) | Texas Instruments CC3200 | [For WiPy](https://docs.micropython.org/en/latest/wipy/quickref.html) | +| [esp8266](ports/esp8266) | Espressif ESP8266 SoC | [here](https://docs.micropython.org/en/latest/esp8266/quickref.html) | +| [pic16bit](ports/pic16bit) | Microchip PIC 16-bit | | +| [powerpc](ports/powerpc) | IBM PowerPC (including Microwatt) | | + +Additional Ports +================ + +In addition to the above there is a Tier M containing ports that are used +primarily for maintenance, development and testing: + +- The ["bare-arm"](ports/bare-arm) port is an example of the absolute minimum + configuration that still includes the compiler, and is used to keep track + of the code size of the core runtime and VM. + +- The ["minimal"](ports/minimal) port provides an example of a very basic + MicroPython port and can be compiled as both a standalone Linux binary as + well as for ARM Cortex-M4. Start with this if you want to port MicroPython + to another microcontroller. + +- The [qemu](ports/qemu) port is a QEMU-based emulated target for Cortex-A, + Cortex-M, RISC-V 32-bit and RISC-V 64-bit architectures. The MicroPython cross-compiler, mpy-cross ----------------------------------------- diff --git a/docs/README.md b/docs/README.md index 892726ba17ffe..9b3b036e063a6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,8 +13,7 @@ Building the documentation locally If you're making changes to the documentation, you may want to build the documentation locally so that you can preview your changes. -Install Sphinx, and optionally (for the RTD-styling), sphinx_rtd_theme, -preferably in a virtualenv: +Install Sphinx and sphinx_rtd_theme, preferably in a virtualenv: pip install sphinx pip install sphinx_rtd_theme @@ -25,6 +24,21 @@ In `micropython/docs`, build the docs: You'll find the index page at `micropython/docs/build/html/index.html`. +Documentation autobuild +----------------------- + +For a more convenient development experience, you can use `sphinx-autobuild` +to automatically rebuild and serve the documentation when you make changes: + + pip install sphinx-autobuild + +Then run from the `micropython/docs` directory: + + sphinx-autobuild . build/html + +This will start a local web server (typically at `http://127.0.0.1:8000`) +and automatically rebuild the documentation whenever you save changes to the source files. + Having readthedocs.org build the documentation ---------------------------------------------- diff --git a/docs/conf.py b/docs/conf.py index 728e3c4285d82..603543aa18c7e 100755 --- a/docs/conf.py +++ b/docs/conf.py @@ -36,6 +36,9 @@ "is_release": micropy_version != "latest", } +# Authors used in various parts of the documentation. +micropy_authors = "MicroPython authors and contributors" + # -- General configuration ------------------------------------------------ @@ -52,6 +55,7 @@ "sphinx.ext.todo", "sphinx.ext.coverage", "sphinxcontrib.jquery", + "sphinx_rtd_theme", ] # Add any paths that contain templates here, relative to this directory. @@ -68,7 +72,7 @@ # General information about the project. project = "MicroPython" -copyright = "- The MicroPython Documentation is Copyright © 2014-2023, Damien P. George, Paul Sokolovsky, and contributors" +copyright = "- The MicroPython Documentation is Copyright © 2014-2025, " + micropy_authors # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -125,20 +129,9 @@ # -- Options for HTML output ---------------------------------------------- -# on_rtd is whether we are on readthedocs.org -on_rtd = os.environ.get("READTHEDOCS", None) == "True" - -if not on_rtd: # only import and set the theme if we're building docs locally - try: - import sphinx_rtd_theme +import sphinx_rtd_theme - html_theme = "sphinx_rtd_theme" - html_theme_path = [sphinx_rtd_theme.get_html_theme_path(), "."] - except: - html_theme = "default" - html_theme_path = ["."] -else: - html_theme_path = ["."] +html_theme = "sphinx_rtd_theme" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the @@ -244,7 +237,7 @@ master_doc, "MicroPython.tex", "MicroPython Documentation", - "Damien P. George, Paul Sokolovsky, and contributors", + micropy_authors, "manual", ), ] @@ -281,7 +274,7 @@ "index", "micropython", "MicroPython Documentation", - ["Damien P. George, Paul Sokolovsky, and contributors"], + [micropy_authors], 1, ), ] @@ -300,7 +293,7 @@ master_doc, "MicroPython", "MicroPython Documentation", - "Damien P. George, Paul Sokolovsky, and contributors", + micropy_authors, "MicroPython", "One line description of project.", "Miscellaneous", diff --git a/docs/develop/cmodules.rst b/docs/develop/cmodules.rst index 75dbc953c06f8..c5aa919b7645a 100644 --- a/docs/develop/cmodules.rst +++ b/docs/develop/cmodules.rst @@ -59,7 +59,7 @@ A MicroPython user C module is a directory with the following files: SRC_USERMOD_LIB_C += $(EXAMPLE_MOD_DIR)/utils/algorithm.c Similarly, use ``SRC_USERMOD_CXX`` and ``SRC_USERMOD_LIB_CXX`` for C++ - source files. + source files. If you want to include assembly files use ``SRC_USERMOD_LIB_ASM``. If you have custom compiler options (like ``-I`` to add directories to search for header files), these should be added to ``CFLAGS_USERMOD`` for C code diff --git a/docs/develop/gettingstarted.rst b/docs/develop/gettingstarted.rst index c1fd338c54cc0..cb479458e1102 100644 --- a/docs/develop/gettingstarted.rst +++ b/docs/develop/gettingstarted.rst @@ -106,7 +106,7 @@ See the `ARM GCC toolchain `_ for the latest details. -Python is also required. Python 2 is supported for now, but we recommend using Python 3. +Python 3 is also required. Check that you have Python available on your system: .. code-block:: bash @@ -278,10 +278,78 @@ To run a selection of tests on a board/device connected over USB use: .. code-block:: bash $ cd tests - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 See also :ref:`writingtests`. +Additional make targets for developers +-------------------------------------- + +In all ``make``-based ports, there is a target to print the size of a specific object file. +When a change is confined to a single file, this is useful when testing variations to find smaller alternatives. + +For instance, to print the size of ``objstr.o`` in the ``py/`` directory when making a unix standard build: + +.. code-block:: bash + + $ make build-standard/py/objstr.sz + +Similarly, there is a target to save the preprocessed version of a file: + +.. code-block:: bash + + $ make build-standard/py/objstr.pp + +In ``ports/unix`` there are additional targets related to running tests: + +.. code-block:: bash + + $ make test//int # Run all tests matching the pattern "int" + $ make test/ports/unix # Run all tests in ports/unix + $ make test-failures # Re-run only the failed tests + $ make print-failures # print the differences for failed tests + $ make clean-failures # delete the .exp and .out files from failed tests + +Using ci.sh locally +------------------- + +MicroPython uses GitHub Actions for continuous integration. +To reduce dependence on any specific CI system, the actual build steps for Unix-based builds are in the file ``tools/ci.sh``. +This can also be used as a script on developer desktops, with caveats: + +* For most steps, An Ubuntu/Debian system similar to the one used during CI is assumed. +* Some specific steps assume specific Ubuntu versions. +* The setup steps may invoke the system package manager to install packages, + download and install software from the internet, etc. + +To get a usage message including the list of commands, run: + +.. code-block:: bash + + $ tools/ci.sh --help + +As an example, you can build and test the unix minimal port with: + +.. code-block:: bash + + $ tools/ci.sh unix_minimal_build unix_minimal_run_tests + +If you use the bash shell, you can add a ``ci`` command with tab completion: + +.. code-block:: bash + + $ eval `tools/ci.sh --bash-completion` + $ ci unix_cov + +This will complete the ci step name to ``unix_coverage_``. +Pressing tab a second time will show the list of matching steps: + +.. code-block:: bash + + $ ci unix_coverage_ + unix_coverage_32bit_build + unix_coverage_32bit_run_native_mpy_tests… + Folder structure ---------------- diff --git a/docs/develop/index.rst b/docs/develop/index.rst index 327038f1978bd..1026b43d0c3f2 100644 --- a/docs/develop/index.rst +++ b/docs/develop/index.rst @@ -24,3 +24,4 @@ MicroPython to a new platform and implementing a core MicroPython library. publiccapi.rst extendingmicropython.rst porting.rst + support_tiers.rst diff --git a/docs/develop/natmod.rst b/docs/develop/natmod.rst index 502ea1c4c6873..072d78b2076b4 100644 --- a/docs/develop/natmod.rst +++ b/docs/develop/natmod.rst @@ -39,7 +39,8 @@ options for the ``ARCH`` variable, see below): * ``armv7emsp`` (ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7) * ``armv7emdp`` (ARM Thumb 2, double precision float, eg Cortex-M7) * ``xtensa`` (non-windowed, eg ESP8266) -* ``xtensawin`` (windowed with window size 8, eg ESP32) +* ``xtensawin`` (windowed with window size 8, eg ESP32, ESP32S3) +* ``rv32imc`` (RISC-V 32 bits with compressed instructions, eg ESP32C3, ESP32C6) When compiling and linking the native .mpy file the architecture must be chosen and the corresponding file can only be imported on that architecture. For more @@ -66,14 +67,31 @@ The known limitations are: * static BSS variables are not supported; workaround: use global BSS variables +* thread-local storage variables are not supported on rv32imc; workaround: use + global BSS variables or allocate some space on the heap to store them + So, if your C code has writable data, make sure the data is defined globally, without an initialiser, and only written to within functions. +The native module is not automatically linked against the standard static libraries +like ``libm.a`` and ``libgcc.a``, which can lead to ``undefined symbol`` errors. +You can link the runtime libraries by setting ``LINK_RUNTIME = 1`` +in your Makefile. Custom static libraries can also be linked by adding +``MPY_LD_FLAGS += -l path/to/library.a``. Note that these are linked into +the native module and will not be shared with other modules or the system. + Linker limitation: the native module is not linked against the symbol table of the full MicroPython firmware. Rather, it is linked against an explicit table of exported symbols found in ``mp_fun_table`` (in ``py/nativeglue.h``), that is fixed at firmware build time. It is thus not possible to simply call some arbitrary HAL/OS/RTOS/system -function, for example. +function, for example, unless that resides at a fixed address. In that case, the path +of a linkerscript containing a series of symbol names and their fixed address can be +passed to ``mpy_ld.py`` via the ``--externs`` command line argument. That way symbols +appearing in the linkerscript will take precedence over what is provided from object +files, but at the moment the object files' implementation will still reside in the +final MPY file. The linkerscript parser is limited in its capabilities, and is +currently used only for parsing the ESP8266 port ROM symbols list (see +``ports/esp8266/boards/eagle.rom.addr.v6.ld``). New symbols can be added to the end of the table and the firmware rebuilt. The symbols also need to be added to ``tools/mpy_ld.py``'s ``fun_table`` dict in the @@ -172,7 +190,7 @@ The file ``Makefile`` contains: # Source files (.c or .py) SRC = factorial.c - # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin) + # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin, rv32imc) ARCH = x64 # Include to get the rules for compiling and linking the module @@ -210,6 +228,26 @@ other module, for example:: print(factorial.factorial(10)) # should display 3628800 +Using Picolibc when building modules +------------------------------------ + +Using `Picolibc `_ as your C standard +library is not only supported, but in fact it is the default for the rv32imc +platform. However, there are a couple of things worth mentioning to make sure +you don't run into problems later when building code. + +Some pre-built Picolibc versions (for example, those provided by Ubuntu Linux +as the ``picolibc-arm-none-eabi``, ``picolibc-riscv64-unknown-elf``, and +``picolibc-xtensa-lx106-elf`` packages) assume thread-local storage (TLS) is +available at runtime, but unfortunately MicroPython modules do not support that +on some architectures (namely ``rv32imc``). This means that some +functionalities provided by Picolibc will default to use TLS, returning an +error either during compilation or during linking. + +For an example on how this may affect you, the ``examples/natmod/btree`` +example module contains a workaround to make sure ``errno`` works (look for +``__PICOLIBC_ERRNO_FUNCTION`` in the Makefile and follow the trail from there). + Further examples ---------------- diff --git a/docs/develop/porting.rst b/docs/develop/porting.rst index 99725e1000bef..5dff843a3a983 100644 --- a/docs/develop/porting.rst +++ b/docs/develop/porting.rst @@ -42,7 +42,6 @@ The basic MicroPython firmware is implemented in the main port file, e.g ``main. #include "py/compile.h" #include "py/gc.h" #include "py/mperrno.h" - #include "py/stackctrl.h" #include "shared/runtime/gchelper.h" #include "shared/runtime/pyexec.h" @@ -51,7 +50,7 @@ The basic MicroPython firmware is implemented in the main port file, e.g ``main. int main(int argc, char **argv) { // Initialise the MicroPython runtime. - mp_stack_ctrl_init(); + mp_cstack_init_with_sp_here(2048); gc_init(heap, heap + sizeof(heap)); mp_init(); @@ -162,8 +161,6 @@ The following is an example of an ``mpconfigport.h`` file: // Type definitions for the specific machine. - typedef intptr_t mp_int_t; // must be pointer size - typedef uintptr_t mp_uint_t; // must be pointer size typedef long mp_off_t; // We need to provide a declaration/definition of alloca(). diff --git a/docs/develop/support_tiers.rst b/docs/develop/support_tiers.rst new file mode 100644 index 0000000000000..f49ee4124f834 --- /dev/null +++ b/docs/develop/support_tiers.rst @@ -0,0 +1,68 @@ +MicroPython Support Tiers +========================= + +MicroPython operates with a set of Support Tier levels for the various ports. +Tiers 1, 2 and 3 are the main Tier levels with Tier 1 being the most mature and +actively maintained. There is also Tier M for additional ports used primarily +for maintenance, development and testing. These Tier levels are defined in the +table below. + +.. table:: + :widths: 40 9 9 9 9 + + +-----------------------------------------------+--------+--------+--------+--------+ + | | Tier 1 | Tier 2 | Tier 3 | Tier M | + +-----------------------------------------------+--------+--------+--------+--------+ + | builds pass under CI | ✔ | ✔ | ✔ | ✔ | + +-----------------------------------------------+--------+--------+--------+--------+ + | tests run under CI (where possible) | ✔ | ✔ | ✔ | ✔ | + +-----------------------------------------------+--------+--------+--------+--------+ + | actively maintained | ✔ | ✔ | | ✔ | + +-----------------------------------------------+--------+--------+--------+--------+ + | stable Python API | ✔ | ✔ | | | + +-----------------------------------------------+--------+--------+--------+--------+ + | new features actively developed | ✔ | ✔ | | | + +-----------------------------------------------+--------+--------+--------+--------+ + | tested on hardware for releases | ✔ | ✔ | | | + +-----------------------------------------------+--------+--------+--------+--------+ + | prioritized bug reports | ✔ | | | ✔ | + +-----------------------------------------------+--------+--------+--------+--------+ + | regressions warrant a patch release | ✔ | | | ✔ | + +-----------------------------------------------+--------+--------+--------+--------+ + | has port-specific documentation | ✔ | | | | + +-----------------------------------------------+--------+--------+--------+--------+ + +Lower Tiers may tick more boxes, but the above table defines the minimum requirements +for a port to belong to a Tier. + +Tier 1 ports: + + - esp32 + - mimxrt + - rp2 + - samd + - stm32 + - unix + - windows + +Tier 2 ports: + + - alif + - embed + - nrf + - renesas-ra + - webassembly + - zephyr + +Tier 3 ports: + + - cc3200 + - esp8266 + - pic16bit + - powerpc + +Tier M ports: + + - bare-arm + - minimal + - qemu diff --git a/docs/develop/writingtests.rst b/docs/develop/writingtests.rst index 9bb5178f55f22..a7d033f17e9b9 100644 --- a/docs/develop/writingtests.rst +++ b/docs/develop/writingtests.rst @@ -60,7 +60,7 @@ Then to run on a board: .. code-block:: bash - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 And to run only a certain set of tests (eg a directory): diff --git a/docs/differences/python_36.rst b/docs/differences/python_36.rst index 3315b0594dafc..18da79f8f8431 100644 --- a/docs/differences/python_36.rst +++ b/docs/differences/python_36.rst @@ -25,7 +25,8 @@ Python 3.6 beta 1 was released on 12 Sep 2016, and a summary of the new features +--------------------------------------------------------+--------------------------------------------------+-----------------+ | `PEP 468 `_ | Preserving the order of *kwargs* in a function | | +--------------------------------------------------------+--------------------------------------------------+-----------------+ - | `PEP 487 `_ | Simpler customization of class creation | | + | `PEP 487 `_ | Simpler customization of class creation | Partial | + | | | [#setname]_ | +--------------------------------------------------------+--------------------------------------------------+-----------------+ | `PEP 520 `_ | Preserving Class Attribute Definition Order | | +--------------------------------------------------------+--------------------------------------------------+-----------------+ @@ -198,3 +199,7 @@ Changes to built-in modules: +--------------------------------------------------------------------------------------------------------------+----------------+ | The *compress()* and *decompress()* functions now accept keyword arguments | | +--------------------------------------------------------------------------------------------------------------+----------------+ + +.. rubric:: Notes + +.. [#setname] Currently, only :func:`__set_name__` is implemented. diff --git a/docs/esp32/quickref.rst b/docs/esp32/quickref.rst index 2be1dbadc353a..25e52ec32db5d 100644 --- a/docs/esp32/quickref.rst +++ b/docs/esp32/quickref.rst @@ -18,7 +18,7 @@ working with this board it may be useful to get an overview of the microcontroll general.rst tutorial/index.rst -Note that there are several varieties of ESP32 -- ESP32, ESP32C3, ESP32S2, ESP32S3 -- +Note that there are several varieties of ESP32 -- ESP32, ESP32C3, ESP32C6, ESP32S2, ESP32S3 -- supported by MicroPython, with some differences in functionality between them. Installing MicroPython @@ -61,13 +61,13 @@ The :mod:`esp32` module:: import esp32 esp32.raw_temperature() # read the internal temperature of the MCU, in Fahrenheit - esp32.ULP() # access to the Ultra-Low-Power Co-processor, not on ESP32C3 + esp32.ULP() # access to the Ultra-Low-Power Co-processor, not on ESP32C3/C6 Note that the temperature sensor in the ESP32 will typically read higher than ambient due to the IC getting warm while it runs. This effect can be minimised by reading the temperature sensor immediately after waking up from sleep. -ESP32C3, ESP32S2, and ESP32S3 also have an internal temperature sensor available. +ESP32C3, ESP32C6, ESP32S2, and ESP32S3 also have an internal temperature sensor available. It is implemented a bit differently to the ESP32 and returns the temperature in Celsius:: @@ -79,35 +79,35 @@ Networking WLAN ^^^^ -The :mod:`network` module:: +The :class:`network.WLAN` class in the :mod:`network` module:: import network - wlan = network.WLAN(network.STA_IF) # create station interface - wlan.active(True) # activate the interface - wlan.scan() # scan for access points - wlan.isconnected() # check if the station is connected to an AP + wlan = network.WLAN() # create station interface (the default, see below for an access point interface) + wlan.active(True) # activate the interface + wlan.scan() # scan for access points + wlan.isconnected() # check if the station is connected to an AP wlan.connect('ssid', 'key') # connect to an AP - wlan.config('mac') # get the interface's MAC address - wlan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + wlan.config('mac') # get the interface's MAC address + wlan.ipconfig('addr4') # get the interface's IPv4 addresses - ap = network.WLAN(network.AP_IF) # create access-point interface - ap.config(ssid='ESP-AP') # set the SSID of the access point - ap.config(max_clients=10) # set how many clients can connect to the network - ap.active(True) # activate the interface + ap = network.WLAN(network.WLAN.IF_AP) # create access-point interface + ap.config(ssid='ESP-AP') # set the SSID of the access point + ap.config(max_clients=10) # set how many clients can connect to the network + ap.active(True) # activate the interface A useful function for connecting to your local WiFi network is:: def do_connect(): - import network - wlan = network.WLAN(network.STA_IF) + import machine, network + wlan = network.WLAN() wlan.active(True) if not wlan.isconnected(): print('connecting to network...') wlan.connect('ssid', 'key') while not wlan.isconnected(): - pass - print('network config:', wlan.ifconfig()) + machine.idle() + print('network config:', wlan.ipconfig('addr4')) Once the network is established the :mod:`socket ` module can be used to create and use TCP/UDP sockets as usual, and the ``requests`` module for @@ -121,32 +121,56 @@ calling ``wlan.config(reconnects=n)``, where n are the number of desired reconne attempts (0 means it won't retry, -1 will restore the default behaviour of trying to reconnect forever). +.. _esp32_network_lan: + LAN ^^^ -To use the wired interfaces one has to specify the pins and mode :: +Built-in MAC (original ESP32) +""""""""""""""""""""""""""""" + +The original ESP32 SoC has a built-in Ethernet MAC. Using this MAC requires an +external Ethernet PHY to be wired to the chip's EMAC pins. Most of the EMAC pin +assignments are fixed, consult the ESP32 datasheet for details. + +If the PHY is connected, the internal Ethernet MAC can be configured via +the :class:`network.LAN` constructor:: import network lan = network.LAN(mdc=PIN_MDC, ...) # Set the pin and mode configuration lan.active(True) # activate the interface - lan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + lan.ipconfig('addr4') # get the interface's IPv4 addresses + + +Required keyword arguments for the constructor: + +- ``mdc`` and ``mdio`` - :class:`machine.Pin` objects (or integers) specifying + the MDC and MDIO pins. +- ``phy_type`` - Select the PHY device type. Supported devices are + ``PHY_GENERIC``, + ``PHY_LAN8710``, ``PHY_LAN8720``, ``PHY_IP101``, ``PHY_RTL8201``, + ``PHY_DP83848``, ``PHY_KSZ8041`` and ``PHY_KSZ8081``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. Common values are ``0`` and ``1``. +All of the above keyword arguments must be present to configure the interface. -The keyword arguments for the constructor defining the PHY type and interface are: +Optional keyword arguments: -- mdc=pin-object # set the mdc and mdio pins. -- mdio=pin-object -- reset=pin-object # set the reset pin of the PHY device. -- power=pin-object # set the pin which switches the power of the PHY device. -- phy_type= # Select the PHY device type. Supported devices are PHY_LAN8710, - PHY_LAN8720, PH_IP101, PHY_RTL8201, PHY_DP83848 and PHY_KSZ8041 -- phy_addr=number # The address number of the PHY device. -- ref_clk_mode=mode # Defines, whether the ref_clk at the ESP32 is an input - or output. Suitable values are Pin.IN and Pin.OUT. -- ref_clk=pin-object # defines the Pin used for ref_clk. +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the PHY reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the PHY device. +- ``ref_clk`` - :class:`machine.Pin` object (or integer) specifying the pin used + for the EMAC ``ref_clk`` signal. If not specified, the board default is used + (typically GPIO 0, but may be different if a particular board has Ethernet.) +- ``ref_clk_mode`` - Defines whether the EMAC ``ref_clk`` pin of the ESP32 + should be an input or an output. Suitable values are ``machine.Pin.IN`` and + ``machine.Pin.OUT``. If not specified, the board default is used + (typically input, but may be different if a particular board has Ethernet.) -These are working configurations for LAN interfaces of popular boards:: +These are working configurations for LAN interfaces of some popular ESP32 boards:: # Olimex ESP32-GATEWAY: power controlled by Pin(5) # Olimex ESP32 PoE and ESP32-PoE ISO: power controlled by Pin(12) @@ -171,6 +195,66 @@ These are working configurations for LAN interfaces of popular boards:: lan = network.LAN(id=0, mdc=Pin(23), mdio=Pin(18), power=Pin(5), phy_type=network.PHY_IP101, phy_addr=1) + +.. _esp32_spi_ethernet: + +SPI Ethernet Interface +"""""""""""""""""""""" + +All ESP32 SoCs support external SPI Ethernet interface chips. These are Ethernet +interfaces that connect via a SPI bus, rather than an Ethernet RMII interface. + +.. note:: The only exception is the ESP32 ``d2wd`` variant, where this feature is disabled + to save code size. + +SPI Ethernet uses the same :class:`network.LAN` constructor, with a different +set of keyword arguments:: + + import machine, network + + spi = machine.SPI(1, sck=SCK_PIN, mosi=MOSI_PIN, miso=MISO_PIN) + lan = network.LAN(spi=spi, cs=CS_PIN, ...) # Set the pin and mode configuration + lan.active(True) # activate the interface + lan.ipconfig('addr4') # get the interface's IPv4 addresses + +Required keyword arguments for the constructor: + +- ``spi`` - Should be a :class:`machine.SPI` object configured for this + connection. Note that any clock speed configured on the SPI object is ignored, + the SPI Ethernet clock speed is configured at compile time. +- ``cs`` - :class:`machine.Pin` object (or integer) specifying the CS pin + connected to the interface. +- ``int`` - :class:`machine.Pin` object (or integer) specifying the INT pin + connected to the interface. +- ``phy_type`` - Select the SPI Ethernet interface type. Supported devices are + ``PHY_KSZ8851SNL``, ``PHY_DM9051``, ``PHY_W5500``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. This is usually ``0`` for SPI Ethernet devices. + +All of the above keyword arguments must be present to configure the interface. + +Optional keyword arguments for the constructor: + +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the SPI Ethernet + interface reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the SPI Ethernet interface. + +Here is a sample configuration for a WIZNet W5500 chip connected to pins on +an ESP32-S3 development board:: + + import machine, network + from machine import Pin, SPI + + spi = SPI(1, sck=Pin(12), mosi=Pin(13), miso=Pin(14)) + lan = network.LAN(spi=spi, phy_type=network.PHY_W5500, phy_addr=0, + cs=Pin(10), int=Pin(11)) + +.. note:: WIZnet W5500 Ethernet is also supported on some other MicroPython + ports, but using a :ref:`different software interface + `. + Delay and timing ---------------- @@ -187,8 +271,10 @@ Use the :mod:`time