Replies: 6 comments 14 replies
-
|
One issue for me is having to pass the toolchain using Errors aren't always propagated: Also, when using Edk2ToolsBuild.py, if I remove the brotli submodule code: Looking in BaseTools/BaseToolsBuild/BASETOOLS_BUILD.txt there are no ERROR lines, even at: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for checking. An issue here is that the GNUmakefile errors need to be recognized and logged as |
Beta Was this translation helpful? Give feedback.
-
You can eventually see some help: That also works for other commands supported by Note that I had to know to use The text "Did you mean to use a different kind of invocable?" is a hint but probably not useful to someone looking for help. Did finding this information contribute to the issue you were having? Is the information helpful? |
Beta Was this translation helpful? Give feedback.
-
|
Issue filed for |
Beta Was this translation helpful? Give feedback.
-
|
tianocore/edk2-pytool-extensions#540 has been resolved. Additional error logging has been added when either TOOL_CHAIN_TAG or TARGET has not been set, along with a graceful exit. This will be available when v0.23.3 is released. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@makubacki I was wondering if you could take a look at my stuart_pr_eval failure at tianocore/edk2-pytool-extensions#550 ? I don't know why it's not working. |
Beta Was this translation helpful? Give feedback.
-
|
I replied. Feel free to follow up there if more help is needed. |
Beta Was this translation helpful? Give feedback.
-
A wish of mine would be an "offline" mode, where it does not download the toolchain (gcc, nasm, etc) but uses the tools it can find in I started looking into this a while back but then got distracted.
I get that .NET is convenient in come cases and it is easily available on most Linux environments, but it is a large dependency compared to what it is used for in the end (downloading dependencies via nuget, I think? Or does it do other things too?). It adds quite some size to the container images that we use in the (Linux) CI. |
Beta Was this translation helpful? Give feedback.
-
|
Offline mode seems reasonable, I've heard it before. We're also reducing .NET usage. @Javagedes will follow up. |
Beta Was this translation helpful? Give feedback.
-
|
@osteffenrh - the online/offline and even .NET dependency comes down to how a platform uses stuart. For example, if you don't leverage any nuget package based dependencies then you don't actually need .NET or nuget. As @makubacki mentioned because this has been a pain point for some edk2 developers we are trying to move edk2 away from nuget based dependencies. The platform controls the dependencies and tools it uses. You can control this through the scopes. You could easily create an option for an edk2 platform (like ovmf, armvirt, emulatorpkg) or your own platform that disables the scopes and makes it work with tools on the path. I don't believe this is a scalable or sustainable practice in that an end user has no idea what versions of tools are supported and thus we didn't do that when setting up the CI platforms in Edk2. One of the best value points for pytools is that it lets you define a repeatable and reproducible environment using artifacts that are checked into the source tree. But regardless, pytools shouldn't need any changes to support offline mode. Again, I am not a fan of this but can understand it could be ideal or useful for some projects and pytools in general should not require online usage if a "platform" doesn't require it. Finally, since I am not sure we have built a platform using all external tools (in a long time) it is possible something has creeped in. If so please let us know so we can fix it. |
Beta Was this translation helpful? Give feedback.
-
|
As requested from @mdkinney, We've added some stuart CLI usability improvements and will leave them open for a few days for anyone to leave comments. Review each PR to see the changes, but overall they do the following:
tianocore/edk2-pytool-extensions#753 |
Beta Was this translation helpful? Give feedback.
-
|
Was there any update on moving edk2 away from NuGet based dependencies? I agree with @osteffenrh that it seems .NET (e.g. on Linux I don't know if there's been any thought to adding GitHub releases as a supported dependency type in https://github.com/tianocore/edk2-pytool-extensions/tree/master/edk2toolext/environment/extdeptypes ? These can be arbitrarily named releases, containing arbitrarily named zip files (not necessarily builds of source in the associated repo) which I believe is about the right amount of structure to conveniently replace the way nasm, iasl and so on are currently downloaded from NuGet. I've seen other projects use this as a place to store versions of dependencies. EDIT 1: I guess GitHub release files can be fetched already, with the existing web dependency type (haven't yet tested, but it looks like it should be fine), but perhaps it still might be convenient to make explicit the support for named files within named releases in GitHub releases, with a new dependency type, rather then just folding this into a URL? EDIT 2: A release can be from an arbitrarily named tag on a repo, I think it must contain the (auto-generated) zip of the repo source at that tag, but can have any other arbitrary added zip files - I'm suggesting naming the release in the former way, then adding the iasl, masm, etc. zip files in the latter way - probably from a repo without much source, perhaps just a README, for this purpose. This is a very rough-and-ready demo of the idea: https://github.com/mikebeaton/edk2_ci/releases/tag/edk2-release-test-v0.1 |
Beta Was this translation helpful? Give feedback.
-
|
@mikebeaton Hello, as you mentioned, this can already be done with the web dependency exactly as requested as above, however it is not the most ergonomic. We have an example of this in mu_basecore, as we pre-compile our basetools rather than compile them locally: https://github.com/microsoft/mu_basecore/blob/dev/202405/BaseTools/Bin/basetoolsbin_ext_dep.yaml "scope": "global",
"id": "edk-tools-bin",
"type": "web",
"name": "Mu-Basetools",
"source": "https://github.com/microsoft/mu_basecore/releases/download/v2023020002.0.3/basetools-v2023020002.0.3.tar.gz",
"version": "v2023020002.0.3",
"sha256": "7260efe6e4e734ac66b24d6dc5f4908bf7aa444a7827cb405a66131b7b75e86f",
"internal_path": "/basetools/",
"compression_type": "tar",
"flags": ["set_shell_var", "set_path", "host_specific"],
"var_name": "EDK_TOOLS_BIN"I'd be happy to create an external dependency specific to github releases, what are you envisioning in terms of ergonomics compared to the web dependency? I could see us converting the path = f'https://github.com/%3Cspan%20class="pl-s1">{repository}/releases/download/{version}/{name}-{version}.tar.gz'But I'd hope for a bit more ergonomic improvement between web dependencyies and github release dependencies if possible Looking forward to your feedback. I can spend some time to get it up and running once we come up with something good. |
Beta Was this translation helpful? Give feedback.
-
|
The issue with this, however, is that it does not work well in our current use case, which is where we re-package a different tool with a version we wish to preserve, in a release that does not match that version. |
Beta Was this translation helpful? Give feedback.
-
|
Hi! How about: Which is then otherwise the same as a web dependency with https://github.com/tianocore/edk2-ci/releases/download/mu_nasm_2.15.5/mu_nasm_2.15.5.zip Equally: Which acts like a web dependency with https://github.com/tianocore/edk2-ci/releases/download/iasl.20190215.0.0/iasl.20190215.0.0.zip Then the dependency format allows fetching an arbitrary file from an arbitrary release, but the proposed repo https://github.com/tianocore/edk2-ci keeps to one tag-plus-release, per version, per tool? |
Beta Was this translation helpful? Give feedback.
-
|
Yes that seems good to me. I'll prototype it. |
Beta Was this translation helpful? Give feedback.
-
Hi. I have written down the steps I had to follow in order to run the tests.
I think I should be able to get some preliminary results in a couple of weeks. I am in the process of building CI for our Linux environment. I would like to keep the resources to run the tests low. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
A tool called Stuart is integrated with edk2 to help provide a consistent build and dependency management interface across different user environments.
Quick references:
Feedback has been received on numerous occasions that Stuart:
and so on...
This discussion was created to gather constructive feedback that can be channeled into actionable improvements. The tool aspires to provide a Python-based build environment that can accommodate common developer flows in the edk2 project, and this feedback will hopefully help bring it closer to achieving that.
I understand it can be frustrating to use something that may have fallen short of expectations. However, please try to keep feedback constructive, concise, and respectful.
Beta Was this translation helpful? Give feedback.
All reactions