Replies: 46 comments 15 replies
-
Meeting Notes 3/14/2022First meeting in the books (March 14, 2022). Thanks for those that attended. A few basic notes. I am sure I missed some. Please add comments if important. Future meetings will be recorded. Attendees@mdkinney DiscussionsBasic meeting introduction, scope, and goals of the seriesGoal being to get some community wide alignment on tooling, process, etc. with the desire to improve code base quality and lower friction for engagement while broadening the set of individuals contributing ideas, validation, and code. Feedback Edk2 mailing list is full of noise which makes it hard to catch everything of importance - Some have an "extensive" set of rules that makes it work for them. Other just redirect to folder/trash and never review. Brainstorming topics
Basetools Pip - progresstianocore/edk2-basetools: EDK II BaseTools Python tools as a PIP module (github.com) Intel CI system is being updated to support virtual environment usage and the pip based version of the tools (end of week) Edk2 CI system is blocked due to GCC arm tool chain download failingDiscussed CI is blocked on edk2 due to failing download. Seems to be with hosted toolchain. Intrinsic Lib in edk2Some activity happening on mailing list. Impacts this group because different toolchains need different intrinsics. Getting better clarity on supported toolchains/validated toolchains is important. More toolchains cause more CI resources. Etc. ConclusionPlease think about your priorities and what you can add to the TianoCore community as well as what you would like to see changed. |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, I cannot join today. The meeting time is a little late for my time zone (Europe). I worked on getting the CI jobs running on Ubuntu into containers (#2628). It almost works now with a custom Ubuntu 20.04 image. My wishes are:
Cheers, |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 3/21/2022Basic notes. Please add comments to the notes if important. And I forgot to record the meeting. sorry. Attendees@mdkinney New DiscussionsWindows/VS VM Image for
|
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 3/28/2022Basic notes. Please add comments to the notes if important. Attendees@mdkinney New Discussions / OpensClarification on VS2019 and VS2022 and Edk2 CIDefault CI image in Azure DevOps for Windows is changing to only include VS2022. Edk2 CI will need to update soon so that merging isn't blocked once the change happens. ScriptsWhere should edk2 specific dev scripts / helpers be checked in? Since edk2-basetools project is trying to remove the non-fw python code from edk2 (and improve quality and portability) where should other scripts go. @ajfish has a " Agreed to follow up once basetools was successfully moved. Build reproducibility@mdkinney - curious how important this is and if it is something the group wanted to focus on. Some work done in the uncrustify timeframe but more changes would be needed to continue this effort. Asserts filename macro, file paths, PDB path, date/time are all common problems. @spbrogan - Still need a business driver before this is priority. Maybe edk2 releasing prebuilt binaries would be motivation. Very little discussion in group. Google summer of code interns@nate-desimone reported that it looks to be on track for roughly 5 interns. Still would be great to have more mentors. Reach out to nate if interested. Github PR to mailing list edk2 reflector status@mdkinney said maybe have time to focus on in the next few weeks. Questions about need for this given GitHub improvements. Discuss developer workflows, policies, and if edk2 can "fit" into the capabilities of GitHub rather than having to author, run, and maintain a custom tool. See recording for more details. RFC tracking@spbrogan requested discussion/thought of changing the RFC process to be GitHub based. Would like to see more discussion, positive approval process, and a better permanent record for RFCs. See Rust-Lang for a great example. https://github.com/rust-lang/rfcs . Discussion to be moved to April Community meeting. @miki-intel-work -- can you add it? Ongoing DiscussionsAdd Clang to CI@BobCF will review and update group on any progress to add ARM/ARCH64/RISCV support. Toolchain support / maintainersNo Update. Do this week.
Edk2-basetools python project transitionNo updates. Expect more next week. Issue 30 needs to be resolved as it is blocking validation. tianocore/edk2-basetools#30 Next StepsPlease think about your priorities and what you can add to the TianoCore community as well as what you would like to see changed. |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, sorry, can't attend today. Also, are the recordings/transcripts available? Thanks, |
Beta Was this translation helpful? Give feedback.
-
|
The recordings are posted here: devel@edk2.groups.io | Files | EDK2 Community Meeting on Build, Tools, CI |
Beta Was this translation helpful? Give feedback.
-
|
Sean,
I was discussing Microsoft’s crypto lib with the engineering team. They asked if Microsoft was considering getting FIPs certification.
Thanks,
Kevin
…_______________________________________________________________
On Mar 28, 2022, at 6:29 PM, Sean Brogan ***@***.***> wrote:
Meeting Notes 3/28/2022
Basic notes. Please add comments to the notes if important.
Recording available here <Todo: add link>
Transcript available here <Todo: add link>
Attendees
@mdkinney<https://github.com/mdkinney>
@BobCF<https://github.com/BobCF>
@spbrogan<https://github.com/spbrogan>
@ashedesimone<https://github.com/ashedesimone>
@nate-desimone<https://github.com/nate-desimone>
@kevindavisinsyde<https://github.com/kevindavisinsyde>
@miki-intel-work<https://github.com/miki-intel-work>
@hlhsiung<https://github.com/hlhsiung>
@ajfish<https://github.com/ajfish>
@YuweiChen1110<https://github.com/YuweiChen1110>
New Discussions / Opens
Clarification on VS2019 and VS2022 and Edk2 CI
Default CI image in Azure DevOps for Windows is changing to only include VS2022. Edk2 CI will need to update soon so that merging isn't blocked once the change happens.
Scripts
Where should edk2 specific dev scripts / helpers be checked in? Since edk2-basetools project is trying to remove the non-fw python code from edk2 (and improve quality and portability) where should other scripts go. @ajfish<https://github.com/ajfish> has a "git grep" like tool that filters based on a built project that would be useful to others. Additionally there are debug scripts and scripts that have been developed. Ideas:
Create a template repo that makes it easy to develop, validate, and publish python tools for edk2 consumption.
Add to existing repo
Agreed to follow up once basetools was successfully moved.
Build reproducibility
@mdkinney<https://github.com/mdkinney> - curious how important this is and if it is something the group wanted to focus on. Some work done in the uncrustify timeframe but more changes would be needed to continue this effort. Asserts filename macro, file paths, PDB path, date/time are all common problems. @spbrogan<https://github.com/spbrogan> - Still need a business driver before this is priority. Maybe edk2 releasing prebuilt binaries would be motivation. Very little discussion in group.
Google summer of code interns
@nate-desimone<https://github.com/nate-desimone> reported that it looks to be on track for roughly 5 interns. Still would be great to have more mentors. Reach out to nate if interested.
Github PR to mailing list edk2 reflector status
@mdkinney<https://github.com/mdkinney> said maybe have time to focus on in the next few weeks. Questions about need for this given GitHub improvements. Discuss developer workflows, policies, and if edk2 can "fit" into the capabilities of GitHub rather than having to author, run, and maintain a custom tool. See recording for more details.
RFC tracking
@spbrogan<https://github.com/spbrogan> requested discussion/thought of changing the RFC process to be GitHub based. Would like to see more discussion, positive approval process, and a better permanent record for RFCs. See Rust-Lang for a great example. https://github.com/rust-lang/rfcs . Discussion to be moved to April Community meeting. @miki-intel-work<https://github.com/miki-intel-work> -- can you add it?
Ongoing Discussions
Add Clang to CI
@BobCF<https://github.com/BobCF> will review and update group on any progress to add ARM/ARCH64/RISCV support.
Toolchain support / maintainers
No Update. Do this week.
Discussed desire to "retire" older toolchains and remove them from tools_def.txt. Additionally increase automation PR gates, CI, and weekly coverage of supported toolchains.
https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template
First order of business is to find maintainers for all supported toolchains. If there is no maintainer, then a toolchain may be removed.
@spbrogan<https://github.com/spbrogan> - To send edk2 mailing list email to identify maintainers.
Edk2-basetools python project transition
No updates. Expect more next week. Issue 30 needs to be resolved as it is blocking validation. tianocore/edk2-basetools#30<tianocore/edk2-basetools#30>
Next Steps
Please think about your priorities and what you can add to the TianoCore community as well as what you would like to see changed.
—
Reply to this email directly, view it on GitHub<#2614 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIA55TEG7MINZFEJXJMALWDVCJMHVANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I mentioned in the meeting I’d constructed a git extension that benefited from being in the path and depended on the BaseTools build output. The tool currently depends on a build being present and then you can `git grep` based on that build. If you have `git-<cmmandName>` in your path git will treat it as an extended git command.
Here is an example of the command in action:
$ git pgrep OvmfX64 PcdDebugPrintErrorLevel -- '*.dsc'
OvmfX64 only: git grep --full-name --color=always --null PcdDebugPrintErrorLevel -- *.dsc
OvmfPkg/OvmfPkgX64.dsc:528: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
Vs.
$ git grep PcdDebugPrintErrorLevel -- '*.dsc'
ArmPkg/Drivers/ArmCrashDumpDxe/ArmCrashDumpDxe.dsc:25: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
ArmVirtPkg/ArmVirtKvmTool.dsc:116: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000000F
CryptoPkg/CryptoPkg.dsc:142: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000
EmbeddedPkg/EmbeddedPkg.dsc:159: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000
EmbeddedPkg/EmbeddedPkg.dsc:167: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000
EmulatorPkg/EmulatorPkg.dsc:219: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000040
IntelFsp2Pkg/IntelFsp2Pkg.dsc:76: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80080046
IntelFsp2Pkg/Tools/Tests/QemuFspPkg.dsc:163: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel | 0
IntelFsp2Pkg/Tools/Tests/QemuFspPkg.dsc:165: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel | 0x80000047
IntelFsp2WrapperPkg/IntelFsp2WrapperPkg.dsc:97: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80080046
MdePkg/MdePkg.dsc:31: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000
NetworkPkg/NetworkPkg.dsc:95: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000
OvmfPkg/AmdSev/AmdSevX64.dsc:474: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/Bhyve/BhyveX64.dsc:472: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/CloudHv/CloudHvX64.dsc:522: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/Microvm/MicrovmX64.dsc:494: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/OvmfPkgIa32.dsc:523: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/OvmfPkgIa32X64.dsc:527: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/OvmfPkgX64.dsc:528: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
OvmfPkg/OvmfXen.dsc:416: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
StandaloneMmPkg/StandaloneMmPkg.dsc:85: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x800000CF
UefiPayloadPkg/UefiPayloadPkg.dsc:420: gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x8000004F
So git-pgrep is just a git-grep scoped to the OvmfX64 build.
Some extra args get passed to git-grep to help with parsing but generally most git-grep arguments should be supported.
The reason I brought up this tool is it raises some generic interesting things to discuss:
1) For this tool, or any git extension, requires the tool to be in the path.
a) edksetup.sh is generally used to setup path. How do we add things like this than need to set the path relative to the BaseTools?
b) I cheated and just dropped this into BaseTools/BinWrappers/PosixLike/git-pgrep to test.
2) This tool depends on Build output files.
a) This tool does a glob on "**/deps.txt", "**/AutoGenTimeStamp", and "**/AutoGen” so there is a dependency on the name, location, and format of this intermediate output.
3) There are still missing files for things that wrap the build.
a) For example I launched the build via OvmfPkg/build.sh but that file in not though of as part of the build.
b) Should a build have a way to say I also depend on these external scripts, makefiles, etc.
Thanks,
Andrew Fish
PS If you don’t pass any git-grep args to git-pgrep it just dumps the file list for the build.
$ git pgrep OvmfX64
MdePkg/Library/BaseLib/X64/XGetBv.nasm
UefiCpuPkg/CpuMpPei/CpuMpPei.c
UefiCpuPkg/CpuDxe/CpuDxe.h
MdePkg/Library/UefiRuntimeServicesTableLib/UefiRuntimeServicesTableLib.c
Build/OvmfX64/DEBUG_XCODE5/X64/MdeModulePkg/Library/DeviceManagerUiLib/DeviceManagerUiLib/DEBUG/AutoGen.c
...
PPS In case the attachment gets removed….
/Volumes/Case/edk2(master)>cat BaseTools/BinWrappers/PosixLike/git-pgrep
#!/usr/bin/env python3
# ## @file
#
# wrapper for git grep that filters by build target.
# currently DEBUG, RELEASE, and NOOPT are all parsed.
#
# Example: ./git-pgrep OvmfX64 PcdDebugPrintErrorLevel -- "*.dsc"
# If git-pgrep is in the $PATH you can:
# git pgrep OvmfX64 PcdDebugPrintErrorLevel -- "*.dsc"
#
# Copyright (c) 2020 - 2022 Apple Inc. All rights reserved.
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
import sys
import argparse
import os
import glob
import subprocess
import tempfile
# Build - EDKII style
# ./ if target is the actual build dir (everything)
# if you have a custom path add it to the list
build_dirs = ["Build", "./"]
def find_build_dir(target, workspace):
for build_dir_names in build_dirs:
build_path = workspace + build_dir_names + os.path.sep + target
if os.path.isdir(build_path):
build_path = os.path.abspath(build_path)
workspace = build_path.split(build_dir_names)[0]
return (build_path, workspace.rstrip(build_dirs[0]))
return (None, None)
def generate_file_dependency_set(target, workspace=''):
(build_path, workspace) = find_build_dir(target, workspace)
if build_path is None:
print("Taraget build directory not found:", target, file=sys.stderr)
sys.exit(1)
lines = []
glob_path = build_path + os.path.sep + "**/deps.txt"
for filename in glob.glob(glob_path, recursive=True):
with open(filename, 'r') as file:
# Skip build results files as they don't git grep
lines += [line.rstrip('\n') for line in file]
glob_path = build_path + os.path.sep + "**/AutoGenTimeStamp"
for filename in glob.glob(glob_path, recursive=True):
with open(filename, 'r') as file:
# Skip build results files as they don't git grep
lines += [line.rstrip('\n') for line in file]
# This picks up build configuration files. DEBUG_XCODE5/AutoGen
glob_path = build_path + os.path.sep + "**/AutoGen"
for filename in glob.glob(glob_path, recursive=True):
with open(filename, 'r') as file:
lines += [line.rstrip('\n') for line in file]
# like Python 3.9 l.removeprefix(workspace), but the hard way
lines = [line[len(workspace):] if line.startswith(
workspace) else line for line in lines]
return set(lines)
def get_workspace_default():
if 'WORKSPACE' in os.environ:
return os.environ['WORKSPACE']
else:
return ''
def main():
usage_str = """git grep using only the files in a specific build target.
Example: ./git-pgrep OvmfX64 0xAFAF -- *.h. If no git grep arguments are passed
then the list of files used by the build are returned. If git-pgrep is in the
$PATH then this script will get added as a git command: git pgrep OvmfX64
0xAFAF. If you pass in the name of the build directory all build products will
be searched.
"""
target_help = """Build target to git grep. OvmfX64 followed by git grep
arguments"""
parser = argparse.ArgumentParser(description=usage_str)
parser.add_argument('target', type=str, default='', help=target_help)
parser.add_argument('--workspace', default=get_workspace_default(),
type=str, help='Location of the workspace')
parser.add_argument('--break', action='https://github.com/store_true', dest="line_break",
help='git grep --break')
parser.add_argument('--heading', action='https://github.com/store_true', dest="heading",
help='git grep --heading')
args, unknown_args = parser.parse_known_args()
if args.workspace:
if not os.path.isdir(args.workspace):
print("--workspace not found:", args.workspace, file=sys.stderr)
sys.exit(1)
if not args.workspace.endswith(os.sep):
args.workspace = args.workspace + os.sep
if not unknown_args:
# dump the dependency file list
fileset = generate_file_dependency_set(args.target, args.workspace)
print("\n".join(fileset))
sys.exit(0)
# does git-grep print out with color? Ary we talking to a tty?
# sys.stdout.isatty() check makes `git pgrep | cat` do the right thing.
grep_color = subprocess.run(
['git', 'config', '--get-colorbool', 'color.grep',
'true' if sys.stdout.isatty() else 'false'],
check=True,
text=True,
capture_output=True).stdout.strip() == 'true'
# Temp file so we don't overflow the pipe
with tempfile.TemporaryFile() as tfile:
# we use --null to help parsing, but not for file lists.
no_null = ['-l', '--files-with-matches', '--name-only', '-L',
'--files-without-match']
# Start up early so we overlap with edk2 Build output parsing
# --full-name so paths are always workspace relative
# --null to make parsing easier
# --color = always is required to force color down the Popen pipe
# the default is auto so Popen loses the color...
git_cmd = ['git', 'grep', '--full-name',
f"--color={'always' if grep_color else 'never'}"]
if args.heading:
git_cmd.append('--heading')
if set(no_null).isdisjoint(unknown_args):
# Use \0 as the delimiter for pathnames.
# if the output is just filenames we want to skip this.
git_cmd.append('--null')
git_cmd += unknown_args
process = subprocess.Popen(git_cmd, stdout=tfile)
print(args.target, "only:", ' '.join(git_cmd))
if args.line_break:
print()
# parse the edk2 build files in parallel to git grep
fileset = generate_file_dependency_set(args.target, args.workspace)
if len(fileset) == 0:
print("No build output information found!", file=sys.stderr)
sys.exit(1)
# this waits for the git grep to complete.
process.wait(timeout=300)
tfile.seek(0)
# Found some extended ASCII that break strict utf-8
grep_data = tfile.read().decode(errors='replace')
if process.returncode != 0:
# return git grep error and failure code
print(grep_data, file=sys.stderr)
sys.exit(process.returncode)
last_match = ''
heading = False
for ggline in grep_data.split('\n'):
try:
# forced --null puts NULL between items
gnull = ggline.split('\0')
if heading:
# ggline is linenum<null>code
heading = gnull[0].isnumeric()
if heading:
print(ggline)
continue
elif args.line_break:
print()
if gnull[0] in fileset:
try:
# ggline is filename<null>linenum<nul>code
print(f'{gnull[0]}:{gnull[1]}: {gnull[2]}')
except IndexError:
print(ggline)
if args.heading:
heading = True
elif args.line_break and last_match != gnull[0]:
print()
last_match = gnull[0]
elif ggline == '--' and last_match != '--':
# Handle multiline output like -B 3 etc.
print(ggline)
last_match = ggline
except Exception:
heading = False
if __name__ == "__main__":
main()
… On Mar 28, 2022, at 6:29 PM, Sean Brogan ***@***.***> wrote:
Meeting Notes 3/28/2022
Basic notes. Please add comments to the notes if important.
Recording available here <Todo: add link>
Transcript available here <Todo: add link>
Attendees
@mdkinney <https://github.com/mdkinney>
@BobCF <https://github.com/BobCF>
@spbrogan <https://github.com/spbrogan>
@ashedesimone <https://github.com/ashedesimone>
@nate-desimone <https://github.com/nate-desimone>
@kevindavisinsyde <https://github.com/kevindavisinsyde>
@miki-intel-work <https://github.com/miki-intel-work>
@hlhsiung <https://github.com/hlhsiung>
@ajfish <https://github.com/ajfish>
@YuweiChen1110 <https://github.com/YuweiChen1110>
New Discussions / Opens
Clarification on VS2019 and VS2022 and Edk2 CI
Default CI image in Azure DevOps for Windows is changing to only include VS2022. Edk2 CI will need to update soon so that merging isn't blocked once the change happens.
Scripts
Where should edk2 specific dev scripts / helpers be checked in? Since edk2-basetools project is trying to remove the non-fw python code from edk2 (and improve quality and portability) where should other scripts ***@***.*** <https://github.com/ajfish> has a "git grep" like tool that filters based on a built project that would be useful to others. Additionally there are debug scripts and scripts that have been developed. Ideas:
Create a template repo that makes it easy to develop, validate, and publish python tools for edk2 consumption.
Add to existing repo
Agreed to follow up once basetools was successfully moved.
Build reproducibility
@mdkinney <https://github.com/mdkinney> - curious how important this is and if it is something the group wanted to focus on. Some work done in the uncrustify timeframe but more changes would be needed to continue this effort. Asserts filename macro, file paths, PDB path, date/time are all common problems. @spbrogan <https://github.com/spbrogan> - Still need a business driver before this is priority. Maybe edk2 releasing prebuilt binaries would be motivation. Very little discussion in group.
Google summer of code interns
@nate-desimone <https://github.com/nate-desimone> reported that it looks to be on track for roughly 5 interns. Still would be great to have more mentors. Reach out to nate if interested.
Github PR to mailing list edk2 reflector status
@mdkinney <https://github.com/mdkinney> said maybe have time to focus on in the next few weeks. Questions about need for this given GitHub improvements. Discuss developer workflows, policies, and if edk2 can "fit" into the capabilities of GitHub rather than having to author, run, and maintain a custom tool. See recording for more details.
RFC tracking
@spbrogan <https://github.com/spbrogan> requested discussion/thought of changing the RFC process to be GitHub based. Would like to see more discussion, positive approval process, and a better permanent record for RFCs. See Rust-Lang for a great example. https://github.com/rust-lang/rfcs <https://github.com/rust-lang/rfcs> . Discussion to be moved to April Community meeting. @miki-intel-work <https://github.com/miki-intel-work> -- can you add it?
Ongoing Discussions
Add Clang to CI
@BobCF <https://github.com/BobCF> will review and update group on any progress to add ARM/ARCH64/RISCV support.
Toolchain support / maintainers
No Update. Do this week.
Discussed desire to "retire" older toolchains and remove them from tools_def.txt. Additionally increase automation PR gates, CI, and weekly coverage of supported toolchains.
https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template <https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template>
First order of business is to find maintainers for all supported toolchains. If there is no maintainer, then a toolchain may be removed.
@spbrogan <https://github.com/spbrogan> - To send edk2 mailing list email to identify maintainers.
Edk2-basetools python project transition
No updates. Expect more next week. Issue 30 needs to be resolved as it is blocking validation. tianocore/edk2-basetools#30 <tianocore/edk2-basetools#30>
Next Steps
Please think about your priorities and what you can add to the TianoCore community as well as what you would like to see changed.
—
Reply to this email directly, view it on GitHub <#2614 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4ZCO2ZTOV4YHIMIWIXEX3VCJMHTANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.
|
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 4/18/2022Basic notes. Please add comments to the notes if important. Attendees New Discussions / Opensedk2-platforms, ci, and requirements for edk2near the 44 min mark Not really new topic but for this forum it is the first time it has been brought up. @makubacki raised difficulty of getting a edk2 change in that adds new lib and trying to get all changes into edk2-platforms. Both the lack of maintainer engagement and the lack of consistency on build process/tooling makes it impossible for an edk2 dev to test and validate on all the edk2-platforms platforms. Request was to clean up maintain list and active project. Review how to make this process better. Ongoing DiscussionsContainerized builds for CI and Devstart of meeting Some discussion about dockerfile location, container registry, and the way to support CI and dev scenarios Toolchain supportnear the 20-minute mark Concerns about toolchains listed but not actively tested/maintained and don't actually work. |
Beta Was this translation helpful? Give feedback.
-
|
Sean,
I tried to join the meeting but Teams never let me in? I wonder if it is related to me not having a Teams account?
Thanks,
Andrew Fish
… On Apr 19, 2022, at 10:52 AM, Sean Brogan ***@***.***> wrote:
Meeting Notes 4/18/2022
Basic notes. Please add comments to the notes if important.
Recording available at https://edk2.groups.io/g/devel/files/EDK2%20Community%20Meeting%20on%20Build,%20Tools,%20CI <https://edk2.groups.io/g/devel/files/EDK2%20Community%20Meeting%20on%20Build,%20Tools,%20CI>
Attendees
@mdkinney <https://github.com/mdkinney>
@BobCF <https://github.com/BobCF>
@spbrogan <https://github.com/spbrogan>
@kevindavisinsyde <https://github.com/kevindavisinsyde>
@hlhsiung <https://github.com/hlhsiung>
@YuweiChen1110 <https://github.com/YuweiChen1110>
@BCRAN <https://github.com/bcran>
@shijunjing <https://github.com/shijunjing>
@makubacki <https://github.com/makubacki>
New Discussions / Opens
edk2-platforms, ci, and requirements for edk2
near the 44 min mark
Not really new topic but for this forum it is the first time it has been brought up. @makubacki <https://github.com/makubacki> raised difficulty of getting a edk2 change in that adds new lib and trying to get all changes into edk2-platforms. Both the lack of maintainer engagement and the lack of consistency on build process/tooling makes it impossible for an edk2 dev to test and validate on all the edk2-platforms platforms. Request was to clean up maintain list and active project. Review how to make this process better.
@makubacki <https://github.com/makubacki> - to send email out start discussion. Groups IO Link here <https://edk2.groups.io/g/devel/topic/edk2_platforms_important/90563726?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,90563726,previd%3D1650386607569402288,nextid%3D1650353724488088589&previd=1650386607569402288&nextid=1650353724488088589>
Ongoing Discussions
Containerized builds for CI and Dev
start of meeting
Some discussion about dockerfile location, container registry, and the way to support CI and dev scenarios
Additional review of docker/container best practices
@hlhsiung <https://github.com/hlhsiung> - to follow up within Intel OTC for any best practices
Also check and provide some comments here: #2732 <#2732>
Mostly focused on GCC and Linux but should also review CLANG and Windows VS builds.
Toolchain support
near the 20-minute mark
Concerns about toolchains listed but not actively tested/maintained and don't actually work.
Some discussion about ASL compilers.
@spbrogan <https://github.com/spbrogan> - actually send email trying to find "maintainers" for each toolchain
—
Reply to this email directly, view it on GitHub <#2614 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD4ZCO4TGVGEWLO66JDH7OLVF3XFTANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, I won't be joining the meeting again (it is in the middle of my night.) I summarized briefly what I did in the last while here. It would be interesting to hear your thoughts on managing Dockerfiles, building images, which image registry to use, where to place the Dockerfiles, ... It would be nice to be able to switch to newer toolchain versions easily in the CI and to become less dependent on external downloads during CI runs. -- Oliver |
Beta Was this translation helpful? Give feedback.
-
|
For reference, you can see our changes I have been experimenting with to the Project MU pipelines here: microsoft/mu_basecore#108 For the tools, it looks like we are mostly aligned. Looking through your dockerfile there isn't too much delta. However, from what i'm seeing the GCC compilers take up significantly more space in my image then you have quoted above. for the ARM64 GCC compiler for example, I see it requiring almost 500 mb for some reason. My image it totally around 1.6 Gb, once I remove some of the tools specific to some Project MU specific pipeline steps. I'm currently using ubuntu 21.04 which I downgraded to from 22.04 because of a package dependency issue I was running into. I don't have a strong preference on Linux distribution though. I'm going to be on vacation starting next week but will follow up with this when I'm back and if we don't have the TianoCore repo at that point we can start working from your repository. |
Beta Was this translation helpful? Give feedback.
-
|
I opened a PR on the containers repo with my Fedora 36 Dockerfile Do we want/need a versioning scheme to tag the images? |
Beta Was this translation helpful? Give feedback.
-
|
@osteffenrh, In the model you describe using the sha, do you mean that the pipeline files would reference a specific sha version of the container rather than using the 'latest' tag? The only potential problem with sha approach is that if there are changes in the installed packages then there could be changes to the image that would not be reflected by the sha. Now, this probably wont matter since your current job file will only run on a new commit, but I wonder if we should consider a timestamp based tag so we could allow ourselves to update the image manually to pull in installed package updates without overwriting the existing container. I will look over the PR and add further comments there. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @cfernald , What I mean was (sorry, I did not explain very well): Indeed, changes inside the packages (for example bug fixed within a Fedora major release) will not be reflected by this tagging scheme, since the Dockerfile (= list of packages to install) does not change. I am not sold on this approach. Just proposed it because it was easy to implement. I would just avoid that the CI only uses the "latest" tag, because the content behind that might change over time and older branches/PRs then might not be able to run the CI. |
Beta Was this translation helpful? Give feedback.
-
|
Agreed, using consistent tags would be best to avoid unexpected wide scale regressions, for example caused by a python regression which we recently observed in our pipelines. We can experiment some, but I think either would likely be fine. We can continue this discussion on the PR though. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Sean
Is there any particular naming convention?
edk2-containers????
…--
Miki Demeter (she/her/Miki)
Security Researcher / FW Developer
FST
Intel Corporation
Co-Chair, Network of Intel African-Ancestry(NIA) - Oregon
NIA-Oregon<https://soco.intel.com/groups/network-of-intel-african-americans-oregon>
Portland Women in Tech Best Speaker
***@***.******@***.***>
503.712.8030 (office)
971.248.0123 (cell)
From: Sean Brogan ***@***.***>
Date: Monday, April 25, 2022 at 4:54 PM
To: tianocore/edk2 ***@***.***>
Cc: Demeter, Miki ***@***.***>, Mention ***@***.***>
Subject: Re: [tianocore/edk2] Join us for the Tools, CI, Code base construction meeting series (Discussion #2614)
@osteffenrh<https://github.com/osteffenrh> - @miki-intel-work<https://github.com/miki-intel-work> is going to make a repo for docker files and enable (if required) the github container registry for tianocore. In that repo i would suggest we use PRs so that we can have some discussions about the content going in as well as a defined release process so that any container version can be referenced/looked up. @cfernald<https://github.com/cfernald> has also been working on enabling the Tianocore and Project Mu CI for GCC5 to use containers so i would expect he would have some additional feedback.
—
Reply to this email directly, view it on GitHub<#2614 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUDSOAG5QO7XG6IRDU3VBKDVG4WCJANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Sure. Not sure if the edk2 part actually helps given that the organization is already tianocore so i might just suggest "containers" |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 4/25/2022Basic notes. Please add comments to the notes if important. Attendees@BobCF New Discussions / OpensWebinar for DCI and x86 debug by Asset Intertech.near the 10-minute mark Harry shared info about upcoming webinar that might be of interest. Ongoing DiscussionsEdk2-Platforms and CIMinor update. Last week's email generated only minimal response. Pedro and Ard both replied and cannot meet at this time. Michael will use the mailing list and prepare a for a future discussion. Other edk2-platform maintainers please reply. Containerized builds for CI and DevMentioned Oliver's discussion topic and thread. Will work with Miki to get github infrastructure enabled. Tools Def cleanup and maintainersBriefly discussed role of maintainer of a tool chain. @spbrogan will finish and finally send email out to start the process of identifying the supported tool chain tags and collecting the names of those that are willing to support them. |
Beta Was this translation helpful? Give feedback.
-
|
That was quick.. I do not have sufficient permissions ☹
…--
Miki Demeter (she/her/Miki)
Security Researcher / FW Developer
FST
Intel Corporation
Co-Chair, Network of Intel African-Ancestry(NIA) - Oregon
NIA-Oregon<https://soco.intel.com/groups/network-of-intel-african-americans-oregon>
Portland Women in Tech Best Speaker
***@***.******@***.***>
503.712.8030 (office)
971.248.0123 (cell)
From: Sean Brogan ***@***.***>
Date: Monday, April 25, 2022 at 5:13 PM
To: tianocore/edk2 ***@***.***>
Cc: Demeter, Miki ***@***.***>, Mention ***@***.***>
Subject: Re: [tianocore/edk2] Join us for the Tools, CI, Code base construction meeting series (Discussion #2614)
Sure. Not sure if the edk2 part actually helps given that the organization is already tianocore so i might just suggest "containers"
—
Reply to this email directly, view it on GitHub<#2614 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUDSOABH3D4OIWWHQGHRQWTVG4YJLANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 5/2/2022Basic notes. Please add comments to the notes if important. Attendees@spbrogan New Discussions / Opensbinary analysis / dependency analysis Testsnear the 13 minutes https://www.synopsys.com/software-integrity/security-testing/software-composition-analysis.html Ongoing DiscussionsContainerized builds for CI and Dev@miki-intel-work still blocked from creating "containers" repo. Will check with @lgao4 Tools Def cleanup and maintainersMail sent. https://edk2.groups.io/g/rfc/message/750 |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 5/9/2022Basic notes. Please add comments to the notes if important. NO MEETING on 5/16 Attendees@spbrogan New Discussions / Opensnone Ongoing DiscussionsContainerized builds for CI and DevMore discussions on containers and getting the new https://github.com/tianocore/containers repo created. Thanks @miki-intel-work. @osteffenrh and @cfernald please see the above repo. Tools Def cleanup and maintainershttps://edk2.groups.io/g/rfc/message/750 So far @ajfish has signed up for xcode5 tag on mailing list. Worked needed on report format. |
Beta Was this translation helpful? Give feedback.
-
|
Build PASS: use openssl-OpenSSL_1_1_1g tag src , ubuntu 21 OS, target :OvmfPkg/OvmfPkgX64.dsc ; X64; GCC5 . |
Beta Was this translation helpful? Give feedback.
-
|
If the MS Teams meeting link isn't changing very often, could we update the groups.io calendar entry to add it directly, instead of requiring people to click through to his page? |
Beta Was this translation helpful? Give feedback.
-
|
@miki-intel-work or @mdkinney, i think you own the groups.io calendar entry. |
Beta Was this translation helpful? Give feedback.
-
This week 5/29/2023 is a US holiday. Meeting will be skipped for the week. Will catch up next week on 6/5/2023 |
Beta Was this translation helpful? Give feedback.
-
|
Added teams link to the location for the calendar invite.
…-miki
--
Miki Demeter (she/her/Miki)
Security Researcher / FW Developer
Tianocore Community Manager
FST
Intel Corporation
Portland Women in Tech Best Speaker
***@***.******@***.***>
503.712.8030 (office)
971.248.0123 (cell)
From: Sean Brogan ***@***.***>
Date: Monday, May 29, 2023 at 12:45 PM
To: tianocore/edk2 ***@***.***>
Cc: Demeter, Miki ***@***.***>, Mention ***@***.***>
Subject: Re: [tianocore/edk2] Join us for the Tools, CI, Code base construction meeting series (Discussion #2614)
@miki-intel-work<https://github.com/miki-intel-work> or @mdkinney<https://github.com/mdkinney>, i think you own the groups.io calendar entry.
—
Reply to this email directly, view it on GitHub<#2614 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUDSOAAQLUGX3OYLDGYT56LXIT4E7ANCNFSM5QRABOMA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Another very light week with no recording. |
Beta Was this translation helpful? Give feedback.
-
|
Next week is cancelled given the US holiday (July 3rd meeting). |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 7/10/2023We had light attendance and a short meeting today. Due to lack of substantial topics, the meeting was not recorded. It was noted that a GitHub project was created in the TianoCore repo to track enabling Pull Requests in edk2. Next steps are that @mdkinney will update some assigned items and then we'll sync back up on how to plan open work items. Attendees@mdkinney |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 7/17/2023We had light attendance and another short meeting today. A recording was made while discussing improvements to library class validation during build. There were two cases that we noted should have raised awareness:
For now, we propose that these show warning messages during build. In particular because during platform integration, many components are provided by various vendors so a platform integrator will need to time to observe the warning messages and work with vendors to get fixes. Christine is to ensure an issue is created in edk2-basetools for both of these cases. Joey Vagedes has already worked on (1) and posted a diff of the change in an issue created here https://github.com/tianocore/edk2-basetools/issues/104. We also discussed some differences between components and libraries implementing a library class. Attendees@mdkinney |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 7/24/2023Mike Kinney made a change over the weekend with the proper way to handle conflicts between edk2 and Windows includes. It can be used for future reference. There was discussion on the edk2 mailing list about commit messages not being correct in some cases.
It was noted that this is a good resource for reproducible builds - https://reproducible-builds.org/docs/. How to configure things like gcc and VS compilers. How to reduce deltas between builds like these debug records. How to eliminate variation in the binaries and post-processing tools that can help. |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 8/7/2023We discussed the following topics:
Attendees@mdkinney |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 8/14/2023We discussed the following topics:
Attendees@mdkinney |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 8/21/2023We discussed the following topics:
Attendees@mdkinney |
Beta Was this translation helpful? Give feedback.
-
Meeting Notes 11/20/2023We discussed the following topics related to
For reference, a discussion previously started to informally collect feedback on However, if you have actionable feedback you would like to track, please file feedback directly in the edk2-pytool-extensions repo so it can be tracked to implementation and discovered by others. Attendees |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for bringing #4369 to my attention. I added some thoughts: #4369 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
HI
…________________________________
From: Oliver Steffen ***@***.***>
Sent: 21 November 2023 13:41
To: tianocore/edk2 ***@***.***>
Cc: Jose Yoyo | Providence Software ZA ***@***.***>; Manual ***@***.***>
Subject: Re: [tianocore/edk2] Join us for the Tools, CI, Code base construction meeting series (Discussion #2614)
Thank you for bringing #4369<#4369> to my attention.
I added some thoughts: #4369 (comment)<#4369 (comment)>
—
Reply to this email directly, view it on GitHub<#2614 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5NEHWTHQ2YMWGVTSSAUXI3YFSHOBAVCNFSM5QRABOMKU5DIOJSWCZC7NNSXTOKENFZWG5LTONUW63SDN5WW2ZLOOQ5TONRSHE3DOMA>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Providence Software Solutions (Pty) Ltd. If you are not the intended recipient of this email, you must not take any action based upon its contents nor copy or show it to anyone. Please contact the sender if you believe that you have received this email in error. Due to the electronic nature of e-mail, there is a risk that the information contained in this message has been modified. Consequently the sender can accept no responsibility or liability as to the completeness or accuracy of the information.
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Providence Software Solutions (Pty) Ltd. If you are not the intended recipient of this email, you must not take any action based upon its contents nor copy or show it to anyone. Please contact the sender if you believe that you have received this email in error. Due to the electronic nature of e-mail, there is a risk that the information contained in this message has been modified. Consequently the sender can accept no responsibility or liability as to the completeness or accuracy of the information.
|
Beta Was this translation helpful? Give feedback.
-
|
I am closing down this super thread. We are moving the tools and CI meeting to be weekly on demand. In order to manage that I will create a new discussion topic each week titled: "Tools and CI Meeting <date>". This will host the call in info and the latest set of topics. If anyone has a topic for the meeting they can add it there and the meeting will be held. If no comments then meeting will be cancelled for the week. I'll try to add a comment ~24 hours before to indicate the status of the meeting. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
TianoCore community,
Microsoft and Intel will be hosting a series of open meetings to discuss build, CI, tools, and other related topics. If you are interested, have ideas/opinions please join us. These meetings will be Monday 4:30pm Pacific Time on Microsoft Teams. See your time zone here
Anyone is welcome to join.
Microsoft Teams meeting
Join on your computer or mobile app
Click here to join the meeting
Or call in (audio only)
+1 425-616-0754,,330776518# United States, Seattle
Phone Conference ID: 330 776 518#
Find a local number | Reset PIN
Learn More | Meeting options
Beta Was this translation helpful? Give feedback.
All reactions