Rebase to v2.51.1 #5877
Rebase to v2.51.1 #5877
Conversation
dscho
force-pushed
the
rebase-to-v2.51.1
branch
from
83f58f5 to
4fdf899
Compare
Range-diff relative to main
|
On LLP64 systems, such as Windows, the size of `long`, `int`, etc. is only 32 bits (for backward compatibility). Git's use of `unsigned long` for file memory sizes in many places, rather than size_t, limits the handling of large files on LLP64 systems (commonly given as `>4GB`). Provide a minimum test for handling a >4GB file. The `hash-object` command, with the `--literally` and without `-w` option avoids writing the object, either loose or packed. This avoids the code paths hitting the `bigFileThreshold` config test code, the zlib code, and the pack code. Subsequent patches will walk the test's call chain, converting types to `size_t` (which is larger in LLP64 data models) where appropriate. Signed-off-by: Philip Oakley <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Continue walking the code path for the >4GB `hash-object --literally` test. The `hash_object_file_literally()` function internally uses both `hash_object_file()` and `write_object_file_prepare()`. Both function signatures use `unsigned long` rather than `size_t` for the mem buffer sizes. Use `size_t` instead, for LLP64 compatibility. While at it, convert those function's object's header buffer length to `size_t` for consistency. The value is already upcast to `uintmax_t` for print format compatibility. Note: The hash-object test still does not pass. A subsequent commit continues to walk the call tree's lower level hash functions to identify further fixes. Signed-off-by: Philip Oakley <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Continue walking the code path for the >4GB `hash-object --literally`
test to the hash algorithm step for LLP64 systems.
This patch lets the SHA1DC code use `size_t`, making it compatible with
LLP64 data models (as used e.g. by Windows).
The interested reader of this patch will note that we adjust the
signature of the `git_SHA1DCUpdate()` function without updating _any_
call site. This certainly puzzled at least one reviewer already, so here
is an explanation:
This function is never called directly, but always via the macro
`platform_SHA1_Update`, which is usually called via the macro
`git_SHA1_Update`. However, we never call `git_SHA1_Update()` directly
in `struct git_hash_algo`. Instead, we call `git_hash_sha1_update()`,
which is defined thusly:
static void git_hash_sha1_update(git_hash_ctx *ctx,
const void *data, size_t len)
{
git_SHA1_Update(&ctx->sha1, data, len);
}
i.e. it contains an implicit downcast from `size_t` to `unsigned long`
(before this here patch). With this patch, there is no downcast anymore.
With this patch, finally, the t1007-hash-object.sh "files over 4GB hash
literally" test case is fixed.
Signed-off-by: Philip Oakley <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Just like the `hash-object --literally` code path, the `--stdin` code path also needs to use `size_t` instead of `unsigned long` to represent memory sizes, otherwise it would cause problems on platforms using the LLP64 data model (such as Windows). To limit the scope of the test case, the object is explicitly not written to the object store, nor are any filters applied. The `big` file from the previous test case is reused to save setup time; To avoid relying on that side effect, it is generated if it does not exist (e.g. when running via `sh t1007-*.sh --long --run=1,41`). Signed-off-by: Philip Oakley <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
To complement the `--stdin` and `--literally` test cases that verify that we can hash files larger than 4GB on 64-bit platforms using the LLP64 data model, here is a test case that exercises `hash-object` _without_ any options. Just as before, we use the `big` file from the previous test case if it exists to save on setup time, otherwise generate it. Signed-off-by: Philip Oakley <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
To verify that the `clean` side of the `clean`/`smudge` filter code is correct with regards to LLP64 (read: to ensure that `size_t` is used instead of `unsigned long`), here is a test case using a trivial filter, specifically _not_ writing anything to the object store to limit the scope of the test case. As in previous commits, the `big` file from previous test cases is reused if available, to save setup time, otherwise re-generated. Signed-off-by: Philip Oakley <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
In the case of Git for Windows (say, in a Git Bash window) running in a Windows Subsystem for Linux (WSL) directory, the GetNamedSecurityInfoW() call in is_path_owned_By_current_side() returns an error code other than ERROR_SUCCESS. This is consistent behavior across this boundary. In these cases, the owner would always be different because the WSL owner is a different entity than the Windows user. The change here is to suppress the error message that looks like this: error: failed to get owner for '//wsl.localhost/...' (1) Before this change, this warning happens for every Git command, regardless of whether the directory is marked with safe.directory. Signed-off-by: Derrick Stolee <[email protected]>
For Windows builds >= 15063 set $env:TERM to "xterm-256color" instead of "cygwin" because they have a more capable console system that supports this. Also set $env:COLORTERM="truecolor" if unset. $env:TERM is initialized so that ANSI colors in color.c work, see 29a3963 (Win32: patch Windows environment on startup, 2012-01-15). See #3629 regarding problems caused by always setting $env:TERM="cygwin". This is the same heuristic used by the Cygwin runtime. Signed-off-by: Rafael Kitover <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
NtQueryObject under Wine can return a success but fill out no name. In those situations, Wine will set Buffer to NULL, and set result to the sizeof(OBJECT_NAME_INFORMATION). Running a command such as echo "$(git.exe --version 2>/dev/null)" will crash due to a NULL pointer dereference when the code attempts to null terminate the buffer, although, weirdly, removing the subshell or redirecting stdout to a file will not trigger the crash. Code has been added to also check Buffer and Length to ensure the check is as robust as possible due to the current behavior being fragile at best, and could potentially change in the future This code is based on the behavior of NtQueryObject under wine and reactos. Signed-off-by: Christopher Degawa <[email protected]>
Atomic append on windows is only supported on local disk files, and it may cause errors in other situations, e.g. network file system. If that is the case, this config option should be used to turn atomic append off. Co-Authored-By: Johannes Schindelin <[email protected]> Signed-off-by: 孙卓识 <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
From the documentation of said setting: This boolean will enable fsync() when writing object files. This is a total waste of time and effort on a filesystem that orders data writes properly, but can be useful for filesystems that do not use journalling (traditional UNIX filesystems) or that only journal metadata and not file contents (OS X’s HFS+, or Linux ext3 with "data=writeback"). The most common file system on Windows (NTFS) does not guarantee that order, therefore a sudden loss of power (or any other event causing an unclean shutdown) would cause corrupt files (i.e. files filled with NULs). Therefore we need to change the default. Note that the documentation makes it sound as if this causes really bad performance. In reality, writing loose objects is something that is done only rarely, and only a handful of files at a time. Signed-off-by: Johannes Schindelin <[email protected]>
Whith Windows 2000, Microsoft introduced a flag to the PE header to mark executables as "terminal server aware". Windows terminal servers provide a redirected Windows directory and redirected registry hives when launching legacy applications without this flag set. Since we do not use any INI files in the Windows directory and don't write to the registry, we don't need this additional preparation. Telling the OS that we don't need this should provide slightly improved startup times in terminal server environments. When building for supported Windows Versions with MSVC the /TSAWARE linker flag is automatically set, but MinGW requires us to set the --tsaware flag manually. This partially addresses #3935. Signed-off-by: Matthias Aßhauer <[email protected]>
Add FileVersion, which is a required field As not all required fields were present, none were being included Fixes #4090 Signed-off-by: Kiel Hurley <[email protected]>
In f9b7573 (repository: free fields before overwriting them, 2017-09-05), Git was taught to release memory before overwriting it, but 357a03e (repository.c: move env-related setup code back to environment.c, 2018-03-03) changed the code so that it would not _always_ be overwritten. As a consequence, the `commondir` attribute would point to already-free()d memory. This seems not to cause problems in core Git, but there are add-on patches in Git for Windows where the `commondir` attribute is subsequently used and causing invalid memory accesses e.g. in setups containing old-style submodules (i.e. the ones with a `.git` directory within theirs worktrees) that have `commondir` configured. This fixes #4083. Signed-off-by: Andrey Zabavnikov <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
This compile-time option allows to ask Git to load libcurl dynamically at runtime. Together with a follow-up patch that optionally overrides the file name depending on the `http.sslBackend` setting, this kicks open the door for installing multiple libcurl flavors side by side, and load the one corresponding to the (runtime-)configured SSL/TLS backend. Signed-off-by: Johannes Schindelin <[email protected]>
This implements the Windows-specific support code, because everything is slightly different on Windows, even loading shared libraries. Note: I specifically do _not_ use the code from `compat/win32/lazyload.h` here because that code is optimized for loading individual functions from various system DLLs, while we specifically want to load _many_ functions from _one_ DLL here, and distinctly not a system DLL (we expect libcurl to be located outside `C:\Windows\system32`, something `INIT_PROC_ADDR` refuses to work with). Also, the `curl_easy_getinfo()`/`curl_easy_setopt()` functions are declared as vararg functions, which `lazyload.h` cannot handle. Finally, we are about to optionally override the exact file name that is to be loaded, which is a goal contrary to `lazyload.h`'s design. Signed-off-by: Johannes Schindelin <[email protected]>
The previous commits introduced a compile-time option to load libcurl lazily, but it uses the hard-coded name "libcurl-4.dll" (or equivalent on platforms other than Windows). To allow for installing multiple libcurl flavors side by side, where each supports one specific SSL/TLS backend, let's first look whether `libcurl-<backend>-4.dll` exists, and only use `libcurl-4.dll` as a fall back. That will allow us to ship with a libcurl by default that only supports the Secure Channel backend for the `https://` protocol. This libcurl won't suffer from any dependency problem when upgrading OpenSSL to a new major version (which will change the DLL name, and hence break every program and library that depends on it). This is crucial because Git for Windows relies on libcurl to keep working when building and deploying a new OpenSSL package because that library is used by `git fetch` and `git clone`. Note that this feature is by no means specific to Windows. On Ubuntu, for example, a `git` built using `LAZY_LOAD_LIBCURL` will use `libcurl.so.4` for `http.sslbackend=openssl` and `libcurl-gnutls.so.4` for `http.sslbackend=gnutls`. Signed-off-by: Johannes Schindelin <[email protected]>
It is merely a historical wart that, say, `git-commit` exists in the `libexec/git-core/` directory, a tribute to the original idea to let Git be essentially a bunch of Unix shell scripts revolving around very few "plumbing" (AKA low-level) commands. Git has evolved a lot from there. These days, most of Git's functionality is contained within the `git` executable, in the form of "built-in" commands. To accommodate for scripts that use the "dashed" form of Git commands, even today, Git provides hard-links that make the `git` executable available as, say, `git-commit`, just in case that an old script has not been updated to invoke `git commit`. Those hard-links do not come cheap: they take about half a minute for every build of Git on Windows, they are mistaken for taking up huge amounts of space by some Windows Explorer versions that do not understand hard-links, and therefore many a "bug" report had to be addressed. The "dashed form" has been officially deprecated in Git version 1.5.4, which was released on February 2nd, 2008, i.e. a very long time ago. This deprecation was never finalized by skipping these hard-links, but we can start the process now, in Git for Windows. Signed-off-by: Johannes Schindelin <[email protected]>
This will help with Git for Windows' maintenance going forward: It allows Git for Windows to switch its primary libcurl to a variant without the OpenSSL backend, while still loading an alternate when setting `http.sslBackend = openssl`. This is necessary to avoid maintenance headaches with upgrading OpenSSL: its major version name is encoded in the shared library's file name and hence major version updates (temporarily) break libraries that are linked against the OpenSSL library. Signed-off-by: Johannes Schindelin <[email protected]>
In Git for Windows v2.39.0, we fixed a regression where `git.exe` would no longer work in Windows Nano Server (frequently used in Docker containers). This GitHub workflow can be used to verify manually that the Git/Scalar executables work in Nano Server. Signed-off-by: Johannes Schindelin <[email protected]>
When running Git for Windows on a remote APFS filesystem, it would appear that the `mingw_open_append()`/`write()` combination would fail almost exactly like on some CIFS-mounted shares as had been reported in #2753, albeit with a different `errno` value. Let's handle that `errno` value just the same, by suggesting to set `windows.appendAtomically=false`. Signed-off-by: David Lomas <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Windows 10 version 1511 (also known as Anniversary Update), according to https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences introduced native support for ANSI sequence processing. This allows using colors from the entire 24-bit color range. All we need to do is test whether the console's "virtual processing support" can be enabled. If it can, we do not even need to start the `console_thread` to handle ANSI sequences. Or, almost all we need to do: When `console_thread()` does its work, it uses the Unicode-aware `write_console()` function to write to the Win32 Console, which supports Git for Windows' implicit convention that all text that is written is encoded in UTF-8. The same is not necessarily true if native ANSI sequence processing is used, as the output is then subject to the current code page. Let's ensure that the code page is set to `CP_UTF8` as long as Git writes to it. Signed-off-by: Johannes Schindelin <[email protected]>
This was pull request #1645 from ZCube/master Support windows container. Signed-off-by: Johannes Schindelin <[email protected]>
With this patch, Git for Windows works as intended on mounted APFS volumes (where renaming read-only files would fail). Signed-off-by: Johannes Schindelin <[email protected]>
Specify symlink type in .gitattributes
Signed-off-by: Johannes Schindelin <[email protected]>
This patch introduces support to set special NTFS attributes that are interpreted by the Windows Subsystem for Linux as file mode bits, UID and GID. Signed-off-by: Johannes Schindelin <[email protected]>
Handle Ctrl+C in Git Bash nicely Signed-off-by: Johannes Schindelin <[email protected]>
Switch to batched fsync by default
A fix for calling `vim` in Windows Terminal caused a regression and was reverted. We partially un-revert this, to get the fix again. Signed-off-by: Johannes Schindelin <[email protected]>
This topic branch re-adds the deprecated --stdin/-z options to `git reset`. Those patches were overridden by a different set of options in the upstream Git project before we could propose `--stdin`. We offered this in MinGit to applications that wanted a safer way to pass lots of pathspecs to Git, and these applications will need to be adjusted. Instead of `--stdin`, `--pathspec-from-file=-` should be used, and instead of `-z`, `--pathspec-file-nul`. Signed-off-by: Johannes Schindelin <[email protected]>
Originally introduced as `core.useBuiltinFSMonitor` in Git for Windows and developed, improved and stabilized there, the built-in FSMonitor only made it into upstream Git (after unnecessarily long hemming and hawing and throwing overly perfectionist style review sticks into the spokes) as `core.fsmonitor = true`. In Git for Windows, with this topic branch, we re-introduce the now-obsolete config setting, with warnings suggesting to existing users how to switch to the new config setting, with the intention to ultimately drop the patch at some stage. Signed-off-by: Johannes Schindelin <[email protected]>
Start monitoring updates of Git for Windows' component in the open
Add a README.md for GitHub goodness. Signed-off-by: Johannes Schindelin <[email protected]>
dscho
force-pushed
the
rebase-to-v2.51.1
branch
from
4fdf899 to
1ed9925
Compare
@mjcheetham pointed out that these two remaining commits are bogus (the first one adds a line Range-diff
|
|
... and the first bug in v2.51.1 has been reported. I'll wait for confirmation that the fix is good, then integrate it, before releasing Git for Windows v2.51.1. |
@fcharlie Is this another bug, or is it covered by the same fix? I guess there might be more to it (haven't had the time to read the report, patch and discussion in detail yet), as there is another patch on the list now (that seems to be intended to augment, rather than replace, the first fix). |
This is reported as two BUGs, but seems to be related? |
Indeed. The report about if (output_format & DIFF_FORMAT_NO_OUTPUT &&
options->flags.exit_with_status &&
options->flags.diff_from_contents) {
for (i = 0; i < q->nr; i++) {
struct diff_filepair *p = q->queue[i];
if (check_pair_status(p))
diff_flush_patch_quietly(p, options);
if (options->found_changes)
break;
}
}Specifically, the The |
In dry run mode, diff_flush_patch() should not produce any output. However, in commit b55e6d3 (diff: ensure consistent diff behavior with ignore options, 2025-08-08), only the output during the comparison of two file contents was suppressed. For file deletions or mode changes, diff_flush_patch() still produces output. In run_extern_diff(), set quiet to true if in dry run mode. In emit_diff_symbol_from_struct(), directly return if in dry run mode. Signed-off-by: Lidong Yan <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Note: This is _not_ strictly necessary, but adds a layer of protection against bugs similar to the one that was reported very quickly after Git v2.51.1 was released, one time via https://lore.kernel.org/git/CACJRbWjwOQwJB13CwTfvhV3p+Hbn4KrNM9AtBanGtUS4V_1MbQ@mail.gmail.com/ and one time via https://lore.kernel.org/git/[email protected]/. In --quiet mode, since we produce only an exit code for "something was changed" and no actual output, we can often get by with just a tree-level diff. However, certain options require us to actually look at the file contents (e.g., if we are ignoring whitespace changes). We have a flag "diff_from_contents" for that, and if it is set we call diff_flush() on each path. To avoid producing any output (since we were asked to be --quiet), we traditionally just redirected the output to /dev/null. That changed in b55e6d3 (diff: ensure consistent diff behavior with ignore options, 2025-08-08), which replaced that with a "dry_run" flag. In theory, with dry_run set, we should produce no output. But it carries a risk of regression: if we forget to respect dry_run in any of the output paths, we'll accidentally produce output. And indeed, there is at least one such regression in that commit, as it covered only the case where we actually call into xdiff, and not creation or deletion diffs, where we manually generate the headers. We even test this case in t4035, but only with diff-tree, which does not show the bug by default because it does not require diff_from_contents. But git-diff does, because it allows external diff programs by default (so we must dig into each diff filepair to decide if it requires running an external diff that may declare two distinct blobs to actually be the same). We should fix all of those code paths to respect dry_run correctly, but in the meantime we can protect ourselves more fully by restoring the redirection to /dev/null. This gives us an extra layer of protection against regressions dues to other code paths we've missed. Though the original issue was reported with "git diff" (and due to its default of --ext-diff), I've used "diff-tree -w" in the new test. It triggers the same issue, but I think the fact that "-w" implies diff_from_contents is a bit more obvious, and fits in with the rest of t4035. Reported-by: Jake Zimmerman <[email protected]> Signed-off-by: Jeff King <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
|
For the record, I cherry-picked the fix (and the additional "layer of protection", even if that was not strictly necessary to address the reported bug, but if there are similar issues lurking nearby, that layer of protection will paper over them). |
|
/git-artifacts The |
Validate the installer manuallyThe installer was built successfully; |
|
/release The |
|
@dscho, please Share on Bluesky and send the announcement email. |
30 participants
The usual PR before releasing.