From 9361031a09234e38bde724048e64bad765da6843 Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Tue, 5 Jan 2021 13:30:53 -0800 Subject: [PATCH 1/7] first commit on ASan docs --- docs/cpp/address-sanitizer.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 docs/cpp/address-sanitizer.md diff --git a/docs/cpp/address-sanitizer.md b/docs/cpp/address-sanitizer.md new file mode 100644 index 0000000000..c61f8c811f --- /dev/null +++ b/docs/cpp/address-sanitizer.md @@ -0,0 +1,12 @@ +--- +title: "AddressSanitizer and MSVC" +description: "Efficient memory safety checking for C and C++ with MSVC & AddressSanitizer (ASan)." +ms.date: 01/05/2021 +f1_keywords: ["ASan","sanitizers","AddressSanitizer"] +helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] +--- + +# AddressSanitizer and MSVC + +AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. + From e63621aaa3663310d89c9daec9ff1481a69d8e31 Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Tue, 5 Jan 2021 13:57:53 -0800 Subject: [PATCH 2/7] initial add of Custom Allocator section, no inline code --- docs/cpp/address-sanitizer.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/cpp/address-sanitizer.md b/docs/cpp/address-sanitizer.md index c61f8c811f..583df26a3a 100644 --- a/docs/cpp/address-sanitizer.md +++ b/docs/cpp/address-sanitizer.md @@ -10,3 +10,31 @@ helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. + +## Custom allocators and ASan + +ASan provides interceptors for common allocator interfaces, malloc/free, new/delete, HeapAlloc/HeapFree (via RtlAllocateHeap/RtlFreeHeap). Many programs make use of custom allocators for one reason or another. An example would be any program using dlmalloc or a solution using the std::allocator interface and VirtualAlloc. ASan instrumentation does not automatically enlighten these allocators, so it falls to the programmer to 'enlighten' them when using ASan. + +### Manual ASan poisoning interface + +The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes: +`void __asan_poison_memory_region(void const volatile *addr, size_t size);` +`void __asan_unpoison_memory_region(void const volatile *addr, size_t size);` + +These functions are not needed when ASan is disabled, the interface header provides wrapper macros that will check whether ASan is enabled during compilation and omit the functions when ASan instrumentation is not enabled. These macros should be used rather than using the above functions directly: +`#define ASAN_POISON_MEMORY_REGION(addr, size)` +`#define ASAN_UNPOISON_MEMORY_REGION(addr, size)` + +### Manual ASan poisoning and alignment + +ASan poisoning has alignment requirements: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the size + padding for each allocation must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. + +See the included examples for a little background. One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. + +To build and run the allocator example: + + `nmake allocator && allocator.exe` + +To build the 'bad unpoisoning' example: + + `nmake unpoison && unpoison.exe` From a65263e3ad201cd3c8d75138146d6a835532b24a Mon Sep 17 00:00:00 2001 From: Matthew McGovern Date: Thu, 7 Jan 2021 10:13:32 -0800 Subject: [PATCH 3/7] adding additional asan files, expanding runtime section --- ...address-sanitizer-intercepted-functions.md | 0 docs/cpp/address-sanitizer-runtime.md | 40 ++++++ docs/cpp/address-sanitizer.md | 128 ++++++++++++++---- 3 files changed, 143 insertions(+), 25 deletions(-) create mode 100644 docs/cpp/address-sanitizer-intercepted-functions.md create mode 100644 docs/cpp/address-sanitizer-runtime.md diff --git a/docs/cpp/address-sanitizer-intercepted-functions.md b/docs/cpp/address-sanitizer-intercepted-functions.md new file mode 100644 index 0000000000..e69de29bb2 diff --git a/docs/cpp/address-sanitizer-runtime.md b/docs/cpp/address-sanitizer-runtime.md new file mode 100644 index 0000000000..583df26a3a --- /dev/null +++ b/docs/cpp/address-sanitizer-runtime.md @@ -0,0 +1,40 @@ +--- +title: "AddressSanitizer and MSVC" +description: "Efficient memory safety checking for C and C++ with MSVC & AddressSanitizer (ASan)." +ms.date: 01/05/2021 +f1_keywords: ["ASan","sanitizers","AddressSanitizer"] +helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] +--- + +# AddressSanitizer and MSVC + +AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. + + +## Custom allocators and ASan + +ASan provides interceptors for common allocator interfaces, malloc/free, new/delete, HeapAlloc/HeapFree (via RtlAllocateHeap/RtlFreeHeap). Many programs make use of custom allocators for one reason or another. An example would be any program using dlmalloc or a solution using the std::allocator interface and VirtualAlloc. ASan instrumentation does not automatically enlighten these allocators, so it falls to the programmer to 'enlighten' them when using ASan. + +### Manual ASan poisoning interface + +The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes: +`void __asan_poison_memory_region(void const volatile *addr, size_t size);` +`void __asan_unpoison_memory_region(void const volatile *addr, size_t size);` + +These functions are not needed when ASan is disabled, the interface header provides wrapper macros that will check whether ASan is enabled during compilation and omit the functions when ASan instrumentation is not enabled. These macros should be used rather than using the above functions directly: +`#define ASAN_POISON_MEMORY_REGION(addr, size)` +`#define ASAN_UNPOISON_MEMORY_REGION(addr, size)` + +### Manual ASan poisoning and alignment + +ASan poisoning has alignment requirements: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the size + padding for each allocation must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. + +See the included examples for a little background. One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. + +To build and run the allocator example: + + `nmake allocator && allocator.exe` + +To build the 'bad unpoisoning' example: + + `nmake unpoison && unpoison.exe` diff --git a/docs/cpp/address-sanitizer.md b/docs/cpp/address-sanitizer.md index 583df26a3a..82ce756d3f 100644 --- a/docs/cpp/address-sanitizer.md +++ b/docs/cpp/address-sanitizer.md @@ -1,40 +1,118 @@ ---- -title: "AddressSanitizer and MSVC" -description: "Efficient memory safety checking for C and C++ with MSVC & AddressSanitizer (ASan)." -ms.date: 01/05/2021 -f1_keywords: ["ASan","sanitizers","AddressSanitizer"] -helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] ---- +# Address Sanitizer - user interface -# AddressSanitizer and MSVC +## Introduction - Developer Message -AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. +The Address Sanitizer is a compiler based technology [introduced by Google](https://www.usenix.org/conference/atc12/technical-sessions/presentation/serebryany). This compiler and runtime technology has become the "defacto standard" in the industry for finding memory safety issues. We now offer this technology as a fully supported feature in Visual Studio. We shipped this (and up streamed our ASan runtime changes) to accommodate our customers with a simple re-compile strategy. Testing your build from this simple re-compile, can dramatically increase correctness, portability and security. If your existing code compiles with our current Windows compiler, then it will compile with the flag -fsanitize=address under any level of optimization and all other compatible flags (e.g., /RTC and is not currently compatible) +Microsoft recommends using the Address Sanitizer in **three workflows**: -## Custom allocators and ASan +- **Developer inner loop** + - Visual Studio - Command line + - Visual Studio - Project system with integrated IDE error reporting support. -ASan provides interceptors for common allocator interfaces, malloc/free, new/delete, HeapAlloc/HeapFree (via RtlAllocateHeap/RtlFreeHeap). Many programs make use of custom allocators for one reason or another. An example would be any program using dlmalloc or a solution using the std::allocator interface and VirtualAlloc. ASan instrumentation does not automatically enlighten these allocators, so it falls to the programmer to 'enlighten' them when using ASan. + +- **CI/CD** - integrate, build and run unaltered, existing test assets + - CMake + - MSBuild -### Manual ASan poisoning interface +- **Fuzzing** - Use the supported libFuzzer build + - [Azure OneFuzz](https://www.microsoft.com/security/blog/2020/09/15/microsoft-onefuzz-framework-open-source-developer-tool-fix-bugs/) + - Local Machine -The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes: -`void __asan_poison_memory_region(void const volatile *addr, size_t size);` -`void __asan_unpoison_memory_region(void const volatile *addr, size_t size);` +You can customize your builds as covered in this section -These functions are not needed when ASan is disabled, the interface header provides wrapper macros that will check whether ASan is enabled during compilation and omit the functions when ASan instrumentation is not enabled. These macros should be used rather than using the above functions directly: -`#define ASAN_POISON_MEMORY_REGION(addr, size)` -`#define ASAN_UNPOISON_MEMORY_REGION(addr, size)` +- **Customization** + - Enhance - Hooking your allocators + - Remove - ASan at function of variable granularity -### Manual ASan poisoning and alignment +There's extensive documentation for these platform dependent implementations of the Address Sanitizer technology. -ASan poisoning has alignment requirements: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the size + padding for each allocation must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. +- [Google](https://clang.llvm.org/docs/AddressSanitizer.html) +- [Apple](https://developer.apple.com/documentation/xcode/diagnosing_memory_thread_and_crash_issues_early) +- [GCC](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html) -See the included examples for a little background. One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. +This document will cover support for the above workflows on the Microsoft Windows 10 platform. We start with a simple command line use of the compiler and linker. We build upon that simple introduction following the structure of the outline seen in the **three workflows** listed above. -To build and run the allocator example: +For a complete indexed catalogue of errors this re-compile will find, see the [IDE support for errors](#ide-support-for-errors) section. - `nmake allocator && allocator.exe` +**NOTE:** Current support is limited to x86 and AMD64 on Windows10. There is no support for -fsanitize=thread, -fsanitize=leak, -fsanitize=memory or -fsanitize=hwaddress. -To build the 'bad unpoisoning' example: +## Command line + +Adding the flag -fsanitize=address to your command line (with support to emit debug information) is all you need to compile,link and run an .EXE. + + `cl -fsanitize=address /Zi file.cpp file2.cpp my3dparty.lib /Fe My.exe` + +The flag -fsanitize=address is compatible with all existing C or C++ optimization flags (e.g., /Od, /O1, /O2, /O2 /GL and PGO) + +**-fsanitize=address** + +Enable the injection of instrumentation code that inter-ops with the Asan runtime binaries that are automatically linked to the binary you are producing. This is a fast memory safety detector that just requires a recompile. Loads, stores, scopes, alloca, and CRT functions are hooked to detect hidden bugs like out-of-bounds, use-after-free, use-after-scope etc.. See [Error reporting](#error-reporting) for a complete list of errors currently detected at runtime. + +Unlike Clang/LLVM this option enables -fsanitize-address-use-after-scope by default and it can not be turned off at the command line or through the [Runtimes](#runtimes) ASAN_OPTIONS environment variable. The functionality for use-after-return requires code generation under an additional flag and environment variable. + +By default (legacy reasons) the CL driver infers the linker flag [/MT](https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=msvc-160) and that will link **static versions** of both the Asan and CRT libraries. If you want to link to the **dynamic version** of the CRT then use the following: + + `cl -fsanitize=address /Zi /MD file.cpp file2.cpp my3dparty.lib /De My.exe` + +See the [Linker](#linker) section for details on more complex build scenarios. See the [runtime](#runtimes) section for an inventory of the Asan runtime libraries. + + +**-fsanitize-address-use-after-return** + +Create the coded generation that will create a dual stack frame in the heap. The stack frame in the heap will linger after the return from the function that created it. If an address of a local, allocated to a slot in the frame in the heap, then the code generation can later determine a stack-use-after-return error. + + `cl -fsanitize=address -fsanitize-address-use-after-return /Zi file.cpp file2.cpp my3dparty.lib /De My.exe` + +This is an opt in code generation which is much slower than just using -fsanize=address. + +## Visual Studio + +### Project System + +### Error reporting + +## Compiler tool set + +### Compiler + +### Linker +[Notes on linker behavior](https://microsoft.sharepoint.com/teams/DD_VC/_layouts/OneNote.aspx?id=%2Fteams%2FDD_VC%2FShared%20Documents%2FVisual%20C%2B%2B%20Team&wd=target%28BE%20Team%2FSecurity%2FCompiler%20Security%20V-Team.one%7CC2A34F56-6B09-4AB1-869B-DFD77BFD7399%2FNotes%20about%20vcasan%20and%20%5C%2Finferasanlibs%7C6D1BD27A-F55A-44BC-BF7C-AF6404C4C5C1%2F%29) + +## Address Sanitizer Runtimes + +This implementation of AddressSanitizer makes use of the Clang ASan runtime libraries. The runtime library version packaged with Visual Studio may contain features that are not yet available in the version packaged with Clang. + +### Function interception +The runtime libraries intercept common memory management functions. [A complete list of intercepted functions is available here.](.\address-sanitizer-intercepted-functions.md) + +Interception is achieved through an assortment of hotpatching techniques. + +### Runtime features overview +A complete overview of the features and design of the runtime is available here: [AddressSanitizer runtime overview](address-sanitizer-runtime.md) + +### Static (x86,AMD64) +### Dynamic (x86,AMD64) +### Runtime Flags + +Pick only the set we currently support +https://github.com/google/sanitizers/wiki/AddressSanitizerFlags#run-time-flags + +### Allocators + +## Build tools + +### CMake + +### MSBuild + +## Fuzing + +### Azure + +### Local Machine + +## See Also - technical overview + +This will cover a deep dive into the implementation and the runtime as a marketing tool to generate interest for the MSDN web page. - `nmake unpoison && unpoison.exe` From 20e84b2dbf323fbf57c70552880e9dc196be742c Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Thu, 7 Jan 2021 15:16:07 -0800 Subject: [PATCH 4/7] updating additional interceptor and runtime info, adding new introduction paragraph --- ...address-sanitizer-intercepted-functions.md | 93 +++++++++++++++++++ docs/cpp/address-sanitizer-runtime.md | 30 ++++-- docs/cpp/address-sanitizer.md | 8 +- 3 files changed, 116 insertions(+), 15 deletions(-) diff --git a/docs/cpp/address-sanitizer-intercepted-functions.md b/docs/cpp/address-sanitizer-intercepted-functions.md index e69de29bb2..d2f4a34bc7 100644 --- a/docs/cpp/address-sanitizer-intercepted-functions.md +++ b/docs/cpp/address-sanitizer-intercepted-functions.md @@ -0,0 +1,93 @@ +# AddressSanitizer list of intercepted functions (Windows) +The AddressSanitier runtime hotpatches numerous functions to enable memory safety checks during program execution. The following is a non-exhaustive list of the functions that the ASan runtime monitors. + +## Default interceptors +- [__C_specific_handler (x64 only)](https://docs.microsoft.com/en-us/windows/win32/devnotes/--c-specific-handler2) +- [_aligned_free](../c-runtime-library/reference/aligned-free.md) +- [_aligned_malloc](../c-runtime-library/reference/aligned-malloc.md) +- [_aligned_msize](../c-runtime-library/reference/aligned-msize.md) +- [_aligned_realloc](../c-runtime-library/reference/aligned-realloc.md) +- [_calloc_base](../c-runtime-library/reference/calloc.md) +- [_calloc_crt](../c-runtime-library/reference/calloc.md) +- [_calloc_dbg (debug runtimes only)](../c-runtime-library/reference/calloc-dbg.md) +- [_except_handler3 (x86 only)](../c-runtime-library/except-handler3.md) +- _except_handler4 (x86 only)(undocumented) +- [_expand](../c-runtime-library/reference/expand.md) +- _expand_base (undocumented) +- [_expand_dbg (debug runtimes only)](../c-runtime-library/reference/expand-dbg.md) +- _free_base (undocumented) +- [_free_dbg (debug runtimes only)](../c-runtime-library/reference/free-dbg.md) +- _malloc_base (undocumented) +- _malloc_crt (undocumented) +- [_malloc_dbg (debug runtimes only)](../c-runtime-library/reference/malloc-dbg.md) +- [_msize](../c-runtime-library/reference/msize.md) +- _msize_base (undocumented) +- [_msize_dbg (debug runtimes only)](../c-runtime-library/reference/msize-dbg.md) +- _realloc_base (undocumented) +- _realloc_crt (undocumented) +- [_realloc_dbg (debug runtimes only)](../c-runtime-library/reference/realloc-dbg.md) +- [_recalloc](../c-runtime-library/reference/recalloc.md) +- _recalloc_base (undocumented) +- _recalloc_crt (undocumented) +- [_recalloc_dbg (debug runtimes only)](../c-runtime-library/reference/recalloc-dbg.md) +- [_strdup](../c-runtime-library/reference/strdup-wcsdup-mbsdup.md) +- [atoi](../c-runtime-library/reference/atoi-atoi-l-wtoi-wtoi-l.md) +- [atol](../c-runtime-library/reference/atoi-atoi-l-wtoi-wtoi-l.md) +- [calloc](../c-runtime-library/reference/calloc.md) +- [CreateThread](https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread) +- [free](../c-runtime-library/reference/free.md) +- [frexp](../c-runtime-library/reference/frexp.md) +- [longjmp](../c-runtime-library/reference/longjmp.md) +- [malloc](../c-runtime-library/reference/malloc.md) +- [memchr](../c-runtime-library/reference/memchr-wmemchr.md) +- [memcmp](../c-runtime-library/reference/memcmp-wmemcmp.md) +- [memcpy](../c-runtime-library/reference/memcpy-wmemcpy.md) +- [memmove](../c-runtime-library/reference/memmove-wmemmove.md) +- [memset](../c-runtime-library/reference/memset-wmemset.md) +- [RaiseException](https://docs.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-raiseexception) +- [realloc](../c-runtime-library/reference/realloc.md) +- [RtlAllocateHeap](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlallocateheap) +- [RtlCreateHeap](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlcreateheap) +- [RtlDestroyHeap](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlcreateheap) +- [RtlFreeHeap](https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlfreeheap) +- [RtlRaiseException](https://docs.microsoft.com/en-us/windows/win32/api/rtlsupportapi/nf-rtlsupportapi-rtlraiseexception) +- RtlReAllocateHeap (undocumented) +- RtlSizeHeap (undocumented) +- [SetUnhandledExceptionFilter](https://docs.microsoft.com/en-us/windows/win32/api/errhandlingapi/nf-errhandlingapi-setunhandledexceptionfilter) +- [strcat](../c-runtime-library/reference/strcat-wcscat-mbscat.md) +- [strchr](../c-runtime-library/reference/strchr-wcschr-mbschr-mbschr-l.md) +- [strcmp](../c-runtime-library/reference/strcmp-wcscmp-mbscmp.md) +- [strcpy](../c-runtime-library/reference/strcpy-wcscpy-mbscpy.md) +- [strcspn](../c-runtime-library/reference/strcspn-wcscspn-mbscspn-mbscspn-l.md) +- [strdup](../c-runtime-library/reference/strdup-wcsdup.md) +- [strlen](../c-runtime-library/reference/strlen-wcslen-mbslen-mbslen-l-mbstrlen-mbstrlen-l.md) +- [strncat](../c-runtime-library/reference/strncat-strncat-l-wcsncat-wcsncat-l-mbsncat-mbsncat-l.md) +- [strncmp](../c-runtime-library/reference/strncmp-wcsncmp-mbsncmp-mbsncmp-l.md) +- [strncpy](../c-runtime-library/reference/strncpy-strncpy-l-wcsncpy-wcsncpy-l-mbsncpy-mbsncpy-l.md) +- [strnlen](../c-runtime-library/reference/strnlen-strnlen-s.md) +- [strpbrk](../c-runtime-library/reference/strpbrk-wcspbrk-mbspbrk-mbspbrk-l.md) +- [strspn](../c-runtime-library/reference/strspn-wcsspn-mbsspn-mbsspn-l.md) +- [strstr](../c-runtime-library/reference/strstr-wcsstr-mbsstr-mbsstr-l.md) +- [strtok](../c-runtime-library/reference/strtok-strtok-l-wcstok-wcstok-l-mbstok-mbstok-l.md) +- [strtol](../c-runtime-library/reference/strtol-wcstol-strtol-l-wcstol-l.md) +- [wcslen](../c-runtime-library/reference/strlen-wcslen-mbslen-mbslen-l-mbstrlen-mbstrlen-l.md) +- [wcsnlen](../c-runtime-library/reference/strnlen-strnlen.md) + +## Optional interceptors +The following interceptors are only installed when an asan runtime option is enabled, set `windows_hook_legacy_allocators` to `true` to enable legacy allocator interception. +`set ASAN_OPTIONS=windows_hook_legacy_allocators=true` + +- [GlobalAlloc](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-globalalloc) +- [GlobalFree](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalFree) +- [GlobalHandle](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalHandle) +- [GlobalLock](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalLock) +- [GlobalReAlloc](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalReAlloc) +- [GlobalSize](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalSize) +- [GlobalUnlock](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-GlobalUnlock) +- [LocalAlloc](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalAlloc) +- [LocalFree](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalFree) +- [LocalHandle](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalHandle) +- [LocalLock](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalLock) +- [LocalReAlloc](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalReAlloc) +- [LocalSize](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalSize) +- [LocalUnlock](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-LocalUnlock) \ No newline at end of file diff --git a/docs/cpp/address-sanitizer-runtime.md b/docs/cpp/address-sanitizer-runtime.md index 583df26a3a..a3b7f6d5d3 100644 --- a/docs/cpp/address-sanitizer-runtime.md +++ b/docs/cpp/address-sanitizer-runtime.md @@ -10,24 +10,34 @@ helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. +### Function interception + +Interception is achieved through a number of hot-patching techniques, [these are well documented within the source code.](https://github.com/llvm/llvm-project/blob/1a2eaebc09c6a200f93b8beb37130c8b8aab3934/compiler-rt/lib/interception/interception_win.cpp#L11) + +The runtime libraries intercept many common memory management and manipulation functions. [A complete list of intercepted functions is available here.](.\address-sanitizer-intercepted-functions.md) ## Custom allocators and ASan -ASan provides interceptors for common allocator interfaces, malloc/free, new/delete, HeapAlloc/HeapFree (via RtlAllocateHeap/RtlFreeHeap). Many programs make use of custom allocators for one reason or another. An example would be any program using dlmalloc or a solution using the std::allocator interface and VirtualAlloc. ASan instrumentation does not automatically enlighten these allocators, so it falls to the programmer to 'enlighten' them when using ASan. +ASan provides interceptors for common allocator interfaces, malloc/free, new/delete, HeapAlloc/HeapFree (via RtlAllocateHeap/RtlFreeHeap). Many programs make use of custom allocators for one reason or another. An example would be any program using dlmalloc or a solution using the std::allocator interface and VirtualAlloc. ASan instrumentation does not automatically enlighten custom allocators; it is the responsibility of the user to use the provided manual poisoning interface to enable their allocators to function with ASan. + +## Manual ASan poisoning interface -### Manual ASan poisoning interface +The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes are as follows: +``` +void __asan_poison_memory_region(void const volatile *addr, size_t size); +void __asan_unpoison_memory_region(void const volatile *addr, size_t size); +``` -The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes: -`void __asan_poison_memory_region(void const volatile *addr, size_t size);` -`void __asan_unpoison_memory_region(void const volatile *addr, size_t size);` +These functions are not needed when ASan is disabled, for convenience the asan interface header provides wrapper macros that check whether ASan is enabled during compilation and omit the poisoning function calls when ASan instrumentation is disabled. These macros should be preferred over calling the above functions directly: -These functions are not needed when ASan is disabled, the interface header provides wrapper macros that will check whether ASan is enabled during compilation and omit the functions when ASan instrumentation is not enabled. These macros should be used rather than using the above functions directly: -`#define ASAN_POISON_MEMORY_REGION(addr, size)` -`#define ASAN_UNPOISON_MEMORY_REGION(addr, size)` +``` +#define ASAN_POISON_MEMORY_REGION(addr, size) +#define ASAN_UNPOISON_MEMORY_REGION(addr, size) +``` -### Manual ASan poisoning and alignment +## Alignment requirements for ASan poisoning -ASan poisoning has alignment requirements: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the size + padding for each allocation must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. +ASan poisoning has an alignment requirement: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the total size of each allocation including any padding must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. See the included examples for a little background. One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. diff --git a/docs/cpp/address-sanitizer.md b/docs/cpp/address-sanitizer.md index 82ce756d3f..f38066c080 100644 --- a/docs/cpp/address-sanitizer.md +++ b/docs/cpp/address-sanitizer.md @@ -1,5 +1,8 @@ # Address Sanitizer - user interface +AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. + + ## Introduction - Developer Message The Address Sanitizer is a compiler based technology [introduced by Google](https://www.usenix.org/conference/atc12/technical-sessions/presentation/serebryany). This compiler and runtime technology has become the "defacto standard" in the industry for finding memory safety issues. We now offer this technology as a fully supported feature in Visual Studio. We shipped this (and up streamed our ASan runtime changes) to accommodate our customers with a simple re-compile strategy. Testing your build from this simple re-compile, can dramatically increase correctness, portability and security. If your existing code compiles with our current Windows compiler, then it will compile with the flag -fsanitize=address under any level of optimization and all other compatible flags (e.g., /RTC and is not currently compatible) @@ -83,11 +86,6 @@ This is an opt in code generation which is much slower than just using -fsanize= This implementation of AddressSanitizer makes use of the Clang ASan runtime libraries. The runtime library version packaged with Visual Studio may contain features that are not yet available in the version packaged with Clang. -### Function interception -The runtime libraries intercept common memory management functions. [A complete list of intercepted functions is available here.](.\address-sanitizer-intercepted-functions.md) - -Interception is achieved through an assortment of hotpatching techniques. - ### Runtime features overview A complete overview of the features and design of the runtime is available here: [AddressSanitizer runtime overview](address-sanitizer-runtime.md) From de04d196caeb869a8dfdfdb2c91f9ee4fb95bdfe Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Thu, 7 Jan 2021 15:54:01 -0800 Subject: [PATCH 5/7] updating docs to point to external example for poisoning --- docs/cpp/address-sanitizer-runtime.md | 37 +++++++++++++++++---------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/docs/cpp/address-sanitizer-runtime.md b/docs/cpp/address-sanitizer-runtime.md index a3b7f6d5d3..2081b54c16 100644 --- a/docs/cpp/address-sanitizer-runtime.md +++ b/docs/cpp/address-sanitizer-runtime.md @@ -6,15 +6,32 @@ f1_keywords: ["ASan","sanitizers","AddressSanitizer"] helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] --- -# AddressSanitizer and MSVC +# AddressSanitizer (ASan) Runtimes -AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. +ASan requires the installation of interceptors for common memory functions, for both allocators and memory manipulation functions. There are a variety of runtime libraries for targeting different executable types and CRT libraries. Unless the user is in an advanced situation and using the /nodefaultlib flag at link time, the compiler will link the appropriate library so long as the /fsanitize=address flag is passed at compile time. -### Function interception +Below is an inventory of runtime libraries related to ASan, where `arch` is either `i386` or `x86_64`. -Interception is achieved through a number of hot-patching techniques, [these are well documented within the source code.](https://github.com/llvm/llvm-project/blob/1a2eaebc09c6a200f93b8beb37130c8b8aab3934/compiler-rt/lib/interception/interception_win.cpp#L11) +> [!NOTE] +> These libraries keep the Clang conventions for architecture names. The Microsoft Visual C++ convention would be x86 and x64 rather than i386 and x86_64. They refer to the same architectures. -The runtime libraries intercept many common memory management and manipulation functions. [A complete list of intercepted functions is available here.](.\address-sanitizer-intercepted-functions.md) +| CRT Flag | DLL or EXE | DEBUG? | ASan Runtime Libraries | +|----------|------------|--------|------------------------------------------------------------------------------------| +| MT | EXE | NO | `clang_rt.asan-{arch}
clang_rt.asan_cxx-{arch}` | +| MT | DLL | NO | `clang_rt.asan_dll_thunk-{arch}` | +| MD | EITHER | NO | `clang_rt.asan_dynamic-{arch}
clang_rt.asan_dynamic_runtime_thunk-{arch}` | +| MT | EXE | YES | `clang_rt.asan_dbg-{arch}
clang_rt.asan_dbg_cxx-{arch}` | +| MT | DLL | YES | `clang_rt.asan_dbg_dll_thunk-{arch}` | +| MD | EITHER | YES | `clang_rt.asan_dbg_dynamic-{arch}
clang_rt.asan_dbg_dynamic_runtime_thunk-{arch}` | + + + During the ASan compiler pass, the compiler generates metadata needed for stack and global variables and inserts checks for memory accesses throughout the program. The ASan runtime libraries install function interceptors during program initialization; these interceptors generate the necessary metadata and insert checks in common string and memory functions. Combined, this extra data is used to generate ASan reports when a memory corruption situation is detected. + +## Function interception + +Interception is achieved through a number of hot-patching techniques, [these are best documented within the source code itself.](https://github.com/llvm/llvm-project/blob/1a2eaebc09c6a200f93b8beb37130c8b8aab3934/compiler-rt/lib/interception/interception_win.cpp#L11) + +The runtime libraries intercept many common memory management and manipulation functions. [A complete list of intercepted functions is available here.](.\address-sanitizer-intercepted-functions.md) The allocation interceptors manage metadata related to each allocation which is created. Each time an allocation is created or destroyed the allocator interceptors set specific values in the ASan shadow memory to indicate the current state of the allocation. When the compiler generates a memory access, it inserts a check that reads this metadata and determines whether the access is valid. ## Custom allocators and ASan @@ -39,12 +56,4 @@ These functions are not needed when ASan is disabled, for convenience the asan i ASan poisoning has an alignment requirement: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the total size of each allocation including any padding must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. -See the included examples for a little background. One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. - -To build and run the allocator example: - - `nmake allocator && allocator.exe` - -To build the 'bad unpoisoning' example: - - `nmake unpoison && unpoison.exe` +[See the provided example programs for an illustration of the requirement and potential issues.](https://github.com/mcgov/asan_alignment_example) One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. From dc7cccc576659f5143646c80552d03c0283e462f Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Fri, 8 Jan 2021 12:43:46 -0800 Subject: [PATCH 6/7] adding runtime flags and options section --- ...address-sanitizer-intercepted-functions.md | 8 +++++ docs/cpp/address-sanitizer-runtime.md | 36 ++++++++++++++----- docs/cpp/address-sanitizer.md | 3 +- 3 files changed, 37 insertions(+), 10 deletions(-) diff --git a/docs/cpp/address-sanitizer-intercepted-functions.md b/docs/cpp/address-sanitizer-intercepted-functions.md index d2f4a34bc7..a3679ac213 100644 --- a/docs/cpp/address-sanitizer-intercepted-functions.md +++ b/docs/cpp/address-sanitizer-intercepted-functions.md @@ -1,3 +1,11 @@ +--- +title: "AddressSanitizer list of intercepted functions (Ms Visual C++)" +description: "Microsoft Visual C++ AddressSanitizer list of intercepted functions." +ms.date: 01/05/2021 +f1_keywords: ["ASan","sanitizers","AddressSanitizer","interception"] +helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan","intercepted functions", "interception"] +--- + # AddressSanitizer list of intercepted functions (Windows) The AddressSanitier runtime hotpatches numerous functions to enable memory safety checks during program execution. The following is a non-exhaustive list of the functions that the ASan runtime monitors. diff --git a/docs/cpp/address-sanitizer-runtime.md b/docs/cpp/address-sanitizer-runtime.md index 2081b54c16..197a948f9d 100644 --- a/docs/cpp/address-sanitizer-runtime.md +++ b/docs/cpp/address-sanitizer-runtime.md @@ -1,9 +1,9 @@ --- -title: "AddressSanitizer and MSVC" -description: "Efficient memory safety checking for C and C++ with MSVC & AddressSanitizer (ASan)." +title: "AddressSanitizer Runtime" +description: "Technical descrption of the AddressSanitizer runtime for Microsoft Visual C++." ms.date: 01/05/2021 f1_keywords: ["ASan","sanitizers","AddressSanitizer"] -helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan"] +helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","clang_rt.asan","Clang runtime","runtime"] --- # AddressSanitizer (ASan) Runtimes @@ -17,13 +17,12 @@ Below is an inventory of runtime libraries related to ASan, where `arch` is eith | CRT Flag | DLL or EXE | DEBUG? | ASan Runtime Libraries | |----------|------------|--------|------------------------------------------------------------------------------------| -| MT | EXE | NO | `clang_rt.asan-{arch}
clang_rt.asan_cxx-{arch}` | +| MT | EXE | NO | `clang_rt.asan-{arch}, clang_rt.asan_cxx-{arch}` | | MT | DLL | NO | `clang_rt.asan_dll_thunk-{arch}` | -| MD | EITHER | NO | `clang_rt.asan_dynamic-{arch}
clang_rt.asan_dynamic_runtime_thunk-{arch}` | -| MT | EXE | YES | `clang_rt.asan_dbg-{arch}
clang_rt.asan_dbg_cxx-{arch}` | +| MD | EITHER | NO | `clang_rt.asan_dynamic-{arch}, clang_rt.asan_dynamic_runtime_thunk-{arch}` | +| MT | EXE | YES | `clang_rt.asan_dbg-{arch}, clang_rt.asan_dbg_cxx-{arch}` | | MT | DLL | YES | `clang_rt.asan_dbg_dll_thunk-{arch}` | -| MD | EITHER | YES | `clang_rt.asan_dbg_dynamic-{arch}
clang_rt.asan_dbg_dynamic_runtime_thunk-{arch}` | - +| MD | EITHER | YES | `clang_rt.asan_dbg_dynamic-{arch}, clang_rt.asan_dbg_dynamic_runtime_thunk-{arch}` | During the ASan compiler pass, the compiler generates metadata needed for stack and global variables and inserts checks for memory accesses throughout the program. The ASan runtime libraries install function interceptors during program initialization; these interceptors generate the necessary metadata and insert checks in common string and memory functions. Combined, this extra data is used to generate ASan reports when a memory corruption situation is detected. @@ -40,6 +39,7 @@ ASan provides interceptors for common allocator interfaces, malloc/free, new/del ## Manual ASan poisoning interface The interface for enlightening is simple but it imposes alignment restrictions on the user. The interface function prototypes are as follows: + ``` void __asan_poison_memory_region(void const volatile *addr, size_t size); void __asan_unpoison_memory_region(void const volatile *addr, size_t size); @@ -57,3 +57,23 @@ These functions are not needed when ASan is disabled, for convenience the asan i ASan poisoning has an alignment requirement: the user must add padding such that the padding ends on a byte boundary in the shadow memory. Since each bit in the ASan shadow memory encodes the state of a byte in real memory, this means that **the total size of each allocation including any padding must align on a 8 byte boundary.** If this requirement is not satisfied it can lead to incorrect bug reporting, including missed and false-positive reports. [See the provided example programs for an illustration of the requirement and potential issues.](https://github.com/mcgov/asan_alignment_example) One is a small program to show what can go wrong with manual shadow memory poisoning. The second is an example implementation of manual poising using the `std::allocator` interface. + +## Run-time Flags + +Microsoft Visual C++ uses a runtime based on the [Clang ASan runtime from the llvm-project repository.](https://github.com/llvm/llvm-project) Because of this, most runtime flags are shared between the two verions. [A complete list of the public Clang runtime flags is available here.](https://github.com/google/sanitizers/wiki/SanitizerCommonFlags) There are some differences detailed here, if you discover options that do not function as expected please file a feedback ticket in the [Visual Studio developer community.](https://developercommunity.visualstudio.com) + +### Unsupported ASan Options + +- detect_container_overflow +- unmap_shadow_on_exit + +> [!NOTE] +> The ASan option `halt_on_error` does not function the way you might expect. In both the Clang and the Microsoft Visual C++ run-time libraries many error types have been designated as non-continuable, including most memory corruption errors. + +### Microsoft Visual C++ specific ASan runtime flags: + +- `windows_hook_legacy_allocators` + - Boolean, set to `true` to enable interception of [GlobalAlloc](https://docs.microsoft.com/windows/win32/api/winbase/nf-winbase-globalalloc) and [LocalAlloc](https://docs.microsoft.com/windows/win32/api/winbase/nf-winbase-localalloc) allocators. + +> [!NOTE] +> The option `windows_hook_legacy_allocators` is not currently available in the public llvm-project runtime as of this date. The option may eventually be contributed back to the public project; however, this is dependent on code review and community acceptance. diff --git a/docs/cpp/address-sanitizer.md b/docs/cpp/address-sanitizer.md index f38066c080..7ef9f9cf6c 100644 --- a/docs/cpp/address-sanitizer.md +++ b/docs/cpp/address-sanitizer.md @@ -1,8 +1,7 @@ -# Address Sanitizer - user interface +# Address Sanitizer AddressSanitizer (ASan) is an algorithm that can help developers identify memory safety issues in their C and C++ programs. Developers can use the /fsanitize=address flag to add ASan instrumentation to their native programs. At runtime, the ASan runtime replaces common allocation and memory manipulation functions with instrumented versions. Together, the instrumentation and ASan runtime enable developers to gather useful information about any memory safety issues that are encountered during execution. - ## Introduction - Developer Message The Address Sanitizer is a compiler based technology [introduced by Google](https://www.usenix.org/conference/atc12/technical-sessions/presentation/serebryany). This compiler and runtime technology has become the "defacto standard" in the industry for finding memory safety issues. We now offer this technology as a fully supported feature in Visual Studio. We shipped this (and up streamed our ASan runtime changes) to accommodate our customers with a simple re-compile strategy. Testing your build from this simple re-compile, can dramatically increase correctness, portability and security. If your existing code compiles with our current Windows compiler, then it will compile with the flag -fsanitize=address under any level of optimization and all other compatible flags (e.g., /RTC and is not currently compatible) From 5cdf2485d0611fadf4f1b995de5d6616746100d5 Mon Sep 17 00:00:00 2001 From: Matthew G McGovern Date: Fri, 8 Jan 2021 13:53:27 -0800 Subject: [PATCH 7/7] add vcasan doc --- docs/cpp/address-sanitizer-vcasan.md | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 docs/cpp/address-sanitizer-vcasan.md diff --git a/docs/cpp/address-sanitizer-vcasan.md b/docs/cpp/address-sanitizer-vcasan.md new file mode 100644 index 0000000000..61dbaba9f6 --- /dev/null +++ b/docs/cpp/address-sanitizer-vcasan.md @@ -0,0 +1,36 @@ +--- +title: "AddressSanitizer: vcasan.lib" +description: "Technical description of vcasan.lib." +ms.date: 01/05/2021 +f1_keywords: ["ASan","sanitizers","AddressSanitizer", "vcasan"] +helpviewer_keywords: ["ASan","sanitizers","AddressSanitizer","vcasan.lib","vcasan","vcasand.lib","libvcasan.lib","libvcasand.lib"] +--- + +# Visual Studio ASan extended functionality library (VCASan) + +The VCAsan libraries exist to enable extended Visual Studio features when debugging with ASan. The library is linked any time AddressSanitizer is enabled with MSVC. The library allows for Visual Studio to display AddressSanitizer-specific message pop-ups, as well as enabling the executable to generate crash dumps when an ASan report is created. + +## VCAsan library inventory + +| Runtime Flag | VCAsan Version | +|--------------|----------------| +| MT | libvcasan.lib | +| MD | vcasan.lib | +| MTd | libvcasand.lib | +| MDd | vcasand.lib | + +## VCAsan library features + +### Rich ASan error report window in Visual Studio IDE + +VCAsan registers a callback within the ASan runtime with the [ASan interface function `__asan_set_error_report_callback`.](https://github.com/llvm/llvm-project/blob/1ba5ea67a30170053964a28f2f47aea4bb7f5ff1/compiler-rt/include/sanitizer/asan_interface.h#L256) If an ASan report is generated, this callback is used to throw an exception that will be caught by Visual Studio. The data in the exception is used to generate the Visual Studio message that is displayed to the user within the IDE. +> [!NOTE] +> Since VCAsan registers this callback function, if user code calls this function a second time it is possible for the user registered callback to overwrite the VCAsan callback registration. This would result in the loss of the ASan error message window in the Visual Studio IDE. Since the registration occurs in a race, it is also possible for the user's call to register the callback to be lost. If you encounter either problem please file a feedback ticket with the [Visual Studio developer community](https://developercommunity.visualstudio.com). + +### Save crash dumps after ASan error report + +When a VCasan library is linked it is possible for the user to generate a crash dump when an ASan error report is generated. To enable this feature, the user must set an environment variable as follows: + +`set ASAN_SAVE_DUMPS="MyFileName.dmpx"` + +This will save a snapshot file when an error is caught by the AddressSanitizer. The meta-data that is saved in the dump file is parsed by the new Visual Studio IDE. You can set this variable on a per-test basis and store these binary artifacts and then view these in the IDE with proper source indexing.