Skip to content

refactor(profiling): remove legacy Python exporter and enable libdatadog exporter by default #13599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 8, 2025

Conversation

taegyunkim
Copy link
Contributor

@taegyunkim taegyunkim commented Jun 5, 2025

Removes python exporter and enable libdatadog exporter by default. Since stack v2 requires libdatadog exporter and it's the default mode, most customers would not be affected by this.

We are dropping 32-bit Linux support for profiling, but this can be added back once profiling depends on libdatadog via Rust bindings using PyO3.

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@taegyunkim taegyunkim requested review from a team as code owners June 5, 2025 00:52
Copy link
Contributor

github-actions bot commented Jun 5, 2025

CODEOWNERS have been resolved as:

releasenotes/notes/profiling-libdd-default-no-linux-32bit-41e7a4b211de7dd2.yaml  @DataDog/apm-python
ddtrace/internal/datadog/profiling/ddup/__init__.py                     @DataDog/profiling-python
ddtrace/profiling/bootstrap/sitecustomize.py                            @DataDog/profiling-python
ddtrace/profiling/collector/__init__.py                                 @DataDog/profiling-python
ddtrace/profiling/collector/_lock.py                                    @DataDog/profiling-python
ddtrace/profiling/collector/_task.pyx                                   @DataDog/profiling-python
ddtrace/profiling/collector/asyncio.py                                  @DataDog/profiling-python
ddtrace/profiling/collector/memalloc.py                                 @DataDog/profiling-python
ddtrace/profiling/collector/pytorch.py                                  @DataDog/profiling-python
ddtrace/profiling/collector/stack.pyx                                   @DataDog/profiling-python
ddtrace/profiling/collector/threading.py                                @DataDog/profiling-python
ddtrace/profiling/event.py                                              @DataDog/profiling-python
ddtrace/profiling/profiler.py                                           @DataDog/profiling-python
ddtrace/profiling/scheduler.py                                          @DataDog/profiling-python
ddtrace/settings/profiling.py                                           @DataDog/profiling-python
mypy.ini                                                                @DataDog/python-guild @DataDog/apm-core-python
pyproject.toml                                                          @DataDog/python-guild
riotfile.py                                                             @DataDog/apm-python
setup.py                                                                @DataDog/python-guild
tests/profiling/collector/pprof_utils.py                                @DataDog/profiling-python
tests/profiling/collector/test_collector.py                             @DataDog/profiling-python
tests/profiling/collector/test_stack.py                                 @DataDog/profiling-python
tests/profiling/test_profiler.py                                        @DataDog/profiling-python
tests/profiling/test_scheduler.py                                       @DataDog/profiling-python
tests/profiling_v2/collector/test_asyncio.py                            @DataDog/profiling-python
tests/profiling_v2/collector/test_stack.py                              @DataDog/profiling-python
tests/profiling_v2/collector/test_threading.py                          @DataDog/profiling-python
tests/profiling_v2/exporter/test_ddup.py                                @DataDog/profiling-python
tests/profiling_v2/test_accuracy.py                                     @DataDog/profiling-python
tests/profiling_v2/test_profiler.py                                     @DataDog/profiling-python
tests/profiling_v2/test_scheduler.py                                    @DataDog/profiling-python
tests/smoke_test.py                                                     @DataDog/apm-core-python
tests/telemetry/test_writer.py                                          @DataDog/apm-python
tests/profiling/collector/pprof.proto                                   @DataDog/profiling-python
tests/profiling/collector/pprof_312_pb2.py                              @DataDog/profiling-python
tests/profiling/collector/pprof_319_pb2.py                              @DataDog/profiling-python
tests/profiling/collector/pprof_3_pb2.py                                @DataDog/profiling-python
tests/profiling/collector/pprof_421_pb2.py                              @DataDog/profiling-python

@taegyunkim taegyunkim added the Profiling Continous Profling label Jun 5, 2025
@taegyunkim taegyunkim changed the title feat(profiling): enable libdatadog exporter by default refactor(profiling): enable libdatadog exporter by default Jun 5, 2025
Copy link
Contributor

github-actions bot commented Jun 5, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 271 ± 2 ms.

The average import time from base is: 271 ± 2 ms.

The import time difference between this PR and base is: -0.43 ± 0.09 ms.

Import time breakdown

The following import paths have grown:

ddtrace.auto 0.668 ms (0.25%)
ddtrace.bootstrap.sitecustomize 0.668 ms (0.25%)
ddtrace.bootstrap.preload 0.536 ms (0.20%)
ddtrace.settings.profiling 0.222 ms (0.08%)
ddtrace.vendor.psutil 0.164 ms (0.06%)
ddtrace.vendor.psutil._pslinux 0.083 ms (0.03%)
ddtrace.internal.datadog.profiling.ddup 0.058 ms (0.02%)
ddtrace.internal.datadog.profiling.ddup._ddup 0.058 ms (0.02%)
ddtrace.internal.flare.flare 0.123 ms (0.05%)
logging.handlers 0.123 ms (0.05%)
ddtrace.internal.remoteconfig._connectors 0.103 ms (0.04%)
ctypes 0.103 ms (0.04%)
_ctypes 0.103 ms (0.04%)
multiprocessing 0.087 ms (0.03%)
multiprocessing.context 0.087 ms (0.03%)
ddtrace.appsec._common_module_patches 0.070 ms (0.03%)
ddtrace.appsec._metrics 0.070 ms (0.03%)
ddtrace._trace.trace_handlers 0.061 ms (0.02%)

The following import paths have shrunk:

ddtrace.auto 2.331 ms (0.86%)
ddtrace.bootstrap.sitecustomize 1.668 ms (0.62%)
ddtrace.bootstrap.preload 1.626 ms (0.60%)
ddtrace.internal.remoteconfig.client 0.600 ms (0.22%)
ddtrace.internal.remoteconfig.constants 0.073 ms (0.03%)
multiprocessing.sharedctypes 0.501 ms (0.18%)
multiprocessing.heap 0.501 ms (0.18%)
mmap 0.501 ms (0.18%)
ddtrace.settings.profiling 0.200 ms (0.07%)
ddtrace.vendor.psutil 0.065 ms (0.02%)
ddtrace.vendor.psutil._pslinux 0.065 ms (0.02%)
glob 0.065 ms (0.02%)
ddtrace.internal.datadog.profiling.stack_v2 0.064 ms (0.02%)
ddtrace.internal.datadog.profiling.stack_v2._stack_v2 0.064 ms (0.02%)
ddtrace.internal.flare.flare 0.104 ms (0.04%)
multiprocessing 0.079 ms (0.03%)
multiprocessing.context 0.079 ms (0.03%)
multiprocessing.process 0.079 ms (0.03%)
ddtrace.settings.dynamic_instrumentation 0.077 ms (0.03%)
ddtrace.internal.remoteconfig._connectors 0.065 ms (0.02%)
ctypes 0.065 ms (0.02%)
ddtrace._trace.trace_handlers 0.042 ms (0.02%)
ddtrace._trace._inferred_proxy 0.042 ms (0.02%)
ddtrace 0.663 ms (0.24%)
ddtrace.internal._unpatched 0.021 ms (0.01%)

@pr-commenter
Copy link

pr-commenter bot commented Jun 5, 2025

Benchmarks

Benchmark execution time: 2025-06-06 21:06:14

Comparing candidate commit 6ef6e41 in PR branch taegyunkim/prof-11194-remove-exporters with baseline commit f163540 in branch main.

Found 0 performance improvements and 4 performance regressions! Performance is the same for 553 metrics, 7 unstable metrics.

scenario:iastaspects-upper_aspect

  • 🟥 execution_time [+200.147ns; +247.890ns] or [+8.913%; +11.039%]

scenario:iastaspectsospath-ospathnormcase_aspect

  • 🟥 execution_time [+386.160ns; +432.964ns] or [+11.231%; +12.593%]

scenario:iastaspectsospath-ospathsplitdrive_aspect

  • 🟥 execution_time [+329.407ns; +388.811ns] or [+9.034%; +10.663%]

scenario:iastaspectsospath-ospathsplitext_aspect

  • 🟥 execution_time [+559.631ns; +669.043ns] or [+12.451%; +14.886%]

Copy link
Contributor

@nsrip-dd nsrip-dd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳 So much deleted code! This is pretty big so I may have missed something, but it seems like this is mostly just deleting the non-native exporter code. Left one non-blocking suggestion for the release note.

@taegyunkim taegyunkim changed the title refactor(profiling): enable libdatadog exporter by default refactor(profiling): remove legacy Python exporter and enable libdatadog exporter by default Jun 6, 2025
@taegyunkim taegyunkim merged commit 10e09e0 into main Jun 8, 2025
773 of 775 checks passed
@taegyunkim taegyunkim deleted the taegyunkim/prof-11194-remove-exporters branch June 8, 2025 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Profiling Continous Profling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants