Rename service metrics to service transaction metrics

[carsonip]

Motivation/summary

Rename service metrics to service transaction metrics, including all internal and external usages.

External facing changes:

• data stream name change: metrics-apm.service-${interval} -> metrics-apm.service_transaction-${interval}
• field name change: service.aggregation.overflow_count -> service_transaction.aggregation.overflow_count
• metricset name change: service -> service_transaction
• config field name apm-server.aggregation.service -> apm-server.aggregation.service_transactions

Checklist

• Update CHANGELOG.asciidoc
• Update package changelog.yml (only if changes to apmpackage have been made)
• Documentation has been updated

How to test these changes

N/A

Related issues

Part of #9654
Closes #9654

[mergify]

This pull request does not have a backport label. Could you fix it @carsonip? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-7.x is the label to automatically backport to the 7.x branch.
• backport-7./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[ghost]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-01-25T08:41:22.617+0000

• Duration: 11 min 14 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate and publish the docker images.

• /test windows : Build & tests on Windows.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[ghost]

📚 Go benchmark report

Diff with the main branch

goos: linux
goarch: amd64
pkg: github.com/elastic/apm-server/internal/agentcfg
cpu: 12th Gen Intel(R) Core(TM) i5-12500
                                  │ build/main/bench.out │              bench.out               │
                                  │        sec/op        │    sec/op      vs base               │
FetchAndAdd/FetchFromCache-12               41.07n ± ∞ ¹    46.12n ± ∞ ¹  +12.30% (p=0.008 n=5)
FetchAndAdd/FetchAndAddToCache-12           94.85n ± ∞ ¹   105.10n ± ∞ ¹  +10.81% (p=0.008 n=5)
geomean                                     62.41n          69.62n        +11.55%
¹ need >= 6 samples for confidence interval at level 0.95

                                  │ build/main/bench.out │              bench.out              │
                                  │         B/op         │    B/op      vs base                │
geomean                                                ³                +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

                                  │ build/main/bench.out │              bench.out              │
                                  │      allocs/op       │  allocs/op   vs base                │
geomean                                                ³                +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

pkg: github.com/elastic/apm-server/internal/beater/request
                                             │ build/main/bench.out │              bench.out              │
                                             │        sec/op        │    sec/op     vs base               │
ContextResetContentEncoding/empty-12                   122.3n ± ∞ ¹   136.2n ± ∞ ¹  +11.37% (p=0.008 n=5)
ContextResetContentEncoding/uncompressed-12            144.0n ± ∞ ¹   162.2n ± ∞ ¹  +12.64% (p=0.008 n=5)
geomean                                                919.9n         914.0n         -0.64%
¹ need >= 6 samples for confidence interval at level 0.95

                                             │ build/main/bench.out │               bench.out               │
                                             │         B/op         │     B/op       vs base                │
geomean                                                           ³                  +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

                                             │ build/main/bench.out │              bench.out              │
                                             │      allocs/op       │  allocs/op   vs base                │
geomean                                                           ³                +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

pkg: github.com/elastic/apm-server/internal/publish
             │ build/main/bench.out │             bench.out             │
             │        sec/op        │   sec/op     vs base              │
Publisher-12            1.004 ± ∞ ¹   1.003 ± ∞ ¹  -0.08% (p=0.016 n=5)
¹ need >= 6 samples for confidence interval at level 0.95

             │ build/main/bench.out │           bench.out            │
             │         B/op         │     B/op       vs base         │
¹ need >= 6 samples for confidence interval at level 0.95

             │ build/main/bench.out │           bench.out           │
             │      allocs/op       │  allocs/op    vs base         │
¹ need >= 6 samples for confidence interval at level 0.95

pkg: github.com/elastic/apm-server/x-pack/apm-server/aggregation/spanmetrics
                 │ build/main/bench.out │           bench.out           │
                 │        sec/op        │    sec/op     vs base         │
¹ need >= 6 samples for confidence interval at level 0.95

                 │ build/main/bench.out │            bench.out             │
                 │         B/op         │     B/op       vs base           │
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                 │ build/main/bench.out │           bench.out            │
                 │      allocs/op       │  allocs/op   vs base           │
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

pkg: github.com/elastic/apm-server/x-pack/apm-server/aggregation/txmetrics
                        │ build/main/bench.out │             bench.out              │
                        │        sec/op        │    sec/op     vs base              │
AggregateTransaction-12           77.65n ± ∞ ¹   76.79n ± ∞ ¹  -1.11% (p=0.016 n=5)
¹ need >= 6 samples for confidence interval at level 0.95

                        │ build/main/bench.out │           bench.out            │
                        │         B/op         │    B/op      vs base           │
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                        │ build/main/bench.out │           bench.out            │
                        │      allocs/op       │  allocs/op   vs base           │
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

pkg: github.com/elastic/apm-server/x-pack/apm-server/sampling
               │ build/main/bench.out │             bench.out              │
               │        sec/op        │    sec/op     vs base              │
geomean                  653.2n         653.6n        +0.07%
¹ need >= 6 samples for confidence interval at level 0.95

               │ build/main/bench.out │               bench.out               │
               │         B/op         │     B/op       vs base                │
geomean                             ³                  +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

               │ build/main/bench.out │              bench.out              │
               │      allocs/op       │  allocs/op   vs base                │
geomean                             ³                +0.00%               ³
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal
³ summaries must be >0 to compute geomean

pkg: github.com/elastic/apm-server/x-pack/apm-server/sampling/eventstorage
                                            │ build/main/bench.out │               bench.out               │
                                            │        sec/op        │    sec/op      vs base                │
WriteTransaction/json_codec-12                        4.080µ ± ∞ ¹   11.016µ ± ∞ ¹  +170.00% (p=0.008 n=5)
WriteTransaction/json_codec_big_tx-12                 4.897µ ± ∞ ¹   11.584µ ± ∞ ¹  +136.55% (p=0.008 n=5)
ReadEvents/json_codec/0_events-12                     316.2n ± ∞ ¹    354.8n ± ∞ ¹   +12.21% (p=0.024 n=5)
ReadEvents/json_codec_big_tx/0_events-12              310.5n ± ∞ ¹    352.4n ± ∞ ¹   +13.49% (p=0.016 n=5)
ReadEvents/json_codec_big_tx/1_events-12              12.01µ ± ∞ ¹    13.10µ ± ∞ ¹    +9.10% (p=0.032 n=5)
ReadEvents/nop_codec/0_events-12                      307.5n ± ∞ ¹    345.4n ± ∞ ¹   +12.33% (p=0.008 n=5)
ReadEvents/nop_codec_big_tx/0_events-12               314.2n ± ∞ ¹    354.8n ± ∞ ¹   +12.92% (p=0.008 n=5)
IsTraceSampled/sampled-12                             69.48n ± ∞ ¹    76.53n ± ∞ ¹   +10.15% (p=0.008 n=5)
IsTraceSampled/unsampled-12                           71.00n ± ∞ ¹    78.69n ± ∞ ¹   +10.83% (p=0.008 n=5)
IsTraceSampled/unknown-12                             373.3n ± ∞ ¹    416.7n ± ∞ ¹   +11.63% (p=0.008 n=5)
geomean                                               28.89µ          32.70µ         +13.19%
¹ need >= 6 samples for confidence interval at level 0.95

                                            │ build/main/bench.out │               bench.out                │
                                            │         B/op         │      B/op       vs base                │
WriteTransaction/json_codec-12                       2.928Ki ± ∞ ¹    2.930Ki ± ∞ ¹  +0.07% (p=0.048 n=5)
ReadEvents/nop_codec/399_events-12                   876.7Ki ± ∞ ¹    875.4Ki ± ∞ ¹  -0.14% (p=0.016 n=5)
ReadEvents/nop_codec_big_tx/399_events-12            875.5Ki ± ∞ ¹    877.3Ki ± ∞ ¹  +0.20% (p=0.016 n=5)
geomean                                              31.41Ki          31.40Ki        -0.02%
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

                                            │ build/main/bench.out │              bench.out               │
                                            │      allocs/op       │  allocs/op    vs base                │
geomean                                                144.7          144.7        -0.00%
¹ need >= 6 samples for confidence interval at level 0.95
² all samples are equal

report generated with https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

[simitt]

LGTM, please wait for confirmation from the UI team before merging.

[carsonip]

@dgieselaar @sqren This renaming PR should be replacing all occurrences of service metrics related names to service transaction metrics. And this will require corresponding changes on the UI side. What do you think about this change?

External facing changes include:

• data stream name change: metrics-apm.service-${interval} -> metrics-apm.service_transaction-${interval}
• field name change: service.aggregation.overflow_count -> service_transaction.aggregation.overflow_count
• metricset name change: service -> service_transaction
• config field name apm-server.aggregation.service -> apm-server.aggregation.service_transactions

cc @simitt

[dgieselaar]

@carsonip lgtm. Does the field name impact anything else? I think we have a policy setting related to the recording of service metrics

[carsonip]

@carsonip lgtm. Does the field name impact anything else? I think we have a policy setting related to the recording of service metrics

Hmm I'm not aware of that. What policy are we talking about? @simitt do you have any idea?

[dgieselaar]

@carsonip Fleet

[simitt]

@dgieselaar do you mean the setting in the Fleet UI to enable/disable it?

[dgieselaar]

@simitt yes

[simitt]

Sounds good, that's basically why we pinged you. Can you trigger the changes on the UI side? We are going to merge this PR in then.

[dgieselaar]

@simitt what kind of changes would that be? I am not sure how a settings key gets translated to an APM Server config setting. @sqren do you know?

[mergify]

This pull request is now in conflicts. Could you fix it @carsonip? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b rename-service-transaction upstream/rename-service-transaction
git merge upstream/main
git push upstream rename-service-transaction

[sorenlouv]

I am not sure how a settings key gets translated to an APM Server config setting. @sqren do you know?

Not from the top of my head, no.

[carsonip]

@simitt I'm not sure if I completely understand the convo above. Did @dgieselaar want to know impact of the config key change, but our understanding is that in reality that config was never exposed in the UI anyway? Is that right?

[dgieselaar]

@carsonip looks like the key and its interpretation is entirely in APM Server, so no concerns from me: https://github.com/search?q=org%3Aelastic+service_metrics_enabled&type=code

[carsonip]

@carsonip looks like the key and its interpretation is entirely in APM Server, so no concerns from me: https://github.com/search?q=org%3Aelastic+service_metrics_enabled&type=code

I believe you actually found something that we should remove. That option is no longer useful and service (transaction) metrics is always enabled since #10048. I'll go ahead and remove that from apmpackage. Thanks!

Update: removing the policy input var in #10109

[simitt]

@carsonip the PR is good to merge then once all checks are green.