Skip to content

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Dec 16, 2022

What does this PR do?

This PR is to add latency config parameter into aws-cloudwatch input. During testing, I found we are also missing the number_of_workers parameter too.

Here is what the agent policy looks like after installing the integration with specifying latency:

inputs:
  - id: aws-cloudwatch-aws_logs-bcb0ad28-0671-41af-bc9a-9657b6d40f3e
    name: aws_logs-1
    revision: 1
    type: aws-cloudwatch
    use_output: default
    meta:
      package:
        name: aws_logs
        version: 0.3.2
    data_stream:
      namespace: default
    streams:
      - id: aws-cloudwatch-aws_logs.generic-bcb0ad28-0671-41af-bc9a-9657b6d40f3e
        data_stream:
          dataset: aws_logs.generic
        log_group_arn: >-
          arn:aws:logs:eu-west-1:123456789:log-group:/aws/lambda/apm-test-api-gateway-v1:*
        start_position: beginning
        scan_frequency: 1m
        api_sleep: 200ms
        latency: 30m
        access_key_id: a
        secret_access_key: b
        session_token: c
        tags:
          - forwarded
        publisher_pipeline.disable_host: true

and here is what the agent policy looks like with specifying number_of_workers:

inputs:
  - id: aws-cloudwatch-aws_logs-16342400-df06-4e9e-8ef6-9f163c25ef69
    name: aws_logs-1
    revision: 1
    type: aws-cloudwatch
    use_output: default
    meta:
      package:
        name: aws_logs
        version: 0.3.5
    data_stream:
      namespace: default
    streams:
      - id: aws-cloudwatch-aws_logs.generic-16342400-df06-4e9e-8ef6-9f163c25ef69
        data_stream:
          dataset: aws_logs.generic
        log_group_name_prefix: /kaiyan/test
        number_of_workers: 5
        region_name: null
        start_position: beginning
        scan_frequency: 1m
        api_sleep: 200ms
        access_key_id: a
        secret_access_key: b
        session_token: c
        tags:
          - forwarded
        publisher_pipeline.disable_host: true

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues

Screenshot

Screen Shot 2022-12-16 at 2 54 06 PM

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner December 16, 2022 20:20
@elasticmachine
Copy link

elasticmachine commented Dec 16, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-12-16T21:59:15.895+0000

  • Duration: 13 min 58 sec

Test stats 🧪

Test Results
Failed 0
Passed 2
Skipped 0
Total 2

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚 2.251
Classes 100.0% (0/0) 💚 2.251
Methods 66.667% (2/3) 👎 -24.832
Lines 100.0% (0/0) 💚 8.341
Conditionals 100.0% (0/0) 💚

@kaiyan-sheng kaiyan-sheng self-assigned this Dec 16, 2022
@kaiyan-sheng kaiyan-sheng added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Dec 16, 2022
@tommyers-elastic tommyers-elastic merged commit 5cb3b6f into elastic:main Dec 19, 2022
@elasticmachine
Copy link

Package aws_logs - 0.3.1 containing this change is available at https://epr.elastic.co/search?package=aws_logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Cloud-Monitoring Label for the Cloud Monitoring team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[AWS Custom Logs] Missing latency config parameter for aws-cloudwatch input

4 participants