-
Notifications
You must be signed in to change notification settings - Fork 14.5k
KAFKA-18960: The default type of ssh key used by vagrant is disabled … #19264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: trunk
Are you sure you want to change the base?
Conversation
Due to the limitations of my local hardware resources, I wasn't able to run all the tests. For now, I've selected and run multiple tests from different modules to verify that the adjusted tests can execute properly. Below are the relevant results. Please let me know if anything needs improvement or if additional test results are required.
|
A label of 'needs-attention' was automatically added to this PR in order to raise the |
…d-by-vagrant-is-disabled-by-OpenSSH-8.8
A label of 'needs-attention' was automatically added to this PR in order to raise the |
…d-by-vagrant-is-disabled-by-OpenSSH-8.8
A label of 'needs-attention' was automatically added to this PR in order to raise the |
…d-by-vagrant-is-disabled-by-OpenSSH-8.8
A label of 'needs-attention' was automatically added to this PR in order to raise the |
tests/README.md
Outdated
@@ -166,6 +166,7 @@ https://cwiki.apache.org/confluence/display/KAFKA/tutorial+-+set+up+and+run+Kafk | |||
$ virtualenv -p python3 venv | |||
$ . ./venv/bin/activate | |||
$ python3 -m pip install --editable . | |||
$ pip3 install setuptools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could setuptools
be added to setup.py
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your review.
I have updated setup.py
to declare setuptools
as a dependency. After making this change, I verified that setuptools
is installed automatically by running:
cd kafka/tests \
&& rm -rf venv \
&& virtualenv -p python3 venv \
&& . ./venv/bin/activate \
&& python3 -m pip install --editable .
Then:
➜ pip3 freeze | grep setuptools
setuptools==78.1.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also re-ran some tests to ensure this new change didn’t introduce issues. PTAL, Thanks!
=================================================================================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.12.0
session_id: 2025-04-25--006
run time: 25.012 seconds
tests run: 1
passed: 1
flaky: 0
failed: 0
ignored: 0
=================================================================================================================================================
test_id: kafkatest.tests.client.pluggable_test.PluggableConsumerTest.test_start_stop.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 24.900 seconds
-------------------------------------------------------------------------------------------------------------------------------------------------
==================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.12.0
session_id: 2025-04-25--008
run time: 1 minute 36.449 seconds
tests run: 1
passed: 1
flaky: 0
failed: 0
ignored: 0
==================================================================================================================================================================================================================================
test_id: kafkatest.tests.client.compression_test.CompressionTest.test_compressed_topic.compression_types=.snappy.gzip.lz4.zstd.none.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 1 minute 36.336 seconds
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SESSION REPORT (ALL TESTS)
ducktape version: 0.12.0
session_id: 2025-04-25--004
run time: 4 minutes 32.461 seconds
tests run: 8
passed: 8
flaky: 0
failed: 0
ignored: 0
========================================================================================================================================================================================================================================================
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_describe_consumer_group.security_protocol=PLAINTEXT.metadata_quorum=ISOLATED_KRAFT.group_protocol=classic
status: PASS
run time: 30.493 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_describe_consumer_group.security_protocol=PLAINTEXT.metadata_quorum=ISOLATED_KRAFT.group_protocol=consumer
status: PASS
run time: 32.633 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_describe_consumer_group.security_protocol=SSL.metadata_quorum=ISOLATED_KRAFT.group_protocol=classic
status: PASS
run time: 38.879 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_describe_consumer_group.security_protocol=SSL.metadata_quorum=ISOLATED_KRAFT.group_protocol=consumer
status: PASS
run time: 38.867 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_list_consumer_groups.security_protocol=PLAINTEXT.metadata_quorum=ISOLATED_KRAFT.group_protocol=classic
status: PASS
run time: 30.817 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_list_consumer_groups.security_protocol=PLAINTEXT.metadata_quorum=ISOLATED_KRAFT.group_protocol=consumer
status: PASS
run time: 29.089 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_list_consumer_groups.security_protocol=SSL.metadata_quorum=ISOLATED_KRAFT.group_protocol=classic
status: PASS
run time: 35.645 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.consumer_group_command_test.ConsumerGroupCommandTest.test_list_consumer_groups.security_protocol=SSL.metadata_quorum=ISOLATED_KRAFT.group_protocol=consumer
status: PASS
run time: 35.149 seconds
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
=================================================================================================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.12.0
session_id: 2025-04-25--005
run time: 1 minute 12.090 seconds
tests run: 2
passed: 2
flaky: 0
failed: 0
ignored: 0
=================================================================================================================================================
test_id: kafkatest.tests.core.controller_mutation_quota_test.ControllerMutationQuotaTest.test_controller_mutation_quota.metadata_quorum=COMBINED_KRAFT
status: PASS
run time: 30.361 seconds
-------------------------------------------------------------------------------------------------------------------------------------------------
test_id: kafkatest.tests.core.controller_mutation_quota_test.ControllerMutationQuotaTest.test_controller_mutation_quota.metadata_quorum=ISOLATED_KRAFT
status: PASS
run time: 41.500 seconds
-------------------------------------------------------------------------------------------------------------------------------------------------
…d-by-vagrant-is-disabled-by-OpenSSH-8.8
…d-by-vagrant-is-disabled-by-OpenSSH-8.8
Summary
This change modernizes the Vagrant configuration used in our system
tests to address compatibility issues with OpenSSH 8.8, which has
disabled the default SSH key type used by Vagrant.
Description
JIRA
The existing environment uses Ubuntu 14.04, which comes with an older
version of Paramiko. This
outdated setup causes issues when performing SSH agent forwarding on
modern systems. For example, on newer hosts using OpenSSH 8.8+,
authentication errors occur due to missing support for
rsa-sha2-256
andrsa-sha2-512
signature algorithms, orbackground retries may happen repeatedly and fail silently. A sample
error encountered is shown below:
Upgrading the OS allows us to use a newer version of Paramiko by
default, which includes built-in support for these signature
algorithms
This ensures compatibility with modern SSH implementations and prevents
connection issues due to requirements in OpenSSH 8.8+.
As part of the OS upgrade, several outdated components were also
updated. These changes include updated package installation commands and
the replacement of legacy NTP utilities with Chrony.
Additionally, the current test setup has a dependency on
distutils
, which is no longer included by default in some modern Python distributions. Instead of requiring users to manually install setuptools, this PR modifiessetup.py
to handle this requirement automatically.Updates
Base Box: Changed the Vagrant base box from
ubuntu/trusty64
toubuntu/jammy64
to ensure support for newer environments.Package Installation: Updated the package installation commands in
the Vagrant base script by replacing deprecated packages.
Time Synchronization: Replaced the legacy
ntpdate/ntp
setup withChrony
for time synchronization, aligning with modern best practices.Modified
setup.py
to automatically installsetuptools
Reviewers: Chia-Ping Tsai [email protected]