From fea0fd884ef250d16b97b7c5e26a4c8f10567f6b Mon Sep 17 00:00:00 2001 From: Ed Santiago Date: Tue, 20 Feb 2018 11:56:32 -0700 Subject: [PATCH 1/5] Ed's podman_fixes branch --- docker_daemon.pid() and .cmdline(): handle podman ...that is, handle the situation where dockerd is not running. --- add special-case exit codes for podman Detect if we're running podman (vs docker), which behaves slightly differently in some conditions. Change the expected exit codes: podman often exits 125 where docker exits 1. Also, skip some tests that will never work in podman. --- add a warning when we can't find pid --- new subtests for podman mount --- Try a different way of handling podman. When testing podman, and looking up config settings, for any given 'docker_xxx' key see if there's a 'podman_xxx' and use it instead. --- More podman changes This is a placeholder commit for more podman-specific changes; the purpose is to make it easier to review smaller changes --- login (and push) test: handle podman Basically: disable TLS verification, and use explicit auth file path podman performs TLS verification by default; that won't work on a self-generated and unsigned cert. podman also uses a different auth file by default, and of course different exit codes / error messages. Also: whoops, add a missing mustpass() and OutputGood() in logout. Also: remove the --email option, let's assume that docker-autotest is no longer usable on docker 1.9 or below. Also: fix push tests (expected stdout, stderr, exit codes). --- selinux_labels test: work with podman Skip tests that check for dockerd or containerd when testing podman; we detect by adding a 'podman-V-R' string to the 'version' output in our filter --- podman: deal with inconsistent 'sha256:' prefix --- DO NOT MERGE! This is the evil unclean commit. A placeholder for debugging printfs, for hardcoded podmanisms that Ed hasn't yet figured out how or if to integrate cleanly. --- DO NOT MERGE - podman documentation, ansible stuff Quick README describing how to use docker-autotest to test podman. Includes ansible playbooks, all of which are duplicates of ones that Ed has checked in elsewhere. This is a crap commit. I am holding my nose for the sake of expediency. checkpoint: merge with other ansible commit --- disable tests that require RHEL subscriptions --- docker version test: skip for podman podman has no daemon; don't try the socket-connect thing --- dockerhelp: remove cp and events (not implemented in podman) find a clean way to do this such that we can coexist with docker --- another podman workaround: strip colons from filename ...libpod issue #489: podman can't deal with colons in filenames --- run_cgroups - support for podman Kept as a separate commit because it's a little more complicated than the others, and because it includes some refactoring of unnecessary code. Signed-off-by: Ed Santiago --- cgroup_parent: force cgroup-manager=systemd (default is currently cgroups; mheon has indicated that the default will be switching back to systemd, but it isn't yet. Once it is, please remove this commit) Signed-off-by: Ed Santiago --- README-podman.md | 167 ++++++++++++++++++ .../_copr_baude-Upstream_CRIO_Family.repo | 10 ++ ansible-files/control.ini | 25 +++ ansible-files/defaults.ini | 15 ++ ansible-files/nginx-default.conf | 3 + ansible-files/tests.ini | 54 ++++++ config_custom/known_failures.txt | 49 +++++ config_defaults/subtests/docker_cli/build.ini | 16 +- .../subtests/docker_cli/create.ini | 2 +- .../subtests/docker_cli/dockerhelp.ini | 2 - .../subtests/docker_cli/kill_bad.ini | 2 + config_defaults/subtests/docker_cli/login.ini | 20 +++ config_defaults/subtests/docker_cli/mount.ini | 2 + .../subtests/docker_cli/negativeusage.ini | 88 ++++----- config_defaults/subtests/docker_cli/pull.ini | 2 + config_defaults/subtests/docker_cli/rmi.ini | 1 + config_defaults/subtests/docker_cli/run.ini | 2 +- .../subtests/docker_cli/run_cgroup_parent.ini | 4 +- .../subtests/docker_cli/run_cgroups.ini | 6 - .../subtests/docker_cli/run_cidfile.ini | 3 + .../subtests/docker_cli/run_exec.ini | 3 +- dockertest/config.py | 18 ++ dockertest/docker_daemon.py | 8 + dockertest/dockercmd.py | 4 + dockertest/output/dockerversion.py | 11 ++ kpod-filter | 146 +++++++++++++++ setup-podman-autotest.yml | 142 +++++++++++++++ subtests/docker_cli/attach/attach.py | 19 +- subtests/docker_cli/build/build.py | 11 ++ subtests/docker_cli/commit/commit.py | 5 +- .../docker_cli/create/create_remote_tag.py | 5 +- subtests/docker_cli/create/create_signal.py | 5 +- subtests/docker_cli/dockerimport/empty.py | 5 +- subtests/docker_cli/dockerimport/truncated.py | 8 +- .../docker_cli/dockerinspect/dockerinspect.py | 8 +- subtests/docker_cli/events/events.py | 5 +- subtests/docker_cli/images_all/images_all.py | 10 +- subtests/docker_cli/import_url/import_url.py | 18 +- subtests/docker_cli/iptable/iptable.py | 6 +- subtests/docker_cli/kill/kill_utils.py | 4 +- subtests/docker_cli/kill_bad/kill_bad.py | 6 +- subtests/docker_cli/login/login.py | 42 +++-- subtests/docker_cli/mount/mount.py | 138 +++++++++++++++ .../docker_cli/negativeusage/negativeusage.py | 6 +- subtests/docker_cli/pull/pull.py | 8 +- subtests/docker_cli/rmi/rmi.py | 6 +- .../run_cgroup_parent/run_cgroup_parent.py | 26 ++- .../docker_cli/run_cgroups/cgroups_base.py | 21 ++- subtests/docker_cli/run_cgroups/cpushares.py | 6 +- subtests/docker_cli/run_cgroups/memory.py | 34 +--- .../docker_cli/run_cidfile/run_cidfile.py | 7 +- subtests/docker_cli/run_exec/run_exec.py | 3 +- subtests/docker_cli/run_user/run_user.py | 6 +- subtests/docker_cli/selinux_labels/test.sh | 16 +- subtests/docker_cli/start/simple.py | 7 +- subtests/docker_cli/version/version.py | 4 + 56 files changed, 1078 insertions(+), 172 deletions(-) create mode 100644 README-podman.md create mode 100644 ansible-files/_copr_baude-Upstream_CRIO_Family.repo create mode 100644 ansible-files/control.ini create mode 100644 ansible-files/defaults.ini create mode 100644 ansible-files/nginx-default.conf create mode 100644 ansible-files/tests.ini create mode 100644 config_custom/known_failures.txt create mode 100644 config_defaults/subtests/docker_cli/mount.ini create mode 100755 kpod-filter create mode 100644 setup-podman-autotest.yml create mode 100644 subtests/docker_cli/mount/mount.py diff --git a/README-podman.md b/README-podman.md new file mode 100644 index 000000000..9c69dbf96 --- /dev/null +++ b/README-podman.md @@ -0,0 +1,167 @@ +Instructions for testing podman with docker-autotest +==================================================== + +2018-04-11 : This document describes how to set up docker-autotest +on a Fedora system, then run it against podman. + +*This is a work in progress*. In particular, much of it is +crappy code intended solely to work around docker/podman +incompatibilities, some is debugging code intended for figuring +out real failures, and some is workarounds for ways in which +podman and docker will presumably forever differ. I (Ed) reserve +the right to force-push new commits to this branch without notice. + +Set up a virt +------------- + +I (Ed) have done all my testing on f27, but f28 is worth testing too. + +Setting up the virt is left as an exercise for the reader. I suggest +creating one inside the Red Hat network: one of the `pulls` from +registry.access.redhat.com fails if run from outside, and you don't +want to waste time debugging that. + + +Configure it +------------ + +Check out this repo on a Fedora 27 host: + + $ cd /some/nice/directory/ + $ git clone --branch podman_fixes https://github.com/edsantiago/autotest-docker.git + $ cd autotest-docker + +Run this command: + + $ HOSTNAME=name-or-ip-address-of-your-virt + $ ansible-playbook -i $HOSTNAME, setup-podman-autotest.yml -v + ^--- do not omit the comma + +(I suggest Fedora 27 because ansible is unreliably chaotic: things that +work on one version don't work on the next, and vice-versa. I do all +my development on f27 and am not interested in testing any of this on +RHEL or Fedora Rawhide; at least not until ansible calms down). +(Also: maybe there's some way to run the ansible stuff on the same +virt, using localhost or something. I dunno. That's not my M.O. so +I'm not going to spend any time trying it. If you like working that way, +and can give me exact instructions for doing so, please do and I +will update this doc) + +This will take about 5 minutes and will leave your Fedora virt configured +to run docker-autotest. This command is safe to run on any system, +e.g. your laptop. It does not require root on the host nor does it +muck with the host system in any way, um, unless you screw up HOSTNAME. + +Run docker-autotest +------------------- + +Log in to your virt as `root`, then run: + + virt# cd /var/lib/autotest/client + virt# unset XDG_RUNTIME_DIR ! podman won't run if this is set + virt# ./autotest-local run docker + +This will take 25-35 minutes depending on the virt horsepower and +the version of podman (some earlier versions hang for a long time). + + +Install GreaseMonkey scripts +---------------------------- + +While docker-autotest is running, I strongly encourage you to install +my [docker-autotest-highlight](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-highlight) +and [docker-autotest-job-report](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-job-report) GreaseMonkey scripts. (You only need to do this once). + +They are not mandatory, but it would be insane to try to read +docker-autotest results without them. docker-autotest results +are horrible and unreadable; these tools make them somewhat less so. +Seriously: you will be in a world of hurt if you try to read +docker-autotest results without these. + +I'm not quite sure how to install into GM, honestly, but try these +direct links and lmk if they work: +[docker-autotest-highlight](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-highlight/raw/master/docker-autotest-highlight.user.js), +[docker-autotest-job-report](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-job-report/raw/master/docker-autotest-job-report.user.js). + +This should download the scriptlet, and GM will ask for confirmation. +Please let me know if that works (or doesn't), and please help me +improve this documentation. +(Installing GreaseMonkey is left as an exercise for the reader). + + +Review results +-------------- + +Point your web browser at `http://IP-ADDRESS-OF-VIRT/default/job_report.html` + +You will see a table of results, with poorly-chosen non-colorblind-friendly +green/red for GOOD/FAIL respectively. Click on the `Debug` link for any +result of interest: if you installed `docker-autotest-job-report`, this +takes you straight to the debug log; otherwise, you see a list of files +and have to click the right one. Sorry, I can't remember which one and +can't be bothered to check -- again, I recommend installing my GM scripts. + +Look for boldface red; that shows the actual failure. Sometimes the actual +error is in orange (warning), especially if you see 'Postprocess' in +orange. Those are magic tests that don't actually fail in the moment +and are hard to find. The `build` test is an example of this. + +(Note that some orange warnings are OK and expected. Sorry, but you'll +have to develop a sense for these yourself.) + +Debugging results +----------------- + +This is a nightmare. + +Find the source code of the test: for a given test `foo` it will +be under `subtests/docker_cli/foo/` but there may be many files +in there - often it might be `foo.py` but it could also be +`subsubtestname.py` for a given subsubtest. Or, for a kill-related +test, `kill_utils.py` + +In the py file, the sequence is: + + initialize() + run_once() + postprocess() + +It can be hard to know exactly what is failing or why. My usual +process is to look in the web browser for **boldface**; this +indicates the docker commands being run. I'll see if I can +figure out what's going on just by looking at the docker commands. +Most of the time I can't, and I have to delve into the source +code. Here, again, you're on your own. Each subtest is an +unhappy family in its own unique way, and there's not really +any way I can prepare you for debugging any particular +failing test. Sorry. + +Also note that I've already picked off the low-hanging fruit: any +subtest that can be easily figured out, I've already filed podman +issues for. The remaining failures are the hard ones, those that +are not immediately obvious from the debug log, and require +delving deep into the code. Again -- sorry. + +Rerunning tests +--------------- + +First: *save your run*: + + virt# mv results/default results/$(date --iso-8601).01.full + (or .02.full or .03.foo-test or whatever) + +This way you can revisit in your web browser. + +If you want to run just one short subtest, use --args: + + virt# ./autotest-local run docker --args docker_cli/foobar + (replace foobar with attach, images_all, build, etc) + +Short subtests can finish in 1-3 minutes instead of 20. You can +use this to instrument the code (adding printfs) then reload +in your web browser to view results. + +Happy hacking. And please, please: don't do it alone. Whatever +stumbling block you run into, I've almost certainly already +encountered. Ping me (esm) on IRC, or send me email. Even if you +cost me three minutes, it's in order to save you thirty. It's worth it. diff --git a/ansible-files/_copr_baude-Upstream_CRIO_Family.repo b/ansible-files/_copr_baude-Upstream_CRIO_Family.repo new file mode 100644 index 000000000..a21228f8c --- /dev/null +++ b/ansible-files/_copr_baude-Upstream_CRIO_Family.repo @@ -0,0 +1,10 @@ +[baude-Upstream_CRIO_Family] +name=Copr repo for Upstream_CRIO_Family owned by baude +baseurl=https://copr-be.cloud.fedoraproject.org/results/baude/Upstream_CRIO_Family/fedora-$releasever-$basearch/ +type=rpm-md +skip_if_unavailable=True +gpgcheck=1 +gpgkey=https://copr-be.cloud.fedoraproject.org/results/baude/Upstream_CRIO_Family/pubkey.gpg +repo_gpgcheck=0 +enabled=1 +enabled_metadata=1 diff --git a/ansible-files/control.ini b/ansible-files/control.ini new file mode 100644 index 000000000..017ff562f --- /dev/null +++ b/ansible-files/control.ini @@ -0,0 +1,25 @@ +[Control] +include = +exclude = pretest_example,intratest_example,posttest_example,example,subexample + ,docker_cli/build/git_path + ,docker_cli/build/dockerignore + ,docker_cli/cp + ,docker_cli/info +subthings = +pretests = pretests +subtests = subtests +intratests = intratests +posttests = posttests + +[Bugzilla] +url = +username = +password = +excluded = +key_field = status_whiteboard +key_match = docker-autotest + +[Query] +product = Red Hat Enterprise Linux 7 +component = docker +status = NEW, ASSIGNED, POST, MODIFIED, ON_DEV diff --git a/ansible-files/defaults.ini b/ansible-files/defaults.ini new file mode 100644 index 000000000..ef49a0525 --- /dev/null +++ b/ansible-files/defaults.ini @@ -0,0 +1,15 @@ +[DEFAULTS] +config_version = 0.8.4 +autotest_version = @!NOVERSIONCHECK!@ +docker_path = /usr/bin/docker +docker_options = +daemon_options = daemon,--selinux-enabled,--storage-driver,devicemapper,--storage-opt,dm.fs=xfs,--storage-opt,dm.thinpooldev=/dev/mapper/atomicos-docker--pool,--icc=false,--add-registry,registry.access.redhat.com,--block-registry,docker.io, +docker_timeout = 600 +__example__ = docker_repo_name, docker_repo_tag, preserve_fqins, preserve_cnames +docker_repo_name = rhel +docker_repo_tag = latest +docker_registry_host = registry.access.redhat.com +docker_registry_user = rhel7 +remove_after_test = yes +preserve_fqins = docker.io/stackbrew/centos:7,docker.io/stackbrew/centos:centos7,docker.io/stackbrew/centos:latest,registry.access.redhat.com/rhel7:latest,registry.access.redhat.com/rhel7/rhel-tools:latest,registry-1.docker.io/distribution/registry:2.1 +preserve_cnames = ,dummy_container_name diff --git a/ansible-files/nginx-default.conf b/ansible-files/nginx-default.conf new file mode 100644 index 000000000..7f316158a --- /dev/null +++ b/ansible-files/nginx-default.conf @@ -0,0 +1,3 @@ +root /var/lib/autotest/client/results; + default_type text/plain; + autoindex on; diff --git a/ansible-files/tests.ini b/ansible-files/tests.ini new file mode 100644 index 000000000..2feda2404 --- /dev/null +++ b/ansible-files/tests.ini @@ -0,0 +1,54 @@ + + +# Note: This template will be overwritten by any job-specific template +# with the same dest name + +[docker_cli/create/create_remote_tag] +config_version=0.8.3 +remote_image_fqin = registry.access.redhat.com/rhel7/rhel:latest +run_options_csv = + +[docker_cli/pull] +config_version=0.8.3 +docker_pull_timeout = 600 +subsubtests = good,wrong_tag,wrong_registry_addr +docker_expected_result = PASS + +[docker_cli/run/run_ping] +config_version=0.8.3 +generate_generic = yes +ping_url = code.engineering.redhat.com +cmd = 'yum install -y iputils && ping -c 10 -q %(ping_url)s' +exit_status = 0 + +[docker_cli/run/run_remote_tag] +config_version=0.8.3 +remote_image_fqin = registry.access.redhat.com/rhel7/rhel:latest +run_options_csv = +cmd = /bin/true + +[docker_cli/build/git_path] +__example__ = dockerfile_dir_path, postproc_cmd_csv +use_config_repo = no +dockerfile_dir_path = git@gitlab.osas.lab.eng.rdu2.redhat.com:cevich/dockertest_dockerfiles.git +postproc_cmd_csv = positive(), + rx_out('^Schazam!$'), + cnt_count('0'), + img_count('2'), + img_exst(), + intr_exst(), + file_exist('/var/tmp/testfile'), + rx_file('/var/tmp/testfile:Schazam!') + +[docker_cli/build/https_file] +__example__ = dockerfile_dir_path, postproc_cmd_csv +use_config_repo = no +dockerfile_dir_path = http://gitlab.osas.lab.eng.rdu2.redhat.com/cevich/dockertest_dockerfiles/raw/master/Dockerfile +postproc_cmd_csv = positive(), + rx_out('^Schazam!$'), + cnt_count('0'), + img_count('2'), + img_exst(), + intr_exst(), + file_exist('/var/tmp/testfile'), + rx_file('/var/tmp/testfile:Schazam!') diff --git a/config_custom/known_failures.txt b/config_custom/known_failures.txt new file mode 100644 index 000000000..7b050619f --- /dev/null +++ b/config_custom/known_failures.txt @@ -0,0 +1,49 @@ +"Full NVRA, may be N-V-* as wildcard" "subsubtest path" comment describing the reason for this test exception +docker-1.12.5-4.el7.x86_64 foo/bar/fubar comment +docker-1.12.5-4.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah +docker-1.12.5-5.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah +docker-1.12.5-* i docker_cli/negativeusage/iv4 bz1393572: expect fix in docker-1.13? +docker-latest-1.12.5-5.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah +docker-latest-1.12.5-* docker_cli/negativeusage/iv4 bz1393572: expect fix in docker-1.13? +docker-1.12.3-10.el7.centos docker_cli/iptable/iptable_remove bz1406460: fixed in 1.12.5 +docker-1.12.3-10.el7.centos docker_cli/login/login_ok bz1406460: fixed in 1.12.5 +docker-1.12.3-10.el7.centos docker_cli/login/login_fail bz1406460: fixed in 1.12.5 +docker-1.12.3-10.el7.centos docker_cli/login/logout_ok bz1406460: fixed in 1.12.5 +docker-1.12.3-10.el7.centos docker_cli/login/push_ok bz1406460: fixed in 1.12.5 +docker-1.12.3-10.el7.centos docker_cli/login/push_fail bz1406460: fixed in 1.12.5 +docker-1.10.3-55.gite03ddb8.fc24 docker_cli/events No idea. Just ignoring. +docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run/run_install Bad repo? ADEPT bug? No idea, just ignoring. +docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_cgroup_parent/run_cgroup_parent_path No idea. Looks like a real bug, possibly kernel-related but I don't care. I just want to silence this. +docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_cgroup_parent/run_cgroup_parent_path_with_hyphens No idea. Looks like a real bug, possibly kernel-related but I don't care. I just want to silence this. +docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_exec/exec_pid_count Argh. Fedora docker image doesn't have 'find'. This probably needs to be fixed in the test itself but just ignore for now. +docker-1.12.6-49.1.hotfix.git90e29fe.el7.x86_64 docker_cli/run_volumes/oci_umount no idea +podman-0.3.2-* docker_cli/create/create_tmpfs "Key Error" - podman inspect doesn't show Tmpfs +podman-0.3.3-* docker_cli/create/create_remote_tag podman can't deal with colons in output filename +podman-0.3.5-* docker_cli/restart/stopped #575 stop an already-stopped container +podman-0.3.4-* docker_cli/kill/run_sigproxy_ttyoff sigproxy not implemented yet +podman-0.3.4-* docker_cli/kill/attach_sigproxy_ttyoff sigproxy not implemented yet +podman-0.3.4-* docker_cli/kill_stress/run_sigproxy_ttyoff sigproxy not implemented yet +podman-0.3.4-* docker_cli/kill_stress/attach_sigproxy_ttyoff sigproxy not implemented yet +podman-0.3.5-* docker_cli/run_dns DOS newlines in output +podman-0.3.3-* docker_cli/events events not implemented, and not likely to be +podman-0.3.4-* docker_cli/run_cgroup_parent/run_cgroup_parent_invalid_name FIXME: file issue on this +podman-0.3.2-* docker_cli/run_cgroup_parent/run_cgroup_parent_path cgroup-parent not implemented +podman-0.3.3-* docker_cli/rmi/only_tag issue #387 +podman-0.4.1-* docker_cli/rm/forced issue #527: unable to get exit code +podman-0.5.4-* docker_cli/images_all/two_images_with_parents #520 podman-history loses history +podman-0.5.4-* docker_cli/images_all/with_unused_containers #520 podman-history loses history +podman-0.4.1-* docker_cli/run/run_install no entitlements +podman-0.4.1-* docker_cli/run/run_update no entitlements +podman-0.4.1-* docker_cli/negativeusage/iv1 one of our dashes is missing +podman-0.4.1-* docker_cli/stop/force #389 exit status is 0 after signal +podman-0.3.5-* docker_cli/stop/stopped #575 stop an already-stopped container +podman-0.4.1-* docker_cli/stop/zerotime #389 exit status is 0 after signal +podman-0.4.1-* docker_cli/attach/simple #580 attach: There Can Be Only One +podman-0.4.1-* docker_cli/attach/no_stdin #580 attach: There Can Be Only One +podman-0.3.5-* docker_cli/kill/attach_sigproxy_ttyoff #580 attach: There Can Be Only One +podman-0.5.4-* docker_cli/commit/check_default_cmd #586 podman-commit doesn't create entrypoint +podman-0.4.4-* docker_cli/run_cgroups/cpu_overflow #633 typo in podman-inspect 'CpuShares' +podman-0.4.4-* docker_cli/load #639 podman prepends 'docker.io' to local image +podman-0.5.4-* docker_cli/build/expose #735 podman exposes both udp and tcp +podman-0.5.4-* docker_cli/build/onbuild buildah #658 ONBUILD not supported +podman-0.5.4-* docker_cli/create/create_remote_tag #649 podman save, multi-tag handling diff --git a/config_defaults/subtests/docker_cli/build.ini b/config_defaults/subtests/docker_cli/build.ini index 8171fc882..044b7107f 100644 --- a/config_defaults/subtests/docker_cli/build.ini +++ b/config_defaults/subtests/docker_cli/build.ini @@ -117,8 +117,8 @@ postproc_cmd_csv = negative(), !rx_out('foobar$'), !rx_err('Successfully built') rx_err('returned a non-zero code'), - cnt_count('1'), - img_count('1'), + cnt_count('docker=1;podman=0'), + img_count('docker=1;podman=0'), intr_exst('2'), last_cnt(), !img_exst() @@ -131,8 +131,8 @@ postproc_cmd_csv = negative(), !rx_out('foobar$'), !rx_err('Successfully built') rx_err('returned a non-zero code'), - cnt_count('1'), - img_count('1'), + cnt_count('docker=1;podman=0'), + img_count('docker=1;podman=0'), last_cnt(), !img_exst() @@ -145,7 +145,7 @@ postproc_cmd_csv = negative(), !rx_err('Successfully built') rx_err('returned a non-zero code'), cnt_count('0'), - img_count('1'), + img_count('docker=1;podman=0'), !img_exst() [docker_cli/build/rm_false] @@ -153,7 +153,7 @@ dockerfile_dir_path = /full docker_build_options = --rm=false,--no-cache=true,--quiet=false,--force-rm=false postproc_cmd_csv = positive(), rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), - cnt_count('5'), + cnt_count('docker=5;podman=0'), img_count('1'), img_exst(), intr_exst() @@ -176,9 +176,9 @@ dockerfile_dir_path2 = /full docker_build_options2 = --rm=true,--no-cache=false,--quiet=false,--force-rm=false #: Second build ``postproc_cmd_csv``. postproc_cmd_csv2 = positive(), - rx_out('Using cache') + rx_out('docker=Using cache;podman=') rx_out('\s*Successfully built\s*(\w{64}|\w{12})'), - !rx_out('Schazam!$'), + !rx_out('docker=Schazam!$;podman=Nonsense_String'), cnt_count('0'), img_count('2'), img_exst(), diff --git a/config_defaults/subtests/docker_cli/create.ini b/config_defaults/subtests/docker_cli/create.ini index a3c7f90b2..02368752e 100644 --- a/config_defaults/subtests/docker_cli/create.ini +++ b/config_defaults/subtests/docker_cli/create.ini @@ -26,7 +26,7 @@ run_options_csv = --tmpfs,%(tmpfs_arg)s [docker_cli/create/create_names] cmd = sleep 2s -run_options_csv = --net,none,--restart,always +run_options_csv = --net,none #: Select whether or not the final --name value is used or the first last_name_sticks = yes #: Number of --name options to pass to create command diff --git a/config_defaults/subtests/docker_cli/dockerhelp.ini b/config_defaults/subtests/docker_cli/dockerhelp.ini index 7c9778cd1..6963a5179 100644 --- a/config_defaults/subtests/docker_cli/dockerhelp.ini +++ b/config_defaults/subtests/docker_cli/dockerhelp.ini @@ -15,9 +15,7 @@ generate_subsubtest_list = yes help_commands = attach build commit - cp diff - events export history images diff --git a/config_defaults/subtests/docker_cli/kill_bad.ini b/config_defaults/subtests/docker_cli/kill_bad.ini index 0fbca081b..bf2da9735 100644 --- a/config_defaults/subtests/docker_cli/kill_bad.ini +++ b/config_defaults/subtests/docker_cli/kill_bad.ini @@ -10,3 +10,5 @@ exec_cmd = 'while :; do sleep 0.1; done' [docker_cli/kill_bad/bad] #: csv of bad signals which will be used in test bad_signals = 0,-1,-78,96,SIGBADSIGNAL,SIG,%%,!,\\,\,"''",'""', , +docker_exit_status = 1 +podman_exit_status = 125 diff --git a/config_defaults/subtests/docker_cli/login.ini b/config_defaults/subtests/docker_cli/login.ini index c15e25156..312f29ce0 100644 --- a/config_defaults/subtests/docker_cli/login.ini +++ b/config_defaults/subtests/docker_cli/login.ini @@ -10,5 +10,25 @@ push_image = busybox #: config file into which docker login writes credentials. This is not #: likely to change. docker_auth_file = /root/.docker/config.json +podman_auth_file = /root/.podman-auth.json +#: FIXME +docker_args_csv = +podman_args_csv = --tls-verify=false,--authfile,%(podman_auth_file)s #: maximum time, in seconds, to wait for registry container to become ready. wait_for_registry_ready = 10 + +[docker_cli/login/login_fail] +docker_exit_status = 1 +podman_exit_status = 125 +docker_error_message = 401 Unauthorized +podman_error_message = invalid username/password + +[docker_cli/login/push_ok] +docker_expected_stdout_csv = ": Pushed","tag_ok: digest:" +podman_expected_stdout_csv = + +[docker_cli/login/push_fail] +docker_expected_stderr = FIXME-FIXME-FIXME test this with docker +podman_expected_stderr = unauthorized: authentication required +docker_exit_status = 1 +podman_exit_status = 125 diff --git a/config_defaults/subtests/docker_cli/mount.ini b/config_defaults/subtests/docker_cli/mount.ini new file mode 100644 index 000000000..b2ba3b8fe --- /dev/null +++ b/config_defaults/subtests/docker_cli/mount.ini @@ -0,0 +1,2 @@ +[docker_cli/mount] +subsubtests = mount_by_name, mount_by_cid, mount_twice, mount_with_no_args, umount_umounts diff --git a/config_defaults/subtests/docker_cli/negativeusage.ini b/config_defaults/subtests/docker_cli/negativeusage.ini index 6f3187cb2..fb1f2e619 100644 --- a/config_defaults/subtests/docker_cli/negativeusage.ini +++ b/config_defaults/subtests/docker_cli/negativeusage.ini @@ -8,18 +8,18 @@ subsubtests = op1, op2, op3, if1, ov1, iv1, iv3, iv4, ip1 #: {t} * ``%%(STPCNTR)s`` **-** ID of a stopped container{n} #: {t} * ``%%(NAME)s`` **-** Name of a container that doesn't exist{n} #: {n} -subcmd = +docker_subcmd = #: Optional docker subcommand arguments (CSV), also accepts same substitutions -#: as ``subcmd``. -subarg = +#: as ``docker_subcmd``. +docker_subarg = #: Required exit status integer -extcmd = 125 +docker_exit_status = 125 #: Optional regex that must match to stderr (group #1), also accepts same substitutions -#: as ``subcmd``. -stderr = +#: as ``docker_subcmd``. +docker_stderr = #: Optional regex that must match to stdout (group #1), also accepts same substitutions -#: as ``subcmd``. -stdout = +#: as ``docker_subcmd``. +docker_stdout = #op: Omit Positional - leave out a required positional parameter #if: Invalid Flag - Addition of non-existing flag @@ -28,59 +28,65 @@ stdout = #ip: Invalid Positional - Give value to non-existing positional parameter [docker_cli/negativeusage/op1] -subcmd = attach -subarg = --no-stdin,--sig-proxy -stderr = "(docker )?attach" requires( exactly)? 1 argument -extcmd = 1 +docker_subcmd = attach +docker_subarg = --no-stdin,--sig-proxy +docker_stderr = "(docker )?attach" requires( exactly)? 1 argument +podman_stderr = attach requires the name or id of one running container +docker_exit_status = 1 +podman_exit_status = 125 [docker_cli/negativeusage/op2] -subcmd = run -subarg = --interactive,%%(FQIN)s,/usr/local/sbin/fail -stderr = no such file or directory -extcmd = 127 +docker_subcmd = run +docker_subarg = --interactive,%%(FQIN)s,/usr/local/sbin/fail +docker_stderr = no such file or directory +docker_exit_status = 127 [docker_cli/negativeusage/op3] -subcmd = run -subarg = --interactive,/usr/local/sbin/fail -stderr = Unable to find image|is not a valid repository/tag +docker_subcmd = run +docker_subarg = --interactive,/usr/local/sbin/fail +docker_stderr = Unable to find image|is not a valid repository/tag +podman_stderr = error parsing reference to image|invalid reference format [docker_cli/negativeusage/if1] -subcmd = commit -subarg = --authormessage,%%(STPCNTR)s,%%(NOFQIN)s -stderr = flag provided but not defined|unknown flag: --authormessage +docker_subcmd = commit +docker_subarg = --authormessage,%%(STPCNTR)s,%%(NOFQIN)s +docker_stderr = flag provided but not defined|unknown flag: --authormessage [docker_cli/negativeusage/ov1] -subcmd = load -subarg = --input -stderr = flag needs an argument +docker_subcmd = load +docker_subarg = --input +docker_stderr = flag needs an argument [docker_cli/negativeusage/iv1] -subcmd = attach -subarg = --no-stdin=sig-proxy -stderr = invalid boolean value "sig-proxy" for --no-stdin|invalid argument "sig-proxy" for --no-stdin=sig-proxy +docker_subcmd = attach +docker_subarg = --no-stdin=sig-proxy +docker_stderr = invalid boolean value "sig-proxy" for --no-stdin|invalid argument "sig-proxy" for --no-stdin=sig-proxy +podman_stderr = invalid boolean value "sig-proxy" for -no-stdin [docker_cli/negativeusage/iv3] -subcmd = run -subarg = -p,192.168.9.1:9000,%%(FQIN)s,/bin/true -stderr = Invalid hostPort +docker_subcmd = run +docker_subarg = -p,192.168.9.1:9000,%%(FQIN)s,/bin/true +docker_stderr = Invalid hostPort [docker_cli/negativeusage/iv4] -subcmd = run -subarg = -e,PATH=/tmp,%%(FQIN)s,true +docker_subcmd = run +docker_subarg = -e,PATH=/tmp,%%(FQIN)s,true # N/B: This (commented) 'stderr' exposes RH BZ 1393572. # stderr = exec\: \"true\"\: executable file not found in \$PATH # Workaround bug to receive some benefit from this sub-subtest running/passing -stderr = executable file not found -extcmd = 127 +docker_stderr = executable file not found +docker_exit_status = 127 [docker_cli/negativeusage/ip1] -subcmd = tag -subarg = %%(RUNCNTR)s,%%(NOFQIN)s -stderr = no such id: -extcmd = 1 +docker_subcmd = tag +docker_subarg = %%(RUNCNTR)s,%%(NOFQIN)s +docker_stderr = no such id: +podman_stderr = in local storage: unable to find a name and tag match +docker_exit_status = 1 +podman_exit_status = 125 [docker_cli/negativeusage/ip2] docker_options = -subarg = daemon -g fdsafsd -b dsca -s fdsafs -e feae -- --name=flag_cGpa +docker_subarg = daemon -g fdsafsd -b dsca -s fdsafs -e feae -- --name=flag_cGpa # FIXME: BZ 1301147 closed -> add a stderr/stdout check -extcmd = 1 +docker_exit_status = 1 diff --git a/config_defaults/subtests/docker_cli/pull.ini b/config_defaults/subtests/docker_cli/pull.ini index f00e76860..a333f2f87 100644 --- a/config_defaults/subtests/docker_cli/pull.ini +++ b/config_defaults/subtests/docker_cli/pull.ini @@ -15,10 +15,12 @@ docker_registry_user = [docker_cli/pull/wrong_tag] docker_expected_exit_status = 1 +podman_expected_exit_status = 125 #: Additional tag flag **must NOT exist** docker_repo_tag = tag_does_not_exist [docker_cli/pull/wrong_registry_addr] docker_expected_exit_status = 1 +podman_expected_exit_status = 125 #: Name of an invalid remote registry server and port docker_registry_host = registry.does.not.exist.example.com:3 diff --git a/config_defaults/subtests/docker_cli/rmi.ini b/config_defaults/subtests/docker_cli/rmi.ini index 618c7096b..a444e124c 100644 --- a/config_defaults/subtests/docker_cli/rmi.ini +++ b/config_defaults/subtests/docker_cli/rmi.ini @@ -33,3 +33,4 @@ docker_commit_timeout = 60.0 [docker_cli/rmi/delete_wrong_name] docker_expected_exit_status = 1 +podman_expected_exit_status = 125 diff --git a/config_defaults/subtests/docker_cli/run.ini b/config_defaults/subtests/docker_cli/run.ini index b30ead059..f3b084f9a 100644 --- a/config_defaults/subtests/docker_cli/run.ini +++ b/config_defaults/subtests/docker_cli/run.ini @@ -1,5 +1,5 @@ [docker_cli/run] -subsubtests = run_true,run_false,run_interactive,run_attach_stdout,run_remote_tag,run_names,run_tmpfs,run_passwd,run_install,run_update +subsubtests = run_true,run_false,run_interactive,run_attach_stdout,run_remote_tag,run_names,run_tmpfs,run_passwd #: The most basic sub-subtests can be generated from a generic #: class at runtime, set 'yes' to enable this feature. generate_generic = no diff --git a/config_defaults/subtests/docker_cli/run_cgroup_parent.ini b/config_defaults/subtests/docker_cli/run_cgroup_parent.ini index 684b4ad0d..e656358bd 100644 --- a/config_defaults/subtests/docker_cli/run_cgroup_parent.ini +++ b/config_defaults/subtests/docker_cli/run_cgroup_parent.ini @@ -3,4 +3,6 @@ subsubtests = run_cgroup_parent_invalid_name,run_cgroup_parent_path,run_cgroup_p [docker_cli/run_cgroup_parent/run_cgroup_parent_invalid_name] #: Regex describing the error message we expect. Passed to re.match(). -expect_stderr = .*Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"\. +docker_expect_stderr = .*Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"\. +#: Regex describing the error message we expect. Passed to re.match(). +podman_expect_stderr = did not receive systemd slice as cgroup parent when using systemd to manage cgroups: invalid argument diff --git a/config_defaults/subtests/docker_cli/run_cgroups.ini b/config_defaults/subtests/docker_cli/run_cgroups.ini index 1cd7e1cd0..e2240e8ec 100644 --- a/config_defaults/subtests/docker_cli/run_cgroups.ini +++ b/config_defaults/subtests/docker_cli/run_cgroups.ini @@ -1,6 +1,4 @@ [docker_cli/run_cgroups] -#: subsystem path to the docker's cgroups on host -cgroup_path = /sys/fs/cgroup/memory/system.slice/docker #: which key is used in this test cgroup_key_value = memory.limit_in_bytes subsubtests = cpu_positive,cpu_zero,memory_positive,memory_no_cgroup,cpu_none,memory_negative,cpu_overflow @@ -25,24 +23,20 @@ memory_invalid = abcd expect_success = FAIL [docker_cli/run_cgroups/cpu_positive] -cgroup_path = /sys/fs/cgroup/cpu/system.slice/docker cgroup_key_value = cpu.shares #: Set/expected value of the ``cgroup_key_value`` cpushares_value = 10 [docker_cli/run_cgroups/cpu_zero] -cgroup_path = /sys/fs/cgroup/cpu/system.slice/docker cgroup_key_value = cpu.shares #: Set/expected value of the ``cgroup_key_value`` cpushares_value = 0 [docker_cli/run_cgroups/cpu_none] -cgroup_path = /sys/fs/cgroup/cpu/system.slice/docker #: Set/expected value of the ``cgroup_key_value`` cgroup_key_value = cpu.shares [docker_cli/run_cgroups/cpu_overflow] -cgroup_path = /sys/fs/cgroup/cpu/system.slice/docker #: Set/expected value of the ``cgroup_key_value`` cgroup_key_value = cpu.shares #: Set/expected value of the ``cgroup_key_value`` diff --git a/config_defaults/subtests/docker_cli/run_cidfile.ini b/config_defaults/subtests/docker_cli/run_cidfile.ini index 03fead58e..4e089fade 100644 --- a/config_defaults/subtests/docker_cli/run_cidfile.ini +++ b/config_defaults/subtests/docker_cli/run_cidfile.ini @@ -2,3 +2,6 @@ subsubtests = basic #: Additional options to pass when starting container run_options_csv = -i +#: Error message we expect to see if cidfile exists +docker_msg_cidfile_exists = Container ID file found, make sure the other container isn't running or delete +podman_msg_cidfile_exists = container id file exists. ensure another container is not using it or delete diff --git a/config_defaults/subtests/docker_cli/run_exec.ini b/config_defaults/subtests/docker_cli/run_exec.ini index abd61dd83..fbdd1258b 100644 --- a/config_defaults/subtests/docker_cli/run_exec.ini +++ b/config_defaults/subtests/docker_cli/run_exec.ini @@ -13,7 +13,8 @@ exit_status = 0 subsubtests = exec_true, exec_false, exec_pid_count [docker_cli/run_exec/exec_false] -exit_status = 1 +docker_exit_status = 1 +podman_exit_status = 125 [docker_cli/run_exec/exec_pid_count] #: Expected count of pid in container when command using exec is started. diff --git a/dockertest/config.py b/dockertest/config.py index 80158816b..a5d2ae7a2 100644 --- a/dockertest/config.py +++ b/dockertest/config.py @@ -11,6 +11,7 @@ from ConfigParser import SafeConfigParser from collections import MutableMapping +from dockertest.output import DockerVersion import os.path import sys import copy @@ -245,6 +246,7 @@ class ConfigDict(MutableMapping): def __init__(self, section, defaults=None, *args, **dargs): self._config_section = ConfigSection(defaults=defaults, section=section) + self._is_podman = None # pylint: disable=E1101 super(ConfigDict, self).__init__(*args, **dargs) @@ -274,6 +276,13 @@ def __getitem__(self, key): # Don't call more methods than necessary if not self.__contains__(key): raise xceptions.DockerKeyError(key) + # Special case for podman: use 'podman_xxxxx' setting if defined + if key.startswith('docker_'): + if self.is_podman: + podman_key = key.replace('docker_', 'podman_') + if self.__contains__(podman_key): + key = podman_key + # No suffix calls regular get(), boolean wants to gobble '0' and '1' :( for suffix in ('int', 'boolean', 'float', ''): method = getattr(self._config_section, 'get%s' % suffix) @@ -307,6 +316,15 @@ def write(filelike): raise xceptions.DockerIOError("Instance does not permit writing to %s" % filelike.name) + @property + def is_podman(self): + """ + Return true if we're testing podman (instead of docker) + """ + if self._is_podman is None: + self._is_podman = DockerVersion().is_podman + return self._is_podman + class Config(dict): diff --git a/dockertest/docker_daemon.py b/dockertest/docker_daemon.py index 2311be6ca..4eca59c35 100644 --- a/dockertest/docker_daemon.py +++ b/dockertest/docker_daemon.py @@ -119,6 +119,7 @@ def which_docker(): or the name of a known docker-as-system-container service. """ docker = 'docker' + return 'podman' # FIXME FIXME FIXME # Known docker-daemon services as of July 2017 docker_services = ('docker', 'docker-latest', 'container-engine') @@ -172,6 +173,10 @@ def systemd_show(prop): def pid(): """ returns the process ID of currently-running docker daemon """ mainpid = int(systemd_show('MainPID')) + # systemd returns 0 for unknown or non-running units; this is + # expected when testing podman, which has no daemon. + if mainpid == 0: + return 0 cmd = cmdline(mainpid) if 'dockerd' in cmd[0]: return mainpid @@ -198,6 +203,9 @@ def cmdline(process_id=None): """ if process_id is None: process_id = pid() + # daemon won't be running if we're testing podman + if int(process_id) == 0: + return '' ps_command = 'ps -o command= -p %d' % int(process_id) return utils.run(ps_command).stdout.strip().split() diff --git a/dockertest/dockercmd.py b/dockertest/dockercmd.py index 3123af3ce..32c7d05d2 100644 --- a/dockertest/dockercmd.py +++ b/dockertest/dockercmd.py @@ -441,6 +441,10 @@ def container_id(self): c_pid = utils.run('docker inspect --format {{.State.Pid}} ' + cid) if int(c_pid.stdout) == int(pid): return cid + + # No match. FIXME: this is pretty likely because the PID reported by + # docker inspect is actually unrelated to ours; FIXME FIXME + self.subtest.logwarning("Could not find container_id for pid %d" % pid) return None # Override base-class property methods to give up-to-second details diff --git a/dockertest/output/dockerversion.py b/dockertest/output/dockerversion.py index 07c29cba3..7a184a32a 100644 --- a/dockertest/output/dockerversion.py +++ b/dockertest/output/dockerversion.py @@ -31,6 +31,7 @@ class DockerVersion(object): _client_info = None _server_info = None _has_distinct_exit_codes = None + _is_podman = None def __init__(self, version_string=None, docker_path=None): # If called without an explicit version string, run docker to find out @@ -248,3 +249,13 @@ def has_distinct_exit_codes(self): has = (d_run.exit_status > 120) DockerVersion._has_distinct_exit_codes = has return DockerVersion._has_distinct_exit_codes + + @property + def is_podman(self): + """ + Are we testing podman? This is expensive, so cache the result. + """ + if DockerVersion._is_podman is None: + d_help = utils.run('docker --help') + DockerVersion._is_podman = ('podman' in d_help.stdout) + return DockerVersion._is_podman diff --git a/kpod-filter b/kpod-filter new file mode 100755 index 000000000..1360aa5a3 --- /dev/null +++ b/kpod-filter @@ -0,0 +1,146 @@ +#!/usr/bin/perl -w +# +# translate podman output to docker style +# +(our $ME = $0) =~ s|^.*/||; + +use strict; +use List::Util qw(max); + +$| = 1; + +our $Podman_Bin = '/usr/bin/podman'; + +my $handler = 'do_' . $ARGV[0]; +if (main->can($handler)) { + no strict 'refs'; + $handler->() +} +else { + exec $Podman_Bin, @ARGV; + die "$ME: exec podman failed: $!\n"; +} + +exit 0; + + +################################################################ + +sub do_images { + # Remove --all option if present + my @new_argv = grep { $_ !~ /^--all$/ } @ARGV; + + exec $Podman_Bin, @new_argv; +} + + +# FIXME: docker ps emits headers even if there are no running containers +sub do_ps { + my $fmt = "%-64s %-41s %-21s %-19s %-26s %-19s %s\n"; + + open my $podman_fh, '-|', $Podman_Bin, @ARGV + or die "$ME: Cannot fork: $!\n"; + while (my $line = <$podman_fh>) { + chomp $line; + $line =~ s/ CREATED AT / CREATED /; + + # Docker emits '0 B' with a space; podman has no space + if (grep { /^(-s|--size)$/ } @ARGV) { + $line =~ s/(\s\d+(\.\d+)?)(\w*B\s+\()/$1 $3/; + } + + print $line, "\n"; + undef $fmt; + } + close $podman_fh + or die "$ME: Command failed: $Podman_Bin @ARGV: status=$?\n"; + + if ($fmt && !("@ARGV" =~ /-q/)) { + printf $fmt, "CONTAINER ID", "IMAGE", "COMMAND", "CREATED", "STATUS", + "PORTS", "NAMES"; + } + exit 0; +} + + +# 2018-02-14 remove '--volumes' if present +sub do_rm { + my @new_argv = grep { $_ !~ /^--volumes/ } @ARGV; + + exec $Podman_Bin, @new_argv; +} + +sub do_version { + # e.g. --help + exec $Podman_Bin, @ARGV if @ARGV > 1; + + print <<'END_VERSION'; +Client: + Version: 1.12.6 + API version: 1.24 + Package version: podman-V-R + Go version: go1.8.3 + Git commit: 5246e78-unsupported + Built: Wed Jan 31 14:13:27 2018 + OS/Arch: linux/amd64 + +Server: + Version: 1.12.6 + API version: 1.24 + Package version: podman-V-R + Go version: go1.8.3 + Git commit: 5246e78-unsupported + Built: Wed Jan 31 14:13:27 2018 + OS/Arch: linux/amd64 +END_VERSION + + exit 0; +} + + +# Headings in 'podman top' are right-aligned for numbers; docker left-aligns +# everything. Let's match docker. +sub do_top { + my @lines; + my $n_elements = 0; + open my $podman_fh, '-|', $Podman_Bin, @ARGV + or die "$ME: Cannot fork: $!\n"; + while (my $line = <$podman_fh>) { + chomp $line; + push @lines, [ split ' ', $line, $n_elements ]; + $n_elements ||= $#{$lines[0]} + 1; + } + close $podman_fh; + my $status = $?; + + # Reformat, left-justified + my @maxlen = (0); + for my $i (0 .. $#{$lines[0]}) { + $maxlen[$i] = max(map { length $_->[$i] } @lines); + } + # dockertest/output/texttable.py requires TWO spaces between fields + my $fmt = join(' ', map { "%-${_}s" } @maxlen) . "\n"; + + for my $line (@lines) { + printf $fmt, @$line; + } + exit $status / 256; +} + + +# strip --rm option +sub do_build { + my @new_argv = grep { $_ !~ /^--(rm|cache|no-cache|force-rm)/ } @ARGV; + + # Special case for 'docker build URL-to-Dockerfile' + # https://github.com/projectatomic/libpod/issues/682 + if ($new_argv[-1] =~ m!^\w+://.*/Dockerfile$!) { + my $url = pop @new_argv; + push @new_argv, '-f', $url, '.'; + } + system $Podman_Bin, @new_argv; + + # docker-autotest expects this docker message which podman does not emit + print "Successfully built 0123456789ab\n" if $? == 0; + exit $? / 256; +} diff --git a/setup-podman-autotest.yml b/setup-podman-autotest.yml new file mode 100644 index 000000000..35842dfd6 --- /dev/null +++ b/setup-podman-autotest.yml @@ -0,0 +1,142 @@ +--- +# playbook for installing docker-autotest on a Fedora host +# such that it will test *podman* (not docker) +# +# Usage: ansible-playbook -i HOSTNAME, setup-podman-autotest.yml -v +# ^-- the comma is required +# +# HOSTNAME is a Fedora 27 virt; you must have root ssh access to it. +# +- hosts: all + gather_facts: false # ARGH! Fedora has no /usr/bin/python + user: '{{ name | default("fedora") }}' + become: yes + vars: + destdir: /var/lib/autotest + autotest_rpm_deps: [ procps, tar, findutils, bzip2, gdb, bridge-utils, + nfs-utils, git, glibc-devel, python-bugzilla, which, + cloud-utils-growpart, python-nose, python-unittest2, + python2-mock, python-docutils, python-sphinx, + python-sphinxcontrib-httpdomain, + python-sphinx_rtd_theme, + selinux-policy-targeted, libselinux-python ] + + tasks: + # Fedora, as of 23, has no /usr/bin/python (just python3). Ansible + # isn't smart enough to try running python3. This step bootstraps us. + # http://blog.oddbit.com/2015/10/15/bootstrapping-ansible-on-fedora-23/ + - name: install python + raw: dnf -y -e0 -d0 install python python-dnf + + - name: NOW gather facts + setup: + + - name: install rpms needed for autotest + dnf: name={{ item }} state=present + with_items: "{{ autotest_rpm_deps }}" + + # git repos: autotest and autotest-docker. update=no means "chill": it + # lets us modify files in the repos without Ansible barfing on future + # runs. If you ever want to update, just rm -rf the checkout manually. + - name: install autotest + git: repo=https://github.com/autotest/autotest.git + dest={{ destdir }} update=no recursive=no + + - name: install autotest-docker + git: repo=https://github.com/edsantiago/autotest-docker.git + dest="{{ destdir }}/client/tests/docker" update=yes + version=podman_fixes + + - name: autotest-docker | custom config + action: copy src=ansible-files/{{item}} + dest="{{ destdir }}/client/tests/docker/config_custom/{{ item }}" + with_items: [ control.ini, defaults.ini, tests.ini ] + +############################################################################# +# BEGIN podman-specific setup + + - name: make sure docker is NOT installed + dnf: name=docker-client state=absent + + - name: install bbaude COPR repo + action: copy src=ansible-files/_copr_baude-Upstream_CRIO_Family.repo + dest=/etc/yum.repos.d/_copr_baude-Upstream_CRIO_Family.repo + + - name: install podman + dnf: name=podman state=present + + - name: symlink to docker filter + file: src="https://github.com/%7B%7B%20destdir%20%7D%7D/client/tests/docker/kpod-filter" + dest="/usr/bin/docker" + state=link + + # Without this, the systemd test fails + - name: docker filter script | set SELinux context + command: chcon -u system_u -t container_runtime_exec_t "{{ destdir }}/client/tests/docker/kpod-filter" + + # Without this, the systemd-in-container test fails, cgroup perms errors + - name: Magic SELinux boolean allows running systemd in container + seboolean: name=container_manage_cgroup state=yes persistent=yes + +# END podman-specific setup +############################################################################# +# BEGIN nginx configuration + + # So nginx can serve files. We can't seem to do this using the + # ansible file module with setype: it's not recursive. + - name: autotest | SELinux policy 1 + action: file path={{destdir}}/client/results state=directory + + # Sigh. On RHEL, semanage fcontext -a can be run over and over again. + # On Fedora, the second time, it barfs: + # ValueError: File context for [...] already defined + # Let's try using the sefcontext module instead + - name: autotest | SELinux policy 2 | ansible prerequisites + package: name=policycoreutils-python state=present + + - name: autotest | SELinux policy 2 + sefcontext: target="{{destdir}}/client/results(/.*)?" + setype=httpd_sys_content_t + + - name: autotest | SELinux policy 3 + command: restorecon -R {{destdir}}/client/results + + - name: nginx is installed + package: name=nginx state=present + + - name: configure nginx | root + lineinfile: dest=/etc/nginx/nginx.conf + regexp="^\s+root\s" + state=absent + notify: restart nginx + + - name: configure nginx | default + action: copy src=ansible-files/nginx-default.conf + dest=/etc/nginx/default.d/default.conf + owner=root group=root + notify: restart nginx + + # Open port 80 to allow viewing results. + # FIXME: Brent says this isn't necessary on his virts; they default to open + # Grumble. ansible firewalld module is broken on Fedora + # https://github.com/ansible/ansible/issues/38161 + # https://github.com/ansible/ansible/pull/37603 +## - name: firewalld is installed +## package: name=firewalld state=present +## +## - name: firewalld is running +## service: name=firewalld state=started enabled=yes +## +## - name: open http port (now) +## shell: firewall-cmd --add-port=80/tcp +## +## - name: open http port (on reboot) +## shell: firewall-cmd --add-port=80/tcp --permanent + +# END nginx configuration +############################################################################# + + + handlers: + - name: restart nginx + service: name=nginx state=restarted enabled=yes diff --git a/subtests/docker_cli/attach/attach.py b/subtests/docker_cli/attach/attach.py index bcdfcafaa..7326a815c 100644 --- a/subtests/docker_cli/attach/attach.py +++ b/subtests/docker_cli/attach/attach.py @@ -126,16 +126,18 @@ def run_once(self): self.logdebug("Before input should be ignored: %s", dkrcmd.cmdresult) # This input should be ignored. - os.write(self.sub_stuff["run_in_pipe_w"], - self.config['interactive_cmd_run'] + "\n") + send_str = self.config['interactive_cmd_run'] + self.logdebug("About to send '%s' to run_in_pipe" % send_str) + os.write(self.sub_stuff["run_in_pipe_w"], send_str + "\n") self.logdebug("Before input should be passed: %s", dkrcmd.cmdresult) # This input should be passed to container. - os.write(attach_in_pipe_w, - self.config['interactive_cmd_attach'] + "\n") + send_str = self.config['interactive_cmd_attach'] + self.logdebug("About to send '%s' to attach_in_pipe" % send_str) + os.write(attach_in_pipe_w, send_str + "\n") self.wait_interactive_cmd() - self.logdebug("After input was passsed: %s", dkrcmd.cmdresult) + self.logdebug("After input was passed: %s", dkrcmd.cmdresult) def failif_contain(self, check_for, in_output, details): self.failif(check_for in in_output, @@ -145,10 +147,9 @@ def failif_contain(self, check_for, in_output, details): self.logdebug("Output does NOT contain '%s'", check_for) def failif_not_contain(self, check_for, in_output, details): - self.failif(check_for not in in_output, - "Command '%s' output must contain '%s' but doesn't." - " Detail: %s" % (self.config["cmd"], - check_for, details)) + self.failif_not_in(check_for, in_output, + "Output from '%s' command. Details: %s" % + (self.config["cmd"], details)) self.logdebug("Output does contain '%s'", check_for) def verify_output(self): diff --git a/subtests/docker_cli/build/build.py b/subtests/docker_cli/build/build.py index e5e7bc487..065536d7e 100644 --- a/subtests/docker_cli/build/build.py +++ b/subtests/docker_cli/build/build.py @@ -31,6 +31,7 @@ from dockertest.images import DockerImages from dockertest.output import OutputGood from dockertest.output import OutputNotBad +from dockertest.output import DockerVersion from dockertest.config import get_as_list from dockertest.xceptions import DockerTestFail from dockertest.xceptions import DockerTestError @@ -193,6 +194,16 @@ def parse_postprocess_commands(self, build_def): except KeyError: self.logerror("Unknown postprocessing command: %s", command) raise + # Special case for 'docker=this and that;podman=something else' + if param and 'podman=' in param: + tool_dict = {} + for tool_value in param.split(';'): + (tool, value) = tool_value.split('=') + tool_dict[tool] = value + if DockerVersion().is_podman: + param = tool_dict['podman'] + else: + param = tool_dict['docker'] args = (build_def, command, param) result.append((method, args)) commands.append(command) diff --git a/subtests/docker_cli/commit/commit.py b/subtests/docker_cli/commit/commit.py index cb69dcff8..2dee0eb52 100644 --- a/subtests/docker_cli/commit/commit.py +++ b/subtests/docker_cli/commit/commit.py @@ -21,7 +21,7 @@ from dockertest.containers import DockerContainers from dockertest.images import DockerImages from dockertest.images import DockerImage -from dockertest.output import OutputGood +from dockertest.output import OutputGood, DockerVersion from dockertest.dockercmd import AsyncDockerCmd, DockerCmd from dockertest import subtest from dockertest import config @@ -70,7 +70,8 @@ def complete_docker_command_line(self): cmd = [] if c_author: cmd.append("-a %s" % c_author) - if c_msg: + # oci format (default for podman commit) does not support -m + if c_msg and not DockerVersion().is_podman: cmd.append("-m %s" % c_msg) cmd.append(self.sub_stuff["container"]) diff --git a/subtests/docker_cli/create/create_remote_tag.py b/subtests/docker_cli/create/create_remote_tag.py index 758c8079a..2d8281386 100644 --- a/subtests/docker_cli/create/create_remote_tag.py +++ b/subtests/docker_cli/create/create_remote_tag.py @@ -33,8 +33,9 @@ def init_save_images(self): if imgs: long_id = imgs[0].long_id existing_images = di.list_imgs_with_image_id(long_id) - self.sub_stuff['saved_images'] = os.path.join(self.tmpdir, - str(long_id)) + # podman can't deal with ':' in filenames; libpod issue #489 + outfile = os.path.join(self.tmpdir, str(long_id).lstrip("sha256:")) + self.sub_stuff['saved_images'] = outfile subargs = ['--output', self.sub_stuff['saved_images']] for img in existing_images: self.loginfo("Going to save image %s" % img.full_name) diff --git a/subtests/docker_cli/create/create_signal.py b/subtests/docker_cli/create/create_signal.py index ea0c72ee3..2cad26d23 100644 --- a/subtests/docker_cli/create/create_signal.py +++ b/subtests/docker_cli/create/create_signal.py @@ -29,7 +29,10 @@ def run_once(self): sigdkrcmd = DockerCmd(self, 'kill', ['--signal', str(sig), self.get_cid()]) - sigdkrcmd_result = mustfail(sigdkrcmd.execute(), 1) + expect_status = 1 + if DockerVersion().is_podman: + expect_status = 125 + sigdkrcmd_result = mustfail(sigdkrcmd.execute(), expect_status) self.sub_stuff['sigdkrcmd'] = sigdkrcmd_result def postprocess(self): diff --git a/subtests/docker_cli/dockerimport/empty.py b/subtests/docker_cli/dockerimport/empty.py index 6c752c71a..cc82206ae 100644 --- a/subtests/docker_cli/dockerimport/empty.py +++ b/subtests/docker_cli/dockerimport/empty.py @@ -87,7 +87,8 @@ def lookup_image_id(self, image_name, image_tag): fqin = DockerImage.full_name_from_component(image_name, image_tag) imglst = di.list_imgs_with_full_name(fqin) try: - # Don't care about duplicate ID's - return imglst[0].long_id + # Don't care about duplicate ID's. + # podman sometimes (inconsistently) prefixes images with 'sha256:' + return imglst[0].long_id.lstrip("sha256:") except IndexError: return None # expected by some sub-subtests diff --git a/subtests/docker_cli/dockerimport/truncated.py b/subtests/docker_cli/dockerimport/truncated.py index 20940527c..bbd06c2e8 100644 --- a/subtests/docker_cli/dockerimport/truncated.py +++ b/subtests/docker_cli/dockerimport/truncated.py @@ -12,6 +12,7 @@ from empty import empty from dockertest.dockercmd import DockerCmd from dockertest.output import mustfail +from dockertest.output import DockerVersion from dockertest import output @@ -26,7 +27,10 @@ def cleanup(self): image_name = self.sub_stuff['image_name'] # assume id lookup failed if self.parent_subtest.config['remove_after_test']: dkrcmd = DockerCmd(self, 'rmi', [image_name]) - mustfail(dkrcmd.execute(), 1) + expected_status = 1 + if DockerVersion().is_podman: + expected_status = 125 + mustfail(dkrcmd.execute(), expected_status) def run_tar(self, tar_command, dkr_command): tarfile = os.path.join(self.parent_subtest.bindir, self.TARFILENAME) @@ -41,7 +45,7 @@ def check_output(self): self.failif(outputgood, "Unexpected good output - this command was " "supposed to fail: %s" % outputgood) self.failif(cmdresult.exit_status == 0, - "Unexpected exit status: got 0, expected error exit") + "Unexpected exit status: expected error exit, got 0") def check_status(self): successful_exit = self.sub_stuff['cmdresult'].exit_status == 0 diff --git a/subtests/docker_cli/dockerinspect/dockerinspect.py b/subtests/docker_cli/dockerinspect/dockerinspect.py index 9c753cd7d..9bb537e26 100644 --- a/subtests/docker_cli/dockerinspect/dockerinspect.py +++ b/subtests/docker_cli/dockerinspect/dockerinspect.py @@ -18,10 +18,11 @@ from dockertest.containers import DockerContainers from dockertest.dockercmd import DockerCmd from dockertest.output import mustpass +from dockertest.output import DockerVersion from dockertest.images import DockerImage from dockertest.subtest import SubSubtest from dockertest.subtest import SubSubtestCaller -from dockertest.xceptions import DockerTestError +from dockertest.xceptions import DockerTestError, DockerTestNAError class dockerinspect(SubSubtestCaller): @@ -31,6 +32,11 @@ class dockerinspect(SubSubtestCaller): class inspect_base(SubSubtest): + def initialize(self): + if DockerVersion().is_podman: + raise DockerTestNAError + super(inspect_container_simple, self).initialize() + @staticmethod def verify_same_configs(subtest, source, comp, ignore_fields=None): for i in range(len(comp)): # pylint: disable=E0012,C0200 diff --git a/subtests/docker_cli/events/events.py b/subtests/docker_cli/events/events.py index dbd066f69..cbeb31d6f 100644 --- a/subtests/docker_cli/events/events.py +++ b/subtests/docker_cli/events/events.py @@ -32,8 +32,9 @@ from dockertest.images import DockerImage from dockertest.dockercmd import DockerCmd from dockertest.output import mustpass +from dockertest.output import DockerVersion from dockertest.dockercmd import AsyncDockerCmd -from dockertest.xceptions import DockerValueError +from dockertest.xceptions import DockerValueError, DockerTestNAError regexes = { @@ -179,6 +180,8 @@ class events(Subtest): config_section = 'docker_cli/events' def initialize(self): + if DockerVersion().is_podman: + raise DockerTestNAError super(events, self).initialize() dc = self.stuff['dc'] = DockerContainers(self) fullname = dc.get_unique_name() diff --git a/subtests/docker_cli/images_all/images_all.py b/subtests/docker_cli/images_all/images_all.py index 30e8575e5..1a1669da3 100644 --- a/subtests/docker_cli/images_all/images_all.py +++ b/subtests/docker_cli/images_all/images_all.py @@ -59,6 +59,8 @@ def _create_image(self, parent, prefix, cmd): img_name = images.get_unique_name(prefix) dkrcmd = DockerCmd(self, "commit", [cont_name, img_name]) img_id = mustpass(dkrcmd.execute()).stdout.strip() +## if not img_id.startswith('sha256:'): +## img_id = 'sha256:' + img_id # FIXME FIXME FIXME ESM mustpass(DockerCmd(self, 'rm', ['--force', '--volumes', cont_name]).execute()) self.sub_stuff['images'].append(img_id) @@ -89,7 +91,7 @@ def format_err_str(test_images, images, imagesall): out = "Expected images:\n" out += "\n".join((str(_) for _ in test_images)) out += "\nImages:\n" - out += "\n".join(("%s%s" % (_.repo, _.long_id) + out += "\n".join(("%s (%s)" % (_.repo, _.long_id) for _ in images)) out += "\nImages --all:\n" out += "\n".join(("%s (%s)" % (_.repo, _.long_id) @@ -102,9 +104,13 @@ def name_in(name, images): def id_in(long_id, images): """ id in list of images """ - return long_id in (_.long_id for _ in images) +# return long_id in (_.long_id for _ in images) + return long_id.lstrip("sha256:") in ( + _.long_id.lstrip("sha256:") for _ in images) images = self.sub_stuff['di'].list_imgs() imagesall = self.sub_stuff['dia'].list_imgs() + import pprint;print ">>> Images:";pprint.pprint(images) + import pprint;print ">>> ImagesAll:";pprint.pprint(imagesall) err_str = lambda: format_err_str(test_images, images, imagesall) for image in test_images: if image[2:4] == [False, False]: # Non existing (nowhere) diff --git a/subtests/docker_cli/import_url/import_url.py b/subtests/docker_cli/import_url/import_url.py index 41948c9a5..cebc897cb 100644 --- a/subtests/docker_cli/import_url/import_url.py +++ b/subtests/docker_cli/import_url/import_url.py @@ -26,6 +26,8 @@ from dockertest.images import DockerImages from dockertest.dockercmd import DockerCmd from dockertest.output import mustpass +from dockertest.output import DockerVersion +from dockertest.xceptions import DockerTestNAError class import_url(https://github.com/SubSubtestCaller): @@ -70,17 +72,21 @@ def cleanup(self): if self.config['remove_after_test']: preserve_fqins = get_as_list(self.config['preserve_fqins']) preserve_cnames = get_as_list(self.config['preserve_cnames']) - if self.sub_stuff['import_repo'] not in preserve_fqins: - DockerCmd(self, 'rmi', - [self.sub_stuff['import_repo']]).execute() - if self.sub_stuff['run_name'] not in preserve_cnames: - DockerCmd(self, 'rm', - [self.sub_stuff['run_name']]).execute() + if 'import_repo' in self.sub_stuff: + if self.sub_stuff['import_repo'] not in preserve_fqins: + DockerCmd(self, 'rmi', + [self.sub_stuff['import_repo']]).execute() + if 'run_name' in self.sub_stuff: + if self.sub_stuff['run_name'] not in preserve_cnames: + DockerCmd(self, 'rm', + [self.sub_stuff['run_name']]).execute() class md5sum(base): def initialize(self): + if DockerVersion().is_podman: + raise DockerTestNAError super(md5sum, self).initialize() self.sub_stuff['filename'] = basename(self.config['in_tar_file']) diff --git a/subtests/docker_cli/iptable/iptable.py b/subtests/docker_cli/iptable/iptable.py index 7ea259656..a93959306 100644 --- a/subtests/docker_cli/iptable/iptable.py +++ b/subtests/docker_cli/iptable/iptable.py @@ -73,7 +73,11 @@ def cntnr_ip_map(self): result = {} for name in [_ for _ in njs if njs[_]["NetworkSettings"]["IPAddress"] != ""]: - result[name] = njs[name]["NetworkSettings"]["IPAddress"] + addr = njs[name]["NetworkSettings"]["IPAddress"] + # podman returns this as a list, not a string + if isinstance(addr, list): + addr = addr[0] + result[name] = addr self.logdebug("%s -> %s", name, result[name]) return result diff --git a/subtests/docker_cli/kill/kill_utils.py b/subtests/docker_cli/kill/kill_utils.py index c4ee7e628..b236b55ef 100644 --- a/subtests/docker_cli/kill/kill_utils.py +++ b/subtests/docker_cli/kill/kill_utils.py @@ -51,6 +51,7 @@ def get(self, idx=None): idx = self.idx out = self.stuff.stdout.splitlines() self.idx = len(out) + print ">>> got here: idx=%d -> %d : '%s'" % (idx, self.idx, out[idx:]) return out[idx:] @@ -318,12 +319,13 @@ def _check_signal(self, container_out, _check, signal, timeout): check = _check % signal output_matches = lambda: check in container_out.get(_idx) # Wait until the signal gets logged - if wait_for(output_matches, timeout, step=0) is None: + if wait_for(output_matches, timeout, step=0.1) is None: msg = ("Signal %s not handled inside container.\nExpected " "output:\n %s\nActual container output:\n %s" % (signal, check, "\n ".join(container_out.get(_idx)))) self.logdebug(msg) + # FIXME: run 'docker logs container_id' here raise xceptions.DockerTestFail("Unhandled signal(s), see debug" "log for more details") diff --git a/subtests/docker_cli/kill_bad/kill_bad.py b/subtests/docker_cli/kill_bad/kill_bad.py index 781d82262..9da85ce97 100644 --- a/subtests/docker_cli/kill_bad/kill_bad.py +++ b/subtests/docker_cli/kill_bad/kill_bad.py @@ -67,17 +67,19 @@ def run_once(self): self.logdebug("Executing couple of bad kill signals.") self.failif(self.sub_stuff['container_cmd'].done, "Testing container " "died unexpectadly.") + expected_status = self.config.get('docker_exit_status') for signal in self.config['bad_signals'].split(','): mustfail(DockerCmd(self, 'kill', ['-s', signal, self.sub_stuff['container_name']]).execute(), - 1) + expected_status) self.failif(self.sub_stuff['container_cmd'].done, "Testing " "container died after using signal %s." % signal) dkrcnt = DockerContainers(self) nonexisting_name = dkrcnt.get_unique_name() self.logdebug("Killing nonexisting containe.") - mustfail(DockerCmd(self, 'kill', [nonexisting_name]).execute(), 1) + mustfail(DockerCmd(self, 'kill', [nonexisting_name]).execute(), + expected_status) def cleanup(self): super(kill_bad_base, self).cleanup() diff --git a/subtests/docker_cli/login/login.py b/subtests/docker_cli/login/login.py index 1d8dc22fd..48a621c6d 100644 --- a/subtests/docker_cli/login/login.py +++ b/subtests/docker_cli/login/login.py @@ -161,13 +161,12 @@ def docker_login(self, password=None): """ if password is None: password = self.sub_stuff['passwd'] - # FIXME: --email option is deprecated in 1.12, and will be - # removed in 1.13. We still need it for <= 1.10; otherwise - # docker prompts from stdin, and we hang. - dc = DockerCmd(self, 'login', ['--username', self.sub_stuff['user'], - '--password', password, - '--email', 'nobody@redhat.com', - self.sub_stuff['server']]) + login_args = ['--username', self.sub_stuff['user'], + '--password', password] + \ + self.config['docker_args_csv'].split(',') + \ + [self.sub_stuff['server']] + import pprint;pprint.pprint(login_args) + dc = DockerCmd(self, 'login', login_args) return dc.execute() def _check_credentials(self): @@ -214,7 +213,8 @@ def docker_push(self, img_name, tag_name): mustpass(dc.execute()) self.sub_stuff['my_images'] += [pushed_name] - return DockerCmd(self, 'push', [pushed_name]).execute() + subargs = self.config['docker_args_csv'].split(',') + [pushed_name] + return DockerCmd(self, 'push', subargs).execute() def cleanup(self): super(login_base, self).cleanup() @@ -263,8 +263,9 @@ def postprocess(self): super(login_fail, self).postprocess() cmdresult = self.sub_stuff['cmdresult'] OutputGood(cmdresult, ignore_error=True) - mustfail(cmdresult, 1) - self.failif_not_in("401 Unauthorized", cmdresult.stderr, + mustfail(cmdresult, self.config['docker_exit_status']) + self.failif_not_in(self.config['docker_error_message'], + cmdresult.stderr, "stderr from failed docker login") @@ -279,13 +280,17 @@ class logout_ok(login_ok): def run_once(self): super(logout_ok, self).run_once() - logout = DockerCmd(self, 'logout', [self.sub_stuff['server']]) - self.sub_stuff['cmdresult_logout'] = logout.execute() + logout_args = [x for x in self.config['docker_args_csv'].split(',') + if 'tls-verify' not in x] + [self.sub_stuff['server']] + logout = DockerCmd(self, 'logout', logout_args) + self.sub_stuff['cmdresult_logout'] = mustpass(logout.execute()) def postprocess(self): super(logout_ok, self).postprocess() + cmdresult = self.sub_stuff['cmdresult_logout'] + OutputGood(cmdresult) xpect = 'Remove login credentials for | Removing login credentials for' - self.failif_not_in(xpect, self.sub_stuff['cmdresult_logout'].stdout, + self.failif_not_in(xpect, cmdresult.stdout, 'stdout from docker logout') @@ -307,10 +312,9 @@ def postprocess(self): cmdresult = self.sub_stuff['pushresult'] OutputGood(cmdresult) mustpass(cmdresult) - self.failif_not_in(": Pushed", cmdresult.stdout, - "stdout from docker push") - self.failif_not_in("tag_ok: digest:", cmdresult.stdout, - "stdout from docker push") + for expect in self.config['docker_expected_stdout_csv'].split(','): + self.failif_not_in(expect, cmdresult.stdout, + "stdout from docker push") class push_fail(login_base): @@ -330,4 +334,6 @@ def postprocess(self): super(push_fail, self).postprocess() cmdresult = self.sub_stuff['pushresult'] OutputGood(cmdresult, ignore_error=True) - mustfail(cmdresult, 1) + mustfail(cmdresult, self.config['docker_exit_status']) + self.failif_not_in(self.config['docker_expected_stderr'], + cmdresult.stderr, "stderr from docker push") diff --git a/subtests/docker_cli/mount/mount.py b/subtests/docker_cli/mount/mount.py new file mode 100644 index 000000000..9fd108ccb --- /dev/null +++ b/subtests/docker_cli/mount/mount.py @@ -0,0 +1,138 @@ +r""" +Summary +------- + +Test podman mount and umount + +Operational Summary +------------------- + +#. Run a container that creates a specially crafted file +#. Run podman mount +#. Access the desired file within that mount point, verify it +#. Run podman umount +#. Confirm that the file is no longer accessible +""" + +import os +from autotest.client import utils +from dockertest.output import OutputGood, mustpass +from dockertest.output import DockerVersion +from dockertest.dockercmd import DockerCmd +from dockertest.images import DockerImage +from dockertest.containers import DockerContainers +from dockertest.xceptions import DockerTestError, DockerTestNAError +from dockertest import subtest + + +class mount(subtest.SubSubtestCaller): + pass + +class mount_base(subtest.SubSubtest): + + def initialize(self): + if not DockerVersion().is_podman: + raise DockerTestNAError + self._run_container() + + def _run_container(self): + filename = 'rand_%s.txt' % utils.generate_random_string(8) + filedata = utils.generate_random_string(32) + fqin = DockerImage.full_name_from_defaults(self.config) + self.sub_stuff['dc'] = dc = DockerContainers(self) + c_name = dc.get_unique_name() + subargs = [ '--name', c_name, fqin, + 'bash', '-c', '\'echo "%s" >/%s\'' % (filedata, filename) ] + mustpass(DockerCmd(self, 'run', subargs).execute()) + + self.sub_stuff['c_name'] = c_name + self.sub_stuff['cid'] = dc.list_containers_with_name(c_name)[0].long_id + self.sub_stuff['filename'] = filename + self.sub_stuff['filedata'] = filedata + + def _run_mount(self, args): + cmd = DockerCmd(self, 'mount', args) + results = mustpass(cmd.execute()) + return results.stdout.strip() + + def postprocess(self): + super(mount_base, self).postprocess() + mountpoint = self.sub_stuff['mountpoint'] + filepath = os.path.join(mountpoint, self.sub_stuff['filename']) + self.failif(not os.path.isdir(mountpoint), + "Mount point does not exist: %s" % mountpoint) + self.failif(not os.path.isfile(filepath), + "File not found: %s" % filepath) + # Read file, compare contents + actual = open(filepath, 'r').read().strip() + self.failif_ne(actual, self.sub_stuff['filedata'], + "Contents of magic file") + + def cleanup(self): + super(mount_base, self).cleanup() + dc = DockerContainers(self) + c_name = self.sub_stuff.get("c_name") + if c_name: + dc.clean_all([c_name]) + + +class mount_by_name(mount_base): + + def run_once(self): + super(mount_by_name, self).run_once() + c_name = self.sub_stuff['c_name'] + self.sub_stuff['mountpoint'] = self._run_mount([c_name]) + + +class mount_by_cid(mount_base): + + def run_once(self): + super(mount_by_cid, self).run_once() + cid = self.sub_stuff['cid'] + self.sub_stuff['mountpoint'] = self._run_mount([cid]) + + +class mount_twice(mount_base): + + def run_once(self): + super(mount_twice, self).run_once() + m1 = self._run_mount([self.sub_stuff['c_name']]) + m2 = self._run_mount([self.sub_stuff['cid']]) + self.failif_ne(m1, m2, 'mountpoint for same container') + self.sub_stuff['mountpoint'] = m1 + + +class mount_with_no_args(mount_by_name): + """ confirm that 'podman mount' returns all mounted points """ + + def postprocess(self): + # super() checks that file exists as expected + super(mount_with_no_args, self).postprocess() + + # Now run just mount, see if we get what we expect + cmd = DockerCmd(self, 'mount', ['--notruncate']) + results = mustpass(cmd.execute()) + (cid, mountpoint) = results.stdout.strip().split() + self.failif_ne(cid, self.sub_stuff['cid'], 'output of podman-mount') + self.failif_ne(mountpoint, self.sub_stuff['mountpoint'], + 'output of podman-mount') + + +class umount_umounts(mount_by_name): + + def postprocess(self): + # super() checks that file exists as expected + super(umount_umounts, self).run_once() + + # We now run umount, and file should no longer exist + cmd = DockerCmd(self, 'umount', [self.sub_stuff['cid']]) + results = mustpass(cmd.execute()) + + # The mountpoint itself should exist, because the container + # hasn't been removed. But there should be no files in it. + mountpoint = self.sub_stuff['mountpoint'] + filepath = os.path.join(mountpoint, self.sub_stuff['filename']) + self.failif(os.path.exists(filepath), + "filepath exists after umount: %s" % filepath) + contents = os.listdir(mountpoint) + self.failif_ne(contents, [], "sdfsdf") diff --git a/subtests/docker_cli/negativeusage/negativeusage.py b/subtests/docker_cli/negativeusage/negativeusage.py index d9ebd7e86..025f6ff9d 100644 --- a/subtests/docker_cli/negativeusage/negativeusage.py +++ b/subtests/docker_cli/negativeusage/negativeusage.py @@ -143,10 +143,10 @@ def do_substitutions(self): # Some are CSV, some are regex, some are simple strings for key in ('subcmd', 'subarg', 'stderr', 'stdout'): # Ignore empty config values - if self.config[key].strip() != '': + if self.config['docker_' + key].strip() != '': # Catch misspellings / unsupported sutstitution keywords try: - value = self.config[key] % self.sub_stuff + value = self.config['docker_' + key] % self.sub_stuff except KeyError, xcpt: raise KeyError("Configuration error with option '%s' value" "in sub-subtest %s: Invalid substitution " @@ -185,7 +185,7 @@ def run_once(self): def postprocess(self): NoPanic(self.sub_stuff['cmdresult']) - expected_exit_status = self.config['extcmd'] + expected_exit_status = self.config['docker_exit_status'] cmdresult = self.sub_stuff['cmdresult'] mustfail(cmdresult, expected_exit_status) # Same checks for both diff --git a/subtests/docker_cli/pull/pull.py b/subtests/docker_cli/pull/pull.py index f82434007..41bc551dc 100644 --- a/subtests/docker_cli/pull/pull.py +++ b/subtests/docker_cli/pull/pull.py @@ -26,6 +26,8 @@ from dockertest.subtest import SubSubtest from dockertest.images import DockerImages from dockertest.output import OutputGood +from dockertest.output import OutputGood +from dockertest.output import DockerVersion from dockertest.dockercmd import AsyncDockerCmd from dockertest import subtest @@ -99,8 +101,12 @@ def outputcheck(self): OutputGood(self.sub_stuff['dkrcmd'].cmdresult) def exitcheck(self): + expect = self.config["docker_expected_exit_status"] + if DockerVersion().is_podman: + if 'podman_expected_exit_status' in self.config: + expect = self.config["podman_expected_exit_status"] self.failif_ne(self.sub_stuff['dkrcmd'].exit_status, - self.config["docker_expected_exit_status"], + expect, "Exit status from pull command") def existcheck(self): diff --git a/subtests/docker_cli/rmi/rmi.py b/subtests/docker_cli/rmi/rmi.py index 2a3f7ef50..44abd7093 100644 --- a/subtests/docker_cli/rmi/rmi.py +++ b/subtests/docker_cli/rmi/rmi.py @@ -20,6 +20,7 @@ from dockertest.subtest import SubSubtest from dockertest.images import DockerImages from dockertest.output import OutputGood +from dockertest.output import DockerVersion from dockertest.dockercmd import AsyncDockerCmd from dockertest.dockercmd import DockerCmd from dockertest.xceptions import DockerTestNAError @@ -83,6 +84,9 @@ def postprocess(self): super(rmi_base, self).postprocess() # Raise exception if problems found expect = self.config["docker_expected_exit_status"] + if DockerVersion().is_podman: + if 'podman_expected_exit_status' in self.config: + expect = self.config['podman_expected_exit_status'] OutputGood(self.sub_stuff['cmdresult'], ignore_error=(expect != 0)) self.failif_ne(self.sub_stuff['cmdresult'].exit_status, expect, @@ -91,7 +95,7 @@ def postprocess(self): if expect == 0: im = self.check_image_exists(self.sub_stuff["image_name"]) self.sub_stuff['image_list'] = im - self.failif_ne(im, [], "Deleted image still exits: %s" % + self.failif_ne(im, [], "Deleted image still exists: %s" % self.sub_stuff["image_name"]) def cleanup(self): diff --git a/subtests/docker_cli/run_cgroup_parent/run_cgroup_parent.py b/subtests/docker_cli/run_cgroup_parent/run_cgroup_parent.py index 7baebfc98..fe47e0fc3 100644 --- a/subtests/docker_cli/run_cgroup_parent/run_cgroup_parent.py +++ b/subtests/docker_cli/run_cgroup_parent/run_cgroup_parent.py @@ -39,6 +39,9 @@ def _setup(self, cg_parent): self.sub_stuff['rand1'] = utils.generate_random_string(8) self.sub_stuff['rand2'] = utils.generate_random_string(8) self.sub_stuff['cgroup_parent'] = cg_parent.format(**self.sub_stuff) + self.sub_stuff['toolname'] = 'docker' + if DockerVersion().is_podman: + self.sub_stuff['toolname'] = 'libpod' def _expect(self, path=None, stderr=None, exit_status=0): """ @@ -65,7 +68,17 @@ def run_once(self): '--cidfile', cidfile, DockerImage.full_name_from_defaults(self.config), '/bin/cat', '/proc/1/cgroup'] - dc = DockerCmd(self, "run", subargs, + + # docker only supports systemd cgroups, so that's what these tests + # cover. Podman supports systemd and cgroupfs, but the latter are + # the default. The command-line option to use systemd cgroups has + # to precede the action: it's a podman option, not a run option. + # FIXME: 2018-05-21 mheon has indicated that the default will + # be switching back to systemd + cmd = "run" + if DockerVersion().is_podman: + cmd = "--cgroup-manager=systemd run" + dc = DockerCmd(self, cmd, subargs, timeout=self.config['docker_timeout']) self.sub_stuff["cmdresult"] = dc.execute() @@ -77,7 +90,8 @@ def postprocess(self): OutputGood(cmdresult, ignore_error=(expected_status != 0), skip=['nonprintables_check']) - self.sub_stuff["cid"] = self._read_cid() + if expected_status == 0: + self.sub_stuff["cid"] = self._read_cid() path_exp = self.sub_stuff['expected_path'].format(**self.sub_stuff) stderr_exp = self.sub_stuff['expected_stderr'].format(**self.sub_stuff) @@ -178,7 +192,8 @@ class run_cgroup_parent_invalid_name(run_cgroup_parent_base): def initialize(self): super(run_cgroup_parent_invalid_name, self).initialize() self._setup("/{rand1}") - self._expect(stderr=self.config['expect_stderr'], exit_status=125) + self._expect(stderr=self.config['docker_expect_stderr'], + exit_status=125) class run_cgroup_parent_path(run_cgroup_parent_base): @@ -190,7 +205,8 @@ class run_cgroup_parent_path(run_cgroup_parent_base): def initialize(self): super(run_cgroup_parent_path, self).initialize() self._setup("{rand1}.slice") - self._expect(path="/{rand1}.slice/docker-{cid}.scope") + expect = "/{rand1}.slice/{toolname}-{cid}.scope" + self._expect(path=expect) class run_cgroup_parent_path_with_hyphens(run_cgroup_parent_base): @@ -202,4 +218,4 @@ def initialize(self): super(run_cgroup_parent_path_with_hyphens, self).initialize() self._setup("{rand1}-{rand2}.slice") self._expect(path="/{rand1}.slice/{rand1}-{rand2}.slice" - "/docker-{cid}.scope") + "/{toolname}-{cid}.scope") diff --git a/subtests/docker_cli/run_cgroups/cgroups_base.py b/subtests/docker_cli/run_cgroups/cgroups_base.py index 6824a3e00..38aaabd66 100644 --- a/subtests/docker_cli/run_cgroups/cgroups_base.py +++ b/subtests/docker_cli/run_cgroups/cgroups_base.py @@ -4,7 +4,7 @@ import os from dockertest import xceptions -from dockertest.output import mustpass +from dockertest.output import mustpass, DockerVersion from dockertest.dockercmd import DockerCmd from dockertest.containers import DockerContainers from dockertest.subtest import SubSubtest @@ -13,25 +13,30 @@ class cgroups_base(SubSubtest): - @staticmethod - def cgroup_fullpath(long_id, path, content): + def cgroup_fullpath(self, long_id, content): """ Return full cgroup path for a container. :param long_id: a container's long id - :path: the cgroup path of container + :cgroup_type: desired cgroup type ('cpu' or 'memory') :param content: the value need check. """ - return os.path.join("%s-%s.scope" % (path, long_id), content) + cgroup_base_dir = '/sys/fs/cgroup' + # 'cpu' or 'memory', extracted from test name + cgroup_type = self.__class__.__name__.split('_')[0] + (parent, subdir) = ('system.slice', 'docker-{}.scope') + if DockerVersion().is_podman: + (parent, subdir) = ('libpod_parent', 'libpod-{}/ctr') + return os.path.join(cgroup_base_dir, cgroup_type, parent, + subdir.format(long_id), content) - @staticmethod - def read_cgroup(long_id, path, content): + def read_cgroup(self, long_id, content): """ Read container's cgroup file, return its value :param long_id: a container's long id, can get via command --inspect. :param path: the cgroup path of container. :param content: the value need read. """ - cgroup_path = cgroups_base.cgroup_fullpath(long_id, path, content) + cgroup_path = self.cgroup_fullpath(long_id, content) if not os.path.exists(cgroup_path): raise xceptions.DockerIOError("Docker cgroup path " "doesn't exist: %s" diff --git a/subtests/docker_cli/run_cgroups/cpushares.py b/subtests/docker_cli/run_cgroups/cpushares.py index e64695dfa..ba34654d0 100644 --- a/subtests/docker_cli/run_cgroups/cpushares.py +++ b/subtests/docker_cli/run_cgroups/cpushares.py @@ -77,9 +77,11 @@ def run_once(self): long_id = cobjs[0].long_id jzero = dc.json_by_long_id(long_id)[0] json_cpushares = jzero["HostConfig"]["CpuShares"] - cgpath = self.config['cgroup_path'] + # podman leaves this unset if unspecified; docker sets to 0 + if json_cpushares is None: + json_cpushares = 0 cgvalue = self.config['cgroup_key_value'] - cgroup_cpushares = self.read_cgroup(long_id, cgpath, cgvalue) + cgroup_cpushares = self.read_cgroup(long_id, cgvalue) docker_cpushares = self.config.get('cpushares_value', 0) result.update(self.check_cpushares(docker_cpushares, cgroup_cpushares, diff --git a/subtests/docker_cli/run_cgroups/memory.py b/subtests/docker_cli/run_cgroups/memory.py index 2d545f76c..a939c846d 100644 --- a/subtests/docker_cli/run_cgroups/memory.py +++ b/subtests/docker_cli/run_cgroups/memory.py @@ -39,32 +39,11 @@ def check_memory(docker_memory, cgroup_memory, unit): elif unit == 'g' or unit == 'G': container_memory *= 1024 * 1024 * 1024 - if container_memory == 0: - if cgroup_memory == 0: - msg = ("container_memory is %s, " - "unit %s, cgroup_memory is %s" - % (container_memory, unit, cgroup_memory)) - result = {'PASS': msg} - - return result - msg = ("container_memory is %s, " - "unit %s, cgroup_memory is %s, status Unknown" - % (container_memory, unit, cgroup_memory)) - result = {'FAIL': msg} - - return result - - if container_memory != cgroup_memory: - msg = ("container_memory is %s " - ",unit %s, cgroup_memory is %s" - % (container_memory, unit, cgroup_memory)) - result = {'FAIL': msg} - return result - msg = ("container_memory is %s, " - "unit %s, cgroup_memory is %s" - % (container_memory, unit, cgroup_memory)) - result = {'PASS': msg} - return result + msg = ("cgroup memory is 0x%X, expected 0x%X (%s%s)" % + (cgroup_memory, container_memory, docker_memory, unit.strip())) + if container_memory == cgroup_memory: + return {'PASS': msg} + return {'FAIL': msg} @staticmethod def combine_subargs(name, option, image, sub_command): @@ -182,9 +161,8 @@ def check_result(self, subargs, long_id): memory = self.split_unit(memory_arg) memory_value = memory[0] memory_unit = memory[1] - cgpath = self.config['cgroup_path'] cgvalue = self.config['cgroup_key_value'] - cgroup_memory = self.read_cgroup(long_id, cgpath, cgvalue) + cgroup_memory = self.read_cgroup(long_id, cgvalue) return self.check_memory(memory_value, cgroup_memory, memory_unit) def postprocess_positive(self, this_result, passed): diff --git a/subtests/docker_cli/run_cidfile/run_cidfile.py b/subtests/docker_cli/run_cidfile/run_cidfile.py index d1d666897..7fcaa164c 100644 --- a/subtests/docker_cli/run_cidfile/run_cidfile.py +++ b/subtests/docker_cli/run_cidfile/run_cidfile.py @@ -88,8 +88,8 @@ def _init_container(self, subargs_in, cidfile, cmd, check_method=None, :warning: When dkrcmd_cls is of Async type, there is no guarantee that it is going to be up&running after return. """ - def do_nothing(resutls): - return resutls + def do_nothing(results): + return results if custom_dockercmd is None: custom_dockercmd = dockercmd.DockerCmd @@ -157,8 +157,7 @@ def run_once(self): def _check_failure_cidfile_present(self, dkrcmd): """ Check that docker warns about existing cidfile """ - msg_cidfile_exists = ("Container ID file found, make sure the other " - "container isn't running or delete") + msg_cidfile_exists = self.config['docker_msg_cidfile_exists'] self.failif(msg_cidfile_exists not in dkrcmd.cmdresult.stderr, "Msg " "'%s' not present in the container stderr:\n%s" % (msg_cidfile_exists, dkrcmd)) diff --git a/subtests/docker_cli/run_exec/run_exec.py b/subtests/docker_cli/run_exec/run_exec.py index 255b6bb09..4f23772c2 100644 --- a/subtests/docker_cli/run_exec/run_exec.py +++ b/subtests/docker_cli/run_exec/run_exec.py @@ -149,7 +149,8 @@ def run_once(self): def postprocess(self): dkrcmd_exec = self.sub_stuff['dkrcmd_exec'] - mustfail(dkrcmd_exec.cmdresult, 1) + expected_status = self.config['docker_exit_status'] + mustfail(dkrcmd_exec.cmdresult, expected_status) OutputNotBad(dkrcmd_exec.cmdresult) super(exec_false, self).postprocess() diff --git a/subtests/docker_cli/run_user/run_user.py b/subtests/docker_cli/run_user/run_user.py index 13c82ac9b..dbff05150 100644 --- a/subtests/docker_cli/run_user/run_user.py +++ b/subtests/docker_cli/run_user/run_user.py @@ -18,6 +18,7 @@ from dockertest.dockercmd import DockerCmd from dockertest.images import DockerImage from dockertest.output import OutputGood +from dockertest.output import DockerVersion class run_user(subtest.SubSubtestCaller): @@ -236,7 +237,10 @@ def _init_test_dependent(self): pass user = utils.get_unique_name(lambda name: name not in users, "user", length=6) - self.sub_stuff['execution_failure'] = "unable to find user %s" % user + expect_msg = 'unable to find user %s' % user + if DockerVersion().is_podman: + expect_msg = 'unknown user error looking up user "%s"' % user + self.sub_stuff['execution_failure'] = expect_msg self.sub_stuff['subargs'] = ['--rm', '--interactive', '--user=%s' % user] diff --git a/subtests/docker_cli/selinux_labels/test.sh b/subtests/docker_cli/selinux_labels/test.sh index ece9b74f7..e7c6af5ff 100755 --- a/subtests/docker_cli/selinux_labels/test.sh +++ b/subtests/docker_cli/selinux_labels/test.sh @@ -56,13 +56,17 @@ check_label() { } # Actual checks -check_label "docker-containerd" \ - "ps axZ | grep docker-containerd | grep -v grep" \ - "container_runtime_t" "docker_t" -check_label "dockerd" \ - "ps axZ | grep dockerd | grep -v grep" \ - "container_runtime_t" "docker_t" +# dockerd and docker-containerd tests N/A when testing podman +if ! docker version | grep -q podman-V-R; then + check_label "docker-containerd" \ + "ps axZ | grep docker-containerd | grep -v grep" \ + "container_runtime_t" "docker_t" + + check_label "dockerd" \ + "ps axZ | grep dockerd | grep -v grep" \ + "container_runtime_t" "docker_t" +fi check_label "confined container" \ "docker run --rm $image cat /proc/self/attr/current" \ diff --git a/subtests/docker_cli/start/simple.py b/subtests/docker_cli/start/simple.py index 8c36df6ff..c4ebbaa99 100644 --- a/subtests/docker_cli/start/simple.py +++ b/subtests/docker_cli/start/simple.py @@ -18,6 +18,7 @@ from dockertest.containers import DockerContainers from dockertest.dockercmd import AsyncDockerCmd, DockerCmd from dockertest.output import mustpass, mustfail +from dockertest.output import DockerVersion from dockertest.images import DockerImage @@ -56,7 +57,11 @@ def run_once(self): super(simple, self).run_once() name = self.sub_stuff['container_name'] # Container does not yet exist; 'start' should fail. - result = mustfail(DockerCmd(self, "start", [name]).execute(), 1) + expect_status = 1 + if DockerVersion().is_podman: + expect_status = 125 + result = mustfail(DockerCmd(self, "start", [name]).execute(), + expect_status) self.failif_not_in(self.config['missing_msg'], str(result), "'docker start ' failed" " (as expected), but docker error message did not" diff --git a/subtests/docker_cli/version/version.py b/subtests/docker_cli/version/version.py index e83437e92..12670db2a 100644 --- a/subtests/docker_cli/version/version.py +++ b/subtests/docker_cli/version/version.py @@ -43,6 +43,10 @@ def postprocess(self): self.verify_version(docker_version) def verify_version(self, docker_version): + # podman has no daemon + if DockerVersion().is_podman: + return + # TODO: Make URL to daemon configurable client = SocketClient() _version = client.version() From b64f66a5283f51bd4d76c4b0b78bc148366990fd Mon Sep 17 00:00:00 2001 From: Ed Santiago Date: Tue, 20 Feb 2018 11:56:32 -0700 Subject: [PATCH 2/5] Podman-autotest Automation Duct-tape This is simply a small set of changes made ontop of @edsantiago work to position this branch for temporary use by (internal) automation. * Removed unnecessary/disused utilities and documentation * Renamed ``kpod-filter`` -> ``podman_filter`` * Renamed ``docker`` -> ``podman`` in ``results2junit`` * Rename ``README-podman.md`` -> ``README.md`` (w/ some small changes) * Moved ``ansible-files/*.ini`` -> ``config_custom/`` (removed redundant sections and generally simplified) * Renamed ``config_custom/tests.ini`` -> ``config_custom/subtests.ini`` * Removed ``^docker.*`` from ``config_custom/known_failures.txt`` * Relocated Ansible-related bits to ADEPT repo ``jobs/podman_basic/...`` Signed-off-by: Chris Evich --- .gitignore | 1 - .papr.yml | 63 -- Makefile | 157 --- README-podman.md | 167 --- README.md | 90 ++ README.rst | 9 - .../_copr_baude-Upstream_CRIO_Family.repo | 10 - ansible-files/control.ini | 25 - ansible-files/defaults.ini | 15 - ansible-files/nginx-default.conf | 3 - ansible-files/tests.ini | 54 - config_custom/README.rst | 17 - config_custom/control.ini | 6 + config_custom/defaults.ini | 15 + config_custom/known_failures.txt | 19 - config_custom/subtests.ini | 6 + config_value.py | 14 - index.rst | 984 ------------------ kpod-filter => podman_filter | 0 results2junit | 16 +- run_checkdocs.py | 259 ----- run_pylint.sh | 218 ---- run_unittests.sh | 38 - setup-podman-autotest.yml | 142 --- 24 files changed, 125 insertions(+), 2203 deletions(-) delete mode 100644 .papr.yml delete mode 100644 Makefile delete mode 100644 README-podman.md create mode 100644 README.md delete mode 100644 README.rst delete mode 100644 ansible-files/_copr_baude-Upstream_CRIO_Family.repo delete mode 100644 ansible-files/control.ini delete mode 100644 ansible-files/defaults.ini delete mode 100644 ansible-files/nginx-default.conf delete mode 100644 ansible-files/tests.ini delete mode 100644 config_custom/README.rst create mode 100644 config_custom/control.ini create mode 100644 config_custom/defaults.ini create mode 100644 config_custom/subtests.ini delete mode 100755 config_value.py delete mode 100644 index.rst rename kpod-filter => podman_filter (100%) delete mode 100755 run_checkdocs.py delete mode 100755 run_pylint.sh delete mode 100755 run_unittests.sh delete mode 100644 setup-podman-autotest.yml diff --git a/.gitignore b/.gitignore index 556a66b4c..fd63d5c83 100644 --- a/.gitignore +++ b/.gitignore @@ -4,7 +4,6 @@ .*.swp results2junitc docs_build -config_custom/*.ini cscope.files cscope.out subtests.rst diff --git a/.papr.yml b/.papr.yml deleted file mode 100644 index ecc368c46..000000000 --- a/.papr.yml +++ /dev/null @@ -1,63 +0,0 @@ ---- - -# Example format: -# https://github.com/projectatomic/papr/blob/master/docs/sample.papr.yml - -# Internal Job (for realtime monitoring): -# https://hurl.corp.redhat.com/eb0faa3 - -container: - image: centos:latest - -# Mark this testsuite as required. This causes a special -# "required" context to be reported to GitHub on branch -# tests. The result is set to successful only if all -# testsuites marked as required are also successful. This is -# useful for integration with merge bots, so that they only -# need to watch for a single status rather than multiple -# (possibly changing) statuses. If omitted, defaults to -# false. -required: true - -# Time to allow before aborting tests. Must satisfy regex -# '[0-9]+[smh]'. If omitted, defaults to '2h', which is the -# maximum. -timeout: 30m - -# These get dumped into /etc/yum.repos.d/$name.repo -extra-repos: - - name: epel - mirrorlist: https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch - gpgkey: https://getfedora.org/static/352C64E5.txt - -packages: - - git - - make - - python-bugzilla - - pylint - - python-pep8 - - python2-mock - - python-sphinx - - python-unittest2 - - python-docutils - - python-sphinxcontrib-httpdomain - - python-sphinx_rtd_theme - - PyYAML - - libselinux-python - -env: - AUTOTEST_REPO: "https://github.com/autotest/autotest.git" - AUTOTEST_BRANCH: "master" - AUTOTEST_PATH: "/var/lib/autotest" - SPHINXOPTS: "-W" -# Put the tasks to be executed in the 'tests' key. They are -# run from the root of the repo. If the 'build' key is also -# present, tests will run after a successful build. Full -# UTF-8 values are supported. -tests: - - mkdir -p $AUTOTEST_PATH - - git clone --branch "$AUTOTEST_BRANCH" --depth 1 --single-branch "$AUTOTEST_REPO" "$AUTOTEST_PATH" - - make - - ./run_checkdocs.py - - ./run_unittests.sh - - ./run_pylint.sh --FF diff --git a/Makefile b/Makefile deleted file mode 100644 index a93106225..000000000 --- a/Makefile +++ /dev/null @@ -1,157 +0,0 @@ -# Makefile for Sphinx documentation -# - -# You can set these variables from the command line. -SPHINXBUILD = sphinx-build -PAPER = -BUILDDIR = docs_build - -# Internal variables. -PAPEROPT_a4 = -D latex_paper_size=a4 -PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . -# the i18n builder cannot share the environment and doctrees with the others -I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . - -.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext - -all: html - -help: - @echo "Please use \`make ' where is one of" - @echo " html to make standalone HTML files" - @echo " dirhtml to make HTML files named index.html in directories" - @echo " singlehtml to make a single large HTML file" - @echo " pickle to make pickle files" - @echo " json to make JSON files" - @echo " htmlhelp to make HTML files and a HTML help project" - @echo " qthelp to make HTML files and a qthelp project" - @echo " devhelp to make HTML files and a Devhelp project" - @echo " epub to make an epub" - @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" - @echo " latexpdf to make LaTeX files and run them through pdflatex" - @echo " text to make text files" - @echo " man to make manual pages" - @echo " texinfo to make Texinfo files" - @echo " info to make Texinfo files and run them through makeinfo" - @echo " gettext to make PO message catalogs" - @echo " changes to make an overview of all changed/added/deprecated items" - @echo " linkcheck to check all external links for integrity" - @echo " doctest to run all doctests embedded in the documentation (if enabled)" - -clean: - -rm -rf $(BUILDDIR)/* - -rm -rf subtests.rst - -rm -rf additional.rst - -rm -rf defaults.rst - -html: - $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." - -dirhtml: - $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." - -singlehtml: - $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml - @echo - @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." - -pickle: - $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle - @echo - @echo "Build finished; now you can process the pickle files." - -json: - $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json - @echo - @echo "Build finished; now you can process the JSON files." - -htmlhelp: - $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp - @echo - @echo "Build finished; now you can run HTML Help Workshop with the" \ - ".hhp project file in $(BUILDDIR)/htmlhelp." - -qthelp: - $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp - @echo - @echo "Build finished; now you can run "qcollectiongenerator" with the" \ - ".qhcp project file in $(BUILDDIR)/qthelp, like this:" - @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/DockerAutotest.qhcp" - @echo "To view the help file:" - @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/DockerAutotest.qhc" - -devhelp: - $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp - @echo - @echo "Build finished." - @echo "To view the help file:" - @echo "# mkdir -p $$HOME/.local/share/devhelp/DockerAutotest" - @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/DockerAutotest" - @echo "# devhelp" - -epub: - $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub - @echo - @echo "Build finished. The epub file is in $(BUILDDIR)/epub." - -latex: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo - @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." - @echo "Run \`make' in that directory to run these through (pdf)latex" \ - "(use \`make latexpdf' here to do that automatically)." - -latexpdf: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo "Running LaTeX files through pdflatex..." - $(MAKE) -C $(BUILDDIR)/latex all-pdf - @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." - -text: - $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text - @echo - @echo "Build finished. The text files are in $(BUILDDIR)/text." - -man: - $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man - @echo - @echo "Build finished. The manual pages are in $(BUILDDIR)/man." - -texinfo: - $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo - @echo - @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." - @echo "Run \`make' in that directory to run these through makeinfo" \ - "(use \`make info' here to do that automatically)." - -info: - $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo - @echo "Running Texinfo files through makeinfo..." - make -C $(BUILDDIR)/texinfo info - @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." - -gettext: - $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale - @echo - @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." - -changes: - $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes - @echo - @echo "The overview file is in $(BUILDDIR)/changes." - -linkcheck: - $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck - @echo - @echo "Link check complete; look for any errors in the above output " \ - "or in $(BUILDDIR)/linkcheck/output.txt." - -doctest: - $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest - @echo "Testing of doctests in the sources finished, look at the " \ - "results in $(BUILDDIR)/doctest/output.txt." diff --git a/README-podman.md b/README-podman.md deleted file mode 100644 index 9c69dbf96..000000000 --- a/README-podman.md +++ /dev/null @@ -1,167 +0,0 @@ -Instructions for testing podman with docker-autotest -==================================================== - -2018-04-11 : This document describes how to set up docker-autotest -on a Fedora system, then run it against podman. - -*This is a work in progress*. In particular, much of it is -crappy code intended solely to work around docker/podman -incompatibilities, some is debugging code intended for figuring -out real failures, and some is workarounds for ways in which -podman and docker will presumably forever differ. I (Ed) reserve -the right to force-push new commits to this branch without notice. - -Set up a virt -------------- - -I (Ed) have done all my testing on f27, but f28 is worth testing too. - -Setting up the virt is left as an exercise for the reader. I suggest -creating one inside the Red Hat network: one of the `pulls` from -registry.access.redhat.com fails if run from outside, and you don't -want to waste time debugging that. - - -Configure it ------------- - -Check out this repo on a Fedora 27 host: - - $ cd /some/nice/directory/ - $ git clone --branch podman_fixes https://github.com/edsantiago/autotest-docker.git - $ cd autotest-docker - -Run this command: - - $ HOSTNAME=name-or-ip-address-of-your-virt - $ ansible-playbook -i $HOSTNAME, setup-podman-autotest.yml -v - ^--- do not omit the comma - -(I suggest Fedora 27 because ansible is unreliably chaotic: things that -work on one version don't work on the next, and vice-versa. I do all -my development on f27 and am not interested in testing any of this on -RHEL or Fedora Rawhide; at least not until ansible calms down). -(Also: maybe there's some way to run the ansible stuff on the same -virt, using localhost or something. I dunno. That's not my M.O. so -I'm not going to spend any time trying it. If you like working that way, -and can give me exact instructions for doing so, please do and I -will update this doc) - -This will take about 5 minutes and will leave your Fedora virt configured -to run docker-autotest. This command is safe to run on any system, -e.g. your laptop. It does not require root on the host nor does it -muck with the host system in any way, um, unless you screw up HOSTNAME. - -Run docker-autotest -------------------- - -Log in to your virt as `root`, then run: - - virt# cd /var/lib/autotest/client - virt# unset XDG_RUNTIME_DIR ! podman won't run if this is set - virt# ./autotest-local run docker - -This will take 25-35 minutes depending on the virt horsepower and -the version of podman (some earlier versions hang for a long time). - - -Install GreaseMonkey scripts ----------------------------- - -While docker-autotest is running, I strongly encourage you to install -my [docker-autotest-highlight](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-highlight) -and [docker-autotest-job-report](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-job-report) GreaseMonkey scripts. (You only need to do this once). - -They are not mandatory, but it would be insane to try to read -docker-autotest results without them. docker-autotest results -are horrible and unreadable; these tools make them somewhat less so. -Seriously: you will be in a world of hurt if you try to read -docker-autotest results without these. - -I'm not quite sure how to install into GM, honestly, but try these -direct links and lmk if they work: -[docker-autotest-highlight](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-highlight/raw/master/docker-autotest-highlight.user.js), -[docker-autotest-job-report](https://gitlab.cee.redhat.com/greasemonkey/docker-autotest-job-report/raw/master/docker-autotest-job-report.user.js). - -This should download the scriptlet, and GM will ask for confirmation. -Please let me know if that works (or doesn't), and please help me -improve this documentation. -(Installing GreaseMonkey is left as an exercise for the reader). - - -Review results --------------- - -Point your web browser at `http://IP-ADDRESS-OF-VIRT/default/job_report.html` - -You will see a table of results, with poorly-chosen non-colorblind-friendly -green/red for GOOD/FAIL respectively. Click on the `Debug` link for any -result of interest: if you installed `docker-autotest-job-report`, this -takes you straight to the debug log; otherwise, you see a list of files -and have to click the right one. Sorry, I can't remember which one and -can't be bothered to check -- again, I recommend installing my GM scripts. - -Look for boldface red; that shows the actual failure. Sometimes the actual -error is in orange (warning), especially if you see 'Postprocess' in -orange. Those are magic tests that don't actually fail in the moment -and are hard to find. The `build` test is an example of this. - -(Note that some orange warnings are OK and expected. Sorry, but you'll -have to develop a sense for these yourself.) - -Debugging results ------------------ - -This is a nightmare. - -Find the source code of the test: for a given test `foo` it will -be under `subtests/docker_cli/foo/` but there may be many files -in there - often it might be `foo.py` but it could also be -`subsubtestname.py` for a given subsubtest. Or, for a kill-related -test, `kill_utils.py` - -In the py file, the sequence is: - - initialize() - run_once() - postprocess() - -It can be hard to know exactly what is failing or why. My usual -process is to look in the web browser for **boldface**; this -indicates the docker commands being run. I'll see if I can -figure out what's going on just by looking at the docker commands. -Most of the time I can't, and I have to delve into the source -code. Here, again, you're on your own. Each subtest is an -unhappy family in its own unique way, and there's not really -any way I can prepare you for debugging any particular -failing test. Sorry. - -Also note that I've already picked off the low-hanging fruit: any -subtest that can be easily figured out, I've already filed podman -issues for. The remaining failures are the hard ones, those that -are not immediately obvious from the debug log, and require -delving deep into the code. Again -- sorry. - -Rerunning tests ---------------- - -First: *save your run*: - - virt# mv results/default results/$(date --iso-8601).01.full - (or .02.full or .03.foo-test or whatever) - -This way you can revisit in your web browser. - -If you want to run just one short subtest, use --args: - - virt# ./autotest-local run docker --args docker_cli/foobar - (replace foobar with attach, images_all, build, etc) - -Short subtests can finish in 1-3 minutes instead of 20. You can -use this to instrument the code (adding printfs) then reload -in your web browser to view results. - -Happy hacking. And please, please: don't do it alone. Whatever -stumbling block you run into, I've almost certainly already -encountered. Ping me (esm) on IRC, or send me email. Even if you -cost me three minutes, it's in order to save you thirty. It's worth it. diff --git a/README.md b/README.md new file mode 100644 index 000000000..091a4df55 --- /dev/null +++ b/README.md @@ -0,0 +1,90 @@ +# DANGER: CONSTRUCTION-ZONE: + +## Hard-hat, double-underwear, and dark-sunglasses required! + +### Useful documentation has been stripped for your own safety + +# Running doc^H^H^H^podman-autotest + + # cd /var/lib/autotest/client + # unset XDG_RUNTIME_DIR ! podman won't run if this is set + # ./autotest-local run docker + +This will take 25-35 minutes depending on the virt horsepower and +the version of podman (some earlier versions hang for a long time). + + +# Review results + +Open ``/var/lib/autotest/client/results/default/job_report.html`` + +You will see a table of results, with poorly-chosen non-colorblind-friendly +green/red for GOOD/FAIL respectively. Click on the `Debug` link for any +result of interest. You'll see a list of files +and have to click the right one. + +Look for boldface red; that shows the actual failure. Sometimes the actual +error is in orange (warning), especially if you see 'Postprocess' in +orange. Those are magic tests that don't actually fail in the moment +and are hard to find. The `build` test is an example of this. + +(Note that some orange warnings are OK and expected. Sorry, but you'll +have to ignore the smell for now.) + +# Debugging results + +This is a nightmare. + +Find the source code of the test: for a given test `foo` it will +be under `subtests/docker_cli/foo/` but there may be many files +in there - often it might be `foo.py` but it could also be +`subsubtestname.py` for a given subsubtest. Or, for a kill-related +test, `kill_utils.py` + +In the py file, the sequence is: + + initialize() + run_once() + postprocess() + +It can be hard to know exactly what is failing or why. My usual +process is to look in the web browser for **boldface**; this +indicates the docker commands being run. I'll see if I can +figure out what's going on just by looking at the docker commands. +Most of the time I can't, and I have to delve into the source +code. Here, again, you're on your own. Each subtest is an +unhappy family in its own unique way, and there's not really +any way I can prepare you for debugging any particular +failing test. Sorry. + +Also note that I've already picked off the low-hanging fruit: any +subtest that can be easily figured out, I've already filed podman +issues for. The remaining failures are the hard ones, those that +are not immediately obvious from the debug log, and require +delving deep into the code. Again -- sorry. + +# Rerunning tests + +Use the ``--tag`` argument to avoid clobbering ``results/default`` + + # cd /var/lib/autotest/client + # ./autotest-local run --tag=nasty_junk docker + +Then look under ``/var/lib/autotest/client/results/nasty_junk`` for +the mostly useless details. + +If you want to run just one short subtest, use --args: + + virt# ./autotest-local run docker --args docker_cli/foobar + (replace foobar with attach, images_all, build, etc) + +Short subtests can finish in 1-3 minutes instead of 20. You can +use this to instrument the code (adding printfs) then reload +in your web browser to view results. + +# Happy hacking, and please, please: do it alone. + +Whatever stumbling block you run into, I've almost certainly already +encountered it, too bad for you. Don't ping me on IRC, or send me +email. I don't care, just --force merge your change here and +we'll all hope for the best. diff --git a/README.rst b/README.rst deleted file mode 100644 index 5a14ece70..000000000 --- a/README.rst +++ /dev/null @@ -1,9 +0,0 @@ -================================= -Autotest | Client Tests | Docker -================================= - -`Full Docker Autotest documentation`__ - -.. _documentation: http://docker-autotest.readthedocs.org/en/latest/ - -__ documentation_ diff --git a/ansible-files/_copr_baude-Upstream_CRIO_Family.repo b/ansible-files/_copr_baude-Upstream_CRIO_Family.repo deleted file mode 100644 index a21228f8c..000000000 --- a/ansible-files/_copr_baude-Upstream_CRIO_Family.repo +++ /dev/null @@ -1,10 +0,0 @@ -[baude-Upstream_CRIO_Family] -name=Copr repo for Upstream_CRIO_Family owned by baude -baseurl=https://copr-be.cloud.fedoraproject.org/results/baude/Upstream_CRIO_Family/fedora-$releasever-$basearch/ -type=rpm-md -skip_if_unavailable=True -gpgcheck=1 -gpgkey=https://copr-be.cloud.fedoraproject.org/results/baude/Upstream_CRIO_Family/pubkey.gpg -repo_gpgcheck=0 -enabled=1 -enabled_metadata=1 diff --git a/ansible-files/control.ini b/ansible-files/control.ini deleted file mode 100644 index 017ff562f..000000000 --- a/ansible-files/control.ini +++ /dev/null @@ -1,25 +0,0 @@ -[Control] -include = -exclude = pretest_example,intratest_example,posttest_example,example,subexample - ,docker_cli/build/git_path - ,docker_cli/build/dockerignore - ,docker_cli/cp - ,docker_cli/info -subthings = -pretests = pretests -subtests = subtests -intratests = intratests -posttests = posttests - -[Bugzilla] -url = -username = -password = -excluded = -key_field = status_whiteboard -key_match = docker-autotest - -[Query] -product = Red Hat Enterprise Linux 7 -component = docker -status = NEW, ASSIGNED, POST, MODIFIED, ON_DEV diff --git a/ansible-files/defaults.ini b/ansible-files/defaults.ini deleted file mode 100644 index ef49a0525..000000000 --- a/ansible-files/defaults.ini +++ /dev/null @@ -1,15 +0,0 @@ -[DEFAULTS] -config_version = 0.8.4 -autotest_version = @!NOVERSIONCHECK!@ -docker_path = /usr/bin/docker -docker_options = -daemon_options = daemon,--selinux-enabled,--storage-driver,devicemapper,--storage-opt,dm.fs=xfs,--storage-opt,dm.thinpooldev=/dev/mapper/atomicos-docker--pool,--icc=false,--add-registry,registry.access.redhat.com,--block-registry,docker.io, -docker_timeout = 600 -__example__ = docker_repo_name, docker_repo_tag, preserve_fqins, preserve_cnames -docker_repo_name = rhel -docker_repo_tag = latest -docker_registry_host = registry.access.redhat.com -docker_registry_user = rhel7 -remove_after_test = yes -preserve_fqins = docker.io/stackbrew/centos:7,docker.io/stackbrew/centos:centos7,docker.io/stackbrew/centos:latest,registry.access.redhat.com/rhel7:latest,registry.access.redhat.com/rhel7/rhel-tools:latest,registry-1.docker.io/distribution/registry:2.1 -preserve_cnames = ,dummy_container_name diff --git a/ansible-files/nginx-default.conf b/ansible-files/nginx-default.conf deleted file mode 100644 index 7f316158a..000000000 --- a/ansible-files/nginx-default.conf +++ /dev/null @@ -1,3 +0,0 @@ -root /var/lib/autotest/client/results; - default_type text/plain; - autoindex on; diff --git a/ansible-files/tests.ini b/ansible-files/tests.ini deleted file mode 100644 index 2feda2404..000000000 --- a/ansible-files/tests.ini +++ /dev/null @@ -1,54 +0,0 @@ - - -# Note: This template will be overwritten by any job-specific template -# with the same dest name - -[docker_cli/create/create_remote_tag] -config_version=0.8.3 -remote_image_fqin = registry.access.redhat.com/rhel7/rhel:latest -run_options_csv = - -[docker_cli/pull] -config_version=0.8.3 -docker_pull_timeout = 600 -subsubtests = good,wrong_tag,wrong_registry_addr -docker_expected_result = PASS - -[docker_cli/run/run_ping] -config_version=0.8.3 -generate_generic = yes -ping_url = code.engineering.redhat.com -cmd = 'yum install -y iputils && ping -c 10 -q %(ping_url)s' -exit_status = 0 - -[docker_cli/run/run_remote_tag] -config_version=0.8.3 -remote_image_fqin = registry.access.redhat.com/rhel7/rhel:latest -run_options_csv = -cmd = /bin/true - -[docker_cli/build/git_path] -__example__ = dockerfile_dir_path, postproc_cmd_csv -use_config_repo = no -dockerfile_dir_path = git@gitlab.osas.lab.eng.rdu2.redhat.com:cevich/dockertest_dockerfiles.git -postproc_cmd_csv = positive(), - rx_out('^Schazam!$'), - cnt_count('0'), - img_count('2'), - img_exst(), - intr_exst(), - file_exist('/var/tmp/testfile'), - rx_file('/var/tmp/testfile:Schazam!') - -[docker_cli/build/https_file] -__example__ = dockerfile_dir_path, postproc_cmd_csv -use_config_repo = no -dockerfile_dir_path = http://gitlab.osas.lab.eng.rdu2.redhat.com/cevich/dockertest_dockerfiles/raw/master/Dockerfile -postproc_cmd_csv = positive(), - rx_out('^Schazam!$'), - cnt_count('0'), - img_count('2'), - img_exst(), - intr_exst(), - file_exist('/var/tmp/testfile'), - rx_file('/var/tmp/testfile:Schazam!') diff --git a/config_custom/README.rst b/config_custom/README.rst deleted file mode 100644 index 0a6ca0fa6..000000000 --- a/config_custom/README.rst +++ /dev/null @@ -1,17 +0,0 @@ -This subdirectory is searched first for ``defaults.ini``. -If found, it's options will become the defaults for all other -configuration sections. In this case, the -``config_defaults/defaults.ini`` file will be completely ignored. - -Next, all regular configuration files in the ``config_defaults`` -subdirectory are loaded and parsed. - -Finally, all regular configuration files in this subdirectory -are parsed. If any configuration sections clash, the sections -loaded from files in **this** (``config_custom``) directory will -override any previously loaded. - -Except for ``defaults.init``, the parsing order within this -subdirectory (as in ``config_defaults``) is completely undefined. -If multiple files contain duplicate sections with differing options, -the active set will be undefined. diff --git a/config_custom/control.ini b/config_custom/control.ini new file mode 100644 index 000000000..e431407e4 --- /dev/null +++ b/config_custom/control.ini @@ -0,0 +1,6 @@ +[Control] +exclude = pretest_example,intratest_example,posttest_example,example,subexample + ,docker_cli/build/git_path + ,docker_cli/build/dockerignore + ,docker_cli/cp + ,docker_cli/info diff --git a/config_custom/defaults.ini b/config_custom/defaults.ini new file mode 100644 index 000000000..7b76dea67 --- /dev/null +++ b/config_custom/defaults.ini @@ -0,0 +1,15 @@ +[DEFAULTS] +docker_path = /var/lib/autotest/client/tests/podman/podman_filter +docker_timeout = 600 +docker_repo_name = rhel +docker_repo_tag = latest +docker_registry_host = registry.access.redhat.com +docker_registry_user = rhel7 +remove_after_test = yes +preserve_fqins = docker.io/stackbrew/centos:7, + docker.io/stackbrew/centos:centos7, + docker.io/stackbrew/centos:latest, + registry.access.redhat.com/rhel7:latest, + registry.access.redhat.com/rhel7/rhel-tools:latest, + registry-1.docker.io/distribution/registry:2.1 +preserve_cnames = ,dummy_container_name diff --git a/config_custom/known_failures.txt b/config_custom/known_failures.txt index 7b050619f..87cd9ab2a 100644 --- a/config_custom/known_failures.txt +++ b/config_custom/known_failures.txt @@ -1,22 +1,4 @@ "Full NVRA, may be N-V-* as wildcard" "subsubtest path" comment describing the reason for this test exception -docker-1.12.5-4.el7.x86_64 foo/bar/fubar comment -docker-1.12.5-4.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah -docker-1.12.5-5.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah -docker-1.12.5-* i docker_cli/negativeusage/iv4 bz1393572: expect fix in docker-1.13? -docker-latest-1.12.5-5.el7.x86_64 docker_cli/negativeusage/iv4 bz1393572: invalid header field value blah blah -docker-latest-1.12.5-* docker_cli/negativeusage/iv4 bz1393572: expect fix in docker-1.13? -docker-1.12.3-10.el7.centos docker_cli/iptable/iptable_remove bz1406460: fixed in 1.12.5 -docker-1.12.3-10.el7.centos docker_cli/login/login_ok bz1406460: fixed in 1.12.5 -docker-1.12.3-10.el7.centos docker_cli/login/login_fail bz1406460: fixed in 1.12.5 -docker-1.12.3-10.el7.centos docker_cli/login/logout_ok bz1406460: fixed in 1.12.5 -docker-1.12.3-10.el7.centos docker_cli/login/push_ok bz1406460: fixed in 1.12.5 -docker-1.12.3-10.el7.centos docker_cli/login/push_fail bz1406460: fixed in 1.12.5 -docker-1.10.3-55.gite03ddb8.fc24 docker_cli/events No idea. Just ignoring. -docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run/run_install Bad repo? ADEPT bug? No idea, just ignoring. -docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_cgroup_parent/run_cgroup_parent_path No idea. Looks like a real bug, possibly kernel-related but I don't care. I just want to silence this. -docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_cgroup_parent/run_cgroup_parent_path_with_hyphens No idea. Looks like a real bug, possibly kernel-related but I don't care. I just want to silence this. -docker-1.10.3-55.gite03ddb8.fc24 docker_cli/run_exec/exec_pid_count Argh. Fedora docker image doesn't have 'find'. This probably needs to be fixed in the test itself but just ignore for now. -docker-1.12.6-49.1.hotfix.git90e29fe.el7.x86_64 docker_cli/run_volumes/oci_umount no idea podman-0.3.2-* docker_cli/create/create_tmpfs "Key Error" - podman inspect doesn't show Tmpfs podman-0.3.3-* docker_cli/create/create_remote_tag podman can't deal with colons in output filename podman-0.3.5-* docker_cli/restart/stopped #575 stop an already-stopped container @@ -26,7 +8,6 @@ podman-0.3.4-* docker_cli/kill_stress/run_sigproxy_tt podman-0.3.4-* docker_cli/kill_stress/attach_sigproxy_ttyoff sigproxy not implemented yet podman-0.3.5-* docker_cli/run_dns DOS newlines in output podman-0.3.3-* docker_cli/events events not implemented, and not likely to be -podman-0.3.4-* docker_cli/run_cgroup_parent/run_cgroup_parent_invalid_name FIXME: file issue on this podman-0.3.2-* docker_cli/run_cgroup_parent/run_cgroup_parent_path cgroup-parent not implemented podman-0.3.3-* docker_cli/rmi/only_tag issue #387 podman-0.4.1-* docker_cli/rm/forced issue #527: unable to get exit code diff --git a/config_custom/subtests.ini b/config_custom/subtests.ini new file mode 100644 index 000000000..e18bc376f --- /dev/null +++ b/config_custom/subtests.ini @@ -0,0 +1,6 @@ +[docker_cli/run/run_ping] +config_version=0.8.8 +generate_generic = yes +ping_url = code.engineering.redhat.com +cmd = 'yum install -y iputils && ping -c 10 -q %(ping_url)s' +exit_status = 0 diff --git a/config_value.py b/config_value.py deleted file mode 100755 index 9113dadd7..000000000 --- a/config_value.py +++ /dev/null @@ -1,14 +0,0 @@ -#!/usr/bin/env python - -"""Simple utility to extract a value from a set of ini-files""" - -import sys -from ConfigParser import SafeConfigParser - -scp = SafeConfigParser() - -if __name__ == "__main__": - section = sys.argv[1] - key = sys.argv[2] - scp.read(sys.argv[3:]) - print scp.get(section, key) diff --git a/index.rst b/index.rst deleted file mode 100644 index ff90e976b..000000000 --- a/index.rst +++ /dev/null @@ -1,984 +0,0 @@ -:tocdepth: 2 - -.. Docker Autotest documentation master file, created by - sphinx-quickstart on Tue Feb 18 09:56:35 2014. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - -.. Quick reference for reStructuredText: - http://docutils.sourceforge.net/docs/user/rst/quickref.html - -.. Reference for sphinx.ext.autodoc extension: - http://sphinx-doc.org/ext/autodoc.html - - -===================================== -Docker Autotest |version| -===================================== - -.. sectnum:: - -.. toctree:: - :hidden: - :numbered: - - defaults - subtests - additional - -.. contents:: - :depth: 1 - :local: - ----------------- -Introduction ----------------- - -Docker Autotest_ is a sub-framework for standalone testing of docker_. -It does not depend on docker itself. Functionally, testing occurs within -any number of subtest modules, which in some cases also include further -nested sub-subtests. It is designed to support both extremely simple, -linear, and more complex, nested, iterative testing of arbitrary external -docker commands. - -Test content and configuration is fully modular, supporting both included -and external tests. The included content is primarily focused on -continuous-integration testing of the Docker CLI client, with a very -limited amount of negative and stress testing. - -As with other `Autotest Client Tests`_ the main entry point is the test -control file. Docker Autotest's control file utilizes Autotest's -steps-engine. This allows the overall testing state to be stored, -recovered, and resumed should a host-kernel panic or if userspace -become unresponsive. Consequently, rebooting the host system as a -testing step is also supported. - -.. _docker: http://www.docker.io - -.. _Autotest: http://github.com/autotest/autotest/wiki - -.. _python-docker-py: http://github.com/dotcloud/docker-py#readme - -.. _Autotest Client Tests: http://github.com/autotest/autotest-client-tests - -.. _docker_autotest_prereq: - ----------------- -Prerequisites ----------------- - -* Docker - - * **Clean** environment, only test-related images, containers - and dependent services (besides ``docker -d``) at the start - of **every** Autotest run. - * Docker installation (with docker daemon running) - * Default settings for Docker on your platform/OS, - including storage (LVM thin-pool + XFS). - * Sufficient available temporary storage for multiple copies of the - test-related images/container content. - -* `Supported Docker OS platform`_ - - * Red Hat Enterprise Linux 7 Server (preferred for development) - * Red Hat Enterprise Linux Atomic Host (preferred for testing) - * Fedora 22 or later including Atomic (not all tests may function) - * Other platforms (such as CentOs, SLES, Ubuntu) un-maintained but possible. - -* Platform Applications/tools - - * Autotest 0.16.0 or later. - * Coreutils or equivalent (i.e. ``cat``, ``mkdir``, ``tee``, etc.) - * Tar and supported compression programs (``bzip2``, ``gzip``, etc.) - * nfs-utils (nfs-server support daemons) - * Python 2.6 or greater (not 3.0) - * libselinux-python 2.2 or later - * Optional (for building documentation), ``make``, ``python-sphinx``, - and ``docutils`` or equivalent for your platform. - * Optional (for running unittests), ``pylint``, ``pep8``, - ``python-unittest2``, and ``python2-mock``. - * Optional, ``python-bugzilla`` for test exclusion based - upon bug status. See ``control.ini`` file comments for details. - - -* **Any specific requirements for particular** `subtest modules`_. - In particular: - - * Direct root access for running docker, restarting the daemon, - and accessing metadata / storage details. Running through - sudo may work to a degree, but is likely to cause problems. - * Access to a remote (to the host) image registry via ``docker pull``, - ``http``, ``https``, and ``git``. - * External testing dependencies must be usable by tests w/in fixed - timeout periods. If an external resource or connection is too slow, - it should be made more network-local to the test host. - * Most tests with external dependencies will have them flagged as - values to the special ``__example__`` option. See `example values`_ - for more details. - -.. _Supported Docker OS platform: https://www.docker.io/gettingstarted/#h_installation - ----------------- -Quickstart ----------------- - -All platforms -============= - -#. Double-check you meet all the requirements in `docker_autotest_prereq`_. For - the quickstart, either a RHEL 7 or Fedora 25 system is assumed with the Docker - daemon started, device-mapper or overlay2 storage configured, and at - least 10gig of registry space is available. - -#. As ``root``, shallow-clone Autotest (**non-recursive**) into ``/var/lib/autotest``, - and set/export the ``AUTOTEST_PATH`` environment variable to it's location. - - :: - - [root@docker ~]# cd /var/lib - - [root@docker lib]# git clone --single-branch --depth 1 \ - https://github.com/autotest/autotest.git autotest - - [root@docker lib]# export AUTOTEST_PATH=/var/lib/autotest - -#. Change to the autotest client subdirectory. - -#. Clone `autotest-docker`_ repository into the ``docker`` subdirectory. - Based from a `formal release`_ or the latest available. - - :: - - [root@docker lib]# cd $AUTOTEST_PATH/client - - [root@docker client]# git clone --branch $VERSION \ - https://github.com/autotest/autotest-docker.git tests/docker - - Where ``$VERSION`` is the docker-autotest release (e.g. "|version|") - **or** to use the current latest release, omit the --branch option: - - :: - - [root@docker client]# git clone \ - https://github.com/autotest/autotest-docker.git docker - - -#. Make a copy of default configuration, then edit as appropriate. - Particularly, verify the CSV list of full-image names and - container names, ``preserve_fqins`` and ``preserve_cnames`` - are set correctly. **All other images/containers will be destroyed!** - See `default configuration options`_ for more details. - - :: - - [root@docker client]# cp -abi tests/docker/config_defaults/defaults.ini \ - tests/docker/config_custom/ - - [root@docker client]# $EDITOR tests/docker/config_custom/defaults.ini - -.. _autotest-docker: https://github.com/autotest/autotest-docker.git -.. _`formal release`: https://github.com/autotest/autotest-docker/releases - -Fedora platforms -================= - -The Fedora base-images lack some essential tooling necessary for testing. In addition -to the steps above, a custom test-image must be configured for building. - -#. Edit the ``defaults.ini`` file again, and change the registry settings as follows: - - :: - - [root@docker client]# $EDITOR tests/docker/config_custom/defaults.ini - - ... - docker_registry_host = - docker_registry_user = - docker_repo_name = fedora_test_image - docker_repo_tag = latest - ... - -#. Make a copy of the ``docker_test_images.ini`` configuration file and configure - it to build the test image. - - :: - - [root@docker client]# cp -abi tests/docker/config_defaults/docker_test_images.ini \ - tests/docker/config_custom/ - - [root@docker client]# $EDITOR tests/docker/config_custom/docker_test_images.ini - - ... - build_name = fedora_test_image:latest - build_dockerfile = https://github.com/autotest/autotest-docker/raw/master/fedora_test_image.tar.gz - build_opts_csv = --no-cache,--pull,--force-rm - ... - -Execute and examine results -============================ - -For all platforms, use the standalone autotest client to select and execute subtests. -The default behavior is to run all subtests. However, the example below only -executes the ``version`` subtest for demonstration purposes. This will bring in -some additional utility "tests", such as ``docker_test_images`` and ``garbage_check``. -Other subtests may be selected via the ``--args`` `parameter or by customizing`_ -``control.ini``. - -:: - - [root@docker /]# cd $AUTOTEST_PATH/client - - [root@docker client]# ./autotest-local tests/docker/control --args=docker_cli/version - Writing results to /var/lib/autotest/client/results/default - START ---- ---- - Subtest/Sub-subtest requested: - 'docker_cli/version' - - Subtest/sub-subtest exclude list: - 'subexample' - 'pretest_example' - 'example' - 'intratest_example' - 'posttest_example' - - Executing tests: - 'docker/pretests/docker_test_images.1' - 'docker/pretests/log_sysconfig.2' - 'docker/pretests/log_versions.3' - 'docker/subtests/docker_cli/version.4' - 'docker/intratests/garbage_check.4' - - START docker/pretests/docker_test_images.1 - docker_test_images: initialize() - docker_test_images: setup() for subtest version 2055 - docker_test_images: Running sub-subtests... - puller: initialize() - puller: run_once() - puller: Pulling registry.access.redhat.com/rhel7/rhel:latest - puller: Pulling docker.io/stackbrew/centos:latest - puller: postprocess() - puller: cleanup() - builder: initialize() - builder: run_once() - builder: postprocess() - builder: cleanup() - docker_test_images: postprocess_iteration() #0 of #1 - docker_test_images: full_name:registry.access.redhat.com/rhel7/rhel:latest - docker_test_images: full_name:docker.io/stackbrew/centos:latest - docker_test_images: Updated preserve_fqins: docker.io/stackbrew/centos:latest,registry.access.redhat.com/rhel7/rhel:latest - docker_test_images: Postprocess sub-subtest results... - docker_test_images: cleanup() - GOOD docker/pretests/docker_test_images.1 - END GOOD docker/pretests/docker_test_images.1 - START docker/pretests/log_sysconfig.2 - log_sysconfig: initialize() - log_sysconfig: setup() for subtest version 0 - log_sysconfig: run_once() - log_sysconfig: postprocess_iteration() #0 of #1 - log_sysconfig: postprocess() - log_sysconfig: cleanup() - GOOD docker/pretests/log_sysconfig.2 - END GOOD docker/pretests/log_sysconfig.2 - START docker/pretests/log_versions.3 - log_versions: initialize() - log_versions: setup() for subtest version 0 - log_versions: run_once() - log_versions: Found docker version client: 1.12.6 server 1.12.6 - log_versions: postprocess_iteration() #0 of #1 - log_versions: postprocess() - log_versions: cleanup() - GOOD docker/pretests/log_versions.3 - END GOOD docker/pretests/log_versions.3 - START docker/subtests/docker_cli/version.4 - version: initialize() - version: setup() for subtest version 0 - version: run_once() - version: postprocess_iteration() #0 of #1 - version: postprocess() - version: docker version client: 1.12.6 server 1.12.6 - version: Docker cli version matches docker client API version - version: cleanup() - GOOD docker/subtests/docker_cli/version.4 - END GOOD docker/subtests/docker_cli/version.4 - START docker/intratests/garbage_check.4 - GOOD docker/intratests/garbage_check.4 - END GOOD docker/intratests/garbage_check.4 - END GOOD ---- ---- - - -*(timestamps and extra inconsequential text removed for clarity)* - -Examine the test results by changing to the ``results/default`` directory. -*Note:* The name "default" is used when no ``--tag`` option is given to the -``autotest-local`` command. - -:: - - [root@docker client]# cd $AUTOTEST_PATH/client/results/default - - [root@docker default]# ls -1 - control # Copy of the control file used for the run - control.ini # Runtime configuration from default control file - control.state # Used to support mid-test reboot / test resumption - debug # All the client / sydout/stderr recorded by log-level. - docker # Directory-tree of subtest results by name - job_report.html # Autogenerated report web-page. - status # Text-version of test run / results - status.json # Same thing, but in JSON format. - sysinfo # Directory of important log-files for the run. - - [root@docker default]# ls -1 docker/subtests/docker_cli/version.4/ - debug # Same as above, but ONLY logs for this subtest - keyval # Copy of subtest configuration, including defaults - profiling # Not used - results # Not used - status # Same as above, but ONLY for this subtest - sysinfo # Logs captured after this subtest ran. - -If you wish jUnit format results, execute the included conversion script. - -:: - - [root@docker client]# cd $AUTOTEST_PATH/client - [root@docker client]# tests/docker/results2junit --name $HOSTNAME results/default - - [root@docker client]# cat results/default/results.junit - - - - ... - -.. _`parameter or by customizing`: _selecting subthings - - -.. _subtests: - ------------------ -Subtests ------------------ - -All `subtest modules`_ reside beneath the ``subtest`` directory. A subtest -module must have the same name as the directory it is in (minus the ``.py`` -extension). Other files/directories may exist at that level, but they -will not be recognized as subtest modules by the Docker client test. This -ensures each subtest's code is kept separate from all others. Finally, -every subtest is run in its own process and context. It does not have -visibility into any other subtest code, configuration, or runtime. - -Organization and Naming -========================= - -The structure/layout of the ``subtest`` directory tree is relevant for -reference and configuration. The reference and configuration section names -for subtests are formed by its relative location under the ``subtest`` -directory. For example, the subtest module ``subtests/docker_cli/version/version.py`` -matches with the ``[docker_cli/version]`` configuration section and -``docker_cli/version`` subtest name. - -Static Content Setup -=========================== - -Subtests may source their own static content from within their directory and below. -When content needs to be built, or is in some way test-environment specific, the -``setup() method`` should be overridden. Content may be referenced from the subtest's -directory by using its ``bindir`` attribute. A version-specific directory to -contain build/setup output is available as the ``srcdir`` attribute. The ``setup()`` -method will ***only*** be called once per version number (including revisions). -State may be reset by clearing the autotest client ``tmp`` directory. - -:Note: The ``setup()`` method runs **after** the ``initialize()`` method - only once. If the configuration version has not changed, the method - will not be called in subsequent Docker Autotest runs. - -Sub-subtests -============== - -There are provisions for tests that contain, or are composed of multiple child -tests or dependent operations. They may share content and code between each other, -so long as it lives within the subtest directory or below. Optionally, they may -use their own configuration sections, named by appending their class name onto -the parent subtest's name. Sub-subtest configuration inherits undefined values -from the parent subtest configuration. Additionally, there are multiple methods -of executing sub-subtests depending on needs. - --------------------- -Images --------------------- - -It is assumed that any images required for testing are available and built -beforehand. A default image for testing purposes is required by most tests. -Its fully-qualified name is configurable, though test results will be directly -affected if it is or becomes unavailable. Individual subtests may require -specific additional images or content. If so, this will be noted -in the subtest documentation's *Prerequisites* section. - -.. _configuration: - --------------------- -Configuration --------------------- - -The default configuration files are all located under the ``config_defaults`` -subdirectory. These are intended to be bundled with the autotest docker test. -To customize any subtest or global default configuration, copies should -be made manually into the ``config_custom`` subdirectory. Any content -within ``config_custom`` will override all files and sections from -``config_defaults``. - -The subdirectory structure or relative file locations under ``config_custom`` -is irrelevant. Multiple sections may appear in the same file, even for -unrelated tests. The only exception is the ``config_custom/defaults.ini`` file -and the test ``control.ini`` file. - -When customizing subtest and sub-subtest configuration options, it is highly -recommended that you add the option ``config_version`` with the current -version number, into each section. This way, you will receive warnings when -updating Docker Autotest, if the specific custom configuration doesn't match -the API. Generally this only happens between major/minor version updates. - -Organization -============== - -Sections ---------------- - -Configuration files use the familiar *ini* style format with separate -sections (e.g. ``[
]``) preventing option names from colliding. -All configuration files are loaded into a single name-space, divided by -each section. Section names can be arbitrary, however those which exactly -match a subtest or sub-subtest's name, will be automatically loaded (see Subtests_). - -.. _default configuration options: - -Defaults ----------------- - -The Default, global values for **all** sections are located within the -special ``defaults.ini`` file's ``DEFAULTS`` section. These option -names and values are supplied for all sections which do not contain -a identical named option. This file is loaded *either* from the -``config_defaults`` *or* ``config_custom`` directory. - - -.. include:: defaults.rst - - -Formatting -============= - -Long values ----------------- - -Long values may be continued onto the next line by prefixing any run of one or -more horizontal-whitespace characters with a newline. For example: - - option_name = This option value is super duper long, - but will be un-folded into a single string with - <---- all this whitespace replaced by a single space. - -In this case, the runs of multiple whitespace following the newline will -be folded into a single space, and combined with the previous line. - -.. _example values: - -Example Values ------------------- - -In order to help dockertest operate w/o customization, many example or demonstration -value have been configured. While this is fine for development and informal testing -purposes, it adds external dependencies for production testing. Therefore every -option with default example values should be specified in a -comma-separated list to the special ``__example__`` option. - -The ``__example__`` option's value is parsed specially. It is not inherited from -defaults, or from subtest to sub-subtest. Instead, it is compounded at each level -then pruned of modified options (as compared to their default/example value). Any -unmodified options remaining at the beginning of a test will cause loud warnings -to be issued. - -Value substitution ---------------------- - -Within each section, optional inline-value substitution is supported -using the ``%(