PR#32: WIP: PoC: Replace STI tests with upstream FMF tests - rpms/cockpit-podman

rpms / cockpit-podman

#32 WIP: PoC: Replace STI tests with upstream FMF tests

Closed 2 years ago by martinpitt. Opened 3 years ago by martinpitt.

rpms/ martinpitt/cockpit-podman rawhide into rawhide

Martin Pitt • 2 years ago

5b4e96b

.fmf/version

file added

		`@@ -0,0 +1,1 @@`
		`+ 1`

plans/upstream.fmf

file added

		`@@ -0,0 +1,7 @@`
		`+ discover:`
		`+ how: fmf`
		`+ repository: https://github.com/cockpit-project/cockpit-podman`
		`+ # FIXME: get rid of the hardcoding: https://github.com/psss/tmt/issues/585`
		`+ ref: "46"`
		`+ execute:`
		`+ how: tmt`

tests/.gitignore

file removed

-3

		`@@ -1,3 +0,0 @@`
		`- *.retry`
		`- artifacts/`
		`- source/`

tests/roles/test/tasks/main.yml

file removed

-30

		`@@ -1,30 +0,0 @@`
		`- ---`
		`- - name: upgrade critical packages to the latest version`
		`- dnf:`
		`- name:`
		`- - podman`
		`- - conmon`
		`- state: latest`
		`-`
		`- - include_role:`
		`- name: standard-test-source`
		`-`
		`- - include_role:`
		`- name: standard-test-basic`
		`- vars:`
		`- required_packages:`
		`- - cockpit-podman`
		`- - cockpit-ws`
		`- - cockpit-system`
		`- - git`
		`- - libvirt-python3`
		`- - make`
		`- - npm`
		`- - python3`
		`- tests:`
		`- - cockpit-podman:`
		`- dir: "{{ test_script_dir \| default('.') }}/source"`
		`- # directory changed after release 29: https://github.com/cockpit-project/cockpit-podman/pull/684`
		`- # drop this after packaging 30`
		`- run: "if [ -e test/browser/browser.sh ]; then test/browser/browser.sh; else plans/browser.sh; fi"`
		`- save-files: ["logs/*"]`

tests/tests.yml

file removed

-6

		`@@ -1,6 +0,0 @@`
		`- ---`
		`- - hosts: localhost`
		`- tags:`
		`- - classic`
		`- roles:`
		`- - test`

martinpitt commented 3 years ago

Don't land this! We need STI for RHEL's container-tools module.

But this is a learning exercise for me how to run FMF tests in Fedora, and how to share as much code as possible.

martinpitt commented 3 years ago

Locally that doesn't work at all for me:

$ tmt run discover --debug --verbose -n provision --how local
/var/tmp/tmt/run-006

/plans/upstream
    discover
        workdir: /var/tmp/tmt/run-006/plans/upstream/discover
        how: fmf
        url: https://github.com/cockpit-project/cockpit-podman
            Clone 'https://github.com/cockpit-project/cockpit-podman' to '/var/tmp/tmt/run-006/plans/upstream/discover/default/tests'.
        ref: 29
            Checkout ref '29'.
            Check metadata tree in '/var/tmp/tmt/run-006/plans/upstream/discover/default/tests/'.
        summary: 0 tests selected
        warning: No tests found, finishing plan.

Neither does it actually clone the upstream repo to the claimed directory:

 ls -l /var/tmp/tmt/run-006/plans/upstream/discover/default/
total 0

Nor does it seem to find any test in it. I tried with the official recipe, and the same situation with that.

Well, let's see what the official infra has to say about this.

zuul commented 3 years ago

Build succeeded.

check-for-arches : SUCCESS in 1m 09s
rpm-scratch-build : SUCCESS in 6m 26s
rpm-scratch-build-s390x : SKIPPED (non-voting)
rpm-scratch-build-ppc64le : SKIPPED (non-voting)
rpm-scratch-build-i686 : SKIPPED (non-voting)
rpm-scratch-build-armv7hl : SKIPPED (non-voting)
rpm-scratch-build-aarch64 : SKIPPED (non-voting)
rpm-linter : SUCCESS in 3m 10s
rpm-rpminspect : TIMED_OUT in 30m 37s (non-voting)
check-for-tests : SUCCESS in 37s
rpm-install-test : SUCCESS in 2m 07s
rpm-test : SKIPPED

martinpitt commented 3 years ago

Added a temporary fixup to use @psss ' upstream branch from https://github.com/cockpit-project/cockpit-podman/pull/684

1 new commit added

Temporarily switch to https://github.com/cockpit-project/cockpit-podman/pull/684 branch

3 years ago

martinpitt commented 3 years ago

This kinda sorta worked, but it stopped after running just one test, and then interrupted/errored out. Interestingly the upstream PR has exactly the same behaviour. But this is progress!

martinpitt commented 3 years ago

[citest]

zuul commented 3 years ago

Build succeeded.

check-for-arches : SUCCESS in 1m 12s
rpm-scratch-build : SUCCESS in 6m 13s
rpm-scratch-build-s390x : SKIPPED (non-voting)
rpm-scratch-build-ppc64le : SKIPPED (non-voting)
rpm-scratch-build-i686 : SKIPPED (non-voting)
rpm-scratch-build-armv7hl : SKIPPED (non-voting)
rpm-scratch-build-aarch64 : SKIPPED (non-voting)
rpm-linter : SUCCESS in 3m 07s
rpm-rpminspect : TIMED_OUT in 30m 38s (non-voting)
check-for-tests : SUCCESS in 35s
rpm-install-test : SUCCESS in 2m 33s
rpm-test : SKIPPED

martinpitt commented 3 years ago

Nice, this worked! I cleaned up https://github.com/cockpit-project/cockpit-podman/pull/684 and will land that upstream.

The remaining wart here is the hardcoded ref: "29", which is a blocker; and the necessity to check out a remote git, which is a bit of a wart, and less robust than the STI approach of always unpacking the upstream source tarballs in a dist-git.

I filed https://github.com/psss/tmt/issues/585 to discuss this.

rebased onto 119a3e6f45bd8346f47aa7db9e2c298302db7118

3 years ago

martinpitt commented 3 years ago

Rebased to current test adjustments in dist-git and FMF shareable tests now landed upstream.

zuul commented 3 years ago

Build succeeded.