PR#3: package tests, run integration tests in CI - rpms/python-docker

rpms / python-docker

#3 package tests, run integration tests in CI

Merged 6 years ago by ttomecek. Opened 6 years ago by ttomecek.

rpms/ ttomecek/python-docker package-tests into master

package tests, run integr tests in CI

Tomas Tomecek • 6 years ago

832ae42

.gitignore

file modified

		`@@ -7,3 +7,6 @@`
		`/docker-2.5.1.tar.gz`
		`/docker-2.6.0.tar.gz`
		`/docker-2.6.1.tar.gz`
		`+ tests/artifacts/`
		`+ tests/tests.retry`
		`+ tests/atomic.qcow2`

python-docker.spec

file modified

+28 -1

		`@@ -11,7 +11,7 @@`

		`Name: python-%{srcname}`
		`Version: 2.6.1`
		`- Release: 1%{?dist}`
		`+ Release: 2%{?dist}`
		`Summary: A Python library for the Docker Engine API`
		`License: ASL 2.0`
		`URL: https://pypi.python.org/pypi/%{srcname}`
		`@@ -74,6 +74,23 @@`
		`It lets you do anything the docker command does, but from within Python apps –`
		`run containers, manage containers, manage Swarms, etc.`

		`+ %package tests`
		`+ Summary: Unit tests and integration tests for python-docker`
		`+ Requires: python2-%{srcname}`
		`+ Requires: python2-mock >= 1.0.1`
		`+ Requires: %{?fedora:python2-}pytest >= 2.9.1`
		`+ Requires: python%{?fedora:2}-coverage >= 3.7.1`
		`+ Requires: python2-pytest-cov >= 2.1.0`
		`+ Requires: python%{?fedora:2}-flake8 >= 2.4.1`
		`+ Requires: python%{?fedora:2}-requests >= 2.5.2`
		`+ Requires: python%{?fedora:2}-six >= 1.4.0`
		`+ Requires: python-websocket-client >= 0.32.0`
		`+ Requires: python%{?fedora:2}-docker-pycreds >= 0.2.1`
		`+ Requires: python%{?fedora:2}-backports-ssl_match_hostname >= 3.5`
		`+ Requires: python-ipaddress >= 1.0.16`
		`+ %description tests`
		`+ Upstream test-suite (unit, integration) packaged as RPM.`
		`+`
		`%if %{with python3}`
		`%package -n python3-%{srcname}`
		`Summary: A Python library for the Docker Engine API`
		`@@ -126,6 +143,10 @@`
		`%py3_install`
		`%endif # with_python3`

		`+ # copy tests to /usr/libexec/installed-tests`
		`+ mkdir -p %{buildroot}%{_libexecdir}/installed-tests/%{name}`
		`+ cp -avr tests/ %{buildroot}%{_libexecdir}/installed-tests/%{name}/`
		`+`
		`%check`
		`%if %{with tests}`
		`%{__python2} -m pytest -v tests/unit/`
		`@@ -150,7 +171,13 @@`
		`%{python3_sitelib}/*`
		`%endif # with_python3`

		`+ %files tests`
		`+ %{_libexecdir}/installed-tests`
		`+`
		`%changelog`
		`+ * Tue Nov 28 2017 Tomas Tomecek <ttomecek@redhat.com> - 2.6.1-2`
		`+ - Package tests`
		`+`
		`* Thu Nov 09 2017 Tomas Tomecek <ttomecek@redhat.com> - 2.6.1-1`
		`- new upstream release: 2.6.1`

tests/TODO

file added

		`@@ -0,0 +1,1 @@`
		`+ * build RPMs locally and inject them inside atomic host VM (local testing should not rely on repositories)`

tests/integration.yml

file added

+67

		`@@ -0,0 +1,67 @@`
		`+ ---`
		`+ - hosts: localhost`
		`+ vars:`
		`+ - artifacts: ./artifacts`
		`+ - remote_artifacts: /root/artifacts`
		`+ tasks:`
		`+ - name: Make local artifacts directory`
		`+ file: path={{ artifacts }} state=directory owner=root mode=755 recurse=yes`
		`+ tags:`
		`+ - classic`
		`+`
		`+ - name: Make artifacts directory on Atomic Host`
		`+ file: path={{ remote_artifacts }} state=directory owner=root mode=755 recurse=yes`
		`+ tags:`
		`+ - atomic`
		`+`
		`+ - name: Install pytest and the test-suite locally`
		`+ dnf:`
		`+ name: "{{ item }}"`
		`+ state: present`
		`+ tags:`
		`+ - classic`
		`+ with_items:`
		`+ - python-docker-tests`
		`+ - python2-pytest`
		`+`
		`+ - name: Install pytest and the test-suite on Atomic Host`
		`+ shell: rpm-ostree install python2-pytest python-docker-tests && rpm-ostree ex livefs`
		`+ tags:`
		`+ - atomic`
		`+`
		`+ - name: Ensure dockerd is running`
		`+ service:`
		`+ name: docker`
		`+ state: started`
		`+ tags:`
		`+ - atomic`
		`+ - classic`
		`+`
		`+ - name: Atomic Host test block`
		`+ block:`
		`+ - name: Run integration test-suite on Atomic Host`
		`+ # -k "test_create_with_restart_policy" for local testing`
		`+ shell: py.test -v ./tests/integration/ \| tee {{ remote_artifacts }}/test.log`
		`+ args:`
		`+ chdir: /usr/libexec/installed-tests/python-docker/`
		`+ always:`
		`+ - name: Pull out the artifacts`
		`+ fetch:`
		`+ dest: "{{ artifacts }}/"`
		`+ src: "{{ remote_artifacts }}/test.log"`
		`+ tags:`
		`+ - atomic`
		`+`
		`+ - name: Get abspath of artifacts dir`
		`+ command: realpath {{ artifacts }}`
		`+ register: artifacts_abs`
		`+ tags:`
		`+ - classic`
		`+`
		`+ - name: Run integration test-suite locally`
		`+ # -k "test_create_with_restart_policy" for local testing`
		`+ shell: py.test -v ./tests/integration/ \| tee {{ artifacts_abs.stdout }}/test.log`
		`+ args:`
		`+ chdir: /usr/libexec/installed-tests/python-docker/`
		`+ tags:`
		`+ - classic`

tests/tests.yml

file added

		`@@ -0,0 +1,2 @@`
		`+ - name: Run the tests`
		`+ import_playbook: integration.yml`

ttomecek commented 6 years ago

I packaged tests so they can be run in CI easily: https://fedoraproject.org/wiki/CI/Tests

dperpeet commented 6 years ago

I will link my pr with Atomic Host tests later, but just for the record: I use the following to install missing packages for the tests and to activate them immediately without rebooting the atomic host, in this case for the package pesign (kernel-tests):

shell: rpm-ostree install pesign && rpm-ostree ex livefs

rebased onto a7fc24fc2dc1a51b8a677f3e6ba2dbaa6ec2c558

6 years ago

rebased onto 717a8a11bded78789fc64deb59b8206e5db38b46

6 years ago

pingou commented 6 years ago

Do you need help finishing/polishing this PR or just a review? :)

ttomecek commented 6 years ago

@pingou I think it's ready to be merged

ttomecek commented 6 years ago

@pingou but getting a review would be very nice of you

sturivny commented 6 years ago

I've tried to run tests on localhost and got an Error:
https://paste.fedoraproject.org/paste/tDzo1l90d6O9wLUQ6t8ckg/raw

sturivny commented 6 years ago

A possible way to fix:
using pip instead of dnf

pip install python-docker-test

Edited 6 years ago by sturivny

sturivny commented 6 years ago

Please, update PR

[DEPRECATION WARNING]: 'include' for playbook includes. You should use 'import_playbook' instead. This feature will be removed in version 2.8. Deprecation warnings can be disabled by setting 

deprecation_warnings=False in ansible.cfg.

sturivny commented 6 years ago

Those tests also Fails on Atomic Host:
https://paste.fedoraproject.org/paste/QXi0XOL7t9m4rm6K~kkiNw/raw

ttomecek commented 6 years ago

@sturivny regarding the errors: that's correct, the subpackage does not exist yet and is being added as part of this PR. The issue is that there should be an easy way to build the RPMs locally and then copy them inside atomic host VM or install them on localhost. It's true that phoebe CI will provide those when running the tests in the pipeline -- but locally, there should be a way to run the tests -- build RPMs and test them in the test subject. I think the pattern will be very common for these type of tests and should ideally be described in docs or even be part of standard test roles.

Regarding the warning, I actually switched to import_playbook.

Since this PR should not break anything, I'm merging it and we can improve the tests with subsequent PRs.

Serhii, thanks for your help!

rebased onto 832ae42

6 years ago

Pull-Request has been merged by ttomecek

6 years ago