rpms / pyproject-rpm-macros

Clone
Source Code
GIT

#413 Add %_pyproject_check_import_allow_no_modules
Merged 9 months ago by churchyard. Opened 9 months ago by ksurma.
rpms/ ksurma/pyproject-rpm-macros allow-empty-macro  into  rawhide

file modified
+6 
@@ -342,6 +342,12 @@ 
 

  additional qualified module names to check, useful for example if some modules are installed manually.
 

  Note that filtering by `-t`/`-e` also applies to the positional arguments.
 

  

+ Another macro, `%_pyproject_check_import_allow_no_modules` allows to pass the import check,
 

+ even if no Python modules are detected in the package.
 

+ This may be a valid case for packages containing e.g. typing stubs.
 

+ Don't use this macro in Fedora packages.
 

+ It's only intended to be used in automated build environments such as Copr.
 

+ 

  

  Generating Extras subpackages
 

  -----------------------------

file modified
+8 
@@ -133,6 +133,14 @@ 
 

  }
 

  

  

+ %_pyproject_check_import_allow_no_modules(e:t) \
 

+ if [ -z "$(cat %{_pyproject_modules})" ]; then\
 

+   echo "No modules to check found, exiting check"\
 

+ else\
 

+   %pyproject_check_import %{?**}\
 

+ fi
 

+ 

+ 

  %default_toxenv py%{python3_version_nodots}
 

  %toxenv %{default_toxenv}

file modified
+5 -2 
@@ -13,8 +13,8 @@ 
 

  #   Increment Y and reset Z when new macros or features are added
 

  #   Increment Z when this is a bugfix or a cosmetic change
 

  # Dropping support for EOL Fedoras is *not* considered a breaking change
 

- Version:        1.9.0
 

- Release:        2%{?dist}
 

+ Version:        1.10.0
 

+ Release:        1%{?dist}
 

  

  # Macro files
 

  Source001:      macros.pyproject
 
@@ -161,6 +161,9 @@ 
 

  

  

  %changelog
 

+ * Wed Sep 13 2023 Karolina Surma <ksurma@redhat.com> - 1.10.0-1
 

+ - Add %%_pyproject_check_import_allow_no_modules for automated environments
 

+ 

  * Fri Jul 21 2023 Fedora Release Engineering <releng@fedoraproject.org> - 1.9.0-2
 

  - Rebuilt for https://fedoraproject.org/wiki/Fedora_39_Mass_Rebuild

file modified
+31 -2 
@@ -12,7 +12,10 @@ 
 

  

  

  %global _description %{expand:
 

- Test that manpages are correctly processed by %%pyproject_save_files '*' +auto.}
 

+ Test that manpages are correctly processed by %%%%%%%%pyproject_save_files '*' +auto.
 

+ Run %%%%%%%%_pyproject_check_import_allow_no_modules twice
 

+ - exclude all modules and test the check still passes thanks to -M option
 

+ - regression test: test that check imports all modules even if -M option is set}
 

  

  

  %description %_description
 
@@ -41,10 +44,36 @@ 
 

  

  

  %check
 

+ # Internal check for our macros, assert the behavior of the import check macros
 

+ # Both of the macros should succeed
 

  %pyproject_check_import
 

+ %_pyproject_check_import_allow_no_modules
 

+ (%{pyproject_check_import}) 2>pyproject_check_import.stderr
 

+ (%{_pyproject_check_import_allow_no_modules}) 2>_pyproject_check_import_allow_no_modules.stderr
 

+ 

+ # Modules were found, stderrs should include getmac.getmac
 

+ grep '^Check import: getmac\.getmac$' pyproject_check_import.stderr
 

+ grep '^Check import: getmac\.getmac$' _pyproject_check_import_allow_no_modules.stderr
 

+ 

+ # Now let's pretend no modules were found at all
 

+ echo -e '' > %{_pyproject_modules}
 

+ 

+ # This should fail
 

+ (%{pyproject_check_import}) && exit 1 || true
 

+ 

+ # This should succeed and say something about no modules found
 

+ %{_pyproject_check_import_allow_no_modules}
 

+ (%{_pyproject_check_import_allow_no_modules}) 2>_pyproject_check_import_allow_no_modules.stderr
 

+ grep '\bNo modules to check found\b' _pyproject_check_import_allow_no_modules.stderr
 

+ 

+ # We want to ensure the rest of the %%check section is still executed
 

+ # (To avoid a temptation to call `exit 0` from %%_pyproject_check_import_allow_no_modules)
 

+ # We'll touch a marker file here and assert its presence in %%files
 

+ touch %{buildroot}/check-completed-entirely
 

+ 

  # Internal check for our macros, assert there is a manpage:
 

  test -f %{buildroot}%{_mandir}/man1/getmac.1*
 

  

  

  %files -n python3-getmac -f %{pyproject_files}
 

- 

+ /check-completed-entirely

Macro which allows to pass the import check even if no Python modules
are detected in the package.
Only to be used in the automated environments.

rebased onto 9425c553c8f096c22abb8b3d1e79e438ebd461d0
9 months ago

The fact that we can echo modules separated by spaces to this file seems accidental. I've been thinking about this: what is the reason to run the empty check as the first one instead of running it as the last one?

If this is to assert that the macro won't exit the %check section prematurely, it probably does not assert that:

  • When a clean exit (exit 0) happens, the build succeeds even if the rest of the %check section is never executed. AFAIK the only way to assert the %check section continued to the end would be to put a marker file into %{buildroot} at the end of %check and list it in %files (installing files in %check is not a good idea regularly, but with a proper comment should be fine here).
  • When a clean exit (exit 0) happens in a subshell (i.e. (exit 0)), the parent shell does not exit anyway.

My untested proposal:

%check
# Internal check for our macros, assert the behavior of the import check macros
# Both of the macros should succeed:
%{pyproject_check_import} 2>pyproject_check_import.stderr
%{_pyproject_check_import_allow_no_modules} 2>_pyproject_check_import_allow_no_modules.stderr

# Modules were found, the stderrs should match and include getmac.getmac:
diff -u pyproject_check_import.stderr _pyproject_check_import_allow_no_modules.stderr
grep '^Check import: getmac\.getmac$' pyproject_check_import.stderr

# Now let's pretend no modules were found at all:
echo -e '' > %{_pyproject_modules}

# This should fail:
%{pyproject_check_import} && exit 1 || true

# This should succeed and say something about no modules found
%{_pyproject_check_import_allow_no_modules} 2>_pyproject_check_import_allow_no_modules.stderr
grep -i '\bno modules\b' _pyproject_check_import_allow_no_modules.stderr

# We want to ensure the rest of the %%check section is still executed
# (To avoid a temptation to call `exit 0` from %%_pyproject_check_import_allow_no_modules)
# We'll touch a marker file here and assert its presence in %%files
touch %{buildroot}/check-completed-entirely


...

%files -n python3-getmac -f %{pyproject_files}
# Internal check for our macros:
/check-completed-entirely

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/da29a2d51ef34eea8f94216bc8cfccc2

what is the reason to run the empty check as the first one instead of running it as the last one?
If this is to assert that the macro won't exit the %check section prematurely, it probably does not assert that:

  • When a clean exit (exit 0) happens, the build succeeds even if the rest of the %check section is never executed. AFAIK the only way to assert the %check section continued to the end would be to put a marker file into %{buildroot} at the end of %check and list it in %files (installing files in %check is not a good idea regularly, but with a proper comment should be fine here).
  • When a clean exit (exit 0) happens in a subshell (i.e. (exit 0)), the parent shell does not exit anyway.

Yes, that's the reason I had in mind. I focused on unsuccessful exits and didn't think of a clean exit scenario. This explanation makes sense to me.

I tested your proposal step by step.

I can't comprehend why %pyproject_check_import must be run in a subshell in order to populate the file while %_pyproject_check_import_allow_no_modules populates the file no matter whether run in a subshell or not. I mean, %_pyproject_check_import_allow_no_modules just runs %pyproject_check_import under the hood, so I'm missing a piece of a puzzle here. 

# all this works
(%{pyproject_check_import}) 2>pyproject_check_import.stderr
%{_pyproject_check_import_allow_no_modules} 2>_pyproject_check_import_allow_no_modules.stderr
(%{_pyproject_check_import_allow_no_modules}) 2>_pyproject_check_import_allow_no_modules.stderr

# this doesn't populate the file
%{pyproject_check_import} 2>pyproject_check_import.stderr

The outputs of the files differ as _pyproject_check_import_allow_no_modules.stderr contains three more lines.

++ cat /builddir/build/BUILD/python-getmac-0.8.3-0.fc40.x86_64-pyproject-modules
+ '[' -z 'getmac
getmac.getmac' ']'

First I thought of using tail -n +4 to diff the files, but as a second thought I don't like it, it's arbitrary and magical, we'd have to manually adjust it every time macro definition changes. grepping for the Check import lines in both files seems sufficient to me. WDYT?

I also tested the marker file bit and verified the build would indeed fail in case of macro happily calling exit 0.

1 new commit added

  • Refactor the integration test
9 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/f355e5e12f8546059fdfe4f3b37ecbb3

Yeah, the macros don't evaluate to one-liners, so whether or not we can stash > after them is... magical.

Let me see what we have here now, sorry for taking so long.

Contemplation: For consistency, should we subshell both of them?

rebased onto 31ede4632a6f942ccaf8c5d455a8d35869722a5e
9 months ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/f6c3dabf0c544c1292f5d90228d221da

rebased onto 97a7574
9 months ago

I've rebased this using the pagure orange button.

I guess it's time to open PRs for all the branches. Shall I do it, or do you prefer to do it yourself?

I've rebased this using the pagure orange button.

I guess it's time to open PRs for all the branches. Shall I do it, or do you prefer to do it yourself?

Done - let's see what CI thinks about all this.

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/d85214a3453e4c4e936bb1465a68dc9b

All green.

Pull-Request has been merged by churchyard
9 months ago
Metadata
None
No Tags
Changes Summary 4
+6 -0
file changed
README.md
+8 -0
file changed
macros.pyproject
+5 -2
file changed
pyproject-rpm-macros.spec
+31 -2
file changed
tests/python-getmac.spec
Powered by Pagure 5.14.1
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.