rpms / pyproject-rpm-macros

Clone
Source Code
GIT

#398 [f37] Allow passing config_settings to the build backend
Merged a year ago by churchyard. Opened a year ago by churchyard.
rpms/ gotmax23/pyproject-rpm-macros config-settings  into  f37

file modified
+37 
@@ -163,6 +163,43 @@ 
 

  it means "include runtime dependencies" which has been the default since version 0-53.
 

  

  

+ Passing config settings to build backends
 

+ -----------------------------------------
 

+ 

+ The `%pyproject_buildrequires` and `%pyproject_wheel` macros accept a `-C` flag
 

+ to pass [configuration settings][config_settings] to the build backend.
 

+ Options take the form of `-C KEY`, `-C KEY=VALUE`, or `-C--option-with-dashes`.
 

+ Pass `-C` multiple times to specify multiple options.
 

+ This option is equivalent to pip's `--config-settings` flag.
 

+ These are passed on to PEP 517 hooks' `config_settings` argument as a Python
 

+ dictionary.
 

+ 

+ The `%pyproject_buildrequires` macro passes these options to the
 

+ `get_requires_for_build_wheel` and `prepare_metadata_for_build_wheel` hooks.
 

+ Passing `-C` to `%pyproject_buildrequires` is incompatible with `-N` which does
 

+ not call these hooks at all.
 

+ 

+ The `%pyproject_wheel` macro passes these options to the `build_wheel` hook.
 

+ 

+ Consult the project's upstream documentation and/or the corresponding build
 

+ backend's documentation for more information.
 

+ Note that some projects don't use config settings at all
 

+ and other projects may only accept config settings for one of the two steps.
 

+ 

+ Note that the current implementation of the macros uses `pip` to build wheels.
 

+ On some systems (notably on RHEL 9 with Python 3.9),
 

+ `pip` is too old to understand `--config-settings`.
 

+ Using the `-C` option for `%pyproject_wheel` (or `%pyproject_buildrequires -w`)
 

+ is not supported there and will result to an error like:
 

+ 

+     Usage:   

+       /usr/bin/python3 -m pip wheel [options] <requirement specifier> ...
 

+       ...
 

+     no such option: --config-settings
 

+ 

+ [config_settings]: https://peps.python.org/pep-0517/#config-settings
 

+ 

+ 

  Running tox based tests
 

  -----------------------

file modified
+1 -1 
@@ -4,4 +4,4 @@ 
 

  # this macro will cause the package with the real macro to be installed.
 

  # When macros.pyproject is installed, it overrides this macro.
 

  # Note: This needs to maintain the same set of options as the real macro.
 

- %pyproject_buildrequires(rRxtNwe:) echo 'pyproject-rpm-macros' && exit 0
 

+ %pyproject_buildrequires(rRxtNwe:C:) echo 'pyproject-rpm-macros' && exit 0

file modified
+4 -3 
@@ -26,11 +26,11 @@ 
 

  # The value is read and used by the %%pytest and %%tox macros:
 

  %_set_pytest_addopts %global __pytest_addopts --ignore=%{_pyproject_builddir}
 

  

- %pyproject_wheel() %{expand:\\\
 

+ %pyproject_wheel(C:) %{expand:\\\
 

  %_set_pytest_addopts
 

  mkdir -p "%{_pyproject_builddir}"
 

  CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}" TMPDIR="%{_pyproject_builddir}" \\\
 

- %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_wheel.py %{_pyproject_wheeldir}
 

+ %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_wheel.py %{?**} %{_pyproject_wheeldir}
 

  }
 

  

  
@@ -140,7 +140,7 @@ 
 

  

  

  # Note: Keep the options in sync with this macro from macros.aaa-pyproject-srpm
 

- %pyproject_buildrequires(rRxtNwe:) %{expand:\\\
 

+ %pyproject_buildrequires(rRxtNwe:C:) %{expand:\\\
 

  %_set_pytest_addopts
 

  # The _auto_set_build_flags feature does not do this in %%generate_buildrequires section,
 

  # but we want to get an environment consistent with %%build:
 
@@ -158,6 +158,7 @@ 
 

  %{-e:%{error:The -N and -e options are mutually exclusive}}
 

  %{-t:%{error:The -N and -t options are mutually exclusive}}
 

  %{-w:%{error:The -N and -w options are mutually exclusive}}
 

+ %{-C:%{error:The -N and -C options are mutually exclusive}}
 

  }
 

  %{-e:%{expand:%global toxenv %(%{__python3} -s %{_rpmconfigdir}/redhat/pyproject_construct_toxenv.py %{?**})}}
 

  echo 'pyproject-rpm-macros'  # first stdout line matches the implementation in macros.aaa-pyproject-srpm

file modified
+5 -1 
@@ -13,7 +13,7 @@ 
 

  #   Increment Y and reset Z when new macros or features are added
 

  #   Increment Z when this is a bugfix or a cosmetic change
 

  # Dropping support for EOL Fedoras is *not* considered a breaking change
 

- Version:        1.8.1
 

+ Version:        1.9.0
 

  Release:        1%{?dist}
 

  

  # Macro files
 
@@ -161,6 +161,10 @@ 
 

  

  

  %changelog
 

+ * Wed May 31 2023 Maxwell G <maxwell@gtmx.me> - 1.9.0-1
 

+ - Allow passing config_settings to the build backend.
 

+ - Resolves: rhbz#2192581
 

+ 

  * Wed May 31 2023 Miro Hrončok <mhroncok@redhat.com> - 1.8.1-1
 

  - On Python older than 3.11, use tomli instead of deprecated toml
 

  - Fix literal %% handling in %%{pyproject_files} on RPM 4.19

file modified
+20 -6 
@@ -14,6 +14,7 @@ 
 

  import zipfile
 

  

  from pyproject_requirements_txt import convert_requirements_txt
 

+ from pyproject_wheel import parse_config_settings_args
 

  

  

  # Some valid Python version specifiers are not supported.
 
@@ -67,7 +68,7 @@ 
 

  class Requirements:
 

      """Requirement gatherer. The macro will eventually print out output_lines."""
 

      def __init__(self, get_installed_version, extras=None,
 

-                  generate_extras=False, python3_pkgversion='3'):
 

+                  generate_extras=False, python3_pkgversion='3', config_settings=None):
 

          self.get_installed_version = get_installed_version
 

          self.output_lines = []
 

          self.extras = set()
 
@@ -81,6 +82,7 @@ 
 

  

          self.generate_extras = generate_extras
 

          self.python3_pkgversion = python3_pkgversion
 

+         self.config_settings = config_settings
 

  

      def add_extras(self, *extras):
 

          self.extras |= set(e.strip() for e in extras)
 
@@ -269,7 +271,7 @@ 
 

  def generate_build_requirements(backend, requirements):
 

      get_requires = getattr(backend, 'get_requires_for_build_wheel', None)
 

      if get_requires:
 

-         new_reqs = get_requires()
 

+         new_reqs = get_requires(config_settings=requirements.config_settings)
 

          requirements.extend(new_reqs, source='get_requires_for_build_wheel')
 

          requirements.check(source='get_requires_for_build_wheel')
 

  
@@ -303,7 +305,7 @@ 
 

              'Use the provisional -w flag to build the wheel and parse the metadata from it, '
 

              'or use the -R flag not to generate runtime dependencies.'
 

          )
 

-     dir_basename = prepare_metadata('.')
 

+     dir_basename = prepare_metadata('.', config_settings=requirements.config_settings)
 

      with open(dir_basename + '/METADATA') as metadata_file:
 

          name, requires = package_name_and_requires_from_metadata_file(metadata_file)
 

          for key, req in requires.items():
 
@@ -327,7 +329,11 @@ 
 

      wheel = find_built_wheel(wheeldir)
 

      if not wheel:
 

          import pyproject_wheel
 

-         returncode = pyproject_wheel.build_wheel(wheeldir=wheeldir, stdout=sys.stderr)
 

+         returncode = pyproject_wheel.build_wheel(
 

+             wheeldir=wheeldir,
 

+             stdout=sys.stderr,
 

+             config_settings=requirements.config_settings,
 

+         )
 

          if returncode != 0:
 

              raise RuntimeError('Failed to build the wheel for %pyproject_buildrequires -w.')
 

          wheel = find_built_wheel(wheeldir)
 
@@ -415,7 +421,7 @@ 
 

      *, include_runtime=False, build_wheel=False, wheeldir=None, toxenv=None, extras=None,
 

      get_installed_version=importlib.metadata.version,  # for dep injection
 

      generate_extras=False, python3_pkgversion="3", requirement_files=None, use_build_system=True,
 

-     output,
 

+     output, config_settings=None,
 

  ):
 

      """Generate the BuildRequires for the project in the current directory
 

  
@@ -426,7 +432,8 @@ 
 

      requirements = Requirements(
 

          get_installed_version, extras=extras or [],
 

          generate_extras=generate_extras,
 

-         python3_pkgversion=python3_pkgversion
 

+         python3_pkgversion=python3_pkgversion,
 

+         config_settings=config_settings,
 

      )
 

  

      try:
 
@@ -516,6 +523,12 @@ 
 

          metavar='REQUIREMENTS.TXT',
 

          help=('Add buildrequires from file'),
 

      )
 

+     parser.add_argument(
 

+         '-C',
 

+         dest='config_settings',
 

+         action='append',
 

+         help='Configuration settings to pass to the PEP 517 backend',
 

+     )
 

  

      args = parser.parse_args(argv)
 

  
@@ -550,6 +563,7 @@ 
 

              requirement_files=args.requirement_files,
 

              use_build_system=args.use_build_system,
 

              output=args.output,
 

+             config_settings=parse_config_settings_args(args.config_settings),
 

          )
 

      except Exception:
 

          # Log the traceback explicitly (it's useful debug info)

file modified
+37 
@@ -986,3 +986,40 @@ 
 

      python3dist(rightdep)
 

      python3dist(startdep)
 

    result: 0
 

+ 

+ config_settings_control:
 

+   include_runtime: false
 

+   config_settings:
 

+   pyproject.toml: |
 

+     [build-system]
 

+     build-backend = "test_backend"
 

+     backend-path = ["."]
 

+   test_backend.py: |
 

+     def get_requires_for_build_wheel(config_settings=None):
 

+         if not (config_settings is None or isinstance(config_settings, dict)):
 

+             raise TypeError
 

+         if config_settings and "test-config-setting" in config_settings:
 

+             return ["test-config-setting"]
 

+         return ["test-no-config-setting"]
 

+   expected: |
 

+     python3dist(test-no-config-setting)
 

+   result: 0
 

+ 

+ config_settings:
 

+   include_runtime: false
 

+   config_settings:
 

+     test-config-setting: ""
 

+   pyproject.toml: |
 

+     [build-system]
 

+     build-backend = "test_backend"
 

+     backend-path = ["."]
 

+   test_backend.py: |
 

+     def get_requires_for_build_wheel(config_settings=None):
 

+         if not (config_settings is None or isinstance(config_settings, dict)):
 

+             raise TypeError
 

+         if config_settings and "test-config-setting" in config_settings:
 

+             return ["test-config-setting"]
 

+         return ["test-no-config-setting"]
 

+   expected: |
 

+     python3dist(test-config-setting)
 

+   result: 0

file modified
+55 -2 
@@ -1,8 +1,46 @@ 
 

+ import argparse
 

  import sys
 

  import subprocess
 

  

  

- def build_wheel(*, wheeldir, stdout=None):
 

+ def parse_config_settings_args(config_settings):
 

+     """
 

+     Given a list of config `KEY=VALUE` formatted config settings,
 

+     return a dictionary that can be passed to PEP 517 hook functions.
 

+     """
 

+     if not config_settings:
 

+         return config_settings
 

+     new_config_settings = {}
 

+     for arg in config_settings:
 

+         key, _, value = arg.partition('=')
 

+         if key in new_config_settings:
 

+             if not isinstance(new_config_settings[key], list):
 

+                 # convert the existing value to a list
 

+                 new_config_settings[key] = [new_config_settings[key]]
 

+             new_config_settings[key].append(value)
 

+         else:
 

+             new_config_settings[key] = value
 

+     return new_config_settings
 

+ 

+ 

+ def get_config_settings_args(config_settings):
 

+     """
 

+     Given a dictionary of PEP 517 backend config_settings,
 

+     yield --config-settings args that can be passed to pip's CLI
 

+     """
 

+     if not config_settings:
 

+         return
 

+     for key, values in config_settings.items():
 

+         if not isinstance(values, list):
 

+             values = [values]
 

+         for value in values:
 

+             if value == '':
 

+                 yield f'--config-settings={key}'
 

+             else:
 

+                 yield f'--config-settings={key}={value}'
 

+ 

+ 

+ def build_wheel(*, wheeldir, stdout=None, config_settings=None):
 

      command = (
 

          sys.executable,
 

          '-m', 'pip',
 
@@ -15,11 +53,26 @@ 
 

          '--no-clean',
 

          '--progress-bar', 'off',
 

          '--verbose',
 

+         *get_config_settings_args(config_settings),
 

          '.',
 

      )
 

      cp = subprocess.run(command, stdout=stdout)
 

      return cp.returncode
 

  

  

+ def parse_args(argv=None):
 

+     parser = argparse.ArgumentParser(prog='%pyproject_wheel')
 

+     parser.add_argument('wheeldir', help=argparse.SUPPRESS)
 

+     parser.add_argument(
 

+         '-C',
 

+         dest='config_settings',
 

+         action='append',
 

+         help='Configuration settings to pass to the PEP 517 backend',
 

+     )
 

+     args = parser.parse_args(argv)
 

+     args.config_settings = parse_config_settings_args(args.config_settings)
 

+     return args
 

+ 

+ 

  if __name__ == '__main__':
 

-     sys.exit(build_wheel(wheeldir=sys.argv[1]))
 

+     sys.exit(build_wheel(**vars(parse_args())))

file modified
+1 
@@ -63,6 +63,7 @@ 
 

              requirement_files=requirement_files,
 

              use_build_system=use_build_system,
 

              output=output,
 

+             config_settings=case.get('config_settings'),
 

          )
 

      except SystemExit as e:
 

          assert e.code == case['result']

file added
+50 
@@ -0,0 +1,50 @@ 
 

+ Name:           config-settings-test
 

+ Version:        1.0.0
 

+ Release:        1%{?dist}
 

+ Summary:        Test config_settings support
 

+ 

+ License:        MIT
 

+ URL:            ...
 

+ Source0:        config_settings_test_backend.py
 

+ 

+ 

+ %description
 

+ %{summary}.
 

+ 

+ 

+ %prep
 

+ %autosetup -cT
 

+ 

+ cp -p %{sources} .
 

+ 

+ cat <<'EOF' >config_settings.py
 

+ """
 

+ This is a test package
 

+ """
 

+ EOF
 

+ 

+ cat <<'EOF' >pyproject.toml
 

+ [build-system]
 

+ build-backend = "config_settings_test_backend"
 

+ backend-path = ["."]
 

+ requires = ["flit-core", "packaging", "pip"]
 

+ 

+ [project]
 

+ name = "config_settings"
 

+ version = "%{version}"
 

+ dynamic = ["description"]
 

+ EOF
 

+ 

+ 

+ %generate_buildrequires
 

+ %pyproject_buildrequires -C abc=123 -C xyz=456 -C--option-with-dashes=1 -C--option-with-dashes=2
 

+ %{!?el9:%pyproject_buildrequires -C abc=123 -C xyz=456 -C--option-with-dashes=1 -C--option-with-dashes=2 -w}
 

+ 

+ 

+ %build
 

+ %{!?el9:%pyproject_wheel -C abc=123 -C xyz=456 -C--option-with-dashes=1 -C--option-with-dashes=2}
 

+ 

+ 

+ %changelog
 

+ * Fri May 19 2023 Maxwell G <maxwell@gtmx.me>
 

+ - Initial package

file added
+40 
@@ -0,0 +1,40 @@ 
 

+ """
 

+ This is a test backend for pyproject-rpm-macros' integration tests
 

+ It is not compliant with PEP 517 and omits some required hooks.
 

+ """
 

+ 

+ from flit_core import buildapi
 

+ from packaging.version import parse
 

+ from pip import __version__ as pip_version
 

+ 

+ EXPECTED_CONFIG_SETTINGS = [{"abc": "123", "xyz": "456", "--option-with-dashes": ["1", "2"]}]
 

+ # Older pip did not accept multiple values,
 

+ # but we might backport that later,
 

+ # hence we accept it both ways with older pips
 

+ if parse(pip_version) < parse("23.1"):
 

+     EXPECTED_CONFIG_SETTINGS.append(
 

+         EXPECTED_CONFIG_SETTINGS[0] | {"--option-with-dashes": "2"}
 

+     )
 

+ 

+ 

+ def _verify_config_settings(config_settings):
 

+     print(f"config_settings={config_settings}")
 

+     if config_settings not in EXPECTED_CONFIG_SETTINGS:
 

+         raise ValueError(
 

+             f"{config_settings!r} does not match expected {EXPECTED_CONFIG_SETTINGS!r}"
 

+         )
 

+ 

+ 

+ def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
 

+     _verify_config_settings(config_settings)
 

+     return buildapi.build_wheel(wheel_directory, None, metadata_directory)
 

+ 

+ 

+ def get_requires_for_build_wheel(config_settings=None):
 

+     _verify_config_settings(config_settings)
 

+     return buildapi.get_requires_for_build_wheel(None)
 

+ 

+ 

+ def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
 

+     _verify_config_settings(config_settings)
 

+     return buildapi.prepare_metadata_for_build_wheel(metadata_directory, None)

file modified
+3 
@@ -94,6 +94,9 @@ 
 

      - escape_percentages:
 

          dir: .
 

          run: ./mocktest.sh escape_percentages
 

+     - config-settings-test:
 

+         dir: .
 

+         run: ./mocktest.sh config-settings-test
 

      required_packages:
 

      - mock
 

      - rpmdevtools

no initial comment

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/51f768275c374a33b7e8b44b7ef5511f

1 new commit added

  • fixup! Accept multiple values for the same config settings
a year ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/d3cb58ca83144e73979aa729f16bb821

4 new commits added

  • Accept multiple values for the same config settings
  • document config_settings support
  • buildrequires: make -C and -N mutually exclusive
  • Allow passing config_settings to the build backend
a year ago

4 new commits added

  • Accept multiple values for the same config settings
  • document config_settings support
  • buildrequires: make -C and -N mutually exclusive
  • Allow passing config_settings to the build backend
a year ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/f00bc2b1351843299374b3e67086e674

4 new commits added

  • Accept multiple values for the same config settings
  • document config_settings support
  • buildrequires: make -C and -N mutually exclusive
  • Allow passing config_settings to the build backend
a year ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci
https://fedora.softwarefactory-project.io/zuul/buildset/7c9631f86ece4f7591e46f01811ecf05

[citest]

recheck

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/3ca5c53689ee47f9bd33934d7e23762a

Pull-Request has been merged by churchyard
a year ago
Metadata
None
No Tags
Changes Summary 11
+37 -0
file changed
README.md
+1 -1
file changed
macros.aaa-pyproject-srpm
+4 -3
file changed
macros.pyproject
+5 -1
file changed
pyproject-rpm-macros.spec
+20 -6
file changed
pyproject_buildrequires.py
+37 -0
file changed
pyproject_buildrequires_testcases.yaml
+55 -2
file changed
pyproject_wheel.py
+1 -0
file changed
test_pyproject_buildrequires.py
+50
file added
tests/config-settings-test.spec
+40
file added
tests/config_settings_test_backend.py
+3 -0
file changed
tests/tests.yml
Powered by Pagure 5.13.3
Documentation • File an Issue • About this Instance • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.