setup.py test is gone in 3.13 and projects seem to be doing custom
things for replacements. Until wider per project detection is in
place, just remove the %check section for setup.py.
Signed-off-by: William Douglas <william.douglas@intel.com>
Modify license skips to handle the common tempfile prefix of '/tmp/*'
and normalize the case where lines lead with a slash vs not.
Also remove a duplicate test for globlike match.
Signed-off-by: William Douglas <william.douglas@intel.com>
Add support for a new config file with glob like handling of paths as
some projects contain license files that are not the license of the
project.
scan_for_licenses now looks at the configuration and does glob like
matching for all the potential license files it tries to add.
This change caused a few structural adjustments to the code and a
slight style change for importing just the toplevel util submodule
that are superficial.
Signed-off-by: William Douglas <william.douglas@intel.com>
vectorscan uses 'vectorscan/' as a tag prefix which confuses our
version parsing (even more so with v being the tag prefix). Add
support for matching any '*/version' tag names as the top priority.
Signed-off-by: William Douglas <william.douglas@intel.com>
Allow autospec to handle building packages that use a
Jinja2 formatted *.spec.template file. Currently only package_name,
package_version, package_release and package_url fields are supported
but others can be added as needed (patches and archives are good next
steps).
Signed-off-by: William Douglas <william.douglas@intel.com>
print_fatal calls need to be followed up by an exit consistently and
exit's need to use print_fatal before being called.
Signed-off-by: William Douglas <william.douglas@intel.com>
Given more packages are using ninja as the build system of choice
instead of make, add flag to enable ninja usage.
Signed-off-by: William Douglas <william.douglas@intel.com>
Previously support for cmake's find_package parsing was only able to
handle single line entries. This change adds support for multiline
versions and allows for module namespacing (currently qt6 and kf6
support is available).
The change for cmake_modules shows this namespacing in use.
Signed-off-by: William Douglas <william.douglas@intel.com>
Add early support for APX builds in autospec. This change also makes
APX disabled by default for now but may revert back to enabled by
default in the future. The APX prefix will be /VA as the arch flag is
not yet finalized. This also may change in the future though for now
there are no plans to move it.
Currently autotools builds are unable to use this as it detects as a
cross compilation.
Currently only C using GCC is supported.
This change also reworks how R language optimized binaries are built
and distributed. They will now work as all the other tooling does with
the /VX prefix.
Signed-off-by: William Douglas <william.douglas@intel.com>
Previous code would emit:
SyntaxWarning: invalid escape sequence
Correct this by removing the escapes as they are not necessary.
Also add a missing buildreq that is detected correctly after the
fix.
Signed-off-by: William Douglas <william.douglas@intel.com>
Some packages should not be able to be added to any subpkg. Add
handling for this case.
Also as pypi-nose is no longer usable so ban it from being added as a
dependency.
Signed-off-by: William Douglas <william.douglas@intel.com>
Add information about what autospec version was used to create the
spec file.
New header will appear as:
+# autospec version: v2
+# autospec commit: 1234567
With this change we'll restart tagging autospec. Changes that get
tagged (after PR merge, tag likely just pushed via git cli) should be
doing something that would benefit from a larger package ecosystem
rebuild.
Signed-off-by: William Douglas <william.douglas@intel.com>
pypi is deprecating the API and doesn't recommend its usage for
integrity checks so stop trying to make use of it in autospec.
Signed-off-by: William Douglas <william.douglas@intel.com>
When autospec find a requires.txt, set the requirements_path not the
requires_path (renamed to a more fitting pyproject_path). Also fix up
unittests and add a new one for this case.
Signed-off-by: William Douglas <william.douglas@intel.com>
The %doc directive doesn't really align with autospec usage where
paths are automatically managed into subpackages so stop using it.
Signed-off-by: William Douglas <william.douglas@intel.com>
Enable content to be found under %{buildroot}/{V3,V4} and work
seemlessly with bans, compat, excludes, manual extras and patterns.
Eventually the efi-move.py will be retooled to store content in these
locations so preempt this change in autospec.
Signed-off-by: William Douglas <william.douglas@intel.com>
Reorder some items for build pattern detection so that build
requirements for specific patterns are added last. This is to help
prevent build requirements that don't apply to a given pattern from
being added unintentionally.
As part of this remove some outdated bits (catkin and disutils36) that
no longer are used.
Signed-off-by: William Douglas <william.douglas@intel.com>
These patterns are being removed as they are not used or are going to
be reworked.
This change also removes support for multi-versioned packages which
were considered for go and cargo purposes.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of pushing the filename that is matched, use the glob for the
files section as in some cases it is required (rustc) in order to
avoid build failures as the filenames may not be static.
Signed-off-by: William Douglas <william.douglas@intel.com>
This change allows *extras files to contain '*' matches. This somewhat
emulates the shell glob match in that '/foo*bar' will match '/foobar'
and '/foobazbar' but not '/foo/bar'. The globs are only allowed one
per directory such that '/foo*/bar*' is allowed but '/foo*bar*' is
not.
Signed-off-by: William Douglas <william.douglas@intel.com>
Some packages have a files section replacement in place that uses
globs instead of listing all files in the section. This doesn't work
well with extras. Add an option to config to turn off the replacement
so extras are able to function as normal.
Signed-off-by: William Douglas <william.douglas@intel.com>
All sdist tarballs available on pypi appear to have both an md5sum and
sha256sum listed, so switch to verify the stronger hash (sha256)
instead.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add support for (*_)provides_{add,ban} configuration files.
This is occasionally useful for cases where a pypi() provide isn't
picked up as the packages isn't a python package but does provide a
python library. It also can be handy when doing some renames to
prevent mass rebuilds.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of using separate regular expressions to capture the multiple
forms pytest test summaries might take, use one generic regular
expression to match the summary line, and then process comma-separated
components of the line with more specific matches.
Three test cases needed to be updated to accommodate this refactor:
- Two tests omitted the leading '=' characters from the line to match,
even though their summary lines clearly originated from pytest. Fix by
adding some leading '=' characters for those summary lines.
- One test mistakenly counted an "xpass" as an "xfail". Autospec maps
"xpass" results to "pass", so update the test accordingly.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Two gitlab URLs had mis-parsed versions that are derived from upstream
tags with a `VERSION_` prefix...
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The generated packageurl name/version tests were only testing one of
four content states, leading to a gap in test coverage, since some names
or versions might be mis-parsed in a state that was not tested.
Fix the issue by testing all content states for every package URL in the
tests/packageurls file. And ensure that test failure output remains
actionable by using a subTest() context manager and setting the `state`
argument, which will print the value of `state` in the test failure
output header.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When encountering patches prefixed with 'backport-' if they no longer
apply, remove them from the patch series and try to restart the build
without them. The goal is to avoid manual work removing patches known
to be obsolete in the next release.
Signed-off-by: William Douglas <william.douglas@intel.com>
When not handling a multiple version package, there was a case where if
the version wasn't set the multi_version map would be the
config.versions map and that's not intended to outside of multiple
version use case.
So to fix this, make a new map when setting a default version for the
multi_version map and keep config.versions as its own map.
Signed-off-by: William Douglas <william.douglas@intel.com>
If a python requirement has a python_version bounded by a maximum
(python_version < 3.8) than ignore it. The reasoning for this change
is to avoid dependencies that no longer apply with the most recent
python version.
Signed-off-by: William Douglas <william.douglas@intel.com>
The file_restart needs to be reset for the buildroot log if the
buildroot log is not clean the build log won't be parsed and so
file_restart won't be reset otherwise (causing build loops to occur
for packages that can't be found for instance).
Signed-off-by: William Douglas <william.douglas@intel.com>
If a file addition only failure happens and a quick rebuilt is done,
the build log will be missing test results. Check the previous build
log in that case.
Signed-off-by: William Douglas <william.douglas@intel.com>
Try to avoid doing a full rebuild when the build.log only has new
files in it.
This has a two downsides. The source buildroot needs to be distinct
from the binary buildroot. The built rpms are no longer installable as
rpmbuild flags them.
The upside is packages that take forever to build or don't have
build times improved by ccache will not need a duplicate round for
just adding the files.
The rpms can still be generated in an installable fashion with 'make
build' and the autospec generated rpms can be installed by force
installing if need be.
Signed-off-by: William Douglas <william.douglas@intel.com>
This migrates python dependencies (both build and runtime) to use
'pypi()' where possible (when detecting dependencies from setup.py,
setup.cfg, pyproject.toml, requires.txt and requirements.txt).
It also adds a few little cleanups to python detection in order to
make the above work more evenly and removes some python2 cruft.
Signed-off-by: William Douglas <william.douglas@intel.com>
Autospec's DESCRIPTION file parsing for CRAN packages is working well
enough that I think we can unconditionally add the detected requirements
as both build and runtime dependencies.
Also remove the propagation of CRAN build->runtime deps via fail
pattern, since we no longer need it; the deps detected via fail pattern
are either Suggests (optional) or not listed in DESCRIPTION at all.
Either way, they are not required dependencies.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When parsing setup.py files, if already in multiline and install_requires
is used as a variable, don't try and treat the line as if it was an
install_requires assignment.
Signed-off-by: William Douglas <william.douglas@intel.com>
Strip the optional leading character matching "[vVrR]" from a version extracted
from a Gitlab URL.
Update a packageurl test case to accommodate.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The Github API endpoint for generating tarballs from tags recently changed, so
teach autospec about the new URL format.
Also add a new packageurl for testing to confirm correct operation of the
regexp that matches the new format.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
