Enable the use of a distro wide key cache directory for cases where a
signing key is used by more than one package.
Signed-off-by: William Douglas <william.douglas@intel.com>
When pypi's json fails to load for an integrity check, return an empty
dict so that the failure is able to be handled. Also fixup the error
string to indicate the URL that was tried.
Signed-off-by: William Douglas <william.douglas@intel.com>
This change allows *extras files to contain '*' matches. This somewhat
emulates the shell glob match in that '/foo*bar' will match '/foobar'
and '/foobazbar' but not '/foo/bar'. The globs are only allowed one
per directory such that '/foo*/bar*' is allowed but '/foo*bar*' is
not.
Signed-off-by: William Douglas <william.douglas@intel.com>
Some packages have a files section replacement in place that uses
globs instead of listing all files in the section. This doesn't work
well with extras. Add an option to config to turn off the replacement
so extras are able to function as normal.
Signed-off-by: William Douglas <william.douglas@intel.com>
Specifically, this change enables build_prepend for the avx2 builds for
both `pyproject` and `distutils3` patterns.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This in theory is implied with gcc 12 when it makes sense and it
breaks clang as it isn't supported there.
Signed-off-by: William Douglas <william.douglas@intel.com>
* The extra subdirectory created via the `-c` flag to `%setup` is
unnecessary for packages using the R pattern, since all CRAN source
tarballs have a toplevel container directory. Remove the `-c` flag to
avoid creating that subdir and thereby fixing license file
installation.
* Adjust the required argument to `R CMD ...` commands that specifies
the relative path where package sources are found. The package sources
are now found in the current directory instead of a subdir named after
the package, so use `.` as the path.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
All sdist tarballs available on pypi appear to have both an md5sum and
sha256sum listed, so switch to verify the stronger hash (sha256)
instead.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
In order to avoid large diffs just because the package version portion
of the license copying line changed, use the %{version} macro.
Signed-off-by: William Douglas <william.douglas@intel.com>
The three python package patterns (pyproject, distutils3, and
distutils36) were starting the %check section before the end of %build
section.
Fix these issues by starting %check immediately before %install.
Also, use the `write_check()` method wrapper for these patterns to align
with all other patterns.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add support for (*_)provides_{add,ban} configuration files.
This is occasionally useful for cases where a pypi() provide isn't
picked up as the packages isn't a python package but does provide a
python library. It also can be handy when doing some renames to
prevent mass rebuilds.
Signed-off-by: William Douglas <william.douglas@intel.com>
In the event that the virtualenv or pip subprocess commands from this
module return an error code, print more information obtained from stderr
to help diagnose the issue(s).
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The targetpath for elf-move.py now needs to be the %{buildroot} base
as the full path to the optimized-elf directory is now hardcoded into
the elf-move.py tooling.
This change was made for the library files being autoinstalled to the
glibc-hwcaps directory without making use of clr-elf-replace (making
swupd a little happier about missing files). Full details of the
change are in the clr-avx-tools repo
commit (415a400edd05e6fa405b0682f7c5c9895d039e09).
Signed-off-by: William Douglas <william.douglas@intel.com>
Rather than listing all the files in the perl module path (which
includes a minor version that causes updates to be a reautospec rather
than a bump) move to a glob to simplify the update workflow.
Signed-off-by: William Douglas <william.douglas@intel.com>
Add support to escape a leading `#` character in pattern conf file
patterns by interpreting `\#` as a pattern starting with `#`.
Also add documentation for this feature to the existing pattern conf
files where this support might be needed.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The distutils build is changing to no longer create avx2/avx512
libraries in the python build process. Instead build avx2 always as
part of python builds (not checking options.conf settings for
avx2/avx512 at this time). This is achieved by setting up cflags
similar to the other build patterns and making use of elf-move.
The reason for this change is to avoid cumbersome distutils patches
and make the package ecosystem more uniform to hopefully catch some
outlier performance issues due to the idiosyncratic build process.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of using separate regular expressions to capture the multiple
forms pytest test summaries might take, use one generic regular
expression to match the summary line, and then process comma-separated
components of the line with more specific matches.
Three test cases needed to be updated to accommodate this refactor:
- Two tests omitted the leading '=' characters from the line to match,
even though their summary lines clearly originated from pytest. Fix by
adding some leading '=' characters for those summary lines.
- One test mistakenly counted an "xpass" as an "xfail". Autospec maps
"xpass" results to "pass", so update the test accordingly.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When packages have excludes previously we would full build because
there was no method to determine if the excludes list had items
dynamically added to it.
This change adds a new list that is the original exlcudes list
manually configured and is used to compare against the excludes list
that is generated during processing the files list. If the two
lists (compared as sets) are the identical, then short-circuiting is
still possible.
Also fixup some out of date documentation.
Signed-off-by: William Douglas <william.douglas@intel.com>
If a patch applies changes to more than one file, multiple `Skipping
patch` messages may appear in the log. For that case, multiple calls to
`remove_backport_patch()` will occur. Support that case by returning
early if the patch has already been removed from `self.patches`.
Also, add a progress message to indicate a backport patch has been
removed, otherwise no progress message is likely to appear between build
rounds.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Make main the default branch with git init as not all systems will
share the same default branch name value and Clear Linux uses main for
the package ecosystem.
Signed-off-by: William Douglas <william.douglas@intel.com>
Two gitlab URLs had mis-parsed versions that are derived from upstream
tags with a `VERSION_` prefix...
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The generated packageurl name/version tests were only testing one of
four content states, leading to a gap in test coverage, since some names
or versions might be mis-parsed in a state that was not tested.
Fix the issue by testing all content states for every package URL in the
tests/packageurls file. And ensure that test failure output remains
actionable by using a subTest() context manager and setting the `state`
argument, which will print the value of `state` in the test failure
output header.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If unset, the `yum_conf` autospec config variable (representing a
dnf.conf in modern times) should reference the dnf.conf from
`conf/dnf.conf` in the `common` repo.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When encountering patches prefixed with 'backport-' if they no longer
apply, remove them from the patch series and try to restart the build
without them. The goal is to avoid manual work removing patches known
to be obsolete in the next release.
Signed-off-by: William Douglas <william.douglas@intel.com>
Previous change made missing build dependendies fatal, and while
correct, causing the buildreq_cache file to not be updated when
missing dependencies cause the build to exit.
Move back to previous obvserved behavior but change the logic to
return if the buildroot log is not successful.
Signed-off-by: William Douglas <william.douglas@intel.com>
When not handling a multiple version package, there was a case where if
the version wasn't set the multi_version map would be the
config.versions map and that's not intended to outside of multiple
version use case.
So to fix this, make a new map when setting a default version for the
multi_version map and keep config.versions as its own map.
Signed-off-by: William Douglas <william.douglas@intel.com>
More modern `pytest` versions appear to format the time value
differently... So, for the regexps that no longer match, support the
new format.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
It's common to see git tag names with a `<packagename>-` prefix for the
version string, also perhaps appearing before a `v` or `V`.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
In Clear Linux OS, `nmap-extras` (subpackage of `nmap`) has provided an
`nc` symlink to `ncat` since Sep 2018, so this mapping should be
updated...
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If a python requirement has a python_version bounded by a maximum
(python_version < 3.8) than ignore it. The reasoning for this change
is to avoid dependencies that no longer apply with the most recent
python version.
Signed-off-by: William Douglas <william.douglas@intel.com>
add_buildreq doesn't take a packages argument and this was causing
junk to be added to the buildreq_cache.
Signed-off-by: William Douglas <william.douglas@intel.com>
Missing packages should always cause autospec to exit as there is no
further actions it can take. Given missing packages are the only
reason for parse_buildroot_log to fail, exit on failure from there.
Signed-off-by: William Douglas <william.douglas@intel.com>
The file_restart needs to be reset for the buildroot log if the
buildroot log is not clean the build log won't be parsed and so
file_restart won't be reset otherwise (causing build loops to occur
for packages that can't be found for instance).
Signed-off-by: William Douglas <william.douglas@intel.com>
In some packages (kubernetes) builds that skip the final round may
have content in the build.log interpreted as test results. For these
cases try and use the empty test count key as a signal that the
previous build should be used for test results instead.
In some cases this will cause packages that had no tests for an empty
key to use the previous build round for reporting test results but
hopefully that won't cause test reporting failures.
Signed-off-by: William Douglas <william.douglas@intel.com>
The `egrep` command has been deprecated since 2007 and in the next
`grep` release, will begin to print a warning about obsolescence.
Port the one instance of autospec's `egrep` use to `grep -E` to align
with upstream direction.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Arjan van de Ven