Package integrity verification print errors when
the verification fails ans some tests are meant
to test such behaviour, but the errors are
confusing. I'm adding buffer=True to test suite
to suppres prints to stdout from pkg_integrity
Some packages create shared libraries but do not pass flags to linkers
to record the names of the shared librares. Change a fatal error to a
warning and use the filename instead. Based on work by Ikey Doherty
<michael.i.doherty@intel.com>
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
Archives are downloaded then moved to /tmp/<user>/<tarball_prefix>/
If this directory is not removed before an autospec run and a user runs
autospec on the same package twice in a row, the build will fail due to
the archives existing in the same location the new archives are
attempting to be moved to. This directory needs to be removed before an
autospec run to prevent this.
options.conf may exist in a package directory in a format different from
the format defined in config.py due to an out-of-date autospec or
user-creation of the configuration file. Rewriting the file on each run
updates the configuration file to the common format without requiring
manual editing by the user.
Consequently, this also allows the user to provide a partial
configuration file without defining every option required. The
configuration file will be rewritten with all available options and
associated comments. Previously missing options will be added as false
values.
The support for golang in autospec was previously installing packages
into the GOROOT causing failures when go get was run on packages already
in the GOROOT.
This patch moves the install location to /usr/lib/golang-dist which
should be added to the GOPATH when building a go package.
The autospec support still does not allow for building go and instead
just handles adding library installation at this time. It also requires
manual configuration for the go source path and build pattern.
Recent commit d9928f (fix cmake) adds cmake-dev as a build requirement
when cmake is used. Add cmake-dev to the specfile for the libjpeg-turbo
test as libjpeg-turbo uses cmake.
The test package, c-helloworld-opts, sets several options using the
deprecated method of creating various empty configuration files. The
tests then verify that the correct configuration is detected and added
to the options.conf file.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Functional tests now check the generated configuration file against a
user-defined expected configuration file called conf-expectations. The
check is done identically to how the spec-expectations file is checked.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Remove "file exists" type configuration options and replace with an
automatically generated single configuration file. This makes the
autospec configuration easier to understand for the autospec user by
setting a boolean configuration option in a standard file.
Autospec will create this configuration file if it does not exist and
remove the deprecated configuration files, as well as removing them from
git tracking.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Integrity check originally looked for a signature file
in 'package_URL + .asc', however this patterns is not
that common therefore I'm adding a way to check
locally for a signature file before attempting to find
a signature at 'package_URL + .asc'
Signature verification was extended to do hash
validation for ruby gems, so integrity validation
is a better description of module. The module
was changed from verify_sign to pkg_integrity
If urlban is not defined (by a configuration file, for instance), do not
try to remove it from a url. This was causing autospec failures when
urlban was not defined. This also removes the autospec.conf dependency
from the functional tests.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
After fighting python-gnupg and gnupg I decided to
use neither, both have missing features that are
required for what I'm trying to do. I'm using a
wrapper for command line instead (for now). I will
revisit my decision if things change, but for now
a wrapper for command line is enough.
Add test framework and basic tests to Autospec to allow testing of
packages. The tests can be run by invoking `python3 test_autospec.py`
The tests first create a tarball of each source directory within the
testfiles directory and runs autospec against it. Each package is
handled in its own thread, so the total time of the tests is equivalent
to the total time of the longest autospec run.
Once the build is complete, the output of `make autospec` and the
command resulting spec file are searched for terms defined by in a
special file within the test source directories. The special file is
expectations.py and within this file the following terms can be
defined:
- buildreqs (list of strings) - build requirements expected to be
detected by autospec.
- license (string) - name of the license to be detected by autospec.
- output_strings (list of strings) - strings expected to be in the
output of `make autospec`.
- specfile (string) - this string is read in from the spec-expectations
file, which contains the exact text of the specfile expected to be
generated from the autospec run.
The file expectations.py is not included in the tarball created in the
first step of the test process.
There is an additional directory 'autospecdir' within the test package
that contains the configuration files used by an autospec run.
Test results are printed at the end of an autospec run on a package. If
the generated specfile differs from spec-expectations, a diff is
displayed along with the failure.
tarball.py is also updated to remove only the specific tarball that was
downloaded. This allows the tests to be run in parallel without the
"downloaded" tarball being erased by subsequent autospec runs.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
This change prevents the possibility of enabled systemd unit files not
being put into the autostart subpackage by just matching all files in
the system *.target.wants subdirectories. The autostart config file has
thus been reworked to instead be a no_autostart file that controls if
the autostart subpackage is included by the main package or not.
This change allows packages to have an autostart subpackage that is
included by default as one of the main package's dependencies. It is
intended to replace an informal use of the extras subpackage which
previously handled having a separation between systemd unit activation
files and the main package.
To make use of this feature, a packager will likely create the
target.wants folder for the systemd unit and create the symlink to the
systemd unit in make_install_append and add the full path to the
unit file symlink in the autostart file (as described in the
README.rst).
When zip files contain hash information it is displayed in the header
causing failure to detect the zip file prefix. This change adds minimal
zip file header detection in order to determine which line contains the
first compressed file to find the prefix.
An example of the unzip -l output that had previously been the only
correctly handled type is:
Archive: wsgiref-0.1.2.zip
Length Date Time Name
--------- ---------- ----- ----
589 06-11-2006 23:49 wsgiref-0.1.2/PKG-INFO
239 06-07-2006 13:13 wsgiref-0.1.2/README.txt
51 06-07-2006 13:10 wsgiref-0.1.2/setup.cfg
581 06-11-2006 23:48 wsgiref-0.1.2/setup.py
17925 06-11-2006 23:18 wsgiref-0.1.2/test_wsgiref.py
27441 06-09-2006 11:54 wsgiref-0.1.2/docs/libwsgiref.tex
2295 06-09-2006 12:01 wsgiref-0.1.2/docs/ref.tex
614 07-09-2005 00:29 wsgiref-0.1.2/ez_setup/README.txt
7661 06-09-2006 14:43 wsgiref-0.1.2/ez_setup/__init__.py
15596 06-11-2006 23:44 wsgiref-0.1.2/wsgiref/handlers.py
5916 06-11-2006 23:43 wsgiref-0.1.2/wsgiref/headers.py
4789 06-11-2006 23:43 wsgiref-0.1.2/wsgiref/simple_server.py
5598 06-11-2006 23:44 wsgiref-0.1.2/wsgiref/util.py
14737 06-11-2006 23:42 wsgiref-0.1.2/wsgiref/validate.py
586 06-05-2006 13:45 wsgiref-0.1.2/wsgiref/__init__.py
589 06-11-2006 23:49 wsgiref-0.1.2/wsgiref.egg-info/PKG-INFO
343 06-11-2006 23:49 wsgiref-0.1.2/wsgiref.egg-info/SOURCES.txt
8 06-11-2006 23:49 wsgiref-0.1.2/wsgiref.egg-info/top_level.txt
--------- -------
105558 18 files
The first line after the header:
589 06-11-2006 23:49 wsgiref-0.1.2/PKG-INFO
is on the 4th line of the output in this case unlike the output where a
checksum is embedded in the zip file:
Archive: msr-tools-1.3.zip
d9a630da8b626196737d29dff697bfda405de93e
Length Date Time Name
--------- ---------- ----- ----
0 09-10-2013 02:49 msr-tools-master/
231 09-10-2013 02:49 msr-tools-master/MAKEDEV-cpuid-msr
977 09-10-2013 02:49 msr-tools-master/Makefile
4383 09-10-2013 02:49 msr-tools-master/cpuid.c
7645 09-10-2013 02:49 msr-tools-master/rdmsr.c
102 09-10-2013 02:49 msr-tools-master/version.h
3756 09-10-2013 02:49 msr-tools-master/wrmsr.c
--------- -------
17094 7 files
In this case the first line after the header:
0 09-10-2013 02:49 msr-tools-master/
is on the 5th line of the output causing the header separation line to
be erroneously used to get the prefix.
In addition, refactor the code into a function since the unzip logic was
used in multiple places already.
This patch adds the optimize_size configuration file to the list of git
adds that are performed at the end of an autospec run.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Matthew Johnson