For testing purposes only, if developers want to install RPMs they just
built onto their development system, potentially overwriting content
that is tracked by swupd, they can now run 'make install-local' to
install all of the RPMs, or 'make install-debuginfo-local' to just
install the debuginfo package.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
For some commands, it is useful to be able to add custom prerequisites.
We already support this for `make koji` and `make build`, and there was
just a request to add it for `make autospec` and `make autospecnew`.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Make the SPDX puller code more robust by reading from JSON sources
rather than parsing HTML. Also, keep all deprecated license identifiers
for now, since they are still present in the spdx list.
Four identifiers were removed since the last refresh, but we are keeping
them for backwards compatibility. The removed license IDs now reside in
the licenses-extra file. At
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Previously, we always emitted an error and failed when the specfile or
upstream target was requested, breaking the -B switch. Now, we only fail
if the file doesn't actually exist, not just that it was requested via
make.
Since binary RPMs are now stored in the "rpms" directory, follow a
similar method for storing them in the local repo, too.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To improve visibility of which RPMs a user may want to install after a
build, copy (hardlink) them to a separate "rpms" directory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
I often find myself running "make clean proper", when by default, I
think the old build artifacts should be removed so that the results
directory is pristine at the start of a new build.
As a result, if users wish to keep all build artifacts from a previous
"make build" or "make autospec", they must keep manual backups, as the
tooling no longer keeps them around.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Mock will append the binary rpm build logs to the srpm logs, but this is
confusing. After the srpm finishes building, rename the log files to
avoid this behavior.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Because Mock dumps its logs and and resulting rpms in the same directory
(results), shuffling the logs and rpms only after a successful build is
confusing, since they will be untouched for the error case.
It's simpler to avoid moving the files entirely. The only exceptions
will be to avoid clobbering existing log files. Changes of this nature
will appear in later commits and in autospec.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Fixes#6
Not all Clear Linux packages have debuginfo subpackages, so ignore
errors when attempting to move them to the results/debuginfo/ directory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When running 'make koji', the tooling will attempt to create a new git
tag with a name constructed from the spec file's NVR.
However, if there are uncommitted changes to the Name, Version, or
Release fields (collectively, NVR) in that file, the new tag's name will
not match the NVR in the spec file referenced by commit HEAD.
Protect against the mismatch by exiting early from a 'make koji' if any
spec file modifications are not yet committed to the repo, and alert the
user what the problem is.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The debuginfo RPMs are stored in a separate directory now, so the rpm
copy pipeline needed a slight adjustment.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The logs, srpms, and debuginfo should all be split into their own
directory, so it is less confusing about which RPMs can be used when
creating mixes. They are mainly for extra metadata, and should not be
left in the main RPM results location.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
After running 'make autospecnew', the freshly created repo only exists locally,
so the repo remote is not yet initialized. The 'git pull --rebase' (via 'make
koji') will then fail due to the uninitialized remote. Improve this case by
only rebasing when the remote is initialized.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The DOWNLOAD_MIRROR server, the SOURCES_URL server, and upstream
source tarballs may be hosted in many different environments.
Allow users to supply additional curl options via a config to use
whenever connecting to the DOWNLOAD_MIRROR and SOURCES_URL servers and
work better with their particular hosting environment. Because upstream
source tarballs can come from many different locations, do not offer a
global config and instead allow users to define this manually as needed
when the rule is used.
Signed-off-by: George T Kramer <george.t.kramer@intel.com>
Eventually developers - when they know what they're doing - want to
manipulate a little bit the mock arguments to i.e skip a few stages
in the mock build cycle while they're hacking a package.
One example is when developers are testing changes to a spec file but
not touching dependencies and they're [re]running the build multiple times
it would be nice to not cleanup before the build and run a rebuild
for example with: --rebuild --no-clean
With this patch one should only do:
make build MOCK_OPTS="--rebuild --no-clean"
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
My previous change to handling of the 'upstream' file assumed that the
file exists already, but this need not be true.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When SOURCES_URL is not defined, the current behavior of extracting
source files from SRPMs in the latest published Clear Linux release has
proven to not work so well: It is not optimal when users are adding new
packages, modifying existing packages, not building from latest master,
etc.
Instead, run 'generateupstream' automatically so that source files are
downloaded from the remote locations listed in the spec file. This
target creates the 'upstream' file too, which becomes useful if the user
later defines SOURCES_URL, when 'upstream' is actively consumed.
In the event that download errors occur, as a fallback mechanism, try to
find the corresponding SRPM from a published release of Clear Linux OS.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
I will reusing this code (largely unmodified) in the next commit, so
move it into a separate target to easily inspect changes.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If any of the curl commands fail, it's better for the entire target to
fail. In this way, developers will be properly notified.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The curl --no-proxy hint is not needed anymore, since Clear Linux OS has
an autoproxy, and curl will honor proxy environment variables.
Also, pass -L to curl to follow redirects, similarly to other curl calls
in this makefile.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Improve the detection of source URLs by matching only non-commented
Source* lines with "://" present.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Over time, the autospec options used for 'autospec' and 'autospecnew'
commands have diverged. They are meant to be in sync, so do that here,
and format the recipe in a way that eases future changes.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The echo commands at key points of the pullrebase should be sufficient
for users to know what action pullrebase performed.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If a new package repo is created with make autospecnew, and then make
autospec is run immediately afterward, a git fetch is performed, but
the remote repo is not likely to exist yet.
Address this issue by only running the git fetch if the remote origin is
initialized locally via a git push, git pull, etc.
In addition to properly supporting the workflow described above, this
change also enables developers to build packages that are *not* intended
to be included in Clear Linux OS, thus not destined to appear in the
'packages' file. Package repos of this nature might have different,
manually configured remotes, or the remote may never be initialized at
all if builds are only run locally.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Similar to how a notice is printed for 'make build', enable the same
notice for 'make autospec' and 'make autospecnew'.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Local repo support was recently added for 'make build', so do the same
for 'make autospec' and 'make autospecnew'.
Note that to make mock invocations work at both package-level and
toplevel, I am resolving MOCK_CONFIG_VAL with the realpath function.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The name "shared" makes more sense at this point, since the makefile
does not contain only functions, and it is meant to be shared between
Makefile.common and Makefile.toplevel.
Also, reorder the makefile includes, since Makefile.shared should be
able to use variables defined in Makefile.config.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To make sure the conf files are more visible in the local repo
directory, unhide them by installing them without a leading dot
character.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
DNF seems to ignore an uninitialized repo, but Yum does not. To fix,
make sure to run createrepo immediately after creating the local repo
directory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
With this change, 'make build' will automatically use the local repo, if
enabled. A notice is printed in case the user didn't want to use the
local repo and wishes to disable it with 'make repodisable'.
Another functional change here is that 'make build' will use the in-tree
mock config (clear.cfg) instead of the /etc/mock/clear.cfg. If this
behavior is not desired, then MOCK_CONF can be modified in
Makefile.config, or overridden by a Makefile.config.site_local.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
In case the user runs 'make repoclean' followed by 'make repoadd' in a
package repo dir, 'rm' will complain. Add the -f option to silence those
warnings.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This new command displays a helpful summary of what packages exist in
the local RPM repo and whether it is enabled for Yum/DNF, Mock, or both.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To ensure that RPM repo metadata is always up-to-date for the local
repo, make sure createrepo is run after removing the RPMs.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
An interface for a local RPM repository already exists, so extend it by
adding Mock support and the ability to enable and disable the
repository, at package level and at toplevel.
Integration for commands that use Mock will be added in separate
commits.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
PKG_NAME and URL were properly supported, but ARCHIVES needed to be
written back to the package repo makefile.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Because there have been very few situations in the past requiring a
"make koji" from a branch other than master, add a strict check that the
active local branch is "master". This also blocks submissions from a
"detached" state, which was previously permitted.
Also, fix and consolidate the git push commands into one. Now that the
branch check is implemented, the "master" refspec can be used, and
explicitly push *one* tag (the tag created immediately before the push).
The --follow-tags and --tags options don't have the intended effect, so
omit them.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
These are targets to be executed before build and koji targets. They
are meant to execute security and validation checks.
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
As pointed out by Kai, I can avoid one level of shell wrapping. Mock
is already running this via "/bin/sh -c" (function doshell in
/usr/lib/python2.7/site-packages/mockbuild/util.py currently).
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
Change the "make shell" command to invoke a shell as the "mockbuild"
user and starting in ~mockbuild. This is typically what you want when
you enter the build environment.
Add a new target, "rootshell", which gives the old behavior, a root
shell starting at /.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
