The `rpmspec` commands usually specify the same flags aside from the
queryformat and the spec file(s) to query.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Indicating end of options with `--` fixes the command invocation if the
filename begins with a hyphen.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
For most people simply running "make autospec" or "make build", this
will have no effect. The RPM macro %{_smp_mflags} is predefined to be
the number of CPUs in your system anyway (output of getconf
_NPROCESSORS_ONLN).
But if you do have MAKEFLAGS set in your environment to your CPU
number (and you should), that will get passed to rpmbuild, which then
uses it during the build. And if you, like me, have it set to a
slightly higher number, RPM will use that too.
But if you run "make -jN autospec", then that N will be passed to the
build. This is especially important if you have something doing "make
-j1": RPM will be -j1 too! So instead of "make -j1", run "MAKEFLAGS= make"
For commands that modify contents of $(TOPLVL)/repo, wrap them with
`flock`, using a lock file `repo.lock` within the repo directory. This
ensures that the repo directory tree is not otherwise modified while
each command is running.
To keep the implementation simple, I opted to not wrap entire makefile
targets with a lock (e.g. lock the entire operation of `make repoadd`).
If users need that level of control, they can implement locking in a
separate wrapper script.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This change allows specifying short/long mock options via MOCK_OPTS
without a leading backslash escape.
For example, `MOCK_OPTS="\--old-chroot --no-clean"` can now be specified
as `MOCK_OPTS="--old-chroot --no-clean"`.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
In the submmitter's local pacakge directory can have mulptiple spec file present,
or swap file thus to avoid adding multiple specfile in git add better only add package
specific specfile.
Signed-off-by: santoshkumar <santoshkumar.laxminarayan.rai@intel.com>
This tool no longer works and the repo looks abandoned (last commit
04/27/2017). The NVD cve database has changed format since.
Signed-off-by: Alex Jaramillo <alex.v.jaramillo@intel.com>
For the `make autospec` variants, use a simpler calling convention so
that the callers do not need to know the boolean option names.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When building large numbers of packages that are interdependent,
recreating the rpm repository for each package can create a large
overhead during processing. 'repostage' is intended to provide an
intermediate target for adding a package to a repository that can
later be built using localrepocreate.
Any attempt at rebasing is unnecessary when the current branch is
up-to-date with the remote, so skip the rebase in this case.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This tooling is designed to automate 2 main tasks that are part of
the package maintenance workflow of Clear Linux LTS. These tasks are:
- Back-porting of a patch (e.g. security fix) to older branches.
- (Not implemented yet) Building RPMs with the intent of sharing binaries of
older LTS branches to newer branches whenever possible.
2 new targets are defined in Makefile.common.lts:
- lts-show: Show a summary of active LTS branches
- lts-backport: Attempt to fast-forward the previous active branch to the current branch
"Active" branches correspond to LTS releases that currently have support.
They are listed in a flat file "active-branches" in "lts" directory, from
oldest to newest. New entries are added by Clear Linux LTS developers as
new releases become available, and entries removed as releases become
obsolete.
Note: For CVE patching, the tool is not aware of CVE severity levels or
the minimum supported severity level of each LTS branch. For now it is
the user's responsibility to know when a CVE does not apply to older
branches and stop calling "make lts-backport".
Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
Consider the following scenario:
* System being set-up on day 0 (by running user-setup.sh script)
* The github.com/clearlinux/common repo is cloned under projects/common
* Fast-forward in the future
* Clone a package for which there is no reachable upstream URL
(e.g. clr-power-tweaks or clr-systemd-config
* Run 'make sources'
That operation will fail because it will search for the src.rpm files starting
with the Clear Linux build determined by looking up the latest tag in the
project/common repo (and the latest is the one from day 0). So it will only find
older versions of those src.rpm, and will eventually fail.
This patch adds a line that fetches all the latest tags from the upstream
projects/common repo so the 'latest_builds' variable is *really* set to the
latest of the day.
Signed-off-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
The for-review.txt files were being generated with `git show` and thus
not compatible with `git send-email`. Use `git format-patch` instead to
fix this issue.
Also add a more prominent notice about the existence of for-review.txt
and its intended purpose.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
We always want the koji CLI to wait for `make scratch-wait`, regardless
of the environment koji detects it is being run under.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Debuginfo packages now install binary debuginfo to /usr/share/debug and
source debuginfo to /usr/share/debug/src.
The purpose for the migration is for these new locations to serve as
"lookaside" directories used by future versions of clr-debug-info,
enabling select debuginfo packages to be installed in bundles and safely
coexist with the FUSE mounts.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Some packages in the distro do not generate debuginfo subpackages, so
make sure to skip the debuginfo install in this situation.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The caches are not package-specific, and --scrub=all will clean all
caches even with the --uniqueext option. For that, we have 'make
mockclean' instead.
Using both --clean and --scrub=chroot is also unnecessary, as --clean
already removes the chroot.
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
For testing purposes only, if developers want to install RPMs they just
built onto their development system, potentially overwriting content
that is tracked by swupd, they can now run 'make install-local' to
install all of the RPMs, or 'make install-debuginfo-local' to just
install the debuginfo package.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
For some commands, it is useful to be able to add custom prerequisites.
We already support this for `make koji` and `make build`, and there was
just a request to add it for `make autospec` and `make autospecnew`.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Make the SPDX puller code more robust by reading from JSON sources
rather than parsing HTML. Also, keep all deprecated license identifiers
for now, since they are still present in the spdx list.
Four identifiers were removed since the last refresh, but we are keeping
them for backwards compatibility. The removed license IDs now reside in
the licenses-extra file. At
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Previously, we always emitted an error and failed when the specfile or
upstream target was requested, breaking the -B switch. Now, we only fail
if the file doesn't actually exist, not just that it was requested via
make.
Since binary RPMs are now stored in the "rpms" directory, follow a
similar method for storing them in the local repo, too.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To improve visibility of which RPMs a user may want to install after a
build, copy (hardlink) them to a separate "rpms" directory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
I often find myself running "make clean proper", when by default, I
think the old build artifacts should be removed so that the results
directory is pristine at the start of a new build.
As a result, if users wish to keep all build artifacts from a previous
"make build" or "make autospec", they must keep manual backups, as the
tooling no longer keeps them around.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Mock will append the binary rpm build logs to the srpm logs, but this is
confusing. After the srpm finishes building, rename the log files to
avoid this behavior.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Because Mock dumps its logs and and resulting rpms in the same directory
(results), shuffling the logs and rpms only after a successful build is
confusing, since they will be untouched for the error case.
It's simpler to avoid moving the files entirely. The only exceptions
will be to avoid clobbering existing log files. Changes of this nature
will appear in later commits and in autospec.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Fixes#6
Not all Clear Linux packages have debuginfo subpackages, so ignore
errors when attempting to move them to the results/debuginfo/ directory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When running 'make koji', the tooling will attempt to create a new git
tag with a name constructed from the spec file's NVR.
However, if there are uncommitted changes to the Name, Version, or
Release fields (collectively, NVR) in that file, the new tag's name will
not match the NVR in the spec file referenced by commit HEAD.
Protect against the mismatch by exiting early from a 'make koji' if any
spec file modifications are not yet committed to the repo, and alert the
user what the problem is.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The debuginfo RPMs are stored in a separate directory now, so the rpm
copy pipeline needed a slight adjustment.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The logs, srpms, and debuginfo should all be split into their own
directory, so it is less confusing about which RPMs can be used when
creating mixes. They are mainly for extra metadata, and should not be
left in the main RPM results location.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
After running 'make autospecnew', the freshly created repo only exists locally,
so the repo remote is not yet initialized. The 'git pull --rebase' (via 'make
koji') will then fail due to the uninitialized remote. Improve this case by
only rebasing when the remote is initialized.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The DOWNLOAD_MIRROR server, the SOURCES_URL server, and upstream
source tarballs may be hosted in many different environments.
Allow users to supply additional curl options via a config to use
whenever connecting to the DOWNLOAD_MIRROR and SOURCES_URL servers and
work better with their particular hosting environment. Because upstream
source tarballs can come from many different locations, do not offer a
global config and instead allow users to define this manually as needed
when the rule is used.
Signed-off-by: George T Kramer <george.t.kramer@intel.com>
Eventually developers - when they know what they're doing - want to
manipulate a little bit the mock arguments to i.e skip a few stages
in the mock build cycle while they're hacking a package.
One example is when developers are testing changes to a spec file but
not touching dependencies and they're [re]running the build multiple times
it would be nice to not cleanup before the build and run a rebuild
for example with: --rebuild --no-clean
With this patch one should only do:
make build MOCK_OPTS="--rebuild --no-clean"
Signed-off-by: Leandro Dorileo <leandro.maciel.dorileo@intel.com>
My previous change to handling of the 'upstream' file assumed that the
file exists already, but this need not be true.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When SOURCES_URL is not defined, the current behavior of extracting
source files from SRPMs in the latest published Clear Linux release has
proven to not work so well: It is not optimal when users are adding new
packages, modifying existing packages, not building from latest master,
etc.
Instead, run 'generateupstream' automatically so that source files are
downloaded from the remote locations listed in the spec file. This
target creates the 'upstream' file too, which becomes useful if the user
later defines SOURCES_URL, when 'upstream' is actively consumed.
In the event that download errors occur, as a fallback mechanism, try to
find the corresponding SRPM from a published release of Clear Linux OS.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
I will reusing this code (largely unmodified) in the next commit, so
move it into a separate target to easily inspect changes.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
