89 Commits

Author SHA1 Message Date
Patrick McCarty
a734764581 common: default to always use sudo
The conditional setting of WITH_SUDO has been a no-op for several years,
so just set the default value to `sudo`.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-09-26 22:10:53 +00:00
Patrick McCarty
72406d0525 koji: set custom bump message for each package in bump.list
This change will make the commits for bumped packages (from bump.list)
more useful in identifying the original reason for the bumps.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-07-28 06:02:05 +00:00
Patrick McCarty
d612ea1d6d bumpnogit: revise docs to align with bump target
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-07-22 22:38:41 +00:00
Patrick McCarty
36d49108a6 bump: add custom commit summary support
To add a custom commit summary for a `make bump`, set the BUMP_MSG
variable to the desired summary.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-07-22 22:37:18 +00:00
William Douglas
349d8c6645 Add CLEANUP=1 for when update runs autospec 2022-07-08 08:17:29 -07:00
William Douglas
1aecb3dcf5 Add an update target to common
Add an update target to common whose purpose is to be able to check if
a new version exists, update to the new version and push the updated
version to koji.

This target will cause bump.list content to be rebuilt as well.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-07-08 08:03:07 -07:00
William Douglas
e610c46517 Makefile.common: Add bump.list to make koji
Add the ability for make koji to look a bump.list file in the package
directory and make and koji all the packages listed in that file.

This is useful for package's that depend on the updated package and
should be rebuilt after the original package is.

If for some reason this functionality isn't desired make koji-nowait
will not look at the bump.list.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-07-05 12:01:09 -07:00
Patrick McCarty
f78b8dad99 common: add new koji-waitrepo command
This command wraps `koji wait-repo` for convenient use within the common
tooling.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-07-05 18:21:59 +00:00
Patrick McCarty
96fd23e88b Specify branch name of main for git init of package repos
The default package repo branches are now `main`.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-04-11 16:13:18 -07:00
William Douglas
6f5fd11673 Move packages content over to main branch
World has moved to main, start updating accordingly.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-04-08 01:22:35 -07:00
Patrick McCarty
890363f65f Handle CLEANUP variable similarly for make autospec
To make the handling of the variable more uniform, set autospec's
cleanup flag using make conditional constructs rather than shell
expansion.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-03-30 11:07:02 -07:00
Patrick McCarty
e90b6f8c72 Support CLEANUP environment variable for more commands
Now, whenever the `CLEANUP` variable is set in the environment, `make
build` and `make build-nocheck` will automatically clean up the build
chroot after a build completes. The same feature has been supported by
`make autospec` for many years.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2022-03-30 11:06:28 -07:00
Patrick McCarty
3cd143d931 Add make whatrequires target
This target can be used to query direct build and runtime dependencies
of any package in Clear Linux OS. The `make help` documentation
describes the current output format and limitations.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2021-09-07 16:14:15 -07:00
Patrick McCarty
49289f00d9 install-debuginfo-local: fix up symlinks to avoid lookaside directory
Because clr-debug-info doesn't yet use the lookaside location
(/usr/share/debug), we still need to modify the debuginfo symlinks to
point to the location under /usr/lib/debug.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2021-03-16 02:23:32 +00:00
Patrick McCarty
cd55363a3c Improve detection of the debuginfo rpm
It's not likely for a package name to have the substring "-debuginfo",
but just in case, implement more robust detection of the actual
debuginfo rpm for a package by constructing the name more carefully.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2021-03-09 06:19:40 +00:00
Patrick McCarty
e0054520f8 Filter out debuginfo rpms when scanning for banned files
The debuginfo subpackages are automatically generated, with content
installed according to a predefined directory layout, so there is little
value in checking for banned files they may ship.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2021-03-09 06:02:10 +00:00
Patrick McCarty
5946c4dfa9 koji: support local branch names of 'main' as well
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2021-01-06 18:42:00 +00:00
Patrick McCarty
e227e8722b Use %{NVR} queryformat when possible
The `%{NVR}` format is equivalent to `%{NAME}-%{VERSION]-%{RELEASE}`.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2020-12-14 03:52:58 +00:00
Patrick McCarty
b924031ba6 Use parameterized function for most rpmspec commands
The `rpmspec` commands usually specify the same flags aside from the
queryformat and the spec file(s) to query.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2020-12-14 03:49:30 +00:00
Patrick McCarty
2b76e56f69 generateupstream: improve sha1sum invocation
Indicating end of options with `--` fixes the command invocation if the
filename begins with a hyphen.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2020-10-15 03:02:13 +00:00
Thiago Macieira
0dfb2f6ef5 Pass the make's -j option to rpmbuild
For most people simply running "make autospec" or "make build", this
will have no effect. The RPM macro %{_smp_mflags} is predefined to be
the number of CPUs in your system anyway (output of getconf
_NPROCESSORS_ONLN).

But if you do have MAKEFLAGS set in your environment to your CPU
number (and you should), that will get passed to rpmbuild, which then
uses it during the build. And if you, like me, have it set to a
slightly higher number, RPM will use that too.

But if you run "make -jN autospec", then that N will be passed to the
build. This is especially important if you have something doing "make
-j1": RPM will be -j1 too! So instead of "make -j1", run "MAKEFLAGS= make"
2020-05-19 13:47:42 -07:00
Patrick McCarty
a198b2d0a0 Make local repo support more parallel safe
For commands that modify contents of $(TOPLVL)/repo, wrap them with
`flock`, using a lock file `repo.lock` within the repo directory. This
ensures that the repo directory tree is not otherwise modified while
each command is running.

To keep the implementation simple, I opted to not wrap entire makefile
targets with a lock (e.g. lock the entire operation of `make repoadd`).
If users need that level of control, they can implement locking in a
separate wrapper script.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2020-05-18 13:55:19 -07:00
Arzhan Kinzhalin
cbb1d9688e make koji: check remote tag exists beforehand 2020-02-20 10:17:32 -08:00
Patrick McCarty
185dec68b8 Improve handling of autospec's --mock-opts option
This change allows specifying short/long mock options via MOCK_OPTS
without a leading backslash escape.

For example, `MOCK_OPTS="\--old-chroot --no-clean"` can now be specified
as `MOCK_OPTS="--old-chroot --no-clean"`.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-11-11 12:52:26 -08:00
santoshkumar
d7f6ddcce1 make bump : Replaced *.spec to package specfilic specfile variable $(SPECFILE).
In the submmitter's local pacakge directory can have mulptiple spec file present,
or swap file thus to avoid adding multiple specfile in git add better only add package
specific specfile.

Signed-off-by: santoshkumar <santoshkumar.laxminarayan.rai@intel.com>
2019-10-03 10:15:36 -07:00
Alex Jaramillo
22a927e26b Removing cve-check-tool
This tool no longer works and the repo looks abandoned (last commit
04/27/2017). The NVD cve database has changed format since.

Signed-off-by: Alex Jaramillo <alex.v.jaramillo@intel.com>
2019-10-01 22:24:48 +00:00
Patrick McCarty
81ef638151 autospec: establish different calling convention for setting cli booleans
For the `make autospec` variants, use a simpler calling convention so
that the callers do not need to know the boolean option names.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-09-29 13:30:41 -07:00
William Douglas
8a2da3412d Add repostage target
When building large numbers of packages that are interdependent,
recreating the rpm repository for each package can create a large
overhead during processing. 'repostage' is intended to provide an
intermediate target for adding a package to a repository that can
later be built using localrepocreate.
2019-07-29 17:44:24 -07:00
Patrick McCarty
3b1e4e5558 pullrebase: skip rebase if current branch is up-to-date
Any attempt at rebasing is unnecessary when the current branch is
up-to-date with the remote, so skip the rebase in this case.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-07-17 12:54:04 -07:00
Arjan van de Ven
c2d637b438 don't shhow version changes in the 32 bit / avx copies 2019-07-05 13:02:00 +00:00
Tan, Yew Wayne
69cee2fcd7 LTS package maintenance utility (ltsutils): initial commit
This tooling is designed to automate 2 main tasks that are part of
the package maintenance workflow of Clear Linux LTS. These tasks are:
- Back-porting of a patch (e.g. security fix) to older branches.
- (Not implemented yet) Building RPMs with the intent of sharing binaries of
  older LTS branches to newer branches whenever possible.

2 new targets are defined in Makefile.common.lts:
- lts-show: Show a summary of active LTS branches
- lts-backport: Attempt to fast-forward the previous active branch to the current branch

"Active" branches correspond to LTS releases that currently have support.
They are listed in a flat file "active-branches" in "lts" directory, from
oldest to newest. New entries are added by Clear Linux LTS developers as
new releases become available, and entries removed as releases become
obsolete.

Note: For CVE patching, the tool is not aware of CVE severity levels or
the minimum supported severity level of each LTS branch. For now it is
the user's responsibility to know when a CVE does not apply to older
branches and stop calling "make lts-backport".

Signed-off-by: Tan, Yew Wayne <yew.wayne.tan@intel.com>
2019-06-21 09:12:15 -07:00
Geoffroy Van Cutsem
79b17335fa Really set 'latest_builds' to the latest value of the day
Consider the following scenario:
* System being set-up on day 0 (by running user-setup.sh script)
* The github.com/clearlinux/common repo is cloned under projects/common
* Fast-forward in the future
* Clone a package for which there is no reachable upstream URL
	(e.g. clr-power-tweaks or clr-systemd-config
* Run 'make sources'
That operation will fail because it will search for the src.rpm files starting
with the Clear Linux build determined by looking up the latest tag in the
project/common repo (and the latest is the one from day 0). So it will only find
older versions of those src.rpm, and will eventually fail.

This patch adds a line that fetches all the latest tags from the upstream
projects/common repo so the 'latest_builds' variable is *really* set to the
latest of the day.

Signed-off-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
2019-06-19 11:03:00 -07:00
Thiago Macieira
facfc57d55 Update message for the for-review.txt on how to recreate
People SHOULD update commit messages if it's not a simple bump or
version update.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2019-06-12 16:05:52 -07:00
Patrick McCarty
9ed6cf64eb Add notice about for-review.txt; make compatible with git send-email
The for-review.txt files were being generated with `git show` and thus
not compatible with `git send-email`. Use `git format-patch` instead to
fix this issue.

Also add a more prominent notice about the existence of for-review.txt
and its intended purpose.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-06-03 15:17:49 -07:00
Patrick McCarty
f9b7f544be scratch-wait: explicitly specify the wait option
We always want the koji CLI to wait for `make scratch-wait`, regardless
of the environment koji detects it is being run under.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-05-22 00:44:24 -07:00
Patrick McCarty
59dfc2236f install-debuginfo-local: fix installation with new lookaside locations
Debuginfo packages now install binary debuginfo to /usr/share/debug and
source debuginfo to /usr/share/debug/src.

The purpose for the migration is for these new locations to serve as
"lookaside" directories used by future versions of clr-debug-info,
enabling select debuginfo packages to be installed in bundles and safely
coexist with the FUSE mounts.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-05-07 10:58:47 -07:00
Arjan van de Ven
9921115897 git diff not git show 2019-04-25 21:59:43 +00:00
Arjan van de Ven
6bc5718c25 Add patchfilter to autsopecnew 2019-04-25 18:21:11 +00:00
Arjan van de Ven
168c9b8606 add a for-review filtered patch 2019-04-20 00:32:37 +00:00
Patrick McCarty
83a461fe40 Print progress messages for install-local and install-debuginfo-local
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-04-11 10:59:21 -07:00
Patrick McCarty
6876f3d11e Skip debuginfo install if the required rpm does not exist
Some packages in the distro do not generate debuginfo subpackages, so
make sure to skip the debuginfo install in this situation.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2019-04-11 10:46:49 -07:00
California Sullivan
9fc1471023 Makefile.common: mockproper: only remove the chroot
The caches are not package-specific, and --scrub=all will clean all
caches even with the --uniqueext option. For that, we have 'make
mockclean' instead.

Using both --clean and --scrub=chroot is also unnecessary, as --clean
already removes the chroot.

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
2019-03-22 16:39:29 -07:00
Thiago Macieira
3a0c58c76f Add a target to install the RPMs in the image accessed via 'make shell'
Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2019-01-29 09:35:04 -08:00
Auke Kok
b671ab5709 Add cloc target for common tooling.
Produces and displays `results/cloc.txt`. Useful to determine code
metrics.
2018-12-19 09:30:04 -08:00
Patrick McCarty
11e0b5baee Add new commands for installing rpms on the running system
For testing purposes only, if developers want to install RPMs they just
built onto their development system, potentially overwriting content
that is tracked by swupd, they can now run 'make install-local' to
install all of the RPMs, or 'make install-debuginfo-local' to just
install the debuginfo package.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-10-10 13:09:20 -07:00
Patrick McCarty
012a1a3767 Support pre-checks for autospec and autospecnew too
For some commands, it is useful to be able to add custom prerequisites.
We already support this for `make koji` and `make build`, and there was
just a request to add it for `make autospec` and `make autospecnew`.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-09-21 12:12:23 -07:00
Thiago Macieira
f59a650cf4 Also delete the rpms/ subdir when making proper
Autospec has begun putting the files there.

Signed-off-by: Thiago Macieira <thiago.macieira@intel.com>
2018-09-05 16:29:59 -07:00
Patrick McCarty
852b6e83c9 Refresh SPDX license list to latest (version 3.2)
Make the SPDX puller code more robust by reading from JSON sources
rather than parsing HTML. Also, keep all deprecated license identifiers
for now, since they are still present in the spdx list.

Four identifiers were removed since the last refresh, but we are keeping
them for backwards compatibility. The removed license IDs now reside in
the licenses-extra file. At

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-08-29 17:19:34 -07:00
Brett T. Warden
5bc6cc856b Error on specfile and upstreams only when missing
Previously, we always emitted an error and failed when the specfile or
upstream target was requested, breaking the -B switch. Now, we only fail
if the file doesn't actually exist, not just that it was requested via
make.
2018-08-07 09:03:54 -07:00
Patrick McCarty
3739bc0e0f repoadd: use new rpms directory, and hardlink instead
Since binary RPMs are now stored in the "rpms" directory, follow a
similar method for storing them in the local repo, too.

Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
2018-08-01 09:38:27 -07:00