This commit adds the -N/--needs-update option, disabled by default,
to list only packages with newer upstream versions. All other packages
will be excluded from the HTML or JSON output.
Signed-off-by: Kadambini Nema <kadambini.nema@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
When running "make pkg-stats" on a host with Python 3.14 (e.g.
Fedora 43 for example), the execution fails with the error:
Checking URL status
Traceback (most recent call last):
File "/buildroot/support/scripts/pkg-stats", line 1387, in <module>
__main__()
~~~~~~~~^^
File "/buildroot/support/scripts/pkg-stats", line 1368, in __main__
loop = asyncio.get_event_loop()
File "/usr/lib64/python3.14/asyncio/events.py", line 715, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'MainThread'.
This is due to a breaking change introduced in Python 3.14
asyncio.get_event_loop(). See [1]. Before Python 3.14, this call was
creating and setting an event loop if there was none. This situation
is now a runtime error.
In order to fix this issue with newer Python version, while keeping
backward compatibility, this commit replaces the code:
loop = asyncio.get_event_loop()
by an explicit event loop creation:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
This commit was tested on a Fedora 43 host with Python-3.14.2, and
with the Buildroot Docker image plus the python3-aiohttp package
which is a Debian 12 with Python-3.11.2.
[1] https://docs.python.org/3.14/library/asyncio-eventloop.html#asyncio.get_event_loop
Signed-off-by: Julien Olivain <ju.o@free.fr>
Tested-by: Vincent Fazio <vfazio@xes-inc.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The NVD database has CVE entries that are not present but may be
referenced in other security trackers.
For instance the CVE-2024-12455 is documented in the Debian security
tracker [1]. However, the NVD page is empty [2] and this entry is not
present in the NVD database mirror.
The following command would make the script fail:
```
echo '{
"vulnerabilities": [
{
"id": "CVE-2024-12455"
}
]
}' | support/scripts/cve-check --enrich-only
```
No CVEs present in Buildroot ignored CVEs are affected. But when
enriching an SBOM with legitimate CVE not present on NVD, the script
will fail.
This patch change the behavior to just log to stderr unknown CVEs
instead of making the script fail.
[1] https://security-tracker.debian.org/tracker/CVE-2024-12455
[2] https://nvd.nist.gov/vuln/detail/CVE-2024-12455
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
[Peter: Tweak warning message]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
As requested by the gnu.org admins:
"
Since August 2024, we've been under DDoS attacks from common command line
tools.
To fix this, we would need to change the user-agent from "Python/3.11
aiohttp/3.8.4" to "buildroot.org pkg-stats" instead.
"
It indeed probably makes sense to use an unique user-agent string, so rework
the script to do that.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The gnu.org admins have been blocking the IP address of machines running
pkg-stats as the GET requests for the (many) packages with gnu.org URLs are
seen as abusive.
The resource body is not used, so use a HTTP HEAD request instead of a GET
to limit server load and bandwidth use.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When using a custom skeleton where the merged symlinks are missing,
the build fails with errors like:
support/scripts/check-merged -t skeleton -u -b /usr/src/simplek8s/rootfs-skeleton
The skeleton in -t is not properly setup:
- /usr/bin should exist, be a directory, and not be a symlink
- /usr/lib should exist, be a directory, and not be a symlink
The skeleton in skeleton is not properly setup:
- /usr/bin should exist, be a directory, and not be a symlink
- /usr/lib should exist, be a directory, and not be a symlink
[...]
Commit 793ebd5d28 (support/scripts/check-merged: use getopts instead of
getopt) intoduced a flawed use of getopts: unlike getopt, getopts does not
conume the positional arguments. This causes the check for directory
validity to also check each option as if they were directories.
For overlays, this is transparently ignored, because the checks are only lax
for overlays (missing symlinks are OK).
However, for skeletons, the checks are strict. Because of that, a missing
symlink is considered an error, when it should be considered as being OK.
The fix is to actually consume the positional args to only keep the list of
directories to validate, like is done for example in
support/download/dl-wrapper.
Fixes: 793ebd5d28
Reviewed-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Commit 1187c34d88 (support/scripts: move merged-usr errors message
into check-merged-usr.sh) introduced the use of getopt to parse its
options; doing so allowed to use long option (with two leading dashes),
which is more descriptive than the usual one-character options.
However, getopt is part of util-linux; it is not a shell built-in.
util-linux is not a prerequisite of Buildroot, so we may end up running
on a system where it is missing.
We could add host-util-linux as a dependency when the system does not
provide getopt, but that's not very nice; even though host-skeleton does
not need to check for merged-bin for now, it does not need getopt, and
thus we could add host-util-linux (which depends on host-skeleton) as a
dependency of skeleton-custom. But that will not be tenable over the
long run, especially if/when we do a merged-bin in host dir.
Requiring that util-linux be installed system-wide is not nice either;
it's an additional requirement on the host.
We can do like we do in the oter scripts, though: use the shell built-in
getopts. Its usage is slightly different, and does not support long
options. As it's just for use in an internal script, we can live with
the less descriptive options, though.
Switch to using getopts, it removes the need for a new host dependency.
Fixes: 1187c34d88
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Enriches the input CycloneDX SBOM with vulnerability information and
analysis from the NVD database.
The NVD database is cloned using a mirror of it and the content is compared
locally. By default the path 'dl/buildroot-nvd' is used.
Example usage to analyse vulnerabilities of an input CycloneDX SBOM:
$ make show-info | utils/generate-cyclonedx | support/script/cve-check
The 'cve-check' can also be used to only enrich the vulnerabilities
present on the input SBOM with a set metadata (description, cvss,
references, ...) without applying an analysis.
With the following command the vulnerabilities ignored by Buildroot
present in the CycloneDX SBOM are enriched with description, cvss, etc
...
$ make show-info | utils/generate-cyclonedx | support/script/cve-check --enrich-only
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
[Peter: fix minor flake8 issues]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This patch move the 'download_nvd' call to the 'pkg-stats' script
instead of automatically calling 'read_nvd_dir'.
Since the cve.py file can be used as a library it's up to the caller to
decide whether or not to update the NVD database.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we partially accept that a skeleton or a rootfs overlay be
merged:
- for unmerged, we accept all kind of situations: unmerged, partially
merged, badly merged, merged-usr or merged-bin, arbitrary relative
or absolute symlinks, and whatnots;
- for merged-usr, we strictly require a properly set up merged-usr,
and we refuse a merged-bin;
- for merged-bin, we stricty require a properly set up merged-bin.
The unmerged case is inconsistent with the other cases, especially it
allows for arbitrary symlinks that may point to arbitrary locations that
may even not belong to $(TARGET_DIR) at all...
We fix that by ensuring that the skeleton and overlays strictly adhere
to the merge-level of the configuration; i.e. for an unmerged config, we
require that the skeleton and overlays be strictly unmerged, that is,
/bin, /lib, and /sbin, and their counterparts in /usr, are actual
directories.
Thus, for all three types of merge level, the skeleton and overlays must
match the configured merge level.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Starting with version 256 [0], systemd warns when /usr/bin and
/usr/sbin are different directories; in the future, it may even
refuse to boot in such a situation.
Add support for merged-bin, not unlike the support we have for
merged-usr; we also make merged-bin a sub-case of merged-usr
(i.e. it is not possible to do merged-bin without merged-usr).
[0] https://github.com/systemd/systemd/blob/v256/NEWS#L265
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Acked-by: TIAN Yuanhao <tianyuanhao3@163.com>
Cc: Edgar Bonet <bonet@grenoble.cnrs.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
We're going to need it to check merged-bin, so the naming would be
misleading as it would no longer be just about merged-usr.
Also drop the extension, it's useless.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently, we accept that the merged usr is backward, i.e. that the
/usr/bin, /usr/sbin, and /usr/lib entries be symlinks to, resp., /bin,
/sbin, and /lib. We also allow either entries to be symlinks to other
parts of the root. Both are accepted despite the comment at the top of
the script explaining what should be accepted.
However, a properly merged usr is the other way around: /bin, /sbin, and
/lib are the symlinks to resp. /usr/bin, /usr/sbin, and /usr/lib, which
are actual directories.
Fix the check-merged-usr script accordingly: implement the test as we
mean it, by testing the conditions rather than resorting to a convoluted
and incorrect use of stat(1).
Even though the split between test_dir() and test_merged() seems
superfluous, it'll come useful when we introduce support for merged-bin
in a later patch.
For skeletons, we require that the directories do exist, while we allow
them to be missing for overlays; indeed, it is perfectly legit to
provide an overlay that only contains totally unrelated directories
(e.g. /var/www to populate a webroot for example).
Extend the heading-comment to be more explicit (and drop '/' as there is
nothing to say about it).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
By moving the loop over the overlays into the script, we can generate
better error messages about how and why a skeleton or a specific overlay
is improperly setup for merged-usr.
We can also now rely on its exit code to decide whether the skeleton or
the overlays are properly setup, rather than stash the stdout/stderr to
a Makefile variable and test the emptiness thereof.
Introduce a --type option to pass the type of root to verify, for better
error reporting. This will incidentally be usefull in a future commit,
when we need to take different actions based on whether the root is a
skeleton or an overlay.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Rather than repeat the same canned sequence over and over again, move it
to a function that we can reuse as many times as needed. This will come
handy when we later need to check merged-bin.
Switch to using bash as it allows for nicer functions (local variables),
but we anyway require it globally already so that's not an additional
dependency.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Acked-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Given the following criteria: `cpe:2.3:a:oneidentitty:syslog-ng:*:*:*:*:-:*:*:*`.
The former `cpe_matches` implementation would match with the following
CPE: `cpe:2.3:a:oneidentitty:syslog-ng:4.71:*:*:*:premium:*:*:*`.
The 'hyphen' ('-') meaning is "Not Attributed" (NA) a criteria with no
attributed software edition shouldn't match with a CPE with an attributed
software edition:
https://csrc.nist.gov/pubs/ir/7695/final
This patch also create a distinct 'CPE' object that aggregate the
function specifics to CPEs like it's done for 'CVE'.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The `support/script/cve.py` file is used as a library. Depending on how
you use this library you might not want to write content to stdout when
calling its function.
This patch move the 'updating' log to the 'pkg-stats' script and write
the alert when LooseVersion doesn't have a version to stderr.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The each_cpe() method is careful that some CVEs have no
"configurations", but some CVEs such as
https://nvd.nist.gov/vuln/detail/CVE-2025-32915 apparently have a
"configurations" node, but no "nodes" inside the "configurations",
causing an exception:
Traceback (most recent call last):
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 1382, in <module>
__main__()
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 1371, in __main__
check_package_cves(args.nvd_path, packages)
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 679, in check_package_cves
check_package_cve_affects(cve, cpe_product_pkgs)
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 638, in check_package_cve_affects
for product in cve.affected_products:
^^^^^^^^^^^^^^^^^^^^^
File "/home/buildroot/buildroot-stats/support/scripts/cve.py", line 185, in affected_products
return set(cpe_product(p['id']) for p in self.each_cpe())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/buildroot/buildroot-stats/support/scripts/cve.py", line 185, in <genexpr>
return set(cpe_product(p['id']) for p in self.each_cpe())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/buildroot/buildroot-stats/support/scripts/cve.py", line 173, in each_cpe
for node in nodes['nodes']:
~~~~~^^^^^^^^^
KeyError: 'nodes'
Fixes:
54f8d97c91 ("support/scripts/pkg-stats: adapt to NVD v2 json format")
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Romain:
- add reference to buildroot commit introducing the issue
- a similar patch was sent by Daniel Lang (thanks!)
- needed on master to fix "Daily results" email]
(cherry picked from commit 67422b9d9c)
Signed-off-by: Romain Naour <romain.naour@smile.fr>
This issue has been fixed in the Bootlin toolchains thanks to the
Microblaze "atomic fix" in GCC.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The NVD database contains some CPEs that are wrongly not associated
with any version number. They are for example sometimes associated
with very old CVEs.
Those CPEs are annoying, because they pollute our pkg-stat CVE results
with CVE entries which actually don't affect us.
The proper way to solve it is, and should remain, to fix the NVD
database by reporting these issues. Having to deal with a lot of
CVEs/CPEs, the NVD database is however slow to be updated.
To reduce the noise in our pkg-stats results in the meantime, one
possibility is to add <PKG_IGNORE_CVES> entries for those CVEs. This
however comes with the downside that even once the NVD database gets
fixed, those ignored entries risk remaining in Buildroot forever
because they are undetected.
This commit tries to address this downside by checking for and
reporting CVEs that are ignored in Buildroot, but where the
NVD reports our package version as unaffected. Those CVEs will appear
in the 'CVEs Ignored' column as '(stale)', and the cell will be
colored the same way warnings are. This should allow us to detect and
remove those entries.
It can be tested for example by adding the following variable to the
apache package (for a CVE that was recently fixed in the NVD database):
APACHE_IGNORE_CVES = CVE-1999-0236
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The last usage of each_product() was removed in commit
52ae092046 ("support/scripts/cve.py: use
the JSON data in 1.1 schema").
Since it's now unused, remove it.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Annoyingly, using "--disable warning" does not disable the warnings
checks.
It turns out that we look for "warnings" (i.e. with an 's') to know if
we should disable the warnings check, so update the help text
accordingly.
Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The ARMV7-A toolchains are capable to compile binaries for ARMv8-A CPU
in AArch32 execution state.
This adds the BR2_ARM_CPU_ARMV8A option in the 'conditions' to allow
ARMV8-A CPU such as Cortex-A53 or Cortex-A72 to use ARMV7-A toolchains.
Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Running pkg-stats is currently quite verbose, as it shows one line per
package when checking for the upstream URL, and another one line per
package when checking for the latest version on
release-monitoring.org.
This noisy output is a bit annoying when pkg-stats is run in a
cronjob, like we do to update https://autobuild.buildroot.net/stats/
every day. This commit adds a -v/--verbose option, off by default, to
have a less noisy output.
Suggested-by: Peter Korsgaard <peter@korsgaard.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
SC2086 is now reported for auto_id since shellcheck 0.9.0:
In support/scripts/mkusers line 453:
add_one_group "${g}" ${auto_id}
^--------^ SC2086 (info): Double quote to prevent globbing and word splitting.
So quote it to get rid of this error.
Signed-off-by: Romain Naour <romain.naour@smile.fr>
[Peter: quote variable instead of disabling check]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Rather than fork+exec:ing a whole bash process for every file of
interest, split up the work into ${PARALLEL_JOBS} separate file lists,
and then fork off a single worker per list.
Since workers now run in a fork, we also don't have to re-parse the
arguments on every call to patch_file().
In the worker, we also add a low-overhead check for the ELF magic,
which allows us to skip all non-ELFs without having to fork off
additional processes. In the event that we encounter an invalid ELF,
that is still taken care of by the existing exitcode-test in
patch_file().
On a 4-core/8-thread box, this cuts down the time it takes to fix the
RPATHS in the host directory from around 25s to 1.5s.
Signed-off-by: Tobias Waldekranz <tobias@waldekranz.com>
[Peter: fix shellcheck issues]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The commit 6889056f1e (support/scripts/genimage.sh) adds support for
creating bmap images.
Since then, the script genimage.sh looses the error returned by the
genimage command.
As a consequence, the target-finalize target looses the error as well,
and so on up to make.
This adds the shell option -e to exit immediately if a command ends with
an error, so the error is not ignored.
Fixes:
gportay@archlinux ~/src/rtone-br2-external $ make
(...)
>>> Executing post-image script support/scripts/genimage.sh
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"" (stderr):
INFO: cmd: "rm -rf "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"/*" (stderr):
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp"" (stderr):
INFO: cmd: "cp -a "/tmp/tmp.EclKGwtA4R" "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp/root"" (stderr):
INFO: cmd: "mkdir -p "/home/gportay/src/rtone-br2-external/output/images"" (stderr):
INFO: vfat(efi-part.vfat): cmd: "mkdosfs '/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat'" (stderr):
INFO: vfat(efi-part.vfat): adding file 'efi-part/EFI' as 'EFI' ...
INFO: vfat(efi-part.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -sp -i '/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat' '/home/gportay/src/rtone-br2-external/output/images/efi-part/EFI' '::EFI'" (stderr):
Disk full
INFO: vfat(efi-part.vfat): cmd: "rm -f "/home/gportay/src/rtone-br2-external/output/images/efi-part.vfat"" (stderr):
ERROR: vfat(efi-part.vfat): failed to generate efi-part.vfat
INFO: cmd: "rm -rf "/home/gportay/src/rtone-br2-external/output/build/genimage.tmp/"" (stderr):
make[1]: Leaving directory '/home/gportay/src/rtone-br2-external/buildroot'
gportay@archlinux ~/src/rtone-br2-external $ echo $?
0
Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Nios2 will be removed from Buildroot, so Bootlin toolchains will
no longer provide nios2 toolchain in further releases.
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
When I initially made the CVE cells collapsible, I neglected to
count the unsure CVEs. This patch adds unsure CVEs to the cell collapsing
calcualation to ensure that cells with lots of unsure CVEs actually get collapsed.
This patch also removes the "+ 1" from the cve_total calculation,
which fixes the cve_total being off-by-one.
I'm not sure *why* I did that in the first place.
demo:
https://sen-h.codeberg.page/pkg-stats-demos/@pages/add-unsure_cves-to-cve_total-calc.html
Signed-off-by: Sen Hastings <sen@hastings.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
With the introduction of ARM FDPIC support, we need to make sure that
the existing ARM FLAT external toolchains are not made
visible/available when ARM FDPIC is selected. This commit updates the
gen-bootlin-toolchains script to take this into account.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit a87abcf6da (Makefile: run PPD and RPATH fixup in host-fialize)
(sic) moved the fixups peviously done in prepare-sdk, to host-finalize.
This exposed a bug in fix-rpath, when RPATH contains multiple entries,
like: /PPD/host-foo/host/lib:/PPD/host-foo/host/lib/foo
In that situation, we want to get rid of /PPD/host-foo and replace it
with the finale HOST_DIR, so we mangle the RPATH with a sed expression.
However, that sed expression only ever replaces the first match, as it
is missing the 'g' option. Thus, the second (and following) parts of
RPATH are still referring to the PPD, and thus patchelf does not find it
relative to the final HOST_DIR, amd rops it. This eventually lead to a
final RPATH set as $ORIGIN/../lib instead of the expected
$ORIGIN/../lib:$ORIGIN/../lib/foo
This is the case for host-systemd, which installs some of its libraries
in $PREFIX/lib/systemd/ and adds an RPATH set appropriately to
/PPD/host-systemd/host/lib:/PPD/host-systemd/host/lib/systemd and that
gets incorrectly mangled.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/issues/39
Also fix a typo in the comment just above.
Reported-by: José Luis Salvador Rufo @jlsalvador
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Reviewed-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Tested-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Reviewed-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This reverts commit e0ed05cc00.
Since this commit has been applied, the update of the NVD data fails:
Checking packages CVEs
Updating from https://github.com/fkie-cad/nvd-json-data-feeds/
Traceback (most recent call last):
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 1346, in <module>
__main__()
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 1335, in __main__
check_package_cves(args.nvd_path, packages)
File "/home/buildroot/buildroot-stats/./support/scripts/pkg-stats", line 660, in check_package_cves
for cve in cvecheck.CVE.read_nvd_dir(nvd_path):
File "/home/buildroot/buildroot-stats/support/scripts/cve.py", line 105, in read_nvd_dir
CVE.download_nvd(nvd_git_dir)
File "/home/buildroot/buildroot-stats/support/scripts/cve.py", line 74, in download_nvd
subprocess.check_call(
File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'pull', '--depth', '1']' returned non-zero exit status 128.
Since we couldn't immediately figure out what's going on, let's revert
for now until this get investigated and fixed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
As of the 2024.05 toolchain release, the toolchain tarballs are now
xz-compressed instead of bz2-compressed. Since this script is only
meant to update to the latest toolchains, we don't really bother
supporting the older toolchains that are bz2-compressed, and we simply
assume the tarballs are xz-compressed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Just a simple clone and pull with --depth 1 should be enough to parse
the CVE data and generate the pkg-stats report.
From a full clone and a depth-1 clone, and the size delta is 2.9GiB
vs. 2.2GiB.
The download size does change: from 983.55MiB down to 270.78MiB.
It's a net time win too: 2m17s vs 1min7s (on a 100Mbps link).
Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 6889056f1e (support/scripts/genimage.sh: support creating a bmap
image) added unconditional gzip compression of the genimage outputs if
bmap-tools is enabled, which is problematic for a number of reasons:
- The gzip invocation is not reproducable (E.G. does not use -n)
- The original file is removed, dropping the sparse info
- It hardcodes policy, E.G. bmap supports a number of different compression
algorithms / compression may not be desired for all images / different
compression levels may be desired
So drop the compression step. If desired, compression can be done with a
post-image script.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Acked-by: TIAN Yuanhao <tianyuanhao3@163.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The patch adds the possibility to create, in addition to the usual
image, an image of type bmap that drastically reduces the amount of
data that needs to be written to an SD card, resulting in time
savings.
It looks at whether BR2_PACKAGE_HOST_BMAP_TOOLS is enabled to decide
whether it should or not generate bmap tool images. It generates bmap
images for all images referenced in the genimage configuration file,
as long as they exist in $(BINARIES_DIR).
Signed-off-by: Dario Binacchi <dario.binacchi@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The current implementation of buildroot depe dependency graphing
either does forward- or reverse-dependency traversal.
This patch enables buildroot to graph forward and reverse dependencies
on the graph for the same package: (Diagram Credit: Yann E. MORIN)
$ make pkg-d-graph-both-depends
pkg A -. .-> pkg E
\ /
pkg B ----> pkg D ----> pkg F
/ \
pkg C -' '-> pkg G
In the above example a single graph shows pkg {A,B,C} are needed
by pkg D, and pkg D is a dependency of pkg {E,F,G}.
Makefile help and manual are also updated.
Signed-off-by: Steve Hay <me@stevenhay.com>
[Arnout:
- remove DEPTH and RDEPTH, their functionality is already covered by
BR2_GRAPH_DEPS_OPTS;
- remove --rdepth, it was felt to not add sufficient added value;
- add the new target to the manual;
- fix flake8 errors.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Instead of having done_deps as a global variable, pass it as an argument
to the print_pkg_deps function. This will allow the recursive search to
be started multiple times.
Signed-off-by: Steve Hay <me@stevenhay.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Kadambini Nema