Since llama.cpp update in Buildroot commit [1], the test_aichat can
fail for several reasons:
The loop checking for the llama-server availability can fail if curl
succeed, but the returned json data is not formatted as expected.
This can happen if the server is ready but the model is not completely
loaded. In that case, the server returns:
{"error":{"message":"Loading model","type":"unavailable_error","code":503}}
This commit ignore Python KeyError exceptions while doing the
server test, to avoid failing if this message is received.
Also, this new llama-server version introduced a prompt caching, which
uses too much memory. This commit completely disable this prompt
caching by adding "--cache-ram 0" in the llama-server options.
[1] https://gitlab.com/buildroot.org/buildroot/-/commit/05c36d5d875713521f99b7bad48be316dcde2510
Signed-off-by: Julien Olivain <ju.o@free.fr>
Release notes: https://github.com/ggml-org/llama.cpp/releases
Merge BR2_PACKAGE_LLAMA_CPP_SERVER into BR2_PACKAGE_LLAMA_CPP_TOOLS, as
both of these options must be enabled to build tools like llama-cli and
llama-server. See upstream commit [1].
Since the Buildroot option BR2_PACKAGE_LLAMA_CPP_SERVER is removed, this
commit also removes it from support/testing/tests/package/test_aichat.py
which was using it.
[1] https://github.com/ggml-org/llama.cpp/commit/a180ba78c710f52af7c7edee562fbe98c36d6ddd
Signed-off-by: Joseph Kogut <joseph.kogut@gmail.com>
[Julien:
- reindent options in .mk
- remove BR2_PACKAGE_LLAMA_CPP_SERVER in test_aichat.py
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Python setuptools has removed the obsolete pkg_resources python module in
v82.0.0. This module is used by the binman command in U-Boot until
v2025.10.
Since the python-setuptools package has been updated to v82.0.0 in
Buildroot, the python test tests.boot.test_atf.TestATFAllwinner (using
U-Boot v2023.10 and binman) fails to build with the following error [1]:
ModuleNotFoundError: No module named 'pkg_resources'
Update test_atf to use a more recent U-Boot v2026.01 (and add the
dependency on GNU TLS), to fix the build.
Link: https://gitlab.com/buildroot.org/buildroot/-/jobs/13500946337 [1]
Fixes: 51365ff063 ("package/python-setuptools: bump to version 82.0.0")
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: James Hilliard <james.hilliard1@gmail.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Subproject tarballs are not provided anymore:
https://discourse.llvm.org/t/llvm-22-1-0-released/89950
"Please note since the last release the subproject tarballs have been
removed"
https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-tarballs-in-the-release-page/75024/14
Used upstream tarball llvm-project-22.1.0.src.tar.xz for all packages
and linked subproject hash files to ../llvm-project.hash.
Removed patches which fix build errors caused by subproject tarballs.
Removed handling of third-party-21.1.8.src.tar.xz which is included in
the monolithic tarball.
Added _SUBDIR variable or updated _INSTALL_CMDS when needed.
For compiler-rt:
Removed both patches, they are not needed anymore.
Added patch to fix aarch64 build.
Added dependency on gcc >= 15.x because libcxx now depends on gcc >=
15.x: https://github.com/llvm/llvm-project/pull/165684
warning "Libc++ only supports GCC 15 and later"
Building with gcc 14.x causes many build errors like
output/build/compiler-rt-22.1.0/compiler-rt/buildroot-build/lib/fuzzer/libcxx_fuzzer_x86_64/build/include/c++/v1/__type_traits/is_array.h:43:68:
error: expected primary-expression before ')' token
output/build/compiler-rt-22.1.0/compiler-rt/buildroot-build/lib/fuzzer/libcxx_fuzzer_x86_64/build/include/c++/v1/__type_traits/is_array.h:43:44:
error: there are no arguments to '__is_unbounded_array' that depend on
a template parameter, so a declaration of '__is_unbounded_array' must
be available [-fpermissive]
output/build/compiler-rt-22.1.0/compiler-rt/buildroot-build/lib/fuzzer/libcxx_fuzzer_x86_64/build/include/c++/v1/__type_traits/decay.h:22:32:
error: expected type-specifier before '__decay'
A corresponding bug report sent upstream
https://github.com/llvm/llvm-project/issues/174203
was answered:
https://github.com/llvm/llvm-project/issues/174203#issuecomment-3711113919
"Our policy is rather clear: Only the latest GCC is supported."
and an update to supported compiler versions was committed:
https://github.com/llvm/llvm-project/commit/d1146b1ddd03aea3d67ce8f413607e8e8be67f4b
Updated TestClangCompilerRT to use a gcc 15-based toolchain.
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The artifacts URL has been moved to https and domain to .org to
improve security and avoid redirection issues. This ensures that
downloads of kernels and rootfs images during runtime tests are
encrypted and verified.
The change has been tested by running a runtime test, confirming the
correct download and renaming of artifacts from the new URL:
Downloading to .../tmpyotq8uor
Renaming from .../tmpyotq8uor to .../kernel-versatile-5.10.202
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Commit 3d2141bcee("support/testing/run-tests: specify multiprocessing
method") added a call to multiprocessing.set_start_method('fork') as a
workaround for python 3.14, which changed the default start method to
forkserver - Which is incompatible with the nose2 setup.
multiprocessing.set_start_method() is only supposed to be called a maximum
of 1 time per process and throws a RuntimeError if called more than that
(even with the same arguments):
>>> import multiprocessing
>>> multiprocessing.set_start_method('fork')
>>> multiprocessing.set_start_method('fork')
Traceback (most recent call last):
File "<python-input-2>", line 1, in <module>
multiprocessing.set_start_method('fork')
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/usr/lib/python3.13/multiprocessing/context.py", line 247, in set_start_method
raise RuntimeError('context has already been set')
Debian included a similar patch in python3-nose2 0.51.1-2 (currently in
testing/unstable) which adds its own call to set_start_method():
https://salsa.debian.org/python-team/packages/nose2/-/blob/debian/0.15.1-2/debian/patches/0004-plugins-mp-set-context-to-fork-for-Python-3.14-mp-AP.patch?ref_type=tags
Which comes from:
https://github.com/nose-devs/nose2/pull/644
As discussed in the upstream PR, this is not a correct fix is wrong and
breaks various use cases. An issue has been opened to get this fixed in the
Debian packaging at:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1129350
But until that is done, rework the patch to:
- Only override set_start_method() if needed to limit impact
- Monkey patch set_start_method() so additional calls are ignored
To unbreak run-test on affected Debian systems and add some documentation to
make it clear why this is done.
[Peter: use allow_none / force optional arguments as pointed out by Julien]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Removed patch that fixed building with libseccomp and old kernel headers
as systemd has fixed that upstream now.
Notable changes:
- Add -Dlibmount=enabled to host and target build. Disabling this
option disables building most of systemd's tools apart from
libsystemd.
- remove gcrypt and gnutls from dependencies and build options.
openssl is the only supported crypto library now.
- Disable nspawn build for host variant and add a Config.in option
for the target variant.
- systemd now requires kernel >= 5.4 while the recommended kernel
version moved to >=5.7. This make systemd depend on
BR2_TOOLCHAIN_HEADERS_AT_LEAST_5_4, removing various dependencies
on older kernel header versions from sub options.
- The sha256sum of LICENSES/README.md changed due to various
license clarification, removal of some vendored files as well as
changing some file paths. Licensing did not actually change.
For changelog, see:
https://github.com/systemd/systemd/blob/v258.3/NEWS
Tested with `./support/testing/run-tests tests.init.test_systemd`
All tests passed
One of the tests now needs a bigger rootfs size.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
[Peter: fix check-package warnings]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The CPIO filesystem generated by the test_firewalld test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.
The test_firewalld test initially started to fail following a systemd
update [1][3]:
[BRTEST# systemctl is-active firewalld
failed
But really started to crash at boot following a python 3.14 update
[2][4]:
Run /init as init process
/init: exec: line 15: /sbin/init: not found
Also, update TestFirewalldSysVInit to use ext2 instead of cpio.
[1] 926e0504d0
[2] a0a6abc8b1
Fixes:
[3] https://gitlab.com/buildroot.org/buildroot/-/jobs/12944797059
[4] https://gitlab.com/buildroot.org/buildroot/-/jobs/11856840940
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
[Peter: Fix flake8 warning, use http.server instead of relying on
connectivity]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The package imports itself in setup.py to get the package-name and
version number. Dping this during the buildroot build would require
buisling host-python with sqlite support, which we are currently not set
up for. It also seems wasteful for just extracting the version number
and package name, so instead we replace the import by using a hardcoded
package-name and the version number stored in buildroot.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Python 3.14 (not yet in Buildroot) introduced colors, enabled by
default, when the output is a terminal. This behavior can make the
pexpect pattern matching more difficult in some cases. See:
https://docs.python.org/3.14/using/cmdline.html#controlling-color
This commit globally disables the Python interpreter colors in the base
runtime Python test, by setting the NO_COLOR=1 environment variable.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Enhance the Xen python tests to exercise block devices: this boils down
to switching from ramdisks to disk partitions for the domains rootfs.
(Refer to the comments in the python script for block devices details.)
- Add support for PCI and Xen block to the Linux kernel configurations.
- Add a few commands to list the xvda block device for good measure.
- Generate two partitions with the rootfs in the disk images; we use the
same rootfs contents twice, once for each domain.
- Add a paravirtualized block device to the Xen dom1 configurations and
adjust both domains kernel command lines, to specify the rootfs
locations.
- Build host-qemu for Arm v7, to workaround an issue with 32b Arm and
old Qemu versions, which is what we have on CI currently.
- While at it, bump Linux kernel to 6.18.4 and U-Boot to 2026.01.
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Commit 3dd73c3 "package/openjdk: allow compiling without X11 support",
dropped the need for X11, so drop it from TestOpenJdk.
Signed-off-by: Thomas Devoogdt <thomas@devoogdt.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Rasdaemon was added in previous patch.
Add a test_rasdaemon that allows to test it.
Test is done on x86. It first checks the daemon's version, then mounts
debugfs and finally ensures that the init script allows to
start/restart/stop the daemon.
Signed-off-by: Bastien Curutchet <bastien.curutchet@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The run log of this ltp-testsuite test shows:
INFO: runltp script is deprecated, try kirk
https://github.com/linux-test-project/kirk
This commit updates this test to replace this deprecated runltp
shell script with the newer kirk Python script.
The logic of this runtime test remains the same: it runs a small number
of 'read' system call tests, and checks there is no failures and at
least one test succeed.
Cc: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Acked-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The ddresue runtime test is using the `dmsetup` command provided by the
dmraid package. This package is outdated and will be removed. This
command is also provided by the lvm2 package, which is still maintained.
This commit replaces the dmraid package by lvm2 in the test config.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add new runtime test for python-varlink.
As always when we need to start a separate server process inside a test
case this gets slightly fiddly.
We override the test_run() function to first start the varlink example server,
then call the packages cli interface to do a varlink call against the
server. The cli defaults to pretty printing the result, which makes it
more annoying to compare to the expected result in the test case, so we
un-prettyprint it with python's builtin json.tool module.
Signed-off-by: Marcus Hoffmann <buildroot@bubu1.eu>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The tests check if all supported hash algorithms are usable in
mkimage, for both host and target packages. Additionally, as a
necessary tool, it verifies the previous fix for FIT output from
dumpimage.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
[Julien: use builtin kernel for faster testing]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Some host commands need to call other host commands: For example,
"mkimage" from host-uboot-tools needs to run "dtc". This would fail or
call system commands without adding the host bin dir to PATH.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
[Julien: use python functions/constants to build path]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Add a simple test ensuring that
- libldns is correctly built and installed
- drill is correctly built and installed
- drill is able to execute on the target
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
The 5.4.x series is now EOL upstream, so drop the linux-headers
option and add legacy handling for it.
Bump kernel version in test_zfs.py.
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Since version 256, systemd will taint the system if /usr/bin and
/usr/sbin are not merged, known as merged-bin:
# systemctl --no-pager status
● buildroot
State: running
Units: 166 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Mon 2025-07-07 19:48:05 UTC; 19s ago
systemd: 257.7
Tainted: unmerged-bin
CGroup: /
├─init.scope
[...]
Although this is not yet an error, it will be in the future.
To be as ready as we can be when that happens, forcibly enable
merged-bin, like we did when we initially added merged-usr (except
this time we carry the select from systemd, even though it is not
yet strictly required, rather than from the init entry).
Extend the runtime test to catch any tainted flag.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Norbert Lange <nolange79@gmail.com>
Cc: Sen Hastings <sen@hastings.org>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The test tests.package.test_xen.TestXenArmv7 has become unstable after
commit 24ff258905 ("support/testing: test_xen: add networking"),
leading to a ~50% chance of failure on gitlab CI.
This seems to happen only with 32b (not with the 64b TestXenAarch64),
only with Qemu < 9.1, and when this happens the test will hang during a
step involving the simulated network and the dom1 (DHCP or ping).
The root cause is suspected to be a bug in the virtio-net hardware
emulation of the qemu version 7.2.15 included in the Buildroot docker
image.
As a workaround, remove all network interactions between dom1 and the
simulated network:
- Instead of the gateway, ping dom0 from dom1.
- Instead of using DHCP, configure the network with fixed IP addresses,
so that we know the IP address of the dom0.
A possible solution could have been to compile a host-qemu within the
test, by adding BR2_PACKAGE_HOST_QEMU=y and
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y in the test configuration.
This would have used the current Buildroot version (10.1.0) not affected
by this issue, but this would also have significantly increased the test
compilation time.
Since the DHCP is not really needed here to test the Xen networking
capability, this is why the network configuration simplification was
preferred here.
After those changes both Xen tests are stable on CI and locally.
Fixes: https://gitlab.com/buildroot.org/buildroot/-/jobs/11871454648
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a package for Snagboot, the open-source
vendor-neutral recover and reflashing tool for embedded targets. We
install the dependencies needed for snagrecover and snagflash, but we
don't bother installing the dependencies of the GUI-based snagfactory,
as that would require Kivy on the host, which is Qt based.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit adds a new host package for python-pyfatfs, which is
needed by Snagboot.
Homepage: https://pypi.org/project/pyfatfs/
A small test doing some minimal verification that pyfatfs works is
also added.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Julien Olivain