Compare commits

..
Author SHA1 Message Date
chainsx a2250c56a4 fix(security): correct openscap cis eval and fix test execution 2026-09-17 05:33:35 +08:00
woqidaideshi 9d8a140b68 Merge pull request 'fix(performance): adjust benchmark timeouts based on measured runtime' (#127) from chainsx/openruyi-autotest:fix-performance-timeouts-final into main
Reviewed-on: woqidaideshi/openruyi-autotest#127
2026-09-03 10:25:38 +00:00
chainsx 1d7340e96e fix(performance): adjust benchmark timeouts and fix MMTests exit parsing 2026-09-03 16:03:51 +08:00
woqidaideshi 9f0b3f5935 Merge pull request 'fix(security): set reasonable timeout for cve tests' (#130) from chainsx/openruyi-autotest:fix-security-cve-timeouts into main
Reviewed-on: woqidaideshi/openruyi-autotest#130
2026-09-01 07:44:30 +00:00
chainsx 1fb30e5398 fix(security): set reasonable timeout for cve tests 2026-08-27 00:13:55 +08:00
woqidaideshi fac8213444 Merge pull request 'feat: add cmake test for editorconfig-core-c fltk-devel spirv-cross-devel' (#128) from cmake_all_new into main
Reviewed-on: woqidaideshi/openruyi-autotest#128
2026-08-25 07:34:23 +00:00
woqidaideshi 74b4eb4ef2 feat: add three test cases for missing -static
This reverts commit f7ab0a24d4.
2026-08-25 03:28:27 -04:00
woqidaideshi 1c44966f8b Merge pull request 'fix(functional): fix cmake and blktests tests' (#126) from chainsx/openruyi-autotest:fix-functional-realtime-skip into main
Reviewed-on: woqidaideshi/openruyi-autotest#126
2026-08-25 02:09:35 +00:00
chainsx 166498ad14 fix(functional): fix cmake and blktests tests 2026-08-22 23:27:36 +08:00
redrose2100 653327ee0c Merge pull request 'chore(ci): 规范化格式' (#125) from dev into main
Reviewed-on: woqidaideshi/openruyi-autotest#125
2026-08-19 08:43:43 +00:00
honghua 0c4a2bce72 Merge branch 'main' of ssh://git.openruyi.cn:54865/woqidaideshi/openruyi-autotest 2026-08-19 16:23:44 +08:00
honghua 593a557da5 chore(fmf): normalize remaining CRLF files to LF per .gitattributes 2026-08-19 16:23:32 +08:00
43 changed files with 620 additions and 211 deletions
+1 -7
View File
@@ -26,10 +26,4 @@ execute:
how: tmt
report:
how: display
finish:
how: shell
script:
- rm -rf /tmp/ltp-posix
- rm -f /tmp/.beakerlib_ltp_posix_suite
how: display
+1
View File
@@ -120,6 +120,7 @@ ltpPosixSetup() {
fi
rlCleanupAppend "ltpPosixCleanup"
}
+63 -63
View File
@@ -1,63 +1,63 @@
# Functional test - Package tests
This directory contains functional test cases for each package, Uses ACL sub-directory structure standard.
## Test coverage overview
- **202 packages**, **561 test cases**, **1,692 functional points**
- All test scripts have been verified on openEuler RISC-V server
- See [Functional test coverage details](../../docs/coverage/functional-coverage.md)
## directory structure
```
tests/functional/
├── main.fmf # Shared configuration (All subdirectories inherit)
├── README.md # this file
└── pkgs/ # RPM Package functional tests
├── main.fmf # Shared configuration
├── acl/ # acl Package tests (Reference standard)
│ ├── main.fmf
│ ├── test.sh
│ └── test_acl_*/ # Sub-test directories
├── attr/ # attr Package tests
│ ├── main.fmf
│ ├── test.sh
│ └── test_attr_*/ # Sub-test directories
└──... # More package tests
```
## Test structure specification (Refer to ACL)
Each package test directory contains:
- `main.fmf` -- Package-level metadata configuration
- `test.sh` -- Package-level main test script
- `test_<pkg>_<feature>/` -- Sub-test directories split by functional point
- `main.fmf` -- Sub-test metadata
- `test.sh` -- Sub-test script
## Add new package test
1. Create directory `tests/functional/pkgs/<package_name>/`
2. create `main.fmf` (Refer to `pkgs/acl/main.fmf`)
3. create `test.sh` (Refer to template)
4. Create sub-test directories by functional point
## Run tests
```bash
tmt run plan --name /plans/functional test --name /tests/functional/<package_name>
```
- Metadata files are uniformly named `main.fmf`
## Test coverage progress
| soft | Status | Number of test cases | Notes |
|--------|------|-----------|------|
| acl | ✅ Complete | 27+ | First example package |
|... | To be added | - | - |
## Reference documentation
- Package test writing guide:`.trellis/tasks/06-09-acl/package-test-guide.md`
- tmt Official documentation:https://tmt.readthedocs.io/
# Functional test - Package tests
This directory contains functional test cases for each package, Uses ACL sub-directory structure standard.
## Test coverage overview
- **202 packages**, **561 test cases**, **1,692 functional points**
- All test scripts have been verified on openEuler RISC-V server
- See [Functional test coverage details](../../docs/coverage/functional-coverage.md)
## directory structure
```
tests/functional/
├── main.fmf # Shared configuration (All subdirectories inherit)
├── README.md # this file
└── pkgs/ # RPM Package functional tests
├── main.fmf # Shared configuration
├── acl/ # acl Package tests (Reference standard)
│ ├── main.fmf
│ ├── test.sh
│ └── test_acl_*/ # Sub-test directories
├── attr/ # attr Package tests
│ ├── main.fmf
│ ├── test.sh
│ └── test_attr_*/ # Sub-test directories
└──... # More package tests
```
## Test structure specification (Refer to ACL)
Each package test directory contains:
- `main.fmf` -- Package-level metadata configuration
- `test.sh` -- Package-level main test script
- `test_<pkg>_<feature>/` -- Sub-test directories split by functional point
- `main.fmf` -- Sub-test metadata
- `test.sh` -- Sub-test script
## Add new package test
1. Create directory `tests/functional/pkgs/<package_name>/`
2. create `main.fmf` (Refer to `pkgs/acl/main.fmf`)
3. create `test.sh` (Refer to template)
4. Create sub-test directories by functional point
## Run tests
```bash
tmt run plan --name /plans/functional test --name /tests/functional/<package_name>
```
- Metadata files are uniformly named `main.fmf`
## Test coverage progress
| soft | Status | Number of test cases | Notes |
|--------|------|-----------|------|
| acl | ✅ Complete | 27+ | First example package |
|... | To be added | - | - |
## Reference documentation
- Package test writing guide:`.trellis/tasks/06-09-acl/package-test-guide.md`
- tmt Official documentation:https://tmt.readthedocs.io/
@@ -0,0 +1,16 @@
summary: Functional test - editorconfig-core-c-devel cmake verify
test: ./test.sh
tag:
- functional
- cmake
- editorconfig-core-c
duration: 10m
tier: 2
path: /tests/functional/cmake/test_editorconfig-core-c-devel_cmake_verify
require:
- editorconfig-core-c-devel
- editorconfig-core-c-static
- cmake
- gcc
- beakerlib
- gcc-c++
@@ -0,0 +1,58 @@
#!/bin/bash
# Functional test: editorconfig-core-c-devel - cmake export integrity verification
# Verify that all files referenced by .cmake files provided by the package actually exist
#
# Verification principle:
# find_package(CONFIG) -> *Config.cmake -> *Targets.cmake
# Targets.cmake referencing non-existent .a/.so will cause cmake configuration to fail directly
. /usr/share/beakerlib/beakerlib.sh || exit 1
PKG="editorconfig-core-c-devel"
rlJournalStart
rlPhaseStartSetup "Environment Setup"
TmpDir=$(mktemp -d)
rlRun "cd $TmpDir" 0 "Enter temporary test directory"
CMAKE_COUNT=$(rpm -ql "$PKG" 2>/dev/null | grep -c '\.cmake$' || echo 0)
rlLogInfo "$PKG provides $CMAKE_COUNT .cmake file(s)"
rlPhaseEnd
rlPhaseStartTest "find_package(CONFIG) - Verify cmake export integrity"
if [ "$CMAKE_COUNT" -eq 0 ]; then
rlLogWarning "$PKG provides no .cmake files, skipping cmake verification"
else
# --- find_package(EditorConfig) ---
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(EditorConfig REQUIRED CONFIG)
message(STATUS "find_package(EditorConfig) succeeded")
EOF
rlLogInfo "Verifying find_package(EditorConfig CONFIG) ..."
rm -rf "$TmpDir/build_0"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_0" 0 \
"cmake configuration (find_package EditorConfig CONFIG)"; then
rlPass "find_package(EditorConfig CONFIG) verification passed"
else
rlFail "find_package(EditorConfig CONFIG) verification failed"
fi
fi
rlPhaseEnd
rlPhaseStartCleanup "Clean up test environment"
rlRun "cd /" 0 "Leave test directory"
if [ -n "$TmpDir" ] && [ -d "$TmpDir" ]; then
rlRun "rm -rf $TmpDir" 0 "Clean up temporary test directory"
fi
rlPhaseEnd
rlJournalPrintText
rlJournalEnd
@@ -0,0 +1,17 @@
summary: Functional test - fltk-devel cmake verify
test: ./test.sh
tag:
- functional
- cmake
- fltk
duration: 10m
tier: 2
path: /tests/functional/cmake/test_fltk-devel_cmake_verify
require:
- fltk-devel
- fltk-static
- fltk-fluid
- cmake
- gcc
- beakerlib
- gcc-c++
@@ -0,0 +1,58 @@
#!/bin/bash
# Functional test: fltk-devel - cmake export integrity verification
# Verify that all files referenced by .cmake files provided by the package actually exist
#
# Verification principle:
# find_package(CONFIG) -> *Config.cmake -> *Targets.cmake
# Targets.cmake referencing non-existent .a/.so will cause cmake configuration to fail directly
. /usr/share/beakerlib/beakerlib.sh || exit 1
PKG="fltk-devel"
rlJournalStart
rlPhaseStartSetup "Environment Setup"
TmpDir=$(mktemp -d)
rlRun "cd $TmpDir" 0 "Enter temporary test directory"
CMAKE_COUNT=$(rpm -ql "$PKG" 2>/dev/null | grep -c '\.cmake$' || echo 0)
rlLogInfo "$PKG provides $CMAKE_COUNT .cmake file(s)"
rlPhaseEnd
rlPhaseStartTest "find_package(CONFIG) - Verify cmake export integrity"
if [ "$CMAKE_COUNT" -eq 0 ]; then
rlLogWarning "$PKG provides no .cmake files, skipping cmake verification"
else
# --- find_package(FLTK) ---
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(FLTK REQUIRED CONFIG)
message(STATUS "find_package(FLTK) succeeded")
EOF
rlLogInfo "Verifying find_package(FLTK CONFIG) ..."
rm -rf "$TmpDir/build_0"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_0" 0 \
"cmake configuration (find_package FLTK CONFIG)"; then
rlPass "find_package(FLTK CONFIG) verification passed"
else
rlFail "find_package(FLTK CONFIG) verification failed"
fi
fi
rlPhaseEnd
rlPhaseStartCleanup "Clean up test environment"
rlRun "cd /" 0 "Leave test directory"
if [ -n "$TmpDir" ] && [ -d "$TmpDir" ]; then
rlRun "rm -rf $TmpDir" 0 "Clean up temporary test directory"
fi
rlPhaseEnd
rlJournalPrintText
rlJournalEnd
@@ -6,7 +6,7 @@ tag:
- qt6-qttools
duration: 10m
tier: 2
path: /tests/functional/pkgs/cmake_batch/generated/test_qt6-qttools-devel_cmake_verify
path: /tests/functional/cmake/test_qt6-qttools-devel_cmake_verify
require:
- qt6-qttools-devel
- qt6-doctools
@@ -0,0 +1,16 @@
summary: Functional test - spirv-cross-devel cmake verify
test: ./test.sh
tag:
- functional
- cmake
- spirv-cross
duration: 10m
tier: 2
path: /tests/functional/cmake/test_spirv-cross-devel_cmake_verify
require:
- spirv-cross-devel
- spirv-cross-static
- cmake
- gcc
- beakerlib
- gcc-c++
@@ -0,0 +1,247 @@
#!/bin/bash
# Functional test: spirv-cross-devel - cmake export integrity verification
# Verify that all files referenced by .cmake files provided by the package actually exist
#
# Verification principle:
# find_package(CONFIG) -> *Config.cmake -> *Targets.cmake
# Targets.cmake referencing non-existent .a/.so will cause cmake configuration to fail directly
. /usr/share/beakerlib/beakerlib.sh || exit 1
PKG="spirv-cross-devel"
rlJournalStart
rlPhaseStartSetup "Environment Setup"
TmpDir=$(mktemp -d)
rlRun "cd $TmpDir" 0 "Enter temporary test directory"
CMAKE_COUNT=$(rpm -ql "$PKG" 2>/dev/null | grep -c '\.cmake$' || echo 0)
rlLogInfo "$PKG provides $CMAKE_COUNT .cmake file(s)"
rlPhaseEnd
rlPhaseStartTest "find_package(CONFIG) - Verify cmake export integrity"
if [ "$CMAKE_COUNT" -eq 0 ]; then
rlLogWarning "$PKG provides no .cmake files, skipping cmake verification"
else
# --- find_package(spirv_cross_c) ---
# spirv_cross_c depends on: glsl, hlsl, msl, cpp, reflect
# glsl/hlsl/msl/cpp depend on: core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_glsl REQUIRED CONFIG)
find_package(spirv_cross_hlsl REQUIRED CONFIG)
find_package(spirv_cross_msl REQUIRED CONFIG)
find_package(spirv_cross_cpp REQUIRED CONFIG)
find_package(spirv_cross_reflect REQUIRED CONFIG)
find_package(spirv_cross_c REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_c) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_c CONFIG) ..."
rm -rf "$TmpDir/build_0"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_0" 0 \
"cmake configuration (find_package spirv_cross_c CONFIG)"; then
rlPass "find_package(spirv_cross_c CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_c CONFIG) verification failed"
fi
# --- find_package(spirv_cross_c_shared) ---
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_c_shared REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_c_shared) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_c_shared CONFIG) ..."
rm -rf "$TmpDir/build_1"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_1" 0 \
"cmake configuration (find_package spirv_cross_c_shared CONFIG)"; then
rlPass "find_package(spirv_cross_c_shared CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_c_shared CONFIG) verification failed"
fi
# --- find_package(spirv_cross_core) ---
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_core) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_core CONFIG) ..."
rm -rf "$TmpDir/build_2"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_2" 0 \
"cmake configuration (find_package spirv_cross_core CONFIG)"; then
rlPass "find_package(spirv_cross_core CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_core CONFIG) verification failed"
fi
# --- find_package(spirv_cross_cpp) ---
# spirv_cross_cpp depends on: glsl -> core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_glsl REQUIRED CONFIG)
find_package(spirv_cross_cpp REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_cpp) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_cpp CONFIG) ..."
rm -rf "$TmpDir/build_3"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_3" 0 \
"cmake configuration (find_package spirv_cross_cpp CONFIG)"; then
rlPass "find_package(spirv_cross_cpp CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_cpp CONFIG) verification failed"
fi
# --- find_package(spirv_cross_glsl) ---
# spirv_cross_glsl depends on: core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_glsl REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_glsl) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_glsl CONFIG) ..."
rm -rf "$TmpDir/build_4"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_4" 0 \
"cmake configuration (find_package spirv_cross_glsl CONFIG)"; then
rlPass "find_package(spirv_cross_glsl CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_glsl CONFIG) verification failed"
fi
# --- find_package(spirv_cross_hlsl) ---
# spirv_cross_hlsl depends on: glsl -> core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_glsl REQUIRED CONFIG)
find_package(spirv_cross_hlsl REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_hlsl) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_hlsl CONFIG) ..."
rm -rf "$TmpDir/build_5"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_5" 0 \
"cmake configuration (find_package spirv_cross_hlsl CONFIG)"; then
rlPass "find_package(spirv_cross_hlsl CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_hlsl CONFIG) verification failed"
fi
# --- find_package(spirv_cross_msl) ---
# spirv_cross_msl depends on: glsl -> core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_glsl REQUIRED CONFIG)
find_package(spirv_cross_msl REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_msl) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_msl CONFIG) ..."
rm -rf "$TmpDir/build_6"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_6" 0 \
"cmake configuration (find_package spirv_cross_msl CONFIG)"; then
rlPass "find_package(spirv_cross_msl CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_msl CONFIG) verification failed"
fi
# --- find_package(spirv_cross_reflect) ---
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_reflect REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_reflect) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_reflect CONFIG) ..."
rm -rf "$TmpDir/build_7"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_7" 0 \
"cmake configuration (find_package spirv_cross_reflect CONFIG)"; then
rlPass "find_package(spirv_cross_reflect CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_reflect CONFIG) verification failed"
fi
# --- find_package(spirv_cross_util) ---
# spirv_cross_util depends on: core
cat > "$TmpDir/CMakeLists.txt" << 'EOF'
cmake_minimum_required(VERSION 3.13.4)
project(cmake_verify
VERSION "0.1"
LANGUAGES C CXX)
find_package(spirv_cross_core REQUIRED CONFIG)
find_package(spirv_cross_util REQUIRED CONFIG)
message(STATUS "find_package(spirv_cross_util) succeeded")
EOF
rlLogInfo "Verifying find_package(spirv_cross_util CONFIG) ..."
rm -rf "$TmpDir/build_8"
if rlRun "cmake -S $TmpDir -B $TmpDir/build_8" 0 \
"cmake configuration (find_package spirv_cross_util CONFIG)"; then
rlPass "find_package(spirv_cross_util CONFIG) verification passed"
else
rlFail "find_package(spirv_cross_util CONFIG) verification failed"
fi
fi
rlPhaseEnd
rlPhaseStartCleanup "Clean up test environment"
rlRun "cd /" 0 "Leave test directory"
if [ -n "$TmpDir" ] && [ -d "$TmpDir" ]; then
rlRun "rm -rf $TmpDir" 0 "Clean up temporary test directory"
fi
rlPhaseEnd
rlJournalPrintText
rlJournalEnd
@@ -1,13 +1,13 @@
summary: Functional test - dejagnu - GCC compiletestwith
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 10m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_gcc_test
require:
- dejagnu
- gcc
- beakerlib
summary: Functional test - dejagnu - GCC compiletestwith
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 10m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_gcc_test
require:
- dejagnu
- gcc
- beakerlib
@@ -1,13 +1,13 @@
summary: Functional test - dejagnu - G++ compiletestwith
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 10m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_gxx_test
require:
- dejagnu
- gcc-c++
- beakerlib
summary: Functional test - dejagnu - G++ compiletestwith
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 10m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_gxx_test
require:
- dejagnu
- gcc-c++
- beakerlib
@@ -1,12 +1,12 @@
summary: Functional test - dejagnu - runtest availablecheck
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 5m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_runtest_check
require:
- dejagnu
- beakerlib
summary: Functional test - dejagnu - runtest availablecheck
test: ./test.sh
tag:
- functional
- compiler
- dejagnu
duration: 5m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_runtest_check
require:
- dejagnu
- beakerlib
@@ -1,7 +1,7 @@
summary: Functional test - dejagnu - warning (-Wall/-Wextra/-Werror/-pedantic)
test: ./test.sh
tag: [functional, compiler, dejagnu]
duration: 5m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_warnings
require: [dejagnu, gcc, clang, beakerlib]
summary: Functional test - dejagnu - warning (-Wall/-Wextra/-Werror/-pedantic)
test: ./test.sh
tag: [functional, compiler, dejagnu]
duration: 5m
tier: 2
path: /tests/functional/compiler/dejagnu/test_dejagnu_warnings
require: [dejagnu, gcc, clang, beakerlib]
+13 -13
View File
@@ -1,13 +1,13 @@
summary: Functional test - Jotai programcompiletest
tag:
- functional
- compiler
- jotai
duration: 15m
tier: 2
path: /tests/functional/compiler/jotai
require:
- beakerlib
- gcc
- clang
- git
summary: Functional test - Jotai programcompiletest
tag:
- functional
- compiler
- jotai
duration: 15m
tier: 2
path: /tests/functional/compiler/jotai
require:
- beakerlib
- gcc
- clang
- git
@@ -1,14 +1,14 @@
summary: Functional test - jotai - Environmentandrepolibrary
test: ./test.sh
tag:
- functional
- compiler
- jotai
duration: 5m
tier: 2
path: /tests/functional/compiler/jotai/test_jotai_setup
require:
- gcc
- clang
- git
- beakerlib
summary: Functional test - jotai - Environmentandrepolibrary
test: ./test.sh
tag:
- functional
- compiler
- jotai
duration: 5m
tier: 2
path: /tests/functional/compiler/jotai/test_jotai_setup
require:
- gcc
- clang
- git
- beakerlib
@@ -1,14 +1,14 @@
summary: Functional test - yarpgen - source codebuildand
test: ./test.sh
tag:
- functional
- compiler
- yarpgen
duration: 10m
tier: 2
path: /tests/functional/compiler/yarpgen/test_yarpgen_build
require:
- gcc-c++
- cmake
- git
- beakerlib
summary: Functional test - yarpgen - source codebuildand
test: ./test.sh
tag:
- functional
- compiler
- yarpgen
duration: 10m
tier: 2
path: /tests/functional/compiler/yarpgen/test_yarpgen_build
require:
- gcc-c++
- cmake
- git
- beakerlib
@@ -1,7 +1,7 @@
summary: Functional test - yarpgen - multifileseparate compilation + C/C++ link
test: ./test.sh
tag: [functional, compiler, yarpgen]
duration: 5m
tier: 2
path: /tests/functional/compiler/yarpgen/test_yarpgen_multifile
require: [gcc, gcc-c++, clang, beakerlib]
summary: Functional test - yarpgen - multifileseparate compilation + C/C++ link
test: ./test.sh
tag: [functional, compiler, yarpgen]
duration: 5m
tier: 2
path: /tests/functional/compiler/yarpgen/test_yarpgen_multifile
require: [gcc, gcc-c++, clang, beakerlib]
+1 -1
View File
@@ -96,7 +96,7 @@ EOF" 2>/dev/null || true
timeout --signal=KILL --kill-after=10 600 \
bash./check "$test_name" 2>&1 | tee "$out"
bash ./check "$test_name" 2>&1 | tee "$out"
local rc=${PIPESTATUS[0]}
@@ -4,7 +4,7 @@ tag:
- functional
- kernel
- blktests
duration: 5m
duration: 15m
tier: 2
path: /tests/functional/kernel/blktests/scsi/test_blktests_scsi_007
require:
+11 -11
View File
@@ -1,11 +1,11 @@
summary: Functional test - kernel - LTP test
tag:
- functional
- kernel
- realtime
- ltp
duration: 30m
tier: 1
path: /tests/functional/kernel/realtime
require:
- beakerlib
summary: Functional test - kernel - LTP test
tag:
- functional
- kernel
- realtime
- ltp
duration: 30m
tier: 1
path: /tests/functional/kernel/realtime
require:
- beakerlib
+1 -1
View File
@@ -116,7 +116,7 @@ _mmtestsRunCase() {
local exit_lines
exit_lines=$(grep "test exit:" "$out" 2>/dev/null)
exit_lines=$(grep '^test exit :: ' "$out" 2>/dev/null)
if [ -z "$exit_lines" ]; then
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 90m
tier: 1
path: /tests/performance/mmtests/test_mmtests_io_blogbench
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 120m
tier: 1
path: /tests/performance/mmtests/test_mmtests_io_trunc
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 60m
tier: 1
path: /tests/performance/mmtests/test_mmtests_memdb_redis_benchmark_small
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 55m
tier: 1
path: /tests/performance/mmtests/test_mmtests_pagereclaim_stutterp
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 45m
tier: 1
path: /tests/performance/mmtests/test_mmtests_workload_ebizzy
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 60m
tier: 1
path: /tests/performance/mmtests/test_mmtests_workload_usemem_stress_numa_compact
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 120m
tier: 1
path: /tests/performance/mmtests/test_mmtests_workload_will_it_scale_io_processes
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 120m
tier: 1
path: /tests/performance/mmtests/test_mmtests_workload_will_it_scale_io_threads
require:
@@ -4,7 +4,7 @@ tag:
- performance
- kernel
- mmtests
duration: 30m
duration: 55m
tier: 1
path: /tests/performance/mmtests/test_mmtests_workload_will_it_scale_sys_processes
require:
@@ -3,7 +3,7 @@ test: ./test.sh
tag:
- performance
- unixbench
duration: 10m
duration: 35m
tier: 2
path: /tests/performance/unixbench/test_unixbench_half_cpu
require:
@@ -3,7 +3,7 @@ test: ./test.sh
tag:
- performance
- unixbench
duration: 10m
duration: 35m
tier: 2
path: /tests/performance/unixbench/test_unixbench_multi_thread
require:
@@ -3,7 +3,7 @@ test: ./test.sh
tag:
- performance
- unixbench
duration: 10m
duration: 20m
tier: 2
path: /tests/performance/unixbench/test_unixbench_quick_mode
require:
@@ -3,7 +3,7 @@ test: ./test.sh
tag:
- performance
- unixbench
duration: 10m
duration: 35m
tier: 2
path: /tests/performance/unixbench/test_unixbench_single_thread
require:
@@ -4,7 +4,7 @@ tag:
- security
- cve
- ltp
duration: 3m
duration: 15m
tier: 2
path: /tests/security/cve/test_cve-2016-8655
require:
@@ -4,7 +4,7 @@ tag:
- security
- cve
- ltp
duration: 3m
duration: 15m
tier: 2
path: /tests/security/cve/test_cve-2017-15649
require:
@@ -4,7 +4,7 @@ tag:
- security
- cve
- ltp
duration: 3m
duration: 15m
tier: 2
path: /tests/security/cve/test_cve-2018-18559
require:
@@ -4,7 +4,7 @@ tag:
- security
- cve
- ltp
duration: 3m
duration: 15m
tier: 2
path: /tests/security/cve/test_cve-2020-14416
require:
@@ -1,10 +1,11 @@
summary: Security test - OpenSCAP CIS - andreport
framework: beakerlib
test: ./test.sh
tag:
- security
- openscap
- cis
duration: 10m
duration: 30m
tier: 2
path: /tests/security/openscap/cis/test_openscap_cis_eval_report
require:
@@ -16,21 +16,21 @@ rlJournalStart
if [ ! -f "$CIS_DS" ]; then
rlFail "Data stream not found"
else
local out=/tmp/cis_eval_$$
local xml=/tmp/cis_result_$$.xml
local html=/tmp/cis_report_$$.html
out=/tmp/cis_eval_$$
xml=/tmp/cis_result_$$.xml
html=/tmp/cis_report_$$.html
oscap xccdf eval --profile "$CIS_PROFILE" --results-arf "$xml" --report "$html" "$CIS_DS" 2>&1 | tee $out
local rc=${PIPESTATUS[0]}
if [ $rc -ne 0 ]; then
rc=${PIPESTATUS[0]}
if [ $rc -ne 0 ] && [ $rc -ne 2 ]; then
rlFail "oscap eval failed (exit=$rc)"
elif [ ! -f "$xml" ]; then
rlFail "ARF result file not generated"
elif [ ! -f "$html" ]; then
rlFail "HTML report not generated"
else
local pass=$(grep -c '<result>pass</result>' "$xml" 2>/dev/null || echo 0)
local fail=$(grep -c '<result>fail</result>' "$xml" 2>/dev/null || echo 0)
local na=$(grep -c '<result>notapplicable</result>' "$xml" 2>/dev/null || echo 0)
pass=$(grep -c '<result>pass</result>' "$xml" 2>/dev/null || echo 0)
fail=$(grep -c '<result>fail</result>' "$xml" 2>/dev/null || echo 0)
na=$(grep -c '<result>notapplicable</result>' "$xml" 2>/dev/null || echo 0)
rlPass "CIS eval complete: pass=$pass fail=$fail notapplicable=$na"
fi
rm -f $out $xml $html
@@ -1,10 +1,11 @@
summary: Security test - OpenSCAP CIS - scriptGenerate
framework: beakerlib
test: ./test.sh
tag:
- security
- openscap
- cis
duration: 10m
duration: 30m
tier: 2
path: /tests/security/openscap/cis/test_openscap_cis_fix
require:
@@ -16,15 +16,15 @@ rlJournalStart
if [ ! -f "$CIS_DS" ]; then
rlFail "Data stream not found"
else
local out=/tmp/cis_fix_$$
local xml=/tmp/cis_fix_xml_$$.xml
local fix=/tmp/cis_fix_$$.sh
out=/tmp/cis_fix_$$
xml=/tmp/cis_fix_xml_$$.xml
fix=/tmp/cis_fix_$$.sh
oscap xccdf eval --profile "$CIS_PROFILE" --results "$xml" "$CIS_DS" 2>&1 | tee $out
local rc=${PIPESTATUS[0]}
if [ $rc -ne 0 ] || [ ! -f "$xml" ]; then
rc=${PIPESTATUS[0]}
if { [ $rc -ne 0 ] && [ $rc -ne 2 ]; } || [ ! -f "$xml" ]; then
rlFail "oscap eval failed (exit=$rc)"
else
local rid=$(grep -oP 'id="\K[^"]+' "$xml" | head -1)
rid=$(grep -oP '<TestResult[^>]*\bid="\K[^"]+' "$xml" | head -1)
if [ -z "$rid" ]; then
rlFail "Cannot extract TestResult id"
else