Commit Graph

2598 Commits

Author SHA1 Message Date
Yann Collet
e51a6be463
Merge pull request #1002 from t-mat/dev
Fix package related GH-Actions errors
2021-06-15 22:30:03 -07:00
Takayuki Matsuoka
a9b93547a0 Fix package related errors
Add explicit "apt-get update" for all test cases which uses "apt-get".
- It may help to stabilize fetching from apt archive.

As for "CC=gcc-11"
- Add "g++-11" package.
- "g++-11" has been removed from the default virtual environment.
  https://github.com/actions/virtual-environments/issues/3467
2021-06-16 12:44:48 +09:00
Yann Collet
355f609529
Merge pull request #996 from t-mat/gha-3
Improve CI script
2021-06-02 18:31:56 -07:00
Takayuki Matsuoka
f0ea466c36 Fix error in ci.yml
Fix the following error
```
lz4 CI : .github#L1
Error when evaluating 'runs-on' for job 'lz4-check-tag'. (Line: 593, Col: 14): Unexpected value ''
```
2021-06-02 04:59:23 +09:00
Takayuki Matsuoka
5b01b5805a Update document for .github/workflows 2021-06-01 20:03:48 +09:00
Takayuki MATSUOKA
02a41178b7 Merge branch 'lz4:dev' into gha-3 2021-06-01 12:00:00 +09:00
Takayuki Matsuoka
8d4005a1ca Add label for checkTag test 2021-05-31 13:54:58 +09:00
Takayuki Matsuoka
44b8b10f84 Add make -C tests checkTag to ci.yml 2021-05-31 13:28:49 +09:00
Yann Collet
145a92bcc5
Merge pull request #998 from t-mat/travis-remove-tests-2
Remove tests from .travis.yml (part 2/2)
2021-05-30 21:06:40 -07:00
Takayuki Matsuoka
ba752cfec9 Disable "(Trusty) gcc-4.4 compilation"
[lz4-c-compilers] contains the following commands

```
make all
CFLAGS=-fPIC LDFLAGS='-pie -fPIE -D_FORTIFY_SOURCE=2' make -C programs
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:25:21 +09:00
Takayuki Matsuoka
60b1f8dfbd Disable "(Trusty) scan-build + cppcheck"
[lz4-scan-build] contains the following command

```
make staticAnalyze
```

[lz4-cppcheck]contains the following command

```
make cppcheck
```

We ignore the exit code of cppcheck.  We refer its output as a guideline.

[lz4-scan-build] 025c3ce8f9/.github/workflows/ci.yml (L285-L302)
[lz4-cppcheck] 025c3ce8f9/.github/workflows/ci.yml (L268-L282)
2021-05-31 11:24:11 +09:00
Takayuki Matsuoka
0e3819cadb Disable "(Trusty) clang-3.8 compilation"
[lz4-c-compilers] contains the following command for clang-3.8.

```
make -C tests test-lz4
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:23:10 +09:00
Takayuki Matsuoka
42c3351af6 Disable "(Xenial) gcc-5 compilation"
[lz4-c-compilers] contains the following commands for gcc-5

```
make -C tests test-lz4
make -C tests test-lz4c32
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:22:25 +09:00
Takayuki Matsuoka
f2a5599c8d Disable "(Trusty) gcc-6 standard C compilation"
Disable "(Trusty) gcc-6 standard C compilation"

[lz4-c-compilers] contains the following commands for gcc-6

```
make c_standards
make -C tests test-lz4
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:21:41 +09:00
Takayuki Matsuoka
3887bc6d81 Disable "(Trusty) i386 frame + fuzzer test"
[lz4-frame] contains the following command

```
make -C tests test-frametest32
```
`
[lz4-fuzzer] contains the following command

```
make -C tests test-fuzzer32
```

[lz4-frame] 025c3ce8f9/.github/workflows/ci.yml (L224-L237)
[lz4-fuzzer] 025c3ce8f9/.github/workflows/ci.yml (L192-L208)
2021-05-31 11:20:45 +09:00
Takayuki Matsuoka
febc3ec9b2 Disable "(Trusty) i386 benchmark + version test"
[lz4-c-compilers] contains the following command

```
make -C tests test-lz4c32
```

[lz4-benchmark] contains the following command

```
make -C tests test-fullbench32
```

[lz4-versions] contains the following command

```
make -C tests versionsTest
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
[lz4-benchmark] 025c3ce8f9/.github/workflows/ci.yml (L167-L189)
[lz4-versions] 025c3ce8f9/.github/workflows/ci.yml (L211-L221)
2021-05-31 11:19:54 +09:00
Takayuki Matsuoka
40f4acb6ce Disable "(Trusty) c-to-c++ test"
[lz4-c-compilers] contains the following command

```
make ctocpptest
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:18:17 +09:00
Takayuki Matsuoka
81ee36cf40 Disable "(Trusty) valgrind test"
[lz4-c-compilers] contains the following command

```
make c_standards
```

[lz4-valgrind] contains the following command

```
make -C tests test-lz4 test-mem
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
[lz4-valgrind] 025c3ce8f9/.github/workflows/ci.yml (L305-L316)
2021-05-31 11:17:35 +09:00
Takayuki Matsuoka
e88b2b14d0 Disable "(Trusty) USan test"
[lz4-ubsan-x64] contains the following command

```
make usan MOREFLAGS=-Wcomma -Werror
```

[lz4-ubsan-x64] 025c3ce8f9/.github/workflows/ci.yml (L322-L336)
2021-05-31 11:04:58 +09:00
Takayuki Matsuoka
abbd46b0d0 Disable "x32 compatibility test"
[lz4-c-compilers] contains the following command

```
make -C tests test MOREFLAGS=-mx32
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
2021-05-31 11:03:04 +09:00
Takayuki Matsuoka
33580eb4a6 Disable "(Precise) g++ and clang CMake test"
[lz4-compilers] contains the following commands.
Note that `make all` contains `make examples`

```
make cxxtest
make clean
make examples
make clean cmake
make clean clangtest
```

[lz4-build-make-travis-install] contains the following commands.

```
make travis-install
```

[lz4-c-compilers] 025c3ce8f9/.github/workflows/ci.yml (L17-L154)
[lz4-build-make-travis-install] 025c3ce8f9/.github/workflows/ci.yml (L498-L510)
2021-05-31 11:00:53 +09:00
Takayuki Matsuoka
9e72f2c01f Disable "Custom LZ4_DISTANCE_MAX ; ..."
[lz4-custom-distance] contains the following command

```
MOREFLAGS='-DLZ4_DISTANCE_MAX=8000' make V=1 check
make V=1 -C programs lz4-wlib
make V=1 -C tests fullbench-wmalloc  # test LZ4_USER_MEMORY_FUNCTIONS
CC="c++ -Wno-deprecated" make V=1 -C tests fullbench-wmalloc  # stricter function signature check
```

Actually, there're 3+1 tests

(1) Test for Custom LZ4_DISTANCE_MAX.  6654c2c, #753, #755
(2) Test for dynamic link library.  3dd34df, #888
(3-1) Test for LZ4_USER_MEMORY_FUNCTIONS.  52646e8, #937, #946
(3-2) Same as (3-1), but compile it with c++.

After we'll stabilize CI tests, this test should be split into 3 individual parts for clarity.

[lz4-custom-distance] 025c3ce8f9/.github/workflows/ci.yml (L241-L255)
2021-05-31 10:57:16 +09:00
Takayuki Matsuoka
485907563f Disable "ASAN tests with fuzzer and frametest"
[lz4-asan-x64] contains the following command

```
sudo sysctl -w vm.mmap_min_addr=4096
CC=clang MOREFLAGS=-fsanitize=address make -C tests test-frametest test-fuzzer
```

[lz4-asan-x64] 025c3ce8f9/.github/workflows/ci.yml (L361-L376)
2021-05-31 10:56:38 +09:00
Takayuki Matsuoka
8704584f6c Disable "(Precise) frame and fuzzer test"
[lz4-frame] in ci.yml contains the following command

```
make -C tests test-frametest
```

[lz4-fuzzer] in ci.yml contains the following command

```
make -C tests test-fuzzer
```
2021-05-31 10:50:40 +09:00
Takayuki Matsuoka
a44ef30cd5 Disable "(Precise) benchmark test"
Disable "(Precise) benchmark test"

lz4-benchmark[1] in ci.yml contains the following command

```
make -C tests test-lz4 test-lz4c test-fullbench
```

[1] 025c3ce8f9/.github/workflows/ci.yml (L167-L189)
2021-05-31 10:47:59 +09:00
Takayuki Matsuoka
4392dc3261 Disable "(macOS) General Test"
lz4-platform-macos-latest[1] in ci.yml contains the following test command

```
make   # test library build
make test MOREFLAGS='-Werror -Wconversion -Wno-sign-conversion' | tee
```

[1] 025c3ce8f9/.github/workflows/ci.yml (L450)
2021-05-31 10:45:07 +09:00
Takayuki Matsuoka
025c3ce8f9 Enable c_standards as per #994 2021-05-31 10:04:50 +09:00
Takayuki Matsuoka
cccdc8fe21 Enable cxxtest as per #993 2021-05-31 10:03:28 +09:00
Yann Collet
bc8230aab7
Merge pull request #993 from t-mat/fix-enum-unsigned-cmp
Fix g++-4.4 warning
2021-05-30 10:25:21 -07:00
Yann Collet
1519312d24
Merge pull request #995 from t-mat/travis-remove-qemu
Remove QEMU test from .travis.yml
2021-05-30 10:24:52 -07:00
Yann Collet
31fb0bb4b8
Merge pull request #994 from t-mat/makefile-split-cstds
Split c_standards into multiple Makefile targets
2021-05-30 10:24:14 -07:00
Takayuki Matsuoka
243f6b5233 Improve CI script
## Added compilers

- gcc: 4.[4678], 5, 6, 11
- clang: 3.[56789], 4, 5, 12

## Known issue

- make -C tests test-lz4c32
  - Fails with all versions of clang.  See #991 for details.
- CFLAGS='-O3 -mx32' make -C tests test-lz4c32
  - Fails with all versions of clang
  - Fails with gcc-11
- `make cxxtest`
  - Disabled for now.  Will be enabled after #993.
- `make c_standards_c90`, `make c_standards_c11`
  - Disabled for now.  Will be enabled after #994.

## Difference with `.travis.yml`

The following tests are not included yet.

- name: Compile OSS-Fuzz targets
- name: tag build

The following tests won't be included due to lmitation of the CI environment.

- name: aarch64 real-hw tests
- name: PPC64LE real-hw tests
- name: IBM s390x real-hw tests

Except above, all other features in `.travis.yml` has been included in this change set.

The following post describes details.
2021-05-30 23:41:43 +09:00
Takayuki Matsuoka
bf140842d9 Remove QEMU test from .travis.yml
We have same test in GitHub Actions.
2021-05-30 18:39:33 +09:00
Yann Collet
e8a8fd6f0b
Merge pull request #992 from t-mat/fix-clang-warning-1
Fix -Wshorten-64-to-32 warning
2021-05-30 01:28:36 -07:00
Takayuki Matsuoka
6443b85af4 Split c_standards into multiple Makefile targets
To support older compiler which doesn't have explicit dialect option
for C90 (gcc-4.4),  this change set split "c_standards" into multiple
part.

Original "make c_standards" still works as intended.  But this change
gives extra freedom of choice for external program  For example, CI
can choose test for standards which is supported by specimen compiler.

With this separation, we can also introduce C17 smoothly.
2021-05-30 17:10:19 +09:00
Takayuki Matsuoka
3c3624c3a6 Fix g++-4.4 warning
g++-4.4 creates the following warning for this line.

```
g++-4.4 -Wno-deprecated -O3 -Wall -Wextra -Wundef -Wshadow -Wcast-align -Werror -I../lib -I../programs -DXXH_NAMESPACE=LZ4_  lz4frame.o lz4.o lz4hc.o xxhash.o checkFrame.c -o checkFrame
checkFrame.c: In function ‘int frameCheck(cRess_t, FILE*, unsigned int, size_t)’:
checkFrame.c:156: error: comparison between signed and unsigned integer expressions
```
2021-05-30 16:53:16 +09:00
Takayuki Matsuoka
28cb94f53a Fix -Wshorten-64-to-32 warning
Fix -Wshorten-64-to-32 warning

The following CI test (macOS) reports "-Wshorten-64-to-32" warning

make V=1 clean test MOREFLAGS='-Werror -Wconversion -Wno-sign-conversion'

```
blockStreaming_lineByLine.c:68:54: error: implicit conversion loses integer precision: 'const size_t' (aka 'const unsigned long') to 'int' [-Werror,-Wshorten-64-to-32]
                lz4Stream, inpPtr, cmpBuf, inpBytes, cmpBufBytes, 1);
                                                     ^~~~~~~~~~~
```
2021-05-30 16:49:24 +09:00
Yann Collet
634fa2047d
Merge pull request #989 from lz4/fix_nullub_lz4
Fix NULL ptr arithmetic in lz4.c
2021-05-28 09:42:01 -07:00
Yann Collet
dfc431fb3d fix NULL ptr arithmetic at lz4:2299 2021-05-28 01:10:41 -07:00
Yann Collet
539c783c98 fix NULL ptr arithmetic in lz4:1680
only do arithmetic if offset > 0
2021-05-28 01:08:18 -07:00
Yann Collet
e73c318a79 Merge branch 'dev' into fix_nullub_lz4 2021-05-28 00:56:46 -07:00
Yann Collet
59273b7127 fix UB lz4:988 and lz4:1178
ensure `dictBase` is only used
when there is an actual dictionary content.
2021-05-28 00:56:26 -07:00
Yann Collet
cacdcf1f51
Merge pull request #987 from t-mat/tmp-disable-meson
Disable Meson + clang build at travis-ci
2021-05-28 00:12:59 -07:00
Takayuki Matsuoka
1f6d2efda1 Disable Meson + clang build at travis-ci 2021-05-28 16:02:07 +09:00
Yann Collet
6ac3b7ba94
Merge pull request #985 from t-mat/add-github-actions
Add GitHub Actions script ci.yml
2021-05-27 23:57:55 -07:00
Yann Collet
dd7a6fbdc4
Merge pull request #986 from lz4/fixub907
fix UB of lz4frame:907
2021-05-27 23:57:19 -07:00
Takayuki Matsuoka
54b695800d Temporary ignore make usan
We must enable this test when all make usan errors will be resolved properly.
2021-05-28 15:28:59 +09:00
Yann Collet
c2c0c47d5f fix NULL ptr arithmetic of lz4:1572
was blindly adding an offset (0) to `dictionary` which could be `NULL`.
2021-05-27 23:20:28 -07:00
Yann Collet
8e46846287 fix UB of lz4frame:907
now line 912
by ensuring pointer arithmetic is only performed
if there is a reason for an internal buffer to be used.
2021-05-27 22:59:22 -07:00
Yann Collet
5b86e4e5ed
Merge pull request #982 from t-mat/fix-make-usan
Fix 'make usan'
2021-05-27 10:24:41 -07:00