Yann Collet
4086b2871b
largeNbDicts compatible with multiple source files
...
splitting is disabled by default, but can be re-enabled using usual command -B#
update commands to look like zstd ones
2018-08-30 14:38:49 -07:00
Yann Collet
a5a77965d3
make all includes contrib/largeNbDicts
2018-08-29 16:17:22 -07:00
Yann Collet
d89fa814c1
added a README
...
for documentation
2018-08-28 18:19:19 -07:00
Yann Collet
6444c50035
increases randomness of ddict ptrs
2018-08-28 18:13:46 -07:00
Yann Collet
6c398df241
level, block size and nb dicts can be set on command line
2018-08-28 18:05:31 -07:00
Yann Collet
0c66a44d1b
first working test program
...
measures :
- compression ratio with / without dictionary
- create one dictionary per block
- memory budget for dictionaries
- decompression speed, using one different dictionary per block
current limitations :
- only one file
- 4K blocks only
- automatic dictionary built with 4K size
dictionary can be selected on command line, with -D
2018-08-28 15:47:07 -07:00
Yann Collet
274b60e6e6
largeNbDicts can compress and compare dict vs noDict
2018-08-27 17:08:44 -07:00
Yann Collet
6782725155
first sketch for largeNbDicts test program
2018-08-26 19:29:12 -07:00
Yann Collet
36d6165a2d
Makefile: added variable SCANBUILD
...
so that a different version of scan-build can be selected
2018-08-16 16:44:13 -07:00
Yann Collet
42a02ab745
fixed minor warnings issued by scan-build
2018-08-15 14:36:02 -07:00
Jennifer Liu
0acb0abd1e
Add non-optimize FASTCOVER ( #1260 )
...
* Add non-optimize FASTCOVER
* Minor fix
* Pass param as value instead of pointer
2018-08-01 11:06:16 -07:00
Jennifer Liu
4e29bc2469
Use CDict instead of CCtx in analyzeEntropy
2018-07-31 10:36:45 -07:00
Jennifer Liu
31229e527b
Increment frequency for every dmer occurence within same sample instead of at most once per sample
2018-07-30 12:54:22 -07:00
Jennifer Liu
51b109c1b5
Delete old benchmarking result
2018-07-27 17:31:33 -07:00
Jennifer Liu
53ef22a4bc
Undo deleting clean in make
2018-07-27 16:56:50 -07:00
Jennifer Liu
96d84ee235
Revert test.sh
2018-07-27 16:54:05 -07:00
Jennifer Liu
61262f6c0d
Save segmentFreqs in ctx instead of malloc and memset in SelectSegment
2018-07-27 16:51:38 -07:00
Jennifer Liu
49b398e93f
Use same param after optimizing cover and fastCover and record k and d for benchmarking
2018-07-27 13:39:19 -07:00
Jennifer Liu
759c543312
Rerun cover and fastCover with optimized values
2018-07-26 19:03:01 -07:00
Jennifer Liu
3d7941ce41
Benchmark different f values
2018-07-26 16:24:13 -07:00
Jennifer Liu
3b163e0b5b
Add array to keep track of frequency within active segment, fix malloc bug, update benchmarking result
2018-07-26 13:53:13 -07:00
Jennifer Liu
2333ecb173
Allow d=6
2018-07-25 18:10:09 -07:00
Jennifer Liu
1e85f314d8
Benchmark fast cover optimize vs k=200
2018-07-25 17:53:38 -07:00
Jennifer Liu
d1fc507ef9
Initial benchmarking result for fastCover
2018-07-25 17:05:54 -07:00
Jennifer Liu
f5407e398a
Make hash value const
2018-07-25 16:54:08 -07:00
Jennifer Liu
7f3f70f766
Add Fast Cover Dictionary Builder
2018-07-25 16:34:07 -07:00
Nick Terrell
77068a8447
Merge pull request #1246 from jennifermliu/benchmark
...
Benchmark dictionary builders
2018-07-20 18:09:31 -07:00
Jennifer Liu
b6c5d4982c
Minor fix
2018-07-20 17:41:22 -07:00
Jennifer Liu
71e767ac09
Refactoring and benchmark without dictionary
2018-07-20 17:03:47 -07:00
Jennifer Liu
470c8d42f4
Benchmark dictionary builders
2018-07-20 11:32:39 -07:00
Nick Terrell
4d1ad5cdb2
Merge pull request #1238 from jennifermliu/random
...
Add random dictionary builder
2018-07-19 13:52:15 -07:00
Jennifer Liu
0c5eaef248
Update Makefile
2018-07-19 13:44:27 -07:00
Jennifer Liu
5bb46a898e
Rename cleanup
2018-07-18 12:15:49 -07:00
Jennifer Liu
52e7cf0e40
Add cleanup to trainfromFiles and move RANDOM_segment_t declaration
2018-07-18 10:40:13 -07:00
Jennifer Liu
ce09fb723d
Update freeSampleInfo
2018-07-17 16:13:40 -07:00
Jennifer Liu
896ff0644a
Fix deallocation problem and add documentation
2018-07-17 16:01:44 -07:00
Jennifer Liu
e6fe405838
Make test PHONY target
2018-07-17 12:42:53 -07:00
Jennifer Liu
49acfaeaec
Move file loading functions to new file for access by benchmarking tool
2018-07-17 12:35:09 -07:00
Jennifer Liu
4d32339b75
Remove CLevel cli option which was accidentally added back in the last commit
2018-07-16 18:59:18 -07:00
Jennifer Liu
1f7fa5cdd6
Fix spacing and Edit Makefile (now run with make instead of make run)
2018-07-16 16:31:59 -07:00
Jennifer Liu
b5806d33db
Refactor RANDOM
2018-07-16 16:03:04 -07:00
Codecat
044cd81ce6
Fix wrong conditions
2018-07-14 12:34:03 +02:00
Codecat
1a61bdb9c0
Update zstd.lua
2018-07-14 12:27:42 +02:00
Jennifer Liu
0e5fbc10fa
Update README
2018-07-13 17:41:09 -07:00
Jennifer Liu
31731df4da
Remove clevel and update documentation
2018-07-13 17:38:53 -07:00
Jennifer Liu
a23a3b95f9
Add random dictionary builder
2018-07-13 16:05:14 -07:00
Codecat
23d77c531e
Added premake4/GENie script to contrib folder
2018-07-11 18:02:18 +02:00
Jon Turney
87579d51ee
meson: fix build
2018-07-02 15:06:46 +01:00
Ryan Schmidt
bf30b9caf4
Add CXXFLAGS to ALL_LDFLAGS ( #1178 )
...
pzstd requires C++11, which older C++ standard libraries like libstdc++
as used on OS X 10.8 and earlier don't support. The user might address
this by setting "CXXFLAGS=-stdlib=libc++". This flag must be used both
at compile time and at link time. Asking the user to also put the flag
in LDFLAGS is undesirable because then the flag would also be used when
linking C code, which would be inappropriate.
2018-06-14 13:05:33 -07:00
Yann Collet
750ee87a92
Merge pull request #1175 from ryandesign/macos
...
Fix name of macOS
2018-06-13 11:32:06 -04:00
