Yann Collet
0ff67511e6
fixed link order for old compilers
2018-08-30 16:43:28 -07:00
Yann Collet
f76253bb70
minor : createDictionaryBuffer() can create dictionaries of different sizes
2018-08-30 16:24:44 -07:00
Yann Collet
39c55a118f
fixed minor compatibility issues with older compilers
2018-08-30 16:00:57 -07:00
Yann Collet
39ef91a599
-std=c99 for largeNbDicts
2018-08-30 14:59:23 -07:00
Yann Collet
4086b2871b
largeNbDicts compatible with multiple source files
...
splitting is disabled by default, but can be re-enabled using usual command -B#
update commands to look like zstd ones
2018-08-30 14:38:49 -07:00
Yann Collet
a5a77965d3
make all includes contrib/largeNbDicts
2018-08-29 16:17:22 -07:00
Yann Collet
d89fa814c1
added a README
...
for documentation
2018-08-28 18:19:19 -07:00
Yann Collet
6444c50035
increases randomness of ddict ptrs
2018-08-28 18:13:46 -07:00
Yann Collet
6c398df241
level, block size and nb dicts can be set on command line
2018-08-28 18:05:31 -07:00
Yann Collet
0c66a44d1b
first working test program
...
measures :
- compression ratio with / without dictionary
- create one dictionary per block
- memory budget for dictionaries
- decompression speed, using one different dictionary per block
current limitations :
- only one file
- 4K blocks only
- automatic dictionary built with 4K size
dictionary can be selected on command line, with -D
2018-08-28 15:47:07 -07:00
Yann Collet
274b60e6e6
largeNbDicts can compress and compare dict vs noDict
2018-08-27 17:08:44 -07:00
Yann Collet
6782725155
first sketch for largeNbDicts test program
2018-08-26 19:29:12 -07:00
Yann Collet
36d6165a2d
Makefile: added variable SCANBUILD
...
so that a different version of scan-build can be selected
2018-08-16 16:44:13 -07:00
Yann Collet
42a02ab745
fixed minor warnings issued by scan-build
2018-08-15 14:36:02 -07:00
Jennifer Liu
0acb0abd1e
Add non-optimize FASTCOVER ( #1260 )
...
* Add non-optimize FASTCOVER
* Minor fix
* Pass param as value instead of pointer
2018-08-01 11:06:16 -07:00
Jennifer Liu
4e29bc2469
Use CDict instead of CCtx in analyzeEntropy
2018-07-31 10:36:45 -07:00
Jennifer Liu
31229e527b
Increment frequency for every dmer occurence within same sample instead of at most once per sample
2018-07-30 12:54:22 -07:00
Jennifer Liu
51b109c1b5
Delete old benchmarking result
2018-07-27 17:31:33 -07:00
Jennifer Liu
53ef22a4bc
Undo deleting clean in make
2018-07-27 16:56:50 -07:00
Jennifer Liu
96d84ee235
Revert test.sh
2018-07-27 16:54:05 -07:00
Jennifer Liu
61262f6c0d
Save segmentFreqs in ctx instead of malloc and memset in SelectSegment
2018-07-27 16:51:38 -07:00
Jennifer Liu
49b398e93f
Use same param after optimizing cover and fastCover and record k and d for benchmarking
2018-07-27 13:39:19 -07:00
Jennifer Liu
759c543312
Rerun cover and fastCover with optimized values
2018-07-26 19:03:01 -07:00
Jennifer Liu
3d7941ce41
Benchmark different f values
2018-07-26 16:24:13 -07:00
Jennifer Liu
3b163e0b5b
Add array to keep track of frequency within active segment, fix malloc bug, update benchmarking result
2018-07-26 13:53:13 -07:00
Jennifer Liu
2333ecb173
Allow d=6
2018-07-25 18:10:09 -07:00
Jennifer Liu
1e85f314d8
Benchmark fast cover optimize vs k=200
2018-07-25 17:53:38 -07:00
Jennifer Liu
d1fc507ef9
Initial benchmarking result for fastCover
2018-07-25 17:05:54 -07:00
Jennifer Liu
f5407e398a
Make hash value const
2018-07-25 16:54:08 -07:00
Jennifer Liu
7f3f70f766
Add Fast Cover Dictionary Builder
2018-07-25 16:34:07 -07:00
Nick Terrell
77068a8447
Merge pull request #1246 from jennifermliu/benchmark
...
Benchmark dictionary builders
2018-07-20 18:09:31 -07:00
Jennifer Liu
b6c5d4982c
Minor fix
2018-07-20 17:41:22 -07:00
Jennifer Liu
71e767ac09
Refactoring and benchmark without dictionary
2018-07-20 17:03:47 -07:00
Jennifer Liu
470c8d42f4
Benchmark dictionary builders
2018-07-20 11:32:39 -07:00
Nick Terrell
4d1ad5cdb2
Merge pull request #1238 from jennifermliu/random
...
Add random dictionary builder
2018-07-19 13:52:15 -07:00
Jennifer Liu
0c5eaef248
Update Makefile
2018-07-19 13:44:27 -07:00
Jennifer Liu
5bb46a898e
Rename cleanup
2018-07-18 12:15:49 -07:00
Jennifer Liu
52e7cf0e40
Add cleanup to trainfromFiles and move RANDOM_segment_t declaration
2018-07-18 10:40:13 -07:00
Jennifer Liu
ce09fb723d
Update freeSampleInfo
2018-07-17 16:13:40 -07:00
Jennifer Liu
896ff0644a
Fix deallocation problem and add documentation
2018-07-17 16:01:44 -07:00
Jennifer Liu
e6fe405838
Make test PHONY target
2018-07-17 12:42:53 -07:00
Jennifer Liu
49acfaeaec
Move file loading functions to new file for access by benchmarking tool
2018-07-17 12:35:09 -07:00
Jennifer Liu
4d32339b75
Remove CLevel cli option which was accidentally added back in the last commit
2018-07-16 18:59:18 -07:00
Jennifer Liu
1f7fa5cdd6
Fix spacing and Edit Makefile (now run with make instead of make run)
2018-07-16 16:31:59 -07:00
Jennifer Liu
b5806d33db
Refactor RANDOM
2018-07-16 16:03:04 -07:00
Codecat
044cd81ce6
Fix wrong conditions
2018-07-14 12:34:03 +02:00
Codecat
1a61bdb9c0
Update zstd.lua
2018-07-14 12:27:42 +02:00
Jennifer Liu
0e5fbc10fa
Update README
2018-07-13 17:41:09 -07:00
Jennifer Liu
31731df4da
Remove clevel and update documentation
2018-07-13 17:38:53 -07:00
Jennifer Liu
a23a3b95f9
Add random dictionary builder
2018-07-13 16:05:14 -07:00
