Yann Collet
5373e44ba7
fixed contrib/adaptive-compression
2018-03-15 17:10:15 -07:00
Yann Collet
355cb645bf
fixed seekable format example
2018-03-15 16:29:28 -07:00
Yann Collet
38cbcb5f1a
removed LRM exploratory experiment
2018-03-15 16:26:08 -07:00
Yann Collet
e916b9090e
gen_html: changed CFLAGS for CXXFLAGS
...
since it's associated with $(CXX)
2018-03-09 11:52:14 -08:00
Shawn Landden
ea41b580eb
meson: allow -Dlegacy_support=true, fix -Dlegacy_support=0
2017-12-29 10:18:33 -08:00
Shawn Landden
6ff43c0051
get soversion right
2017-12-24 10:05:43 -08:00
Shawn Landden
daffe435c0
meson: support differn't legacy levels.
...
Default to v0.4.0+
2017-12-21 15:47:38 -08:00
Shawn Landden
3ddfa42fe8
meson: fix build
...
used absolute paths which are deprecated in meson, also missing some sources
that got split
also move source files each to their own line so future diffs are clearer.
2017-12-19 22:02:03 -08:00
Yann Collet
c173dbd6e7
no longer supported starting C++17
2017-12-04 18:00:53 -08:00
Yann Collet
0d58aaf6f0
/contrib: fixed license header
...
removed last reference to PATENTS file
2017-10-02 02:07:17 -07:00
Yann Collet
f1a3ce34b7
Merge pull request #826 from terrelln/license
...
[linux-kernel] Update license
2017-08-31 12:52:23 -07:00
Nick Terrell
f9252d8347
[linux-kernel] Update license
2017-08-31 12:48:36 -07:00
Yann Collet
b0cb081dc8
last batch of header files changed to reflect new license ( #825 )
...
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
Yann Collet
e21384fffb
fixed more file headers after license change ( #825 )
2017-08-31 12:11:57 -07:00
Yann Collet
e9dc204f42
fixed a bunch of headers after license change ( #825 )
2017-08-31 11:24:54 -07:00
Yann Collet
394bdd7db9
changed license for examples
...
intentionnally this time
2017-08-29 09:24:11 -07:00
Yann Collet
dd63533b29
Merge pull request #797 from paulcruz74/documentation
...
Updating README.md
2017-08-16 00:37:55 -07:00
Paul Cruz
733ca51360
Updating README.md
2017-08-15 17:48:23 -07:00
Yann Collet
2daeab12f4
Merge pull request #795 from terrelln/squashfs-tools2
...
[kernel] Update squashfs-tools patch
2017-08-15 11:59:41 -07:00
Nick Terrell
57e2df6651
[kernel] Update squashfs-tools patch
2017-08-14 22:43:36 -07:00
Paul Cruz
fc90469587
updated program name print statement
2017-08-10 16:11:59 -07:00
Paul Cruz
75c3ad947c
Merge remote-tracking branch 'upstream/dev' into dev
2017-08-10 14:46:31 -07:00
Yann Collet
8915b9b518
Merge pull request #786 from terrelln/squashfs-tools
...
[linux-kernel] Update patches for v4 and v5
2017-08-09 16:00:11 -07:00
Nick Terrell
8b6702a00d
[linux-kernel] Update patches for v5
2017-08-09 13:03:40 -07:00
Yann Collet
a1fe656165
Merge pull request #779 from paulcruz74/adapt-approach-4
...
Patching warnings, adding ability to bound compression variation
2017-08-07 14:50:38 -07:00
Paul Cruz
7069bb9458
Merge branch 'adapt-approach-4' into bug-fixes
2017-08-07 13:18:53 -07:00
Paul Cruz
e100a311eb
removed direct assignment of 22, used ZSTD_maxCLevel() instead
2017-08-07 13:11:07 -07:00
Nick Terrell
7393b49fbd
[linux-kernel] Update patches for v4
2017-08-04 16:57:03 -07:00
Paul Cruz
01237e3b35
changed multi to zstd-adaptive in the help menu
2017-08-03 15:13:49 -07:00
Paul Cruz
8be7bba08c
added mutex for compression level to avoid data race
2017-08-02 10:27:33 -07:00
Paul Cruz
69ef22c0ac
added detach statements to prevent resource leak
2017-08-01 17:36:13 -07:00
Paul Cruz
f8c7b191e7
Merge branch 'adapt-approach-4' into dev
2017-07-31 15:42:57 -07:00
Paul Cruz
0295737ad7
change signal to broadcast for jobCompressed condition varaible since multiple threads waiting
2017-07-31 13:43:03 -07:00
Paul Cruz
cf92221671
Merge remote-tracking branch 'upstream/dev' into dev
2017-07-31 11:43:40 -07:00
Paul Cruz
c2f1bda8a1
Merge branch 'adapt-approach-4' into dev
2017-07-31 11:43:01 -07:00
Paul Cruz
9ea7df03de
add install target in makefile
2017-07-31 11:04:17 -07:00
Paul Cruz
f60cd3f99b
print defaults and range, remove EXT
2017-07-31 09:47:09 -07:00
Paul Cruz
e22b60cb76
removed ternary operation, added assert statement, check to make sure initial compression level is within bounds
2017-07-28 17:46:51 -07:00
Paul Cruz
cb9af53e77
delete empty line
2017-07-28 17:28:25 -07:00
Paul Cruz
51788225db
remove exe extension from makefile, reinclude pthread flag
2017-07-28 17:27:36 -07:00
Yann Collet
ee27f6937c
Merge pull request #758 from stellamplau/ldm
...
Add working prototype of a long distance matcher
2017-07-28 17:07:21 -07:00
Paul Cruz
4d904ac800
add flags for multithreading
2017-07-28 16:12:58 -07:00
Paul Cruz
0f4cb67b00
add tests for compression bounds, fix another warning
2017-07-28 15:55:02 -07:00
Paul Cruz
ff54fced64
patched style errors, add ability to bound compression level variation
2017-07-28 15:30:46 -07:00
Stella Lau
8fae41c412
Return error code in verify() and minor code cleanup
2017-07-27 17:14:05 -07:00
Stella Lau
1294a4a897
Fix typo
2017-07-27 15:49:46 -07:00
Stella Lau
627621839c
Add checks in initialization code
2017-07-27 15:37:37 -07:00
Stella Lau
c105f605e6
Update README
2017-07-27 11:11:35 -07:00
Yann Collet
e1222544be
Merge pull request #753 from paulcruz74/adapt-approach-3
...
adaptive compression v1
2017-07-27 10:00:10 -07:00
Paul Cruz
2320e7378a
remove unused variable, add documentation for context fields
2017-07-26 17:02:47 -07:00
Stella Lau
9eaf3d22d0
Allow HASH_ONLY_EVERY_LOG to be configured in ldm.h
2017-07-26 16:43:25 -07:00
Paul Cruz
ab5a78547e
fix leaky abstraction regarding measuring completion
2017-07-26 16:40:05 -07:00
Paul Cruz
715f36ca81
added definitions for conversion constants, moved forced compression check to top of adaptCompressionLevel, used ZSTD_BLOCKSIZE_MAX
2017-07-26 15:52:15 -07:00
Paul Cruz
6c1c1242fc
set the window log value before performing compression
2017-07-26 14:29:59 -07:00
Stella Lau
40759bade9
Add README and clean up code
2017-07-26 13:24:03 -07:00
Paul Cruz
a959cc881a
moved reset of completion to right after wait
2017-07-26 10:34:48 -07:00
Paul Cruz
305d5ee70f
change to >= convergence counter
2017-07-26 10:20:29 -07:00
Paul Cruz
be92a38d6a
decrease completion requirements for change, move create thread wait, merge cases where compression thread should wait
2017-07-26 10:05:10 -07:00
Stella Lau
e9161637b2
Allow parameters to be modified from a separate file
2017-07-25 18:13:27 -07:00
Paul Cruz
0b18d21e03
building on readme, added another help tip in the menu
2017-07-25 17:47:02 -07:00
Paul Cruz
7cc74e0b27
adding more to readme
2017-07-25 16:55:16 -07:00
Paul Cruz
8dbb07d822
updated progress bar with better representation of time, added const
2017-07-25 16:03:43 -07:00
Paul Cruz
9a132707af
changing time units to seconds
2017-07-25 15:26:26 -07:00
Stella Lau
629c300118
Rename and remove unneeded files
2017-07-25 15:17:36 -07:00
Paul Cruz
31a9ed9883
updated const values, added more comments
2017-07-25 14:53:40 -07:00
Paul Cruz
5cfbf609a4
removed old debug statements no longer being used
2017-07-25 14:31:48 -07:00
Paul Cruz
0882cd1981
progress bar -- don't print num jobs, time elapsed shown in seconds
2017-07-25 14:26:55 -07:00
Paul Cruz
310c12d07e
moved debug statements to a compiler flag
2017-07-25 14:08:39 -07:00
Paul Cruz
e02c79f833
started using decrease cooldown so that compression level would not decrease several times in a row
2017-07-25 11:16:27 -07:00
Paul Cruz
85d7c919f6
created independent function for controlling how completion relates to compression level change
2017-07-25 10:32:14 -07:00
Paul Cruz
6f1e260edd
added mechanism for getting rid of spikes
2017-07-25 10:01:10 -07:00
Paul Cruz
700758d676
added help statement for -p, switched it to hide progress bar now that progress bar is default
2017-07-24 16:26:20 -07:00
Paul Cruz
df3754b6ed
add quiet option, make progress bar default
2017-07-24 16:19:07 -07:00
Stella Lau
0295a27133
Experiment with not using a checksum
2017-07-24 15:26:44 -07:00
Paul Cruz
4dc83ca64c
compression thread should take measurements independently based on whether or not the create/write thread will actually bottleneck performance
2017-07-24 15:14:58 -07:00
Paul Cruz
0ee3f8c2f8
adding more debug
2017-07-24 15:06:11 -07:00
Paul Cruz
8328f8192a
updating debug statements again
2017-07-24 14:40:23 -07:00
Paul Cruz
d3d759301f
changing position of endline for debug
2017-07-24 13:47:39 -07:00
Stella Lau
08a6e9a141
Minor code cleanup
2017-07-24 13:22:00 -07:00
Stella Lau
6eefa32911
Deduplicate code
2017-07-24 12:40:59 -07:00
Stella Lau
8ed9220102
Experiment with eviction policies and minor code cleanup
2017-07-24 12:05:43 -07:00
Paul Cruz
e508f632d6
updated comments and debug statements
2017-07-24 11:01:36 -07:00
Stella Lau
eb16da647d
Minor clean up
2017-07-24 10:20:53 -07:00
Stella Lau
1a188fe864
Fix overflow bug when calculating hash
2017-07-24 10:20:53 -07:00
Stella Lau
0b8fb1703b
Experiment with 64-bit hash insertion policy
2017-07-24 10:20:30 -07:00
Stella Lau
273c17b350
Experiment with 64-bit hash and checksum
2017-07-24 10:19:50 -07:00
Paul Cruz
483d936b87
reduced competition for completion mutex by separating mutex use based on which values is updated
2017-07-23 14:09:16 -07:00
Paul Cruz
880f08d104
change how completion is measured in compression thread
2017-07-23 10:18:54 -07:00
Paul Cruz
08d9e42ec6
removed useless measurements
2017-07-21 18:02:55 -07:00
Paul Cruz
95bef759b3
switched over to model where reading only waits on compression thread
2017-07-21 17:49:39 -07:00
Paul Cruz
6455ec482c
taking the maximum of the completion level reads in order to determine which one was waiting more
2017-07-21 16:05:01 -07:00
Paul Cruz
05fe8dd47c
updating debug statements
2017-07-21 14:06:24 -07:00
Paul Cruz
db109f8fef
measure multiple completion levels during each wait
2017-07-21 13:38:24 -07:00
Paul Cruz
721c6a8b97
added bounding to compression level change
2017-07-21 09:30:24 -07:00
Paul Cruz
e929d3b787
added priority decision making for adapt compression level
2017-07-21 09:26:35 -07:00
Paul Cruz
9259c7afa4
semi working version that stabilizes
2017-07-20 18:45:33 -07:00
Paul Cruz
82e488770c
fixed bug where writeSize could be zero
2017-07-20 16:38:02 -07:00
Paul Cruz
a19916425d
reworked adaptCompressionLevel to only account for completion information
2017-07-20 16:19:16 -07:00
Nick Terrell
7d3ac0710d
[linux] Update patches for v3
2017-07-20 13:33:55 -07:00
Paul Cruz
7ab758a640
changed how completion is actually sampled
2017-07-20 10:53:51 -07:00
