Adjust timeout of tests with "no resend" assertions
There are currently 4 tests in ssl-opt.sh with either -C "resend" or -S "resend", that is, asserting that no retransmission will occur. They sometimes fail on loaded CI machines as one side doesn't send a message fast enough, causing the other side to retransmit, causing the test to fail. (For the "reconnect" test there was an other issue causing random failures, fixed in a previous commit, but even after that fix the test would still sometimes randomly fail, even if much more rarely.) While it's a hard problem to fix in a general and perfect way, in practice the probability of failures can be drastically reduced by making the timeout values much larger. For some tests, where retransmissions are actually expected, this would have the negative effect of increasing the average running time of the test, as each side would wait for longer before it starts retransmission, so we have a trade-off between average running time and probability of spurious failures. But for tests where retransmission is not expected, there is no such trade-off as the expected running time of the test (assuming the code is correct most of the time) is not impacted by the timeout value. So the only negative effect of increasing the timeout value is on the worst-case running time on the test, which is much less important, as test should only fail quite rarely. This commit addresses the easy case of tests that don't expect retransmission by increasing the value of their timeout range to 10s-20s. This value corresponds to the value used for tests that assert `-S "autoreduction"` which are in the same case and where the current value seems acceptable so far. It also represents an increase, compared to the values before this commit, of a factor 20 for the "reconnect" tests which were frequently observed to fail in the CI, and of a factor 10 for the first two "DTLS proxy" tests, which were observed to fail much less frequently, so hopefully the new values are enough to reduce the probability of spurious failures to an acceptable level. Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This commit is contained in:
parent
baad2de6d8
commit
b6929891d6
@ -7279,8 +7279,8 @@ run_test "DTLS cookie: enabled, nbio" \
|
||||
|
||||
not_with_valgrind # spurious resend
|
||||
run_test "DTLS client reconnect from same port: reference" \
|
||||
"$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
|
||||
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000" \
|
||||
"$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
|
||||
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000" \
|
||||
0 \
|
||||
-C "resend" \
|
||||
-S "The operation timed out" \
|
||||
@ -7288,8 +7288,8 @@ run_test "DTLS client reconnect from same port: reference" \
|
||||
|
||||
not_with_valgrind # spurious resend
|
||||
run_test "DTLS client reconnect from same port: reconnect" \
|
||||
"$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
|
||||
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000 reconnect_hard=1" \
|
||||
"$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
|
||||
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000 reconnect_hard=1" \
|
||||
0 \
|
||||
-C "resend" \
|
||||
-S "The operation timed out" \
|
||||
@ -8395,8 +8395,8 @@ run_test "DTLS fragmenting: 3d, openssl client, DTLS 1.0" \
|
||||
not_with_valgrind # spurious resend due to timeout
|
||||
run_test "DTLS proxy: reference" \
|
||||
-p "$P_PXY" \
|
||||
"$P_SRV dtls=1 debug_level=2" \
|
||||
"$P_CLI dtls=1 debug_level=2" \
|
||||
"$P_SRV dtls=1 debug_level=2 hs_timeout=10000-20000" \
|
||||
"$P_CLI dtls=1 debug_level=2 hs_timeout=10000-20000" \
|
||||
0 \
|
||||
-C "replayed record" \
|
||||
-S "replayed record" \
|
||||
@ -8413,8 +8413,8 @@ run_test "DTLS proxy: reference" \
|
||||
not_with_valgrind # spurious resend due to timeout
|
||||
run_test "DTLS proxy: duplicate every packet" \
|
||||
-p "$P_PXY duplicate=1" \
|
||||
"$P_SRV dtls=1 dgram_packing=0 debug_level=2" \
|
||||
"$P_CLI dtls=1 dgram_packing=0 debug_level=2" \
|
||||
"$P_SRV dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
|
||||
"$P_CLI dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
|
||||
0 \
|
||||
-c "replayed record" \
|
||||
-s "replayed record" \
|
||||
|
Loading…
Reference in New Issue
Block a user