Adjust timeout of tests with "no resend" assertions

There are currently 4 tests in ssl-opt.sh with either -C "resend" or -S
"resend", that is, asserting that no retransmission will occur. They sometimes
fail on loaded CI machines as one side doesn't send a message fast enough,
causing the other side to retransmit, causing the test to fail.

(For the "reconnect" test there was an other issue causing random failures,
fixed in a previous commit, but even after that fix the test would still
sometimes randomly fail, even if much more rarely.)

While it's a hard problem to fix in a general and perfect way, in practice the
probability of failures can be drastically reduced by making the timeout
values much larger.

For some tests, where retransmissions are actually expected, this would have
the negative effect of increasing the average running time of the test, as
each side would wait for longer before it starts retransmission, so we have a
trade-off between average running time and probability of spurious failures.

But for tests where retransmission is not expected, there is no such trade-off
as the expected running time of the test (assuming the code is correct most of
the time) is not impacted by the timeout value. So the only negative effect of
increasing the timeout value is on the worst-case running time on the test,
which is much less important, as test should only fail quite rarely.

This commit addresses the easy case of tests that don't expect retransmission
by increasing the value of their timeout range to 10s-20s. This value
corresponds to the value used for tests that assert `-S "autoreduction"` which
are in the same case and where the current value seems acceptable so far.

It also represents an increase, compared to the values before this commit, of
a factor 20 for the "reconnect" tests which were frequently observed to fail
in the CI, and of a factor 10 for the first two "DTLS proxy" tests, which were
observed to fail much less frequently, so hopefully the new values are enough
to reduce the probability of spurious failures to an acceptable level.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
This commit is contained in:
Manuel Pégourié-Gonnard 2019-09-09 11:14:37 +02:00
parent baad2de6d8
commit b6929891d6

View File

@ -7279,8 +7279,8 @@ run_test "DTLS cookie: enabled, nbio" \
not_with_valgrind # spurious resend
run_test "DTLS client reconnect from same port: reference" \
"$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000" \
"$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000" \
0 \
-C "resend" \
-S "The operation timed out" \
@ -7288,8 +7288,8 @@ run_test "DTLS client reconnect from same port: reference" \
not_with_valgrind # spurious resend
run_test "DTLS client reconnect from same port: reconnect" \
"$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000 reconnect_hard=1" \
"$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
"$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000 reconnect_hard=1" \
0 \
-C "resend" \
-S "The operation timed out" \
@ -8395,8 +8395,8 @@ run_test "DTLS fragmenting: 3d, openssl client, DTLS 1.0" \
not_with_valgrind # spurious resend due to timeout
run_test "DTLS proxy: reference" \
-p "$P_PXY" \
"$P_SRV dtls=1 debug_level=2" \
"$P_CLI dtls=1 debug_level=2" \
"$P_SRV dtls=1 debug_level=2 hs_timeout=10000-20000" \
"$P_CLI dtls=1 debug_level=2 hs_timeout=10000-20000" \
0 \
-C "replayed record" \
-S "replayed record" \
@ -8413,8 +8413,8 @@ run_test "DTLS proxy: reference" \
not_with_valgrind # spurious resend due to timeout
run_test "DTLS proxy: duplicate every packet" \
-p "$P_PXY duplicate=1" \
"$P_SRV dtls=1 dgram_packing=0 debug_level=2" \
"$P_CLI dtls=1 dgram_packing=0 debug_level=2" \
"$P_SRV dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
"$P_CLI dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
0 \
-c "replayed record" \
-s "replayed record" \