AuroraMiddleware/v8
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity

[tools] Default to 4 retries for all perf tests

Browse Source 
Overall, total test runtime that was wasted due to timeouts is 3420 seconds in
the last 2 weeks. Even with 4 retries, assuming all of them time out, needed
additional capacity is under 2 hours per week. Based on this analysis, I think
it's safe to land this CL.

Note that this is not intended as a long-term solution of the timeout problem,
but rather a temporary solution to prevent ongoing errors. Proper investigation
and correct long-term solution are still needed and tracked in the bug.

R=machenbach@chromium.org, tmrts@chromium.org

Bug: chromium:841700
Change-Id: Id16e6b784fa85bb9e28ed8c6b267b583636e2dc1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1593342
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61224}
This commit is contained in:
Sergiy Belozorov 2019-05-02 22:49:36 +02:00 committed by Commit Bot
parent 140c1e51ae
commit b3e2bcf95a
3 changed files with 7 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
test/js-perf-test/JSTests5.json
View File

@ -40,7 +40,6 @@
"main": "run.js",
"resources": [],
"results_regexp": "^%s\\-ArrayLiteralInitialSpreadLargePacked\\(Score\\): (.+)$",
"retry_count_arm64": 1,
"tests": [
{"name": "Spread"},
{"name": "ForLength"},
@ -57,7 +56,6 @@
"main": "run.js",
"resources": [],
"results_regexp": "^%s\\-ArrayLiteralInitialSpreadLargeHoley\\(Score\\): (.+)$",
"retry_count_arm64": 1,
"tests": [
{"name": "Spread"},
{"name": "ForLength"},
@ -74,7 +72,6 @@
"main": "run.js",
"resources": [],
"results_regexp": "^%s\\-ArrayLiteralInitialSpreadLargeDoublePacked\\(Score\\): (.+)$",
"retry_count_arm64": 1,
"tests": [
{"name": "Spread"},
{"name": "ForLength"},
@ -91,7 +88,6 @@
"main": "run.js",
"resources": [],
"results_regexp": "^%s\\-ArrayLiteralInitialSpreadLargeDoubleHoley\\(Score\\): (.+)$",
"retry_count_arm64": 1,
"tests": [
{"name": "Spread"},
{"name": "ForLength"},

2
tools/run_perf.py
View File

@ -284,7 +284,7 @@ class DefaultSentinel(Node):
self.binary = binary
self.run_count = 10
self.timeout = 60
self.retry_count = 0
self.retry_count = 4
self.path = []
self.graphs = []
self.flags = []

8
tools/unittests/run_perf_test.py
View File

@ -403,9 +403,12 @@ class PerfTest(unittest.TestCase):
os.path.join('out', 'x64.release', 'd7'), '--flag', 'run.js')
def testOneRunCrashed(self):
self._WriteTestInput(V8_JSON)
test_input = dict(V8_JSON)
test_input['retry_count'] = 1
self._WriteTestInput(test_input)
self._MockCommand(
['.'], ['x\nRichards: 1.234\nDeltaBlue: 10657567\ny\n'], exit_code=-1)
['.'], ['x\nRichards: 1.234\nDeltaBlue: 10657567\ny\n', ''],
exit_code=-1)
self.assertEqual(1, self._CallMain())
self._VerifyResults('test', 'score', [])
self._VerifyErrors([])
@ -415,6 +418,7 @@ class PerfTest(unittest.TestCase):
def testOneRunTimingOut(self):
test_input = dict(V8_JSON)
test_input['timeout'] = 70
test_input['retry_count'] = 0
self._WriteTestInput(test_input)
self._MockCommand(['.'], [''], timed_out=True)
self.assertEqual(1, self._CallMain())