v8/tools/testrunner/testproc/loader.py
Tamer Tas 3f83accb09 [testrunner] load tests concurrently into test execution processor
loading every test up-front into the processing queue costs about 224MB for a
x64 testsuite run.

This CL eliminates that overhead by utilizing generators and threading.

LoadingProc now loads test after receiving the results of the loaded tests.

R=machenbach@chromium.org
CC=​yangguo@chromium.org,sergiyb@chromium.org

Bug: v8:8174,v8:8731
Change-Id: Ifee79c3e213da568f092de0f1623016174e9410c
Reviewed-on: https://chromium-review.googlesource.com/c/1439240
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59223}
2019-01-31 08:19:06 +00:00

43 lines
1.1 KiB
Python

# Copyright 2018 the V8 project authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
from . import base
class LoadProc(base.TestProc):
"""First processor in the chain that passes all tests to the next processor.
"""
def __init__(self, tests):
super(LoadProc, self).__init__()
self.tests = tests
def load_initial_tests(self, initial_batch_size):
"""
Args:
exec_proc: execution processor that the tests are being loaded into
initial_batch_size: initial number of tests to load
"""
loaded_tests = 0
while loaded_tests < initial_batch_size:
try:
t = next(self.tests)
except StopIteration:
return
if self._send_test(t):
loaded_tests += 1
def next_test(self, test):
assert False, 'Nothing can be connected to the LoadProc'
def result_for(self, test, result):
try:
while not self._send_test(next(self.tests)):
pass
except StopIteration:
# No more tests to load.
pass