AuroraMiddleware/v8
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity

[test] Add more documentation about test-specifications

Browse Source 
NOTRY=true
NOPRESUBMIT=true
NOTREECHECKS=true

Bug: chromium:788104
Change-Id: I1918d4ba69b2c00151ac51ee9426d980e24ab871
Reviewed-on: https://chromium-review.googlesource.com/808389
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49863}
This commit is contained in:
Michael Achenbach 2017-12-05 12:54:21 +01:00 committed by Commit Bot
parent 2520c33642
commit 19d040eaa7
1 changed files with 28 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
infra/testing/README.md
View File

@ -1,8 +1,15 @@
# Src-side test specifications
The infra/testing folder in V8 contains test specifications, consumed and
executed by the continuous infrastructure. Every master has an optional file
named `<mastername>.pyl`. E.g. `tryserver.v8.pyl`.
Src-side test specifications enable developers to quickly add tests running on
specific bots on V8's continuous infrastructure (CI) or tryserver. Features to
be tested must live behind runtime flags, which are mapped to named testing
variants specified [here](https://chromium.googlesource.com/v8/v8/+/master/tools/testrunner/local/variants.py).
Changes to src-side test specifications go through CQ like any other CL and
require tests added for specific trybots to pass.
The test specifications are defined in a V8-side folder called infra/testing.
Every master has an optional file named `<mastername>.pyl`. E.g.
`tryserver.v8.pyl`.
The structure of each file is:
```
@ -21,10 +28,10 @@ The structure of each file is:
The `<buildername>` is a string name of the builder to execute the tests.
`<test-spec name>` is a label defining a test specification matching the
[infra-side](https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipe_modules/v8/testing.py#58).
The `<variant name>` is a testing variant as specified in
`v8/tools/testrunner/local/variants.py`. `<number of shards>` is optional
(default 1), but can be provided to increase the swarming shards for
long-running tests.
The `<variant name>` is a testing variant specified
[here](https://chromium.googlesource.com/v8/v8/+/master/tools/testrunner/local/variants.py).
`<number of shards>` is optional (default 1), but can be provided to increase
the swarming shards for long-running tests.
Example:
```
@ -47,4 +54,17 @@ tryserver.v8:
client.v8:
V8 Linux64
V8 Linux64 - debug
```
```
Please only add tests that are expected to pass, or skip failing tests via
status file for the selected testing variants only. If you want to add FYI tests
(i.e. not closing the tree and not blocking CQ) you can do so for the following
set of bots:
```
tryserver.v8:
v8_linux64_fyi_rel_ng_triggered
client.v8:
V8 Linux64 - fyi
V8 Linux64 - debug - fyi
```