v8/infra/testing
Michael Achenbach 23ccfa1d94 [test] Turn exemplary configurations into reference comments
This also updates the README with guidelines.

Bug: chromium:788104
Change-Id: I0ca0ea78c5990204b0242be9c7fe6368439a5dd1
Reviewed-on: https://chromium-review.googlesource.com/796311
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49713}
2017-11-29 13:18:22 +00:00
..
client.v8.pyl [test] Turn exemplary configurations into reference comments 2017-11-29 13:18:22 +00:00
OWNERS [test] Add v8-side demo test configuration 2017-11-28 17:21:36 +00:00
README.md [test] Turn exemplary configurations into reference comments 2017-11-29 13:18:22 +00:00
tryserver.v8.pyl [test] Turn exemplary configurations into reference comments 2017-11-29 13:18:22 +00:00

Src-side test specifications

The infra/testing folder in V8 contains test specifications, consumed and executed by the continuous infrastructure. Every master has an optional file named <mastername>.pyl. E.g. tryserver.v8.pyl.

The structure of each file is:

{
  <buildername>: [
    {
      'name': <test-spec name>,
      'variant': <variant name>,
      'shards': <number of shards>,
    },
    ...
  ],
  ...
}

The <buildername> is a string name of the builder to execute the tests. <test-spec name> is a label defining a test specification matching the infra-side. The <variant name> is a testing variant as specified in v8/tools/testrunner/local/variants.py. <number of shards> is optional (default 1), but can be provided to increase the swarming shards for long-running tests.

Example:

{
  'v8_linux64_rel_ng_triggered': [
    {'name': 'v8testing', 'variant': 'nooptimization', 'shards': 2},
  ],
}

Guidelines

Please keep trybots and continuous bots in sync. E.g. add the same configuration for the release and debug CI bots and the corresponding trybot (where applicable). E.g.

tryserver.v8:
  v8_linux64_rel_ng_triggered
client.v8:
  V8 Linux64
  V8 Linux64 - debug