v8/test/message
Tobias Tebbi d8f8a7e210 Reland "[flags] warn about contradictory flags"
This is a reland of b8f9166664
Difference to previous CL: Additional functionality to specify
incompatible flags based on GN variables and extra-flags, used
to fix the issues that came up on the waterfall.

This also changes the rules regarding repeated flags: While
explicitly repeated flags are allowed for boolean values as long
as they are identical, repeated flags or explicit flags in the
presence of an active implication are disallowed for non-boolean
flags. The latter simplifies specifying conflict rules in
variants.py. Otherwise a rule like

INCOMPATIBLE_FLAGS_PER_EXTRA_FLAG = {
  "--gc-interval=*": ["--gc-interval=*"],
}

wouldn't work because specifying the same GC interval twice
wouldn't actually count as a conflict. This was an issue with
test/mjsunit/wasm/gc-buffer.js, which specifies
--gc-interval=500 exactly like the extra flag by the stress bot.

Also, this now expands contradictory flags checking to d8 flags
for consistency.

Original change's description:
> [flags] warn about contradictory flags
>
> Design Doc: https://docs.google.com/document/d/1lkvu8crkK7Ei39qjkPCFijpNyxWXsOktG9GB-7K34jM/
>
> Bug: v8:10577
> Change-Id: Ib9cfdffa401c48c895bf31caed5ee03545beddab
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2154792
> Reviewed-by: Clemens Backes <clemensb@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Georg Neis <neis@chromium.org>
> Reviewed-by: Tamer Tas <tmrts@chromium.org>
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#68168}

Bug: v8:10577
Change-Id: I268e590ee18a535b13dee14eeb15ddd0a9ee8341
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2235115
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68989}
2020-07-22 12:22:03 +00:00
..
fail [d8] Exit with error code upon unhandled promise rejection 2020-06-24 07:21:58 +00:00
mjsunit [d8] Handle recursively rejected promises 2020-06-30 08:27:36 +00:00
regress/fail [scanner] fix bug with reporting invalid numeric separator 2019-08-08 16:36:36 +00:00
asm-assignment-undeclared.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-assignment-undeclared.out [asm.js] Fix assignment with undeclared target. 2017-03-30 08:36:17 +00:00
asm-function-mismatch-def.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-function-mismatch-def.out [asm.js] Test and fix function type checking. 2017-04-07 12:10:45 +00:00
asm-function-mismatch-use.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-function-mismatch-use.out [asm.js] Test and fix function type checking. 2017-04-07 12:10:45 +00:00
asm-function-redefined.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-function-redefined.out [asm.js] Test and fix function name collisions. 2017-04-07 09:03:11 +00:00
asm-function-undefined.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-function-undefined.out [asm.js] Fix handling of bogus code after export statement. 2019-02-21 14:37:37 +00:00
asm-function-variable-collision.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-function-variable-collision.out [asm.js] Test and fix function name collisions. 2017-04-07 09:03:11 +00:00
asm-import-wrong-annotation.js [asm.js] Fix validation failure in module variable import. 2017-05-05 11:39:06 +00:00
asm-import-wrong-annotation.out [asm.js] Fix validation failure in module variable import. 2017-05-05 11:39:06 +00:00
asm-import-wrong-object.js [asm.js] Fix validation failure in module variable import. 2017-05-05 11:39:06 +00:00
asm-import-wrong-object.out [asm.js] Fix validation failure in module variable import. 2017-05-05 11:39:06 +00:00
asm-linking-bogus-heap.js [asm.js] Report module linking failures to the console. 2017-05-19 11:31:12 +00:00
asm-linking-bogus-heap.out Reland "[asmjs] Properly validate asm.js heap sizes" 2018-08-21 09:00:04 +00:00
asm-linking-bogus-stdlib.js [asm.js] Report module linking failures to the console. 2017-05-19 11:31:12 +00:00
asm-linking-bogus-stdlib.out [asm.js] Report module linking failures to the console. 2017-05-19 11:31:12 +00:00
asm-linking-missing-heap.js [asm.js] Report module linking failures to the console. 2017-05-19 11:31:12 +00:00
asm-linking-missing-heap.out [asm.js] Report module linking failures to the console. 2017-05-19 11:31:12 +00:00
asm-missing-parameter-annotation.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-missing-parameter-annotation.out [asm.js] Make validation error messages consistent. 2017-03-29 20:55:12 +00:00
asm-missing-return-annotation.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-missing-return-annotation.out [asm.js] Make validation error messages consistent. 2017-03-29 20:55:12 +00:00
asm-table-mismatch-def.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-table-mismatch-def.out [asm.js] Test and fix function table type checks. 2017-04-10 08:47:34 +00:00
asm-table-mismatch-use.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-table-mismatch-use.out [asm.js] Test and fix function table type checks. 2017-04-10 08:47:34 +00:00
asm-table-redefined.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-table-redefined.out [asm.js] Test and fix function table definition failures. 2017-04-11 16:46:26 +00:00
asm-table-undefined.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-table-undefined.out [asm.js] Fix handling of bogus code after export statement. 2019-02-21 14:37:37 +00:00
asm-table-variable-collision.js [asm.js] Remove deprecated --fast-validate-asm flag. 2017-05-04 13:52:39 +00:00
asm-table-variable-collision.out [asm.js] Test and fix function table definition failures. 2017-04-11 16:46:26 +00:00
BUILD.gn [build] Add data deps for d8 test suites 2018-03-26 13:44:58 +00:00
empty.js [platform] Allow to memory-map empty files 2019-01-21 16:02:47 +00:00
empty.out [platform] Allow to memory-map empty files 2019-01-21 16:02:47 +00:00
message.status [d8] Handle recursively rejected promises 2020-06-30 08:27:36 +00:00
non-use-strict-hex-escape.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
non-use-strict-hex-escape.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
non-use-strict-octal-escape.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
non-use-strict-octal-escape.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
non-use-strict-uhex-escape.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
non-use-strict-uhex-escape.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-arguments.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-arguments.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-eval.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-eval.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-with.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
nonstrict-with.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
object-binding-pattern-await-computed-name.js [parser] classify binding pattern errors when parsing await expression 2017-12-09 16:29:23 +00:00
object-binding-pattern-await-computed-name.out [parser] classify binding pattern errors when parsing await expression 2017-12-09 16:29:23 +00:00
README.md Remove always-true --harmony-function-tostring runtime flag 2018-10-03 15:04:16 +00:00
strict-octal-indirect-regexp.js [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
strict-octal-indirect-regexp.out [testing] Move the last JS tests out of "preparser". 2016-03-14 17:10:49 +00:00
strict-octal-regexp.js [testing] Convert some "preparser" tests into "message". 2016-03-11 11:00:12 +00:00
strict-octal-regexp.out [testing] Convert some "preparser" tests into "message". 2016-03-11 11:00:12 +00:00
testcfg.py [test] Remove module pragma in favor of .mjs 2019-06-27 13:25:48 +00:00
try-catch-finally-no-message.js Changed the message reporting for try { ... } finally { ... } statements to 2008-10-23 08:40:19 +00:00
try-catch-finally-no-message.out First commit of new tools/run-tests.py 2012-09-24 09:38:46 +00:00
try-catch-finally-return-in-finally.js Update V8 to version 3.0 (re-land r5920). 2010-12-07 11:31:57 +00:00
try-catch-finally-return-in-finally.out Update V8 to version 3.0 (re-land r5920). 2010-12-07 11:31:57 +00:00
try-finally-return-in-finally.js Update V8 to version 3.0 (re-land r5920). 2010-12-07 11:31:57 +00:00
try-finally-return-in-finally.out Update V8 to version 3.0 (re-land r5920). 2010-12-07 11:31:57 +00:00
wasm-finish-compilation.js [wasm] merge js constants file in module-builder 2019-01-30 22:10:34 +00:00
wasm-finish-compilation.out Reland "Reland "[d8][wasm] Test wasm compilation completion"" 2018-04-10 08:24:27 +00:00
wasm-function-name-async.js [wasm] merge js constants file in module-builder 2019-01-30 22:10:34 +00:00
wasm-function-name-async.out [stack-trace] Add url to wasm stack traces 2020-02-18 06:55:28 +00:00
wasm-function-name-streaming.js [wasm] Load module name during streaming compile 2018-03-16 09:01:20 +00:00
wasm-function-name-streaming.out [stack-trace] Add url to wasm stack traces 2020-02-18 06:55:28 +00:00
wasm-module-and-function-name-async.js [wasm] merge js constants file in module-builder 2019-01-30 22:10:34 +00:00
wasm-module-and-function-name-async.out [stack-trace] Add url to wasm stack traces 2020-02-18 06:55:28 +00:00
wasm-module-and-function-name-streaming.js [wasm] Load module name during streaming compile 2018-03-16 09:01:20 +00:00
wasm-module-and-function-name-streaming.out [stack-trace] Add url to wasm stack traces 2020-02-18 06:55:28 +00:00
wasm-module-name-async.js [wasm] merge js constants file in module-builder 2019-01-30 22:10:34 +00:00
wasm-module-name-async.out [wasm] WAT-compatible naming for exported function 2020-03-26 17:29:49 +00:00
wasm-module-name-streaming.js [wasm] Load module name during streaming compile 2018-03-16 09:01:20 +00:00
wasm-module-name-streaming.out [wasm] WAT-compatible naming for exported function 2020-03-26 17:29:49 +00:00
wasm-no-name-async.js [wasm] merge js constants file in module-builder 2019-01-30 22:10:34 +00:00
wasm-no-name-async.out [wasm] WAT-compatible naming for exported function 2020-03-26 17:29:49 +00:00
wasm-no-name-streaming.js [wasm] Load module name during streaming compile 2018-03-16 09:01:20 +00:00
wasm-no-name-streaming.out [wasm] WAT-compatible naming for exported function 2020-03-26 17:29:49 +00:00
wasm-trace-liftoff.js [wasm] Fix register uses in wasm tracing 2020-06-15 11:13:18 +00:00
wasm-trace-liftoff.out [wasm] Fix register uses in wasm tracing 2020-06-15 11:13:18 +00:00
wasm-trace-memory-liftoff.js Reland "[flags] warn about contradictory flags" 2020-07-22 12:22:03 +00:00
wasm-trace-memory-liftoff.out [wasm] Add some more test cases to wasm-trace-memory 2020-03-26 17:44:53 +00:00
wasm-trace-memory.js Reland "[flags] warn about contradictory flags" 2020-07-22 12:22:03 +00:00
wasm-trace-memory.out [wasm] Add some more test cases to wasm-trace-memory 2020-03-26 17:44:53 +00:00
wasm-trace-turbofan.js [wasm] Add tracing support in turbofan 2020-06-09 10:35:33 +00:00
wasm-trace-turbofan.out [wasm] Fix register uses in wasm tracing 2020-06-15 11:13:18 +00:00
weakref-finalizationregistry-error.js [weakrefs] Update FinalizationRegistry to take a per-item callback 2020-04-08 17:54:24 +00:00
weakref-finalizationregistry-error.out [weakrefs] Port FinalizationRegistry cleanup loop to Torque 2020-04-16 03:31:04 +00:00

JavaScript tests with expected output

Tests in test/message pass if the output matches the expected output. Message tests are particularly useful when checking for exact error messages.

Tests and their expected output must have the same filename, with the .js and .out extension.

foo.js
foo.out

All tests must end with an exception. The test runner does not handle output from multiple runs, e.g., --stress-opt. Without an exception, the output will be generated several times and the comparison will fail.

You can use a regex in the expected output instead of the exact path:

*%(basename)s:7: SyntaxError: Detected cycle while resolving name 'a'

Empty lines are ignored in the comparison, but whitespaces are not.

Exact details of the test runner are in testcfg.py.