v8/tools/torque/format-torque.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

161 lines
5.6 KiB
Python
Raw Normal View History

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright 2014 the V8 project authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""This program either generates the parser files for Torque, generating
the source and header files directly in V8's src directory."""
# for py2/py3 compatibility
from __future__ import print_function
import subprocess
import sys
import re
from subprocess import Popen, PIPE
kPercentEscape = r'α'; # Unicode alpha
kDerefEscape = r''; # Unicode star
kAddressofEscape = r''; # Unicode house
def preprocess(input):
# Special handing of '%' for intrinsics, turn the percent
# into a unicode character so that it gets treated as part of the
# intrinsic's name if it's already adjacent to it.
input = re.sub(r'%([A-Za-z])', kPercentEscape + r'\1', input)
# Similarly, avoid treating * and & as binary operators when they're
# probably used as address operators.
input = re.sub(r'([^/])\*([a-zA-Z(])', r'\1' + kDerefEscape + r'\2', input)
input = re.sub(r'&([a-zA-Z(])', kAddressofEscape + r'\1', input)
input = re.sub(r'(if\s+)constexpr(\s*\()', r'\1/*COxp*/\2', input)
input = re.sub(r'(\s+)operator\s*(\'[^\']+\')', r'\1/*_OPE \2*/', input)
Reland "[torque] introduce JSAny type for user-accessible JavaScript values" Changes in the reland: Rebased and added a check that JavaScript-linkage builtins use JSAny in parameters and return type, plus the necessary cleanups for this test to pass. Design Doc: https://docs.google.com/document/d/1z6j0pWHnNIfId0v00uWN2HBrGRDJxJfYuCr5K7Kr1xA This reverts commit 4418a7b96afdf8278577211ab0ed555c41ee269e. Original change's description: > Revert "[torque] introduce JSAny type for user-accessible JavaScript values" > > This reverts commit 79b00555eac939bf4a00177ec59e21b856d7301c. > > Reason for revert: needs more discussion > > Original change's description: > > [torque] introduce JSAny type for user-accessible JavaScript values > > > > This CL introduces a JSAny type for user-exposed JavaScript values and > > a few new types to define it. Especially, it splits Symbol into > > PrivateSymbol (not exposed) and PublicSymbol (JavaScript exposed > > symbols). > > > > The change is mostly mechanical, but a few things are interesting: > > - PropertyKey and JSPrimitive were designed to coincide with the spec > > notions of IsPropertyKey() and primitive value, respectively. > > - Since Name is an open type, we define AnyName to be the known > > subtypes of Name. This is not too elegant, but by using AnyName > > instead of Name, typeswitch can properly conclude something if a > > subtype of Name is excluded. > > > > Small drive-by changes, which were necessary: > > - Allow subtyping on label parameters. > > - Fix the formatting of typeswitch, it was broken with union types > > in case types. > > > > Bug: v8:7793 > > Change-Id: I14b10507f8cf316ad85e048fe8d53d1df5e0bb13 > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735322 > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org> > > Reviewed-by: Jakob Gruber <jgruber@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#63114} > > TBR=neis@chromium.org,jgruber@chromium.org,tebbi@chromium.org > > Change-Id: Ifde7881d74afe407628f40047997339d54cb2424 > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: v8:7793 > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1741652 > Reviewed-by: Tobias Tebbi <tebbi@chromium.org> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org> > Cr-Commit-Position: refs/heads/master@{#63115} TBR=neis@chromium.org,jgruber@chromium.org,tebbi@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: v8:7793 Change-Id: Icca34e3824f55009b984d9348fd21884400f0081 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1769316 Commit-Queue: Tobias Tebbi <tebbi@chromium.org> Reviewed-by: Tobias Tebbi <tebbi@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Cr-Commit-Position: refs/heads/master@{#63395}
2019-08-26 12:07:01 +00:00
input = re.sub(r'\btypeswitch\s*(\([^{]*\))\s{', r' if /*tPsW*/ \1 {', input)
input = re.sub(r'\bcase\s*(\([^{]*\))\s*:\s*deferred\s*{', r' if /*cAsEdEfF*/ \1 {', input)
input = re.sub(r'\bcase\s*(\([^{]*\))\s*:\s*{', r' if /*cA*/ \1 {', input)
[torque] enable multiple inheritance from Torque-generated assemblers This enables more seamless interop between Torque and CSA: Since CodeStubAssembler can now inherit from the Torque base namespace, macros defined in the base namespace can be used in CodeStubAssembler macros, even without qualification. At the same time, macros in the base namespace can refer to CodeStubAssembler macros. The only new limitation is that types defined in code-stub-assembler.h cannot be referenced in the signature of macros defined in the base namespace, since this would produce a cyclic header dependency. A work-around for this woud be to put such types (like int31 in this CL) into a separate header included by both. I (mis-)used code-assembler.h for that. Another side-effec is that types and enums defined in CodeStubAssembler have to be accessed in a qualified way from Torque. Other assemblers can now inherit from their Torque equivalent, so porting macros into the corresponding Torque namespace doesn't require any change to the existing use-sites. To avoid C++ ambiguities, the Torque-generated assemblers must not define anything also defined in Code(Stub)Assembler. This includes the type aliases for TNode, PLabel, ... My workaround is to qualify everything in the generated C++. As a drive-by fix, I had to change the formatter to avoid a situation where it doesn't compute a fixed point: putting a keyword at the beginning of a line removes the '\s' in front of it, so I replaced that with '\b'. Bug: v8:7793 Change-Id: If3b9e9ad967a181b380a10d5673615606abd1041 Reviewed-on: https://chromium-review.googlesource.com/c/1341955 Reviewed-by: Daniel Clifford <danno@chromium.org> Commit-Queue: Tobias Tebbi <tebbi@chromium.org> Cr-Commit-Position: refs/heads/master@{#57645}
2018-11-20 10:17:24 +00:00
input = re.sub(r'\bgenerates\s+\'([^\']+)\'\s*',
[torque] Generate instance types Design doc: https://docs.google.com/document/d/1ZU6rCvF2YHBGMLujWqqaxlPsjFfjKDE9C3-EugfdlAE/edit Changes from the design doc: - Changed to use 'class' declarations rather than 'type' declarations for things that need instance types but whose layout is not known to Torque. These declarations end with a semicolon rather than having a full set of methods and fields surrounded by {}. If the class's name should not be treated as a class name in generated output (because it's actually a template, or doesn't exist at all), we use the standard 'generates' clause to declare the most appropriate C++ class. - Removed @instanceTypeName. - @highestInstanceType became @highestInstanceTypeWithinParentClassRange to indicate a semantic change: it no longer denotes the highest instance type globally, but only within the range of values for its immediate parent class. This lets us use it for Oddball, which is expected to be the highest primitive type. - Added new abstract classes JSCustomElementsObject and JSSpecialObject to help with some range checks. - Added @lowestInstanceTypeWithinParentClassRange so we can move the new classes JSCustomElementsObject and JSSpecialObject to the beginning of the JSObject range. This seems like the least-brittle way to establish ranges that also include JSProxy (and these ranges are verified with static assertions in instance-type.h). - Renamed @instanceTypeValue to @apiExposedInstanceTypeValue. - Renamed @instanceTypeFlags to @reserveBitsInInstanceType. This change introduces the new annotations and adds the ability for Torque to assign instance types that satisfy those annotations. Torque now emits two new macros: - TORQUE_ASSIGNED_INSTANCE_TYPES, which is used to define the InstanceType enumeration - TORQUE_ASSIGNED_INSTANCE_TYPE_LIST, which replaces the non-String parts of INSTANCE_TYPE_LIST The design document mentions a couple of other macro lists that could easily be replaced, but I'd like to defer those to a subsequent checkin because this one is already pretty large. Bug: v8:7793 Change-Id: Ie71d93a9d5b610e62be0ffa3bb36180c3357a6e8 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1757094 Commit-Queue: Seth Brenith <seth.brenith@microsoft.com> Reviewed-by: Tobias Tebbi <tebbi@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org> Cr-Commit-Position: refs/heads/master@{#64258}
2019-10-11 21:52:06 +00:00
r'_GeNeRaTeS00_/*\1@*/', input)
[torque] enable multiple inheritance from Torque-generated assemblers This enables more seamless interop between Torque and CSA: Since CodeStubAssembler can now inherit from the Torque base namespace, macros defined in the base namespace can be used in CodeStubAssembler macros, even without qualification. At the same time, macros in the base namespace can refer to CodeStubAssembler macros. The only new limitation is that types defined in code-stub-assembler.h cannot be referenced in the signature of macros defined in the base namespace, since this would produce a cyclic header dependency. A work-around for this woud be to put such types (like int31 in this CL) into a separate header included by both. I (mis-)used code-assembler.h for that. Another side-effec is that types and enums defined in CodeStubAssembler have to be accessed in a qualified way from Torque. Other assemblers can now inherit from their Torque equivalent, so porting macros into the corresponding Torque namespace doesn't require any change to the existing use-sites. To avoid C++ ambiguities, the Torque-generated assemblers must not define anything also defined in Code(Stub)Assembler. This includes the type aliases for TNode, PLabel, ... My workaround is to qualify everything in the generated C++. As a drive-by fix, I had to change the formatter to avoid a situation where it doesn't compute a fixed point: putting a keyword at the beginning of a line removes the '\s' in front of it, so I replaced that with '\b'. Bug: v8:7793 Change-Id: If3b9e9ad967a181b380a10d5673615606abd1041 Reviewed-on: https://chromium-review.googlesource.com/c/1341955 Reviewed-by: Daniel Clifford <danno@chromium.org> Commit-Queue: Tobias Tebbi <tebbi@chromium.org> Cr-Commit-Position: refs/heads/master@{#57645}
2018-11-20 10:17:24 +00:00
input = re.sub(r'\bconstexpr\s+\'([^\']+)\'\s*',
r' _CoNsExP_/*\1@*/', input)
input = re.sub(r'\notherwise',
r'\n otherwise', input)
input = re.sub(r'(\n\s*\S[^\n]*\s)otherwise',
r'\1_OtheSaLi', input)
input = re.sub(r'@if\(', r'@iF(', input)
input = re.sub(r'@export', r'@eXpOrT', input)
input = re.sub(r'js-implicit[ \n]+', r'jS_iMpLiCiT_', input)
input = re.sub(r'^(\s*namespace\s+[a-zA-Z_0-9]+\s*{)(\s*)$', r'\1}\2', input, flags = re.MULTILINE)
# includes are not recognized, change them into comments so that the
# formatter ignores them first, until we can figure out a way to format cpp
# includes within a JS file.
input = re.sub(r'^#include', r'// InClUdE', input, flags=re.MULTILINE)
return input
def postprocess(output):
output = re.sub(r'\/\*COxp\*\/', r'constexpr', output)
output = re.sub(r'(\S+)\s*: type([,>])', r'\1: type\2', output)
output = re.sub(r'(\n\s*)labels( [A-Z])', r'\1 labels\2', output)
output = re.sub(r'\/\*_OPE \'([^\']+)\'\*\/', r"operator '\1'", output)
Reland "[torque] introduce JSAny type for user-accessible JavaScript values" Changes in the reland: Rebased and added a check that JavaScript-linkage builtins use JSAny in parameters and return type, plus the necessary cleanups for this test to pass. Design Doc: https://docs.google.com/document/d/1z6j0pWHnNIfId0v00uWN2HBrGRDJxJfYuCr5K7Kr1xA This reverts commit 4418a7b96afdf8278577211ab0ed555c41ee269e. Original change's description: > Revert "[torque] introduce JSAny type for user-accessible JavaScript values" > > This reverts commit 79b00555eac939bf4a00177ec59e21b856d7301c. > > Reason for revert: needs more discussion > > Original change's description: > > [torque] introduce JSAny type for user-accessible JavaScript values > > > > This CL introduces a JSAny type for user-exposed JavaScript values and > > a few new types to define it. Especially, it splits Symbol into > > PrivateSymbol (not exposed) and PublicSymbol (JavaScript exposed > > symbols). > > > > The change is mostly mechanical, but a few things are interesting: > > - PropertyKey and JSPrimitive were designed to coincide with the spec > > notions of IsPropertyKey() and primitive value, respectively. > > - Since Name is an open type, we define AnyName to be the known > > subtypes of Name. This is not too elegant, but by using AnyName > > instead of Name, typeswitch can properly conclude something if a > > subtype of Name is excluded. > > > > Small drive-by changes, which were necessary: > > - Allow subtyping on label parameters. > > - Fix the formatting of typeswitch, it was broken with union types > > in case types. > > > > Bug: v8:7793 > > Change-Id: I14b10507f8cf316ad85e048fe8d53d1df5e0bb13 > > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1735322 > > Commit-Queue: Tobias Tebbi <tebbi@chromium.org> > > Reviewed-by: Jakob Gruber <jgruber@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#63114} > > TBR=neis@chromium.org,jgruber@chromium.org,tebbi@chromium.org > > Change-Id: Ifde7881d74afe407628f40047997339d54cb2424 > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: v8:7793 > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1741652 > Reviewed-by: Tobias Tebbi <tebbi@chromium.org> > Commit-Queue: Tobias Tebbi <tebbi@chromium.org> > Cr-Commit-Position: refs/heads/master@{#63115} TBR=neis@chromium.org,jgruber@chromium.org,tebbi@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: v8:7793 Change-Id: Icca34e3824f55009b984d9348fd21884400f0081 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1769316 Commit-Queue: Tobias Tebbi <tebbi@chromium.org> Reviewed-by: Tobias Tebbi <tebbi@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Cr-Commit-Position: refs/heads/master@{#63395}
2019-08-26 12:07:01 +00:00
output = re.sub(r'\bif\s*\/\*tPsW\*\/', r'typeswitch', output)
output = re.sub(r'\bif\s*\/\*cA\*\/\s*(\([^{]*\))\s*{', r'case \1: {', output)
output = re.sub(r'\bif\s*\/\*cAsEdEfF\*\/\s*(\([^{]*\))\s*{', r'case \1: deferred {', output)
output = re.sub(r'\n_GeNeRaTeS00_\s*\/\*([^@]+)@\*\/',
r"\n generates '\1'", output)
output = re.sub(r'_GeNeRaTeS00_\s*\/\*([^@]+)@\*\/',
r"generates '\1'", output)
output = re.sub(r'_CoNsExP_\s*\/\*([^@]+)@\*\/',
r"constexpr '\1'", output)
output = re.sub(r'\n(\s+)otherwise',
r"\n\1 otherwise", output)
output = re.sub(r'\n(\s+)_OtheSaLi',
r"\n\1otherwise", output)
output = re.sub(r'_OtheSaLi',
r"otherwise", output)
output = re.sub(r'@iF\(', r'@if(', output)
output = re.sub(r'@eXpOrT',
r"@export", output)
output = re.sub(r'jS_iMpLiCiT_',
r"js-implicit ", output)
output = re.sub(r'}\n *label ', r'} label ', output);
output = re.sub(r'^(\s*namespace\s+[a-zA-Z_0-9]+\s*{)}(\s*)$', r'\1\2', output, flags = re.MULTILINE);
output = re.sub(kPercentEscape, r'%', output)
output = re.sub(kDerefEscape, r'*', output)
output = re.sub(kAddressofEscape, r'&', output)
output = re.sub( r'^// InClUdE',r'#include', output, flags=re.MULTILINE)
return output
def process(filename, lint, should_format):
with open(filename, 'r') as content_file:
content = content_file.read()
original_input = content
if sys.platform.startswith('win'):
p = Popen(['clang-format', '-assume-filename=.ts'], stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=True)
else:
p = Popen(['clang-format', '-assume-filename=.ts'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
output, err = p.communicate(preprocess(content))
output = postprocess(output)
rc = p.returncode
if (rc != 0):
print("error code " + str(rc) + " running clang-format. Exiting...")
sys.exit(rc);
if (output != original_input):
if lint:
print(filename + ' requires formatting', file=sys.stderr)
if should_format:
output_file = open(filename, 'wb')
output_file.write(output);
output_file.close()
def print_usage():
print('format-torque -i file1[, file2[, ...]]')
print(' format and overwrite input files')
print('format-torque -l file1[, file2[, ...]]')
print(' merely indicate which files need formatting')
def Main():
if len(sys.argv) < 3:
print("error: at least 2 arguments required")
print_usage();
sys.exit(-1)
def is_option(arg):
return arg in ['-i', '-l', '-il']
should_format = lint = False
use_stdout = True
flag, files = sys.argv[1], sys.argv[2:]
if is_option(flag):
if '-i' == flag:
should_format = True
elif '-l' == flag:
lint = True
else:
lint = True
should_format = True
else:
print("error: -i and/or -l flags must be specified")
print_usage();
sys.exit(-1);
for filename in files:
process(filename, lint, should_format)
return 0
if __name__ == '__main__':
sys.exit(Main());