AuroraMiddleware/protobuf
1
0
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity

Merge pull request #1215 from haberman/commonjs

Browse Source 
Added support for CommonJS require()
This commit is contained in:
Joshua Haberman 2016-02-19 14:39:17 -08:00
commit 32daf513ce
14 changed files with 668 additions and 80 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

165
js/README.md
View File

@ -1,14 +1,159 @@
This directory contains Protocol Buffer support for JavaScript. This code works
in browsers and in Node.js.
Protocol Buffers - Google's data interchange format
===================================================
The packaging work for this is still in-progress. For now you can just run the
tests. First you need to build the main C++ distribution because the code
generator for JavaScript is written in C++:
[![Build Status](https://travis-ci.org/google/protobuf.svg?branch=master)](https://travis-ci.org/google/protobuf)
$ ./autogen.sh
$ ./configure
$ make
Copyright 2008 Google Inc.
Then you can run the JavaScript tests in this directory:
This directory contains the JavaScript Protocol Buffers runtime library.
$ cd js && gulp test
The library is currently compatible with:
1. CommonJS-style imports (eg. `var protos = require('my-protos');`)
2. Closure-style imports (eg. `goog.require('my.package.MyProto');`)
Support for ES6-style imports is not implemented yet. Browsers can
be supported by using Browserify, webpack, Closure Compiler, etc. to
resolve imports at compile time.
To use Protocol Buffers with JavaScript, you need two main components:
1. The protobuf runtime library. You can install this with
`npm install google-protobuf`, or use the files in this directory.
2. The Protocol Compiler `protoc`. This translates `.proto` files
into `.js` files. The compiler is not currently available via
npm, but you can download a pre-built binary
[on GitHub](https://github.com/google/protobuf/releases)
(look for the `protoc-*.zip` files under **Downloads**).
Setup
=====
First, obtain the Protocol Compiler. The easiest way is to download
a pre-built binary from [https://github.com/google/protobuf/releases](https://github.com/google/protobuf/releases).
If you want, you can compile `protoc` from source instead. To do this
follow the instructions in [the top-level
README](https://github.com/google/protobuf/blob/master/src/README.md).
Once you have `protoc` compiled, you can run the tests by typing:
$ cd js
$ npm install
$ npm test
# If your protoc is somewhere else than ../src/protoc, instead do this.
# But make sure your protoc is the same version as this (or compatible)!
$ PROTOC=/usr/local/bin/protoc npm test
This will run two separate copies of the tests: one that uses
Closure Compiler style imports and one that uses CommonJS imports.
You can see all the CommonJS files in `commonjs_out/`.
If all of these tests pass, you know you have a working setup.
Using Protocol Buffers in your own project
==========================================
To use Protocol Buffers in your own project, you need to integrate
the Protocol Compiler into your build system. The details are a
little different depending on whether you are using Closure imports
or CommonJS imports:
Closure Imports
---------------
If you want to use Closure imports, your build should run a command
like this:
$ protoc --js_out=library=myproto_libs,binary:. messages.proto base.proto
For Closure imports, `protoc` will generate a single output file
(`myproto_libs.js` in this example). The generated file will `goog.provide()`
all of the types defined in your .proto files. For example, for the unit
tests the generated files contain many `goog.provide` statements like:
goog.provide('proto.google.protobuf.DescriptorProto');
goog.provide('proto.google.protobuf.DescriptorProto.ExtensionRange');
goog.provide('proto.google.protobuf.DescriptorProto.ReservedRange');
goog.provide('proto.google.protobuf.EnumDescriptorProto');
goog.provide('proto.google.protobuf.EnumOptions');
The generated code will also `goog.require()` many types in the core library,
and they will require many types in the Google Closure library. So make sure
that your `goog.provide()` / `goog.require()` setup can find all of your
generated code, the core library `.js` files in this directory, and the
Google Closure library itself.
Once you've done this, you should be able to import your types with
statements like:
goog.require('proto.my.package.MyMessage');
var message = proto.my.package.MyMessage();
CommonJS imports
----------------
If you want to use CommonJS imports, your build should run a command
like this:
$ protoc --js_out=import_style=commonjs,binary:. messages.proto base.proto
For CommonJS imports, `protoc` will spit out one file per input file
(so `messages_pb.js` and `base_pb.js` in this example). The generated
code will depend on the core runtime, which should be in a file called
`google-protobuf.js`. If you are installing from `npm`, this file should
already be built and available. If you are running from GitHub, you need
to build it first by running:
$ gulp dist
Once you've done this, you should be able to import your types with
statements like:
var messages = require('./messages_pb');
var message = new messages.MyMessage();
The `--js_out` flag
-------------------
The syntax of the `--js_out` flag is:
--js_out=[OPTIONS:]output_dir
Where `OPTIONS` are separated by commas. Options are either `opt=val` or
just `opt` (for options that don't take a value). The available options
are specified and documented in the `GeneratorOptions` struct in
[src/google/protobuf/compiler/js/js_generator.h](https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/js/js_generator.h#L53).
Some examples:
- `--js_out=library=myprotos_lib.js,binary:.`: this contains the options
`library=myprotos.lib.js` and `binary` and outputs to the current directory.
The `import_style` option is left to the default, which is `closure`.
- `--js_out=import_style=commonjs,binary:protos`: this contains the options
`import_style=commonjs` and `binary` and outputs to the directory `protos`.
API
===
The API is not well-documented yet. Here is a quick example to give you an
idea of how the library generally works:
var message = new MyMessage();
message.setName("John Doe");
message.setAge(25);
message.setPhoneNumbers(["800-555-1212", "800-555-0000"]);
// Serializes to a UInt8Array.
bytes = message.serializeBinary();
var message2 = new MyMessage();
message2.deserializeBinary(bytes);
For more examples, see the tests. You can also look at the generated code
to see what methods are defined for your generated messages.

2
js/binary/proto_test.js
View File

@ -31,6 +31,8 @@
// Test suite is written using Jasmine -- see http://jasmine.github.io/
goog.require('goog.testing.asserts');
// CommonJS-LoadFromFile: testbinary_pb proto.jspb.test
goog.require('proto.jspb.test.ExtendsWithMessage');
goog.require('proto.jspb.test.ForeignEnum');
goog.require('proto.jspb.test.ForeignMessage');

22
js/commonjs/export.js Normal file
View File

@ -0,0 +1,22 @@
/**
* @fileoverview Export symbols needed by generated code in CommonJS style.
*
* This effectively is our canonical list of what we publicly export from
* the google-protobuf.js file that we build at distribution time.
*/
goog.require('goog.object');
goog.require('jspb.BinaryReader');
goog.require('jspb.BinaryWriter');
goog.require('jspb.ExtensionFieldInfo');
goog.require('jspb.Message');
exports.Message = jspb.Message;
exports.BinaryReader = jspb.BinaryReader;
exports.BinaryWriter = jspb.BinaryWriter;
exports.ExtensionFieldInfo = jspb.ExtensionFieldInfo;
// These are used by generated code but should not be used directly by clients.
exports.exportSymbol = goog.exportSymbol;
exports.inherits = goog.inherits;
exports.object = {extend: goog.object.extend};

37
js/commonjs/export_asserts.js Normal file
View File

@ -0,0 +1,37 @@
/**
* @fileoverview Exports symbols needed only by tests.
*
* This file exports several Closure Library symbols that are only
* used by tests. It is used to generate a file
* closure_asserts_commonjs.js that is only used at testing time.
*/
goog.require('goog.testing.asserts');
var global = Function('return this')();
// All of the closure "assert" functions are exported at the global level.
//
// The Google Closure assert functions start with assert, eg.
// assertThrows
// assertNotThrows
// assertTrue
// ...
//
// The one exception is the "fail" function.
function shouldExport(str) {
return str.lastIndexOf('assert') === 0 || str == 'fail';
}
for (var key in global) {
if ((typeof key == "string") && global.hasOwnProperty(key) &&
shouldExport(key)) {
exports[key] = global[key];
}
}
// The COMPILED variable is set by Closure compiler to "true" when it compiles
// JavaScript, so in practice this is equivalent to "exports.COMPILED = true".
// This will disable some debugging functionality in debug.js. We could
// investigate whether this can/should be enabled in CommonJS builds.
exports.COMPILED = COMPILED

9
js/commonjs/jasmine.json Normal file
View File

@ -0,0 +1,9 @@
{
"spec_dir": "",
"spec_files": [
"*_test.js",
"binary/proto_test.js"
],
"helpers": [
]
}

92
js/commonjs/rewrite_tests_for_commonjs.js Normal file
View File

@ -0,0 +1,92 @@
/**
* @fileoverview Utility to translate test files to CommonJS imports.
*
* This is a somewhat hacky tool designed to do one very specific thing.
* All of the test files in *_test.js are written with Closure-style
* imports (goog.require()). This works great for running the tests
* against Closure-style generated code, but we also want to run the
* tests against CommonJS-style generated code without having to fork
* the tests.
*
* Closure-style imports import each individual type by name. This is
* very different than CommonJS imports which are by file. So we put
* special comments in these tests like:
*
* // CommonJS-LoadFromFile: test_pb
* goog.require('proto.jspb.test.CloneExtension');
* goog.require('proto.jspb.test.Complex');
* goog.require('proto.jspb.test.DefaultValues');
*
* This script parses that special comment and uses it to generate proper
* CommonJS require() statements so that the tests can run and pass using
* CommonJS imports. The script will change the above statements into:
*
* var test_pb = require('test_pb');
* googleProtobuf.exportSymbol('proto.jspb.test.CloneExtension', test_pb.CloneExtension, global);
* googleProtobuf.exportSymbol('proto.jspb.test.Complex', test_pb.Complex, global);
* googleProtobuf.exportSymbol('proto.jspb.test.DefaultValues', test_pb.DefaultValues, global);
*
* (The "exportSymbol" function will define the given names in the global
* namespace, taking care not to overwrite any previous value for
* "proto.jspb.test").
*/
var lineReader = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
function tryStripPrefix(str, prefix) {
if (str.lastIndexOf(prefix) !== 0) {
throw "String: " + str + " didn't start with: " + prefix;
}
return str.substr(prefix.length);
}
function camelCase(str) {
var ret = '';
var ucaseNext = false;
for (var i = 0; i < str.length; i++) {
if (str[i] == '-') {
ucaseNext = true;
} else if (ucaseNext) {
ret += str[i].toUpperCase();
ucaseNext = false;
} else {
ret += str[i];
}
}
return ret;
}
var module = null;
var pkg = null;
lineReader.on('line', function(line) {
var isRequire = line.match(/goog\.require\('([^']*)'\)/);
var isLoadFromFile = line.match(/CommonJS-LoadFromFile: (\S*) (.*)/);
var isSetTestOnly = line.match(/goog.setTestOnly()/);
if (isRequire) {
if (module) { // Skip goog.require() lines before the first directive.
var fullSym = isRequire[1];
var sym = tryStripPrefix(fullSym, pkg);
console.log("googleProtobuf.exportSymbol('" + fullSym + "', " + module + sym + ', global);');
}
} else if (isLoadFromFile) {
if (!module) {
console.log("var googleProtobuf = require('google-protobuf');");
console.log("var asserts = require('closure_asserts_commonjs');");
console.log("var global = Function('return this')();");
console.log("");
console.log("// Bring asserts into the global namespace.");
console.log("googleProtobuf.object.extend(global, asserts);");
}
module = camelCase(isLoadFromFile[1])
pkg = isLoadFromFile[2];
if (module != "googleProtobuf") { // We unconditionally require this in the header.
console.log("var " + module + " = require('" + isLoadFromFile[1] + "');");
}
} else if (!isSetTestOnly) { // Remove goog.setTestOnly() lines.
console.log(line);
}
});

5
js/debug_test.js
View File

@ -31,13 +31,16 @@
goog.setTestOnly();
goog.require('goog.testing.asserts');
// CommonJS-LoadFromFile: google-protobuf
goog.require('jspb.debug');
// CommonJS-LoadFromFile: test_pb
goog.require('proto.jspb.test.HasExtensions');
goog.require('proto.jspb.test.IsExtension');
goog.require('proto.jspb.test.Simple1');
describe('debugTest', function() {
it('testSimple1', function() {
if (COMPILED) {

79
js/gulpfile.js
View File

@ -1,25 +1,79 @@
var gulp = require('gulp');
var exec = require('child_process').exec;
var glob = require('glob');
gulp.task('genproto', function (cb) {
exec('../src/protoc --js_out=library=testproto_libs,binary:. -I ../src -I . *.proto ../src/google/protobuf/descriptor.proto',
var protoc = process.env.PROTOC || '../src/protoc';
gulp.task('genproto_closure', function (cb) {
exec(protoc + ' --js_out=library=testproto_libs,binary:. -I ../src -I . *.proto ../src/google/protobuf/descriptor.proto',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
})
});
gulp.task('deps', ['genproto'], function (cb) {
gulp.task('genproto_commonjs', function (cb) {
exec('mkdir -p commonjs_out && ' + protoc + ' --js_out=import_style=commonjs,binary:commonjs_out -I ../src -I . *.proto ../src/google/protobuf/descriptor.proto',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
});
gulp.task('dist', function (cb) {
// TODO(haberman): minify this more aggressively.
// Will require proper externs/exports.
exec('./node_modules/google-closure-library/closure/bin/calcdeps.py -i message.js -i binary/reader.js -i binary/writer.js -i commonjs/export.js -p . -p node_modules/google-closure-library/closure -o compiled --compiler_jar node_modules/google-closure-compiler/compiler.jar > google-protobuf.js',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
});
gulp.task('commonjs_asserts', function (cb) {
exec('mkdir -p commonjs_out && ./node_modules/google-closure-library/closure/bin/calcdeps.py -i commonjs/export_asserts.js -p . -p node_modules/google-closure-library/closure -o compiled --compiler_jar node_modules/google-closure-compiler/compiler.jar > commonjs_out/closure_asserts_commonjs.js',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
});
gulp.task('make_commonjs_out', ['dist', 'genproto_commonjs', 'commonjs_asserts'], function (cb) {
// TODO(haberman): minify this more aggressively.
// Will require proper externs/exports.
var cmd = "mkdir -p commonjs_out/binary && ";
function addTestFile(file) {
cmd += 'node commonjs/rewrite_tests_for_commonjs.js < ' + file +
' > commonjs_out/' + file + '&& ';
}
glob.sync('*_test.js').forEach(addTestFile);
glob.sync('binary/*_test.js').forEach(addTestFile);
exec(cmd +
'cp commonjs/jasmine.json commonjs_out/jasmine.json && ' +
'cp google-protobuf.js commonjs_out',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
});
gulp.task('deps', ['genproto_closure'], function (cb) {
exec('./node_modules/google-closure-library/closure/bin/build/depswriter.py *.js binary/*.js > deps.js',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
})
});
gulp.task('test', ['genproto', 'deps'], function (cb) {
gulp.task('test_closure', ['genproto_closure', 'deps'], function (cb) {
exec('JASMINE_CONFIG_PATH=jasmine.json ./node_modules/.bin/jasmine',
function (err, stdout, stderr) {
console.log(stdout);
@ -27,3 +81,16 @@ gulp.task('test', ['genproto', 'deps'], function (cb) {
cb(err);
});
});
gulp.task('test_commonjs', ['make_commonjs_out'], function (cb) {
exec('cd commonjs_out && JASMINE_CONFIG_PATH=jasmine.json NODE_PATH=. ../node_modules/.bin/jasmine',
function (err, stdout, stderr) {
console.log(stdout);
console.log(stderr);
cb(err);
});
});
gulp.task('test', ['test_closure', 'test_commonjs'], function(cb) {
cb();
});

24
js/message_test.js
View File

@ -34,35 +34,47 @@ goog.setTestOnly();
goog.require('goog.json');
goog.require('goog.testing.asserts');
// CommonJS-LoadFromFile: google-protobuf jspb
goog.require('jspb.Message');
// CommonJS-LoadFromFile: test5_pb proto.jspb.exttest.beta
goog.require('proto.jspb.exttest.beta.floatingStrField');
// CommonJS-LoadFromFile: test3_pb proto.jspb.exttest
goog.require('proto.jspb.exttest.floatingMsgField');
// CommonJS-LoadFromFile: test4_pb proto.jspb.exttest
goog.require('proto.jspb.exttest.floatingMsgFieldTwo');
// CommonJS-LoadFromFile: test_pb proto.jspb.test
goog.require('proto.jspb.test.CloneExtension');
goog.require('proto.jspb.test.Complex');
goog.require('proto.jspb.test.DefaultValues');
goog.require('proto.jspb.test.Empty');
goog.require('proto.jspb.test.EnumContainer');
goog.require('proto.jspb.test.ExtensionMessage');
goog.require('proto.jspb.test.floatingMsgField');
goog.require('proto.jspb.test.floatingStrField');
goog.require('proto.jspb.test.HasExtensions');
goog.require('proto.jspb.test.IndirectExtension');
goog.require('proto.jspb.test.IsExtension');
goog.require('proto.jspb.test.OptionalFields');
goog.require('proto.jspb.test.OuterEnum');
goog.require('proto.jspb.test.OuterMessage.Complex');
goog.require('proto.jspb.test.simple1');
goog.require('proto.jspb.test.Simple1');
goog.require('proto.jspb.test.Simple2');
goog.require('proto.jspb.test.SpecialCases');
goog.require('proto.jspb.test.TestClone');
goog.require('proto.jspb.test.TestExtensionsMessage');
goog.require('proto.jspb.test.TestGroup');
goog.require('proto.jspb.test.TestGroup1');
goog.require('proto.jspb.test.TestMessageWithOneof');
goog.require('proto.jspb.test.TestReservedNames');
goog.require('proto.jspb.test.TestReservedNamesExtension');
// CommonJS-LoadFromFile: test2_pb proto.jspb.test
goog.require('proto.jspb.test.ExtensionMessage');
goog.require('proto.jspb.test.TestExtensionsMessage');
goog.require('proto.jspb.test.floatingMsgField');
@ -86,6 +98,12 @@ describe('Message test suite', function() {
assertEquals('some_bytes', data.getBytesField());
});
it('testNestedMessage', function() {
var msg = new proto.jspb.test.OuterMessage.Complex();
msg.setInnerComplexField(5);
assertObjectEquals({innerComplexField: 5}, msg.toObject());
});
it('testComplexConversion', function() {
var data1 = ['a',,, [, 11], [[, 22], [, 33]],, ['s1', 's2'],, 1];
var data2 = ['a',,, [, 11], [[, 22], [, 33]],, ['s1', 's2'],, 1];

7
js/package.json
View File

@ -2,13 +2,16 @@
"name": "google-protobuf",
"version": "3.0.0-alpha.5",
"description": "Protocol Buffers for JavaScript",
"main": "debug.js",
"main": "google-protobuf.js",
"dependencies": {
"google-closure-library": "~20160125.0.0",
"gulp": "~3.9.0",
"jasmine": "~2.4.1"
},
"devDependencies": {},
"devDependencies": {
"google-closure-compiler": "~20151216.2.0",
"glob": "~6.0.4"
},
"scripts": {
"test": "./node_modules/gulp/bin/gulp.js test"
},

4
js/proto3_test.js
View File

@ -29,7 +29,11 @@
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
goog.require('goog.testing.asserts');
// CommonJS-LoadFromFile: testbinary_pb proto.jspb.test
goog.require('proto.jspb.test.ForeignMessage');
// CommonJS-LoadFromFile: proto3_test_pb proto.jspb.test
goog.require('proto.jspb.test.Proto3Enum');
goog.require('proto.jspb.test.TestProto3');

7
js/test.proto
View File

@ -100,6 +100,13 @@ message Complex {
repeated string a_repeated_string = 7;
}
message OuterMessage {
// Make sure this doesn't conflict with the other Complex message.
message Complex {
optional int32 inner_complex_field = 1;
}
}
message IsExtension {
extend HasExtensions {
optional IsExtension ext_field = 100;

277
src/google/protobuf/compiler/js/js_generator.cc
View File

@ -134,12 +134,37 @@ bool IsReserved(const string& ident) {
// Returns a copy of |filename| with any trailing ".protodevel" or ".proto
// suffix stripped.
// TODO(robinson): Unify with copy in compiler/cpp/internal/helpers.cc.
string StripProto(const string& filename) {
const char* suffix = HasSuffixString(filename, ".protodevel")
? ".protodevel" : ".proto";
return StripSuffixString(filename, suffix);
}
// Given a filename like foo/bar/baz.proto, returns the correspoding JavaScript
// file foo/bar/baz.js.
string GetJSFilename(const string& filename) {
const char* suffix = HasSuffixString(filename, ".protodevel")
? ".protodevel" : ".proto";
return StripSuffixString(filename, suffix) + "_pb.js";
}
// Returns the alias we assign to the module of the given .proto filename
// when importing.
string ModuleAlias(const string& filename) {
// This scheme could technically cause problems if a file includes any 2 of:
// foo/bar_baz.proto
// foo_bar_baz.proto
// foo_bar/baz.proto
//
// We'll worry about this problem if/when we actually see it. This name isn't
// exposed to users so we can change it later if we need to.
string basename = StripProto(filename);
StripString(&basename, "-", '$');
StripString(&basename, "/", '_');
return basename + "_pb";
}
// Returns the fully normalized JavaScript path for the given
// file descriptor's package.
string GetPath(const GeneratorOptions& options,
@ -215,6 +240,26 @@ string GetPath(const GeneratorOptions& options,
value_descriptor->type()) + "." + value_descriptor->name();
}
string MaybeCrossFileRef(const GeneratorOptions& options,
const FileDescriptor* from_file,
const Descriptor* to_message) {
if (options.import_style == GeneratorOptions::IMPORT_COMMONJS &&
from_file != to_message->file()) {
// Cross-file ref in CommonJS needs to use the module alias instead of
// the global name.
return ModuleAlias(to_message->file()->name()) + "." + to_message->name();
} else {
// Within a single file we use a full name.
return GetPath(options, to_message);
}
}
string SubmessageTypeRef(const GeneratorOptions& options,
const FieldDescriptor* field) {
GOOGLE_CHECK(field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE);
return MaybeCrossFileRef(options, field->file(), field->message_type());
}
// - Object field name: LOWER_UNDERSCORE -> LOWER_CAMEL, except for group fields
// (UPPER_CAMEL -> LOWER_CAMEL), with "List" (or "Map") appended if appropriate,
// and with reserved words triggering a "pb_" prefix.
@ -952,11 +997,13 @@ string RelativeTypeName(const FieldDescriptor* field) {
}
string JSExtensionsObjectName(const GeneratorOptions& options,
const FileDescriptor* from_file,
const Descriptor* desc) {
if (desc->full_name() == "google.protobuf.bridge.MessageSet") {
// TODO(haberman): fix this for the IMPORT_COMMONJS case.
return "jspb.Message.messageSetExtensions";
} else {
return GetPath(options, desc) + ".extensions";
return MaybeCrossFileRef(options, from_file, desc) + ".extensions";
}
}
@ -1113,19 +1160,24 @@ void Generator::GenerateHeader(const GeneratorOptions& options,
"\n");
}
void Generator::FindProvidesForFile(const GeneratorOptions& options,
io::Printer* printer,
const FileDescriptor* file,
std::set<string>* provided) const {
for (int i = 0; i < file->message_type_count(); i++) {
FindProvidesForMessage(options, printer, file->message_type(i), provided);
}
for (int i = 0; i < file->enum_type_count(); i++) {
FindProvidesForEnum(options, printer, file->enum_type(i), provided);
}
}
void Generator::FindProvides(const GeneratorOptions& options,
io::Printer* printer,
const vector<const FileDescriptor*>& files,
std::set<string>* provided) const {
for (int i = 0; i < files.size(); i++) {
for (int j = 0; j < files[i]->message_type_count(); j++) {
FindProvidesForMessage(options, printer, files[i]->message_type(j),
provided);
}
for (int j = 0; j < files[i]->enum_type_count(); j++) {
FindProvidesForEnum(options, printer, files[i]->enum_type(j),
provided);
}
FindProvidesForFile(options, printer, files[i], provided);
}
printer->Print("\n");
@ -1204,38 +1256,45 @@ void Generator::GenerateRequires(const GeneratorOptions& options,
io::Printer* printer,
const vector<const FileDescriptor*>& files,
std::set<string>* provided) const {
std::set<string> required;
std::set<string> forwards;
bool have_extensions = false;
bool have_message = false;
if (options.import_style == GeneratorOptions::IMPORT_BROWSER) {
return;
} else if (options.import_style == GeneratorOptions::IMPORT_CLOSURE) {
// For Closure imports we need to import every message type individually.
std::set<string> required;
std::set<string> forwards;
bool have_extensions = false;
bool have_message = false;
for (int i = 0; i < files.size(); i++) {
for (int j = 0; j < files[i]->message_type_count(); j++) {
FindRequiresForMessage(options,
files[i]->message_type(j),
&required, &forwards, &have_message);
}
if (!have_extensions && HasExtensions(files[i])) {
have_extensions = true;
for (int i = 0; i < files.size(); i++) {
for (int j = 0; j < files[i]->message_type_count(); j++) {
FindRequiresForMessage(options,
files[i]->message_type(j),
&required, &forwards, &have_message);
}
if (!have_extensions && HasExtensions(files[i])) {
have_extensions = true;
}
for (int j = 0; j < files[i]->extension_count(); j++) {
const FieldDescriptor* extension = files[i]->extension(j);
if (IgnoreField(extension)) {
continue;
}
if (extension->containing_type()->full_name() !=
"google.protobuf.bridge.MessageSet") {
required.insert(GetPath(options, extension->containing_type()));
}
FindRequiresForField(options, extension, &required, &forwards);
have_extensions = true;
}
}
for (int j = 0; j < files[i]->extension_count(); j++) {
const FieldDescriptor* extension = files[i]->extension(j);
if (IgnoreField(extension)) {
continue;
}
if (extension->containing_type()->full_name() !=
"google.protobuf.bridge.MessageSet") {
required.insert(GetPath(options, extension->containing_type()));
}
FindRequiresForField(options, extension, &required, &forwards);
have_extensions = true;
}
GenerateRequiresImpl(options, printer, &required, &forwards, provided,
/* require_jspb = */ have_message,
/* require_extension = */ have_extensions);
} else if (options.import_style == GeneratorOptions::IMPORT_COMMONJS) {
// CommonJS imports are based on files
}
GenerateRequiresImpl(options, printer, &required, &forwards, provided,
/* require_jspb = */ have_message,
/* require_extension = */ have_extensions);
}
void Generator::GenerateRequires(const GeneratorOptions& options,
@ -1406,6 +1465,12 @@ void Generator::GenerateClass(const GeneratorOptions& options,
if (IsExtendable(desc) && desc->full_name() != "google.protobuf.bridge.MessageSet") {
GenerateClassExtensionFieldInfo(options, printer, desc);
}
if (options.import_style != GeneratorOptions:: IMPORT_CLOSURE) {
for (int i = 0; i < desc->extension_count(); i++) {
GenerateExtension(options, printer, desc->extension(i));
}
}
}
// Recurse on nested types.
@ -1623,7 +1688,7 @@ void Generator::GenerateClassToObject(const GeneratorOptions& options,
"obj,\n"
" $extObject$, $class$.prototype.getExtension,\n"
" includeInstance);\n",
"extObject", JSExtensionsObjectName(options, desc),
"extObject", JSExtensionsObjectName(options, desc->file(), desc),
"class", GetPath(options, desc));
}
@ -1652,13 +1717,13 @@ void Generator::GenerateClassFieldToObject(const GeneratorOptions& options,
printer->Print("jspb.Message.toObjectList(msg.get$getter$(),\n"
" $type$.toObject, includeInstance)",
"getter", JSGetterName(field),
"type", GetPath(options, field->message_type()));
"type", SubmessageTypeRef(options, field));
}
} else {
printer->Print("(f = msg.get$getter$()) && "
"$type$.toObject(includeInstance, f)",
"getter", JSGetterName(field),
"type", GetPath(options, field->message_type()));
"type", SubmessageTypeRef(options, field));
}
} else {
// Simple field (singular or repeated).
@ -1723,7 +1788,7 @@ void Generator::GenerateClassFieldFromObject(
" }));\n",
"name", JSObjectFieldName(field),
"index", JSFieldIndex(field),
"fieldclass", GetPath(options, field->message_type()));
"fieldclass", SubmessageTypeRef(options, field));
}
} else {
printer->Print(
@ -1731,7 +1796,7 @@ void Generator::GenerateClassFieldFromObject(
" msg, $index$, $fieldclass$.fromObject(obj.$name$));\n",
"name", JSObjectFieldName(field),
"index", JSFieldIndex(field),
"fieldclass", GetPath(options, field->message_type()));
"fieldclass", SubmessageTypeRef(options, field));
}
} else {
// Simple (primitive) field.
@ -1815,7 +1880,7 @@ void Generator::GenerateClassField(const GeneratorOptions& options,
/* always_singular = */ false),
"rpt", (field->is_repeated() ? "Repeated" : ""),
"index", JSFieldIndex(field),
"wrapperclass", GetPath(options, field->message_type()),
"wrapperclass", SubmessageTypeRef(options, field),
"required", (field->label() == FieldDescriptor::LABEL_REQUIRED ?
", 1" : ""));
printer->Print(
@ -2043,7 +2108,7 @@ void Generator::GenerateClassDeserializeBinary(const GeneratorOptions& options,
" $class$.prototype.getExtension,\n"
" $class$.prototype.setExtension);\n"
" break;\n",
"extobj", JSExtensionsObjectName(options, desc),
"extobj", JSExtensionsObjectName(options, desc->file(), desc),
"class", GetPath(options, desc));
} else {
printer->Print(
@ -2073,7 +2138,7 @@ void Generator::GenerateClassDeserializeBinaryField(
" var value = new $fieldclass$;\n"
" reader.read$msgOrGroup$($grpfield$value,"
"$fieldclass$.deserializeBinaryFromReader);\n",
"fieldclass", GetPath(options, field->message_type()),
"fieldclass", SubmessageTypeRef(options, field),
"msgOrGroup", (field->type() == FieldDescriptor::TYPE_GROUP) ?
"Group" : "Message",
"grpfield", (field->type() == FieldDescriptor::TYPE_GROUP) ?
@ -2149,7 +2214,7 @@ void Generator::GenerateClassSerializeBinary(const GeneratorOptions& options,
printer->Print(
" jspb.Message.serializeBinaryExtensions(this, writer, $extobj$,\n"
" $class$.prototype.getExtension);\n",
"extobj", JSExtensionsObjectName(options, desc),
"extobj", JSExtensionsObjectName(options, desc->file(), desc),
"class", GetPath(options, desc));
}
@ -2222,7 +2287,7 @@ void Generator::GenerateClassSerializeBinaryField(
printer->Print(
",\n"
" $submsg$.serializeBinaryToWriter\n",
"submsg", GetPath(options, field->message_type()));
"submsg", SubmessageTypeRef(options, field));
} else {
printer->Print("\n");
}
@ -2290,9 +2355,9 @@ void Generator::GenerateExtension(const GeneratorOptions& options,
"index", SimpleItoa(field->number()),
"name", JSObjectFieldName(field),
"ctor", (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE ?
GetPath(options, field->message_type()) : string("null")),
SubmessageTypeRef(options, field) : string("null")),
"toObject", (field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE ?
(GetPath(options, field->message_type()) + ".toObject") :
(SubmessageTypeRef(options, field) + ".toObject") :
string("null")),
"repeated", (field->is_repeated() ? "1" : "0"));
@ -2308,11 +2373,11 @@ void Generator::GenerateExtension(const GeneratorOptions& options,
"binaryWriterFn", JSBinaryWriterMethodName(field),
"binaryMessageSerializeFn",
(field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) ?
(GetPath(options, field->message_type()) +
(SubmessageTypeRef(options, field) +
".serializeBinaryToWriter") : "null",
"binaryMessageDeserializeFn",
(field->cpp_type() == FieldDescriptor::CPPTYPE_MESSAGE) ?
(GetPath(options, field->message_type()) +
(SubmessageTypeRef(options, field) +
".deserializeBinaryFromReader") : "null",
"isPacked", (field->is_packed() ? "true" : "false"));
} else {
@ -2324,7 +2389,8 @@ void Generator::GenerateExtension(const GeneratorOptions& options,
"// toObject() will function correctly.\n"
"$extendName$[$index$] = $class$.$name$;\n"
"\n",
"extendName", JSExtensionsObjectName(options, field->containing_type()),
"extendName", JSExtensionsObjectName(options, field->file(),
field->containing_type()),
"index", SimpleItoa(field->number()),
"class", extension_scope,
"name", JSObjectFieldName(field));
@ -2364,6 +2430,19 @@ bool GeneratorOptions::ParseFromOptions(
namespace_prefix = options[i].second;
} else if (options[i].first == "library") {
library = options[i].second;
} else if (options[i].first == "import_style") {
if (options[i].second == "closure") {
import_style = IMPORT_CLOSURE;
} else if (options[i].second == "commonjs") {
import_style = IMPORT_COMMONJS;
} else if (options[i].second == "browser") {
import_style = IMPORT_BROWSER;
} else if (options[i].second == "es6") {
import_style = IMPORT_ES6;
} else {
*error = "Unknown import style " + options[i].second + ", expected " +
"one of: closure, commonjs, browser, es6.";
}
} else {
// Assume any other option is an output directory, as long as it is a bare
// `key` rather than a `key=value` option.
@ -2375,6 +2454,11 @@ bool GeneratorOptions::ParseFromOptions(
}
}
if (!library.empty() && import_style != IMPORT_CLOSURE) {
*error = "The library option should only be used for "
"import_style=closure";
}
return true;
}
@ -2418,6 +2502,63 @@ void Generator::GenerateFileAndDeps(
}
}
void Generator::GenerateFile(const GeneratorOptions& options,
io::Printer* printer,
const FileDescriptor* file) const {
GenerateHeader(options, printer);
// Generate "require" statements.
if (options.import_style == GeneratorOptions::IMPORT_COMMONJS) {
printer->Print("var jspb = require('google-protobuf');\n");
printer->Print("var goog = jspb;\n");
printer->Print("var global = Function('return this')();\n\n");
for (int i = 0; i < file->dependency_count(); i++) {
const std::string& name = file->dependency(i)->name();
printer->Print(
"var $alias$ = require('$file$');\n",
"alias", ModuleAlias(name),
"file", GetJSFilename(name));
}
}
// We aren't using Closure's import system, but we use goog.exportSymbol()
// to construct the expected tree of objects, eg.
//
// goog.exportSymbol('foo.bar.Baz', null, this);
//
// // Later generated code expects foo.bar = {} to exist:
// foo.bar.Baz = function() { /* ... */ }
std::set<std::string> provided;
// Cover the case where this file declares extensions but no messages.
// This will ensure that the file-level object will be declared to hold
// the extensions.
for (int i = 0; i < file->extension_count(); i++) {
provided.insert(file->extension(i)->full_name());
}
FindProvidesForFile(options, printer, file, &provided);
for (std::set<string>::iterator it = provided.begin();
it != provided.end(); ++it) {
printer->Print("goog.exportSymbol('$name$', null, global);\n",
"name", *it);
}
GenerateClassesAndEnums(options, printer, file);
// Extensions nested inside messages are emitted inside
// GenerateClassesAndEnums().
for (int i = 0; i < file->extension_count(); i++) {
GenerateExtension(options, printer, file->extension(i));
}
if (options.import_style == GeneratorOptions::IMPORT_COMMONJS) {
printer->Print("goog.object.extend(exports, $package$);\n",
"package", GetPath(options, file));
}
}
bool Generator::GenerateAll(const vector<const FileDescriptor*>& files,
const string& parameter,
GeneratorContext* context,
@ -2430,10 +2571,14 @@ bool Generator::GenerateAll(const vector<const FileDescriptor*>& files,
}
// We're either generating a single library file with definitions for message
// and enum types in *all* FileDescriptor inputs, or we're generating a single
// file for each type.
if (options.library != "") {
// There are three schemes for where output files go:
//
// - import_style = IMPORT_CLOSURE, library non-empty: all output in one file
// - import_style = IMPORT_CLOSURE, library empty: one output file per type
// - import_style != IMPORT_CLOSURE: one output file per .proto file
if (options.import_style == GeneratorOptions::IMPORT_CLOSURE &&
options.library != "") {
// All output should go in a single file.
string filename = options.output_dir + "/" + options.library + ".js";
google::protobuf::scoped_ptr<io::ZeroCopyOutputStream> output(context->Open(filename));
GOOGLE_CHECK(output.get());
@ -2469,7 +2614,7 @@ bool Generator::GenerateAll(const vector<const FileDescriptor*>& files,
if (printer.failed()) {
return false;
}
} else {
} else if (options.import_style == GeneratorOptions::IMPORT_CLOSURE) {
// Collect all types, and print each type to a separate file. Pull out
// free-floating extensions while we make this pass.
map< string, vector<const FieldDescriptor*> > extensions_by_namespace;
@ -2611,6 +2756,24 @@ bool Generator::GenerateAll(const vector<const FileDescriptor*>& files,
}
}
}
} else {
// Generate one output file per input (.proto) file.
for (int i = 0; i < files.size(); i++) {
const google::protobuf::FileDescriptor* file = files[i];
string filename = options.output_dir + "/" + GetJSFilename(file->name());
google::protobuf::scoped_ptr<io::ZeroCopyOutputStream> output(
context->Open(filename));
GOOGLE_CHECK(output.get());
io::Printer printer(output.get(), '$');
GenerateFile(options, &printer, file);
if (printer.failed()) {
return false;
}
}
}
return true;

18
src/google/protobuf/compiler/js/js_generator.h
View File

@ -67,6 +67,13 @@ struct GeneratorOptions {
bool error_on_name_conflict;
// Enable binary-format support?
bool binary;
// What style of imports should be used.
enum ImportStyle {
IMPORT_CLOSURE, // goog.require()
IMPORT_COMMONJS, // require()
IMPORT_BROWSER, // no import statements
IMPORT_ES6, // import { member } from ''
} import_style;
GeneratorOptions()
: add_require_for_enums(false),
@ -75,7 +82,8 @@ struct GeneratorOptions {
namespace_prefix(""),
library(""),
error_on_name_conflict(false),
binary(false) {}
binary(false),
import_style(IMPORT_CLOSURE) {}
bool ParseFromOptions(
const vector< pair< string, string > >& options,
@ -111,6 +119,10 @@ class LIBPROTOC_EXPORT Generator : public CodeGenerator {
io::Printer* printer,
const vector<const FileDescriptor*>& file,
std::set<string>* provided) const;
void FindProvidesForFile(const GeneratorOptions& options,
io::Printer* printer,
const FileDescriptor* file,
std::set<string>* provided) const;
void FindProvidesForMessage(const GeneratorOptions& options,
io::Printer* printer,
const Descriptor* desc,
@ -168,6 +180,10 @@ class LIBPROTOC_EXPORT Generator : public CodeGenerator {
std::set<string>* required,
std::set<string>* forwards) const;
void GenerateFile(const GeneratorOptions& options,
io::Printer* printer,
const FileDescriptor* file) const;
// Generate definitions for all message classes and enums in all files,
// processing the files in dependence order.
void GenerateFilesInDepOrder(const GeneratorOptions& options,