History

David L. Jones 66e3562aaf Remove uses of pkg_resources in non-namespace packages. (#7902 ) In #713 and #1296, the `google` package in protobuf sources was found to cause conflicts with other Google projects, because it was not properly configured as a namespace package [1]. The initial fix in `786f80f` addressed part of the issue, and #1298 fixed the rest. However, `786f80f` (the initial fix) also made `google.protobuf` and `google.protobuf.pyext` into namespace packages. This was not correct: they are both regular, non-namespace, sub-subpackages. However (still), the follow-up #1298 did not nominate them as namespace packages, so the namespace registration behavior has remained, but without benefit. This change removes the unnecessary namespace registration, which has substantial overhead, thus reducing startup time substantially when using protobufs. Because this change affects the import internals, quantifying the overhead requires a full tear-down/start-up of the Python interpreter. So, to capture the full cost for every run, I measured the time to launching a _fresh_ Python instance in a subprocess, varying the imports and code under test. In other words, I used `timeit` to measure the time to launch a _fresh_ Python subprocess which actually performs the imports. * Reference: normal Python startup (i.e., don't import protobuf at all). ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "pass"])' 10 loops, best of 3: 27.1 msec per loop ``` * Baseline: cost to import `google.protobuf.descriptor`, with extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 133 msec per loop ``` * This change: cost to import `google.protobuf.descriptor`, without extraneous namespace packages. ``` % python3 -m timeit -s 'import subprocess' -r 3 -n 10 'subprocess.call(["python3", "-c", "import google.protobuf.descriptor"])' 10 loops, best of 3: 43.1 msec per loop ``` [1]: https://packaging.python.org/guides/packaging-namespace-packages/		2020-09-22 20:46:07 -07:00
..
compatibility_tests/v2.5.0	Fix lots of spelling errors (#7751 )	2020-08-10 11:08:25 -07:00
docs	Sync from Piper @308829107	2020-04-28 08:40:38 -07:00
google	Remove uses of pkg_resources in non-namespace packages. (#7902 )	2020-09-22 20:46:07 -07:00
protobuf_distutils	Update documentation for protobuf_distutils Python package.	2020-09-14 20:14:32 -07:00
release/wheel	Replace repo links.	2018-08-22 11:55:30 -07:00
.repo-metadata.json	python: publish sphinx docs to googleapis.dev	2020-02-11 13:55:20 -06:00
MANIFEST.in	Include .cc and .h files in source distribution	2017-11-17 15:30:31 -08:00
mox.py	Fix typos (#7050 )	2020-01-08 10:18:20 -08:00
README.md	Sync from Piper @327110949	2020-08-17 15:26:13 -07:00
release.sh	Sync from Piper @327110949	2020-08-17 15:26:13 -07:00
setup.cfg	add setup.cfg for building wheels	2016-07-29 14:02:07 -07:00
setup.py	python/setup.py: remove unneeded wheel dependency	2020-09-21 12:25:14 -07:00
stubout.py	import inspect (#3507 )	2018-06-11 14:17:46 -07:00
tox.ini	delete all duplicate empty blanks (#5758 )	2019-02-20 19:28:50 -08:00

README.md

Protocol Buffers - Google's data interchange format

This directory contains the Python Protocol Buffers runtime library.

Normally, this directory comes as part of the protobuf package, available from:

https://developers.google.com/protocol-buffers/

The complete package includes the C++ source code, which includes the Protocol Compiler (protoc). If you downloaded this package from PyPI or some other Python-specific source, you may have received only the Python part of the code. In this case, you will need to obtain the Protocol Compiler from some other source before you can use this package.

Development Warning

The pure python performance is slow. For better performance please use python c++ implementation.

Installation

Make sure you have Python 2.7 or newer. If in doubt, run:
```
$ python -V
```
If you do not have setuptools installed, note that it will be downloaded and installed automatically as soon as you run setup.py. If you would rather install it manually, you may do so by following the instructions on this page.
Build the C++ code, or install a binary distribution of protoc. If you install a binary distribution, make sure that it is the same version as this package. If in doubt, run:
```
$ protoc --version
```
Build and run the tests:
```
$ python setup.py build
$ python setup.py test
```
To build, test, and use the C++ implementation, you must first compile libprotobuf.so:
```
$ (cd .. && make)
```
On OS X:

If you are running a Homebrew-provided Python, you must make sure another version of protobuf is not already installed, as Homebrew's Python will search /usr/local/lib for libprotobuf.so before it searches ../src/.libs.

You can either unlink Homebrew's protobuf or install the libprotobuf you built earlier:
```
$ brew unlink protobuf
```
or
```
$ (cd .. && make install)
```
On other *nix:

You must make libprotobuf.so dynamically available. You can either install libprotobuf you built earlier, or set LD_LIBRARY_PATH:
```
$ export LD_LIBRARY_PATH=../src/.libs
```
or
```
$ (cd .. && make install)
```
To build the C++ implementation run:
```
$ python setup.py build --cpp_implementation
```
Then run the tests like so:
```
$ python setup.py test --cpp_implementation
```
If some tests fail, this library may not work correctly on your system. Continue at your own risk.

Please note that there is a known problem with some versions of Python on Cygwin which causes the tests to fail after printing the error: sem_init: Resource temporarily unavailable. This appears to be a bug either in Cygwin or in Python.

We do not know if or when it might be fixed. We also do not know how likely it is that this bug will affect users in practice.
Install:
```
$ python setup.py install
```
or:
```
$ (cd .. && make install)
$ python setup.py install --cpp_implementation
```
This step may require superuser privileges. NOTE: To use C++ implementation, you need to export an environment variable before running your program. See the "C++ Implementation" section below for more details.

Usage

The complete documentation for Protocol Buffers is available via the web at:

https://developers.google.com/protocol-buffers/

C++ Implementation

The C++ implementation for Python messages is built as a Python extension to improve the overall protobuf Python performance.

To use the C++ implementation, you need to install the C++ protobuf runtime library, please see instructions in the parent directory.