Newer versions of virtualenv lack the --no-site-packages option, so I
had to remove it to keep the release.sh script working. I read that this
option has already been the default for a long time, so removing it
shouldn't chany any behavior.
For the setup.py script, I was getting some errors about the bdist_wheel
argument to setup.py, but I was able to fix that by adding 'wheel' to
setup_requires.
Since our (public) generated APIs document their use of repeated fields, we should probably document its API, too.
This includes some changes to generate_docs.py to explicitly allow named modules, since the repeated field implementation lives in the "internal" tree.
* Set execute bit on files if and only if they begin with (#!).
Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.
We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.
* Added fix_permissions.sh script to set +x iff a file begins with (#!).
This change updates docstrings and comments so that they will produce nicer
formatting and cross-references from Sphinx. There are a few broad categories of
changes:
- Paramter and attribute docs are updated so that types will be recognized by
Napoleon (https://sphinxcontrib-napoleon.readthedocs.io/en/latest/) This
usually just means moving a colon in the docstring, so
`name: (type) description` becomes `name (type): description`.
- References to other symbols can be cross-references if they have the right
format. For example, "attr_name" might become ":attr:`attr_name`".
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#cross-referencing-python-objects
- For fenced code blocks, adding a double-colon `::` signifies a literal block.
https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#literal-blocks
- Some bits of docstrings move from docstring to comments. For TODOs, this
means we won't be putting stale (or otherwise unrelated) noise into the docs.
For `Message.DESCRIPTOR`, the change means it gets appropriate documentation.
- There are some wording tweaks for consistency, and some new docstrings
(especially for methods in Message).
For types, I used the convention of `list[Foo]` and `dict(foo, bar)`, which seem
to be common among other Python rst docstrings. Sphinx should generally
recognize both, and cross-links them correctly (both internally and to Python
library documentation). Upgrading to Python3-style type annotations would allow
us to use `sphinx-autodoc-typehints`; the changes in this commit are very
similar to typing-based hints.
Background:
This is a follow-up to the PR that adds sphinx docs.
Read the Docs is a hosting platform for documentation, primarily Python
docs. It supports builds at commit time as well as at specific git
labels to support versioned docs. I have claimed the
protobuf.readthedocs.io project and can add any Googlers who need access
to be able to configure and trigger builds.
https://readthedocs.org/projects/protobuf/builds/ It's also relatively
easy to create a new project to test the documentation builds from a
fork, such as https://readthedocs.org/projects/tswast-protobuf/builds/
About this change:
Once web hooks are configured, Read the Docs will automatically build
the docs for the latest changes on the master branch.
I needed to update `python/setup.py` to support installation from the
root of the repository because Read the Docs does not `cd python` before
installing the protobuf package with `setup.py install`. To support
this, I updated the file paths to use the absolute path to files. The
`__file__` special variable comes in handy for this, as it provides the
path to the `setup.py` file.
A banner is added to the docs when published to readthedocs. This links
to the official documentation and the future home of the stable API
reference on googleapis.dev.
**Background:**
This is a follow-up to the PR that adds sphinx docs.
Googleapis.dev hosts client library documentation for several languages as
well as documentation for core packages such as google-api-core. It is owned
and maintained by the Cloud DevRel Core team.
**About this change:**
The `kokoro/docs/publish-python.sh` script builds the docs with sphinx, then
uses the [googleapis/docuploader](https://github.com/googleapis/docuploader)
tool to publish to googleapis.dev. The publish script is triggered manually
with a Kokoro RELEASE type job. Googlers, see go link
protobuf-docs-python-design for additional internal references.
* python: generate documentation with Sphinx and Read the Docs
Background:
Formerly, the Python protobuf reference documentation was built with
[Epydoc](http://epydoc.sourceforge.net/). This package has not been
updated since 2008, and it has inconsistent formatting (see internal
issue 131415575) with most Python documentation. Sphinx is used for the
official docs.python.org docs as well as most other Python packages,
including the Google client libraries and related packages, such as
https://googleapis.dev/python/google-api-core/latest/
To build the docs with Sphinx:
1. Install the needed packages (`sphinx`, `sphinxcontrib-napoleon` for
Google-style docstring support). I've created a conda environment file
to make this easier:
```
conda env create -f python/docs/environment.yml
```
2. (Optional) Generate reference docs files and regenerate index:
```
cd python
python generate_docs.py
cd ..
```
3. Run Sphinx.
```
cd python/docs
make html
```
About this change:
The script at `python/generate_docs.py` creates a ReStructured Text file
for each public module in the protobuf Python package. The script also
updates the table of contents in `python/docs/index.rst` to point to
these module references.
Future work:
Testing the docs build on PRs requires contributors to actually do some
setup work to configure builds on their fork. It'd be better if CI had a
docs build session to verify that the Sphinx docs generation at least
runs.
There are many warnings due to not-quite-correct docstrings in the
actual Python code itself. I'm choosing to ignore these errors to keep
the PR small, but I recommend you fix these and then enable "fail on
warnings" in the docs build on CI.
* add docs to EXTRA_DIST
* add instructions to build documentation to generate_docs.py
* exclude python/odcs from cpp_distcheck
