Commit Graph

144 Commits

Author SHA1 Message Date
David L. Jones
b774da0d42
Change Python kokoro configs to test only their specified version. (#7927)
Using non-versioned scripts for `build_file` means every Python version is
tested (under Tox), and this is multiplied by each Python version running Tox.

For example, when the `python38` config is used to run Tox, the
Tox layer runs on Python 3.8. This then tests all of the Python versions
individually, including, for example, python27 tests.

This change fixes the `build_file` paths to point to the same-pathed build
script. For example, `kokoro/linux/python27/presubmit.cfg` now uses
`kokoro/linux/python27/build.sh`.

Some additional fixes:

*  Use `python -m tox` in tests.sh instead of just `tox`. This helps non-site
   installations of tox, where the `tox` script may not be on `$PATH`.
*  Ensure tox (and other Python build-related packages) are available in
   Python testing images. (New images have been pushed.)
*  Disable `--warnings_as_errors` due to deprecated function.
*  Remove apt lists per [Docker best practices][1].

[1]: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
2020-09-30 15:56:17 -07:00
Adam Cozzette
58f4260e8e Merged 3.13.x into master 2020-08-17 11:06:11 -07:00
Paul Yang
d4ca92962c
Port php c extension to php8 (#7793)
* Only ported c extension to php8.
* Didn't fixed the issue of throwing warnings for missing arginfo in bundled files.
* Tests not fixed, because syntax of phpunit (<7 vs >9.3) are not compatible.
* In next release, needs to drop php5 and php7.0 support (in order to use phpunit > 7)
2020-08-11 19:30:46 -07:00
Bo Yang
8947701be5 Add docker file for php8.0 2020-08-10 13:51:06 -07:00
Peter Newman
e2cc2de304
Fix lots of spelling errors (#7751)
* Fix a typo

* Fix lots of spelling errors

* Fix a few more spelling mistakes

* s/parsable/parseable/

* Don't touch the third party files

* Cloneable is the preferred C# term

* Copyable is the preferred C++ term

* Revert "s/parsable/parseable/"

This reverts commit 534ecf7675.

* Revert unparseable->unparsable corrections
2020-08-10 11:08:25 -07:00
David L. Jones
0f9ae07fe0 Override pip and virtualenv installation for macOS. (#7704)
This commit cleans up our environment setup for macOS builds.

-  Always use `python -m {pip | virtualenv}` form of commands.
-  Only upgrade in-place, and never use `--ignore-installed`.
-  Use latest tox on macos, and test 3.6 on macos presubmit.

These changes ensure that multibuilder won't break the system-installed Python on the Kokoro macOS builder, and thus won't resolve the ancient system version. For Tox, make sure we test with Python 3.6 (the version of Python in current macOS).
2020-08-07 14:44:49 -07:00
Jan Tattermusch
fed7e18193 upgrade dotnet SDK in the CI to 3.1 2020-06-30 08:53:31 -04:00
Thomas Van Lenten
ce8b7af46b Update the brew workflow
- Remove the `brew install`; the kokoro image already has it, just
  update/upgrade instead.
- Remove `prune`, logs had:
    """
    Error: Unknown command: prune
    """
- Remove `uninstall`, logs had:
    """
    Error: Refusing to uninstall /usr/local/Cellar/cmake/3.16.4 and /usr/local/Cellar/icu4c/64.2
    because they are required by ceres-solver, ffmpeg, harfbuzz, libass and opencv, which are currently installed.
    You can override this and force removal with:
      brew uninstall --ignore-dependencies node icu4c cmake wget
    """
- Skip installing some things since they are already in the base image, logs
  had:
    """
    Warning: gflags 2.2.2 is already installed and up-to-date
    To reinstall 2.2.2, run `brew reinstall gflags`
    Warning: openssl@1.1 1.1.1g is already installed and up-to-date
    To reinstall 1.1.1g, run `brew reinstall openssl@1.1`
    Warning: pcre 8.44 is already installed and up-to-date
    To reinstall 8.44, run `brew reinstall pcre`
    """
- Don't install gpg gpg2 as gnupg is already installed, also use gpg instead of
  gpg2 for commands (and update the commands), logs had:
    """
    kokoro/macos/prepare_build_macos_rc: line 44: gpg2: command not found
    kokoro/macos/prepare_build_macos_rc: line 45: gpg2: command not found
    """
- Add env guards to control all the option installs and only request them be
  installed in the cases that need it.  This avoids having to install/update
  the things like ruby when some other tool only needed in some configs is
  install differently and could have conflicts.
- Switch to brew for cocoapods to avoid compat issues on the supporting
  libraries.
2020-06-09 09:57:17 -04:00
Joshua Haberman
c077d949d8 Merge branch '3.12.x' into merge-312 2020-05-26 16:45:43 -07:00
Stanley Cheung
ef22086980
Fix :protobuf_objc bazel target (#7538)
* Potential 3.12 ObjC bazel patch

* Add test targets to bazel test

* Remove :protobuf_objc test target
2020-05-21 12:46:35 -07:00
Daniel Azuma
cf81f38b5d
Restore binary builds of Ruby 2.3 and 2.4 (#7529) (#7531) 2020-05-19 15:52:59 -07:00
Daniel Azuma
ba1ce70744
Restore binary builds of Ruby 2.3 and 2.4 (#7529) 2020-05-19 15:28:57 -07:00
Joshua Haberman
27ba0c5bb0
Fixed the PHP Kokoro build. (#7503)
* Fixed the PHP Kokoro build.

* Renamed all PHP7.4 Mac tests -> PHP7.3.
2020-05-13 15:36:21 -07:00
Joshua Haberman
e19bfc0e28
Stop building binary gems for ruby <2.5. (#7453)
It is no longer possible to "bundle install" these versions:

$ bundle install
Fetching gem metadata from https://rubygems.org/.................
reline-0.1.3 requires ruby version >= 2.5, which is incompatible with the current version, ruby 2.4.9p362
2020-05-01 12:35:09 -07:00
Masaki Hara
f494cb222d
Test in Ruby 2.7 (#7386)
* Test in Ruby 2.7

* Turn off GC.stress for 'require' in Ruby 2.7

* Backport 2c8364b to compatibility_tests/v3.0.0
2020-04-24 09:50:27 -07:00
Paul Yang
f15ac4091f
Drop 3.3, 3.4 and use single version docker images for all python tests (#7396)
* Create a new docker image for python 3.6

The previous one cannot build, because python3.6 was removed from
stretch.

* Drop support for 3.3 and 3.4

Also update all docker file to be single version
2020-04-18 14:39:52 -07:00
Masaki Hara
64f6c59ccb
Build extensions for Ruby 2.7 (#7027)
* Build extensions for Ruby 2.7

* Try installing bundler 2.x

* Try bumping rake-compiler-dock

* Use standard RCD images

* Avoid 'rake cross native' with rake-compiler-dock

* Use Ruby 2.5 for building Ruby <= 2.6

* Use rake-compiler 1.1.0

* Specify target

* Don't update Ruby test image for now
2020-04-16 23:31:47 -07:00
Joshua Haberman
c649397029
Set execute bit on files if and only if they begin with (#!). (#7347)
* Set execute bit on files if and only if they begin with (#!).

Git only tracks the 'x' (executable) bit on each file. Prior to this
CL, our files were a random mix of executable and non-executable.
This change imposes some order by making files executable if and only
if they have shebang (#!) lines at the beginning.

We don't have any executable binaries checked into the repo, so
we shouldn't need to worry about that case.

* Added fix_permissions.sh script to set +x iff a file begins with (#!).
2020-04-01 15:28:25 -07:00
David L. Jones
42903f87f0
Merge pull request #7154 from tswast/issue4498-googleapis.dev
python: publish sphinx docs to googleapis.dev
2020-02-27 14:30:54 -08:00
Rafi Kamal
0eb476b696 Merge branch 3.11.x to master 2020-02-18 16:42:47 -08:00
Rafi Kamal
df2bce345d
Remove 32-bit Mac protoc reference from collect_all_artifacts (#7228) 2020-02-18 12:55:01 -08:00
Rafi Kamal
d314101531
Remove protoc release for 32-bit Macs (#7209)
* Remove protoc release for 32-bit Macs 

Apple has been removing the support for 32-bit Mac apps: https://support.apple.com/en-us/HT208436

Our release infrastructure no longer supports building for 32-bit architecture.

* Remove protoc artifact for MacOS 32-bit
2020-02-13 14:00:19 -08:00
Yannic
f0cb9cdb95
[bazel] Move Java runtime/toolchains into //java (#7190)
* [bazel] Move Java runtime/toolchains into //java

This change moves `java_library` targets from the top-level BUILD file
into `//java/{core,lite,util}` and declares `alias` targets to point to
their new locations (hence, this is not a breaking change).

This will allow users that don't use Java to stop depending on
`@rules_java` (e.g. as requested in
https://github.com/bazelbuild/rules_scala/pull/989#issuecomment-583405161).

Note that there is no intention to deprecate + remove the top-level
targets in the foreseeable future.

* Add BUILD files in //java to java_EXTRA_DIST
2020-02-13 13:04:14 -08:00
Rafi Kamal
1440569231
Update the Xcode version number in Kokoro (#7202) (#7203)
Currently all of our MacOS tests are failing with the error: "autom4te: need GNU m4 1.4 or later: /usr/bin/m4". This is likely to be happening because Kokoro has updated their Xcode version to 11.3.
2020-02-12 11:34:51 -08:00
Rafi Kamal
a24e594833
Update the Xcode version number in Kokoro (#7202)
Currently all of our MacOS tests are failing with the error: "autom4te: need GNU m4 1.4 or later: /usr/bin/m4". This is likely to be happening because Kokoro has updated their Xcode version to 11.3.
2020-02-12 11:28:38 -08:00
Adam Cozzette
802e1b99b2 Attempt to mitigate "docker pull" failures
Lots of CI tests have been failing recently with error messages like
this from "docker pull":

  Error response from daemon: unexpected EOF

This seems to be an issue affecting other projects as well:
https://forums.docker.com/t/docker-pull-causes-error-response-from-daemon-unexpected-eof/88608

This commit tries to mitigate the problem by periodically retrying
failed Docker pulls.
2020-02-11 12:59:49 -08:00
Tim Swast
d1f4e25e15
python: publish sphinx docs to googleapis.dev
**Background:**

This is a follow-up to the PR that adds sphinx docs.

Googleapis.dev hosts client library documentation for several languages as
well as documentation for core packages such as google-api-core. It is owned
and maintained by the Cloud DevRel Core team.

**About this change:**

The `kokoro/docs/publish-python.sh` script builds the docs with sphinx, then
uses the [googleapis/docuploader](https://github.com/googleapis/docuploader)
tool to publish to googleapis.dev. The publish script is triggered manually
with a Kokoro RELEASE type job. Googlers, see go link
protobuf-docs-python-design for additional internal references.
2020-02-11 13:55:20 -06:00
Rafi Kamal
bc237eb672
Install RubyGem bundler version specified in Gemfile.lock to fix release failure (#7144) (#7156)
Ref: https://bundler.io/blog/2019/05/14/solutions-for-cant-find-gem-bundler-with-executable-bundle.html
2020-01-31 09:43:58 -08:00
Rafi Kamal
4cf5bfee95
Install RubyGem bundler version specified in Gemfile.lock to fix release failure (#7144)
* Update Gem version to fix Ruby release failure

Ref: https://bundler.io/blog/2019/05/14/solutions-for-cant-find-gem-bundler-with-executable-bundle.html
2020-01-30 13:08:31 -08:00
Paul Yang
85219578fa
Add continuous test for php7.4 on mac (#7153) 2020-01-30 12:45:47 -08:00
Yannic
948740bc9d [bazel] Fix blacklisted_protos in cc_toolchain and add test (#7075) 2020-01-15 10:27:35 -08:00
Yannic
29cd005ce1 [bazel] Fix blacklisted_protos in cc_toolchain and add test (#7096) 2020-01-15 10:22:16 -08:00
salamaniibm
c3ffdabed5 correcting the s390x Arch name 2020-01-10 16:07:15 -08:00
Rafi Kamal
ed8688de72
Merge pull request #7064 from rafi-kamal/3.11.x-202001071136
Merge branch 3.11.x to master
2020-01-08 12:24:20 -08:00
Brian Wignall
a104dffcb6 Fix typos (#7050)
Uses https://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings/For_machines to find likely typos, with https://github.com/bwignall/typochecker to help automate the checking.
2020-01-08 10:18:20 -08:00
Rafi Kamal
4500f31a6a Merge branch 3.11.x to master 2020-01-07 11:52:29 -08:00
Jie Luo
63cfdafacb
Fix python release breakages by set MB_PYTHON_OSX_VER (#7038) 2019-12-19 14:06:29 -08:00
Jie Luo
03ba4ac171
Add python3.8 in build_artifacts for linux and macos (#7019) 2019-12-19 10:27:39 -08:00
Jie Luo
7faab5eeeb add python38 (#7009)
* add python38

* add python3.8

* fix tox
2019-12-13 15:49:18 -08:00
Paul Yang
7bb8b108d1
Add php 7.4 to docker image (#6971) (#6975)
* Add php 7.4 to docker image

* Use https links
2019-12-02 16:05:55 -08:00
Paul Yang
ef210164aa
Add php 7.4 to docker image (#6971)
* Add php 7.4 to docker image

* Use https links
2019-12-02 15:33:05 -08:00
Jan Tattermusch
96e9ff3863 Revert "Revert "C# upgrade dotnet SDK (#6877)" (#6888)" (#6920)
This reverts commit f06800524f.
2019-11-25 11:29:47 -08:00
Rafi Kamal
bb0c543934
Update Xcode version needed to run Kokoro tests and replace deprecated rubygem options (#6890)
* Update Xcode version needed to run Kokoro tests

* Replace deprecated --no-ri --no-rdoc rubygem options with --no-document
2019-11-14 13:44:33 -08:00
Jie Luo
f06800524f
Revert "C# upgrade dotnet SDK (#6877)" (#6888)
This reverts commit 0c8acb6c8d.
2019-11-14 09:49:59 -08:00
Jan Tattermusch
0c8acb6c8d C# upgrade dotnet SDK (#6877)
* upgrade dotnet SDK

* upgrade dotnet SDK version in global.json
2019-11-14 09:39:31 -08:00
salamaniibm
7ba4a99e00 adding protoc support for s390x 2019-08-30 11:44:43 -07:00
Paul Yang
fa8f155dcf
Reinstall wget on mac (#6505) 2019-08-13 11:36:17 -07:00
Yannic Bonenberger
edbc27fb81 Print logs of failing tests 2019-07-24 10:23:55 -07:00
Yannic Bonenberger
4979489451 Add test to verify BUILD- and cmake-files are in sync with src/Makefile.am 2019-07-24 10:23:55 -07:00
Paul Yang
d28cf02ee0 Drop building wheel for python 3.4 (#6406)
https://github.com/matthew-brett/multibuild/pull/240
2019-07-22 14:56:18 -07:00