[infra] Add BazelBuild task to build CanvasKit on the CI with Bazel

For additional context, see "Codifying Certain Build Options"
and "Building on the CI" in the design doc go/skia-bazel

Suggested review order:
 - builder_name_schema.json to see the three required and
   one optional part of BazelBuild jobs.
 - jobs.json to see one new BazelBuild job added. In an
   ideal world, this job would have been named
   BazelBuild-//modules/canvaskit:canvaskit_wasm-debug-linux_x64
   but Buildbucket (?) requires jobs match the regex
   ^[a-zA-Z0-9\\-_.\\(\\) ]{1,128}$
   so we use spaces instead of slashes or colons.
 - gen_tasks_logic.go; noting the makeBazelLabel function
   expands most of the spaces to / and the last one to a
   colon to make a single-target label. If there are three
   dots, then it is a multi-target label, and we do not
   need to add a colon.
 - bazel_build.go; This is a very simple task driver, and
   I do not anticipate getting too much more complex.
   The place where we decide which args to augment
   a build with depend on the host platform and thus
   should be set in gen_tasks_logic.go.
 - bazel/buildrc to see some initial configurations set,
   one of which, "debug", is used by the new job.
   The "release" version of CanvasKit probably works on
   3.1.10 which had a bugfix, but we are still on
   3.1.9
 - .bazelrc to see a rename of the linux-rbe config to
   linux_rbe (our configs should have no dashes if
   we want to specify them verbatim in our Job names).
   It also imports the Skia-specified build configs
   from //bazel/buildrc and supports the user-specified
   //bazel/user/buildrc file if it exists.
 - All other files in any order.

Change-Id: Ib954dd6045100eadcbbf4ffee0888f6fbce65fa7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537797
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Jorge Betancourt <jmbetancourt@google.com>

.bazelrc

@@ -90,11 +90,19 @@ build:remote --remote_instance_name projects/skia-rbe/instances/default_instance
 # These settings are specific to compiling on our Linux RBE workers. For example,
 # Use the worker pool as specified by the gce_linux_platform platform in
 # //bazel/rbe/BUILD.bazel.
-build:linux-rbe --config=remote
+build:linux_rbe --config=remote
 # Use our hermetic toolchain instead of the clang in the toolchain.
-build:linux-rbe --crosstool_top=//toolchain:clang_suite_linux
+build:linux_rbe --crosstool_top=//toolchain:clang_suite_linux
 # We want to run on this RBE platform
-build:linux-rbe --extra_execution_platforms=//bazel/rbe:gce_linux_platform
+build:linux_rbe --extra_execution_platforms=//bazel/rbe:gce_linux_platform
 # On the RBE instances, this Java and C++ toolchain are available
-build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/java:all
-build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/config:cc-toolchain
+build:linux_rbe --extra_toolchains=//bazel/rbe/gce_linux/java:all
+build:linux_rbe --extra_toolchains=//bazel/rbe/gce_linux/config:cc-toolchain
+
+# Import our specified build configurations
+# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file
+# We chose to split our build configurations into their own file to have better organization
+# because we anticipate that file growing large.
+import %workspace%/bazel/buildrc
+# Import User's custom builds if they have defined any
+try-import %workspace%/bazel/user/buildrc

.gitignore

@@ -53,3 +53,6 @@ node_modules
 tools/lottiecap/filmstrip.png
 
 bazel-*
+
+# A user is free to specify custom builds or options here without them being checked in.
+bazel/user/buildrc

PRESUBMIT.py

@@ -264,7 +264,10 @@ def _CheckBazelBUILDFiles(input_api, output_api):
   for affected_file in input_api.AffectedFiles(include_deletes=False):
     affected_file_path = affected_file.LocalPath()
     is_bazel = affected_file_path.endswith('BUILD.bazel')
-    if is_bazel:
+    # This list lines up with the one in autoroller_lib.py (see G3).
+    excluded_paths = ["infra/", "bazel/rbe/"]
+    is_excluded = any(affected_file_path.startswith(n) for n in excluded_paths)
+    if is_bazel and not is_excluded:
       with open(affected_file_path, 'r') as file:
         contents = file.read()
         if 'exports_files_legacy()' not in contents:

bazel/Makefile

@@ -40,24 +40,24 @@ known_good_builds:
 		--features skia_enforce_iwyu
 
 rbe_known_good_builds:
-	bazelisk build //experimental/bazel_test/... --config=linux-rbe
-	bazelisk run //experimental/bazel_test:bazel_test_exe --config=linux-rbe
-	bazelisk build //:skia_core --config=linux-rbe
-	bazelisk build //src/sksl/lex:sksllex --config=linux-rbe
-	bazelisk build //tools/skdiff --config=linux-rbe
-	bazelisk build //tools/skslc --config=linux-rbe
+	bazelisk build //experimental/bazel_test/... --config=linux_rbe
+	bazelisk run //experimental/bazel_test:bazel_test_exe --config=linux_rbe
+	bazelisk build //:skia_core --config=linux_rbe
+	bazelisk build //src/sksl/lex:sksllex --config=linux_rbe
+	bazelisk build //tools/skdiff --config=linux_rbe
+	bazelisk build //tools/skslc --config=linux_rbe
 	# TODO(kjlubick) CanvasKit in release mode (i.e. with Closure) requires
 	# https://github.com/emscripten-core/emscripten/pull/16640 to land
-	bazelisk build //modules/canvaskit:canvaskit_wasm --compilation_mode dbg --config=linux-rbe \
+	bazelisk build //modules/canvaskit:canvaskit_wasm --compilation_mode dbg --config=linux_rbe \
 		--jobs 100
 
 	# Test the enforcement of include what you use
-	bazelisk build //example:hello_world_gl --config=linux-rbe --features skia_enforce_iwyu
+	bazelisk build //example:hello_world_gl --config=linux_rbe --features skia_enforce_iwyu
  	# Both with and without a GPU backend should be error free (i.e. IWYU should let us
  	# conditionally import things.
-	bazelisk build //src/svg/... --config=linux-rbe --features skia_enforce_iwyu \
+	bazelisk build //src/svg/... --config=linux_rbe --features skia_enforce_iwyu \
 		--gpu_backend=gl_backend --include_decoder=jpeg_decode_codec
-	bazelisk build //src/svg/... --config=linux-rbe --features skia_enforce_iwyu
-	bazelisk build //tools/debugger --config=linux-rbe --gpu_backend=gl_backend \
+	bazelisk build //src/svg/... --config=linux_rbe --features skia_enforce_iwyu
+	bazelisk build //tools/debugger --config=linux_rbe --gpu_backend=gl_backend \
 		--features skia_enforce_iwyu
-	bazelisk build //:skia_core --config=linux-rbe --features skia_enforce_iwyu
+	bazelisk build //:skia_core --config=linux_rbe --features skia_enforce_iwyu

bazel/buildrc

@@ -0,0 +1,6 @@
+build:release --compilation_mode=opt
+build:debug --compilation_mode=dbg
+
+# Setting the spawn_strategy like this makes emscripten compiler work faster
+build:ck_release --config=release --spawn_strategy=local
+build:ck_debug --config=debug --spawn_strategy=local

bazel/user/README.md

@@ -0,0 +1,6 @@
+If you wish to define custom Bazel configurations (e.g. custom builds), make a text file in this
+folder called buildrc. It should follow the
+[.bazelrc conventions](https://bazel.build/docs/bazelrc#config).
+
+Users are free to put their custom builds in the $HOME/.bazelrc file as per usual, but if they
+wish to avoid conflicts with other Bazel projects, this is a safer place to store them.

infra/bots/BUILD.bazel

@@ -17,6 +17,7 @@ exports_files_legacy()
 genrule(
     name = "all_task_drivers",
     srcs = [
+        "//infra/bots/task_drivers/bazel_build",
         "//infra/bots/task_drivers/bazel_check_includes",
         "//infra/bots/task_drivers/check_generated_bazel_files",
         "//infra/bots/task_drivers/codesize",

infra/bots/build_task_drivers.sh

@@ -24,7 +24,7 @@ PLATFORM=${2:-linux_amd64} # use linux_amd64 if not specified
 # https://bazel.build/docs/output_directories#layout
 bazelisk --output_user_root=/mnt/pd0/bazel_cache \
     build //infra/bots:all_task_drivers --platforms=@io_bazel_rules_go//go/toolchain:${PLATFORM} \
-    --config=linux-rbe
+    --config=linux_rbe
 
 tar -xf bazel-bin/infra/bots/built_task_drivers.tar -C ${1}
 # Bazel outputs are write-protected, so we make sure everybody can write them. This way there

infra/bots/gen_tasks_logic/gen_tasks_logic.go

@@ -2075,3 +2075,57 @@ func (b *jobBuilder) runWasmGMTests() {
 		)
 	})
 }
+
+// Maps a shorthand version of a label (which can be an arbitrary string) to an absolute Bazel
+// label or "target pattern" https://bazel.build/docs/build#specifying-build-targets
+// The reason we need this mapping is because Buildbucket build names cannot have / or : in them.
+var shorthandToLabel = map[string]string{
+	"modules_canvaskit_canvaskit_wasm": "//modules/canvaskit:canvaskit_wasm",
+}
+
+// bazelBuild adds a task which builds the specified single-target label (//foo:bar) or
+// multi-target label (//foo/...) using Bazel. Depending on the host we run this on, we may
+// specify additional Bazel args to build faster.
+func (b *jobBuilder) bazelBuild() {
+	shorthand, config, host, cross := b.parts.bazelBuildParts()
+	label, ok := shorthandToLabel[shorthand]
+	if !ok {
+		panic("unsupported Bazel label shorthand " + shorthand)
+	}
+	b.addTask(b.Name, func(b *taskBuilder) {
+		cmd := []string{"./bazel_build",
+			"--project_id=skia-swarming-bots",
+			"--task_id=" + specs.PLACEHOLDER_TASK_ID,
+			"--task_name=" + b.Name,
+			"--label=" + label,
+			"--config=" + config,
+		}
+		if cross != "" {
+			// The cross (and host) platform is expected to be defined in
+			// //bazel/common_config_settings/BUILD.bazel
+			cross = "//bazel/common_config_settings:" + cross
+			cmd = append(cmd, "--cross="+cross)
+		}
+		if host == "linux_x64" {
+			b.linuxGceDimensions(MACHINE_TYPE_MEDIUM)
+			b.dep(b.buildTaskDrivers("linux", "amd64"))
+
+			// We want all Linux Bazel Builds to use RBE
+			cmd = append(cmd, "--bazel_arg=--config=linux_rbe")
+			cmd = append(cmd, "--bazel_arg=--jobs=100")
+			cmd = append(cmd, "--bazel_arg=--remote_download_minimal")
+		} else {
+			panic("unsupported Bazel host " + host)
+		}
+		b.cmd(cmd...)
+
+		// TODO(kjlubick) I believe this bazelisk package is just the Linux one. To support
+		//   more hosts, we need to have platform-specific bazelisk binaries.
+		b.cipd(b.MustGetCipdPackageFromAsset("bazelisk"))
+		b.addToPATH("bazelisk")
+		b.idempotent()
+		b.cas(CAS_COMPILE)
+		b.attempts(1)
+		b.serviceAccount(b.cfg.ServiceAccountCompile)
+	})
+}

infra/bots/gen_tasks_logic/job_builder.go

@@ -218,6 +218,11 @@ func (b *jobBuilder) genTasksForJob() {
 		return
 	}
 
+	if b.role("BazelBuild") {
+		b.bazelBuild()
+		return
+	}
+
 	log.Fatalf("Don't know how to handle job %q", b.Name)
 }
 

infra/bots/gen_tasks_logic/schema.go

@@ -249,6 +249,12 @@ func (p parts) isLinux() bool {
 	return p.matchOs("Debian", "Ubuntu")
 }
 
+// bazelParts returns all parts from the BazelBuild schema. label, config, and host are required;
+// cross is optional.
+func (p parts) bazelBuildParts() (label string, config string, host string, cross string) {
+	return p["label"], p["config"], p["host"], p["cross"]
+}
+
 // TODO(borenet): The below really belongs in its own file, probably next to the
 // builder_name_schema.json file.
 

infra/bots/jobs.json

@@ -1,4 +1,5 @@
 [
+  {"name": "BazelBuild-modules_canvaskit_canvaskit_wasm-debug-linux_x64"},
   {"name": "Build-Debian9-Clang-arm-Release-Flutter_Android_Docker"},
   {"name": "Build-Debian10-GCC-x86-Debug-Docker"},
   {"name": "Build-Debian10-GCC-x86-Release-Docker"},

infra/bots/recipe_modules/builder_name_schema/builder_name_schema.json

@@ -1,5 +1,15 @@
 {
   "builder_name_schema": {
+    "BazelBuild": {
+      "keys": [
+        "label",
+        "config",
+        "host"
+      ],
+      "optional_keys": [
+        "cross"
+      ]
+    },
     "Build": {
       "keys": [
         "os",

infra/bots/recipe_modules/builder_name_schema/builder_name_schema.py

@@ -20,6 +20,7 @@ BUILDER_NAME_SCHEMA = None
 BUILDER_NAME_SEP = None
 
 # Builder roles.
+BUILDER_ROLE_BAZELBUILD = 'BazelBuild'
 BUILDER_ROLE_BUILD = 'Build'
 BUILDER_ROLE_BUILDSTATS = 'BuildStats'
 BUILDER_ROLE_CANARY = 'Canary'
@@ -30,7 +31,8 @@ BUILDER_ROLE_PERF = 'Perf'
 BUILDER_ROLE_TEST = 'Test'
 BUILDER_ROLE_FM = 'FM'
 BUILDER_ROLE_UPLOAD = 'Upload'
-BUILDER_ROLES = (BUILDER_ROLE_BUILD,
+BUILDER_ROLES = (BUILDER_ROLE_BAZELBUILD,
+                 BUILDER_ROLE_BUILD,
                  BUILDER_ROLE_BUILDSTATS,
                  BUILDER_ROLE_CANARY,
                  BUILDER_ROLE_CODESIZE,

infra/bots/task_drivers/bazel_build/BUILD.bazel

@@ -0,0 +1,23 @@
+load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
+
+licenses(["notice"])
+
+go_library(
+    name = "bazel_build_lib",
+    srcs = ["bazel_build.go"],
+    importpath = "go.skia.org/skia/infra/bots/task_drivers/bazel_build",
+    visibility = ["//visibility:private"],
+    deps = [
+        "@org_skia_go_infra//go/common",
+        "@org_skia_go_infra//go/exec",
+        "@org_skia_go_infra//task_driver/go/lib/bazel",
+        "@org_skia_go_infra//task_driver/go/lib/os_steps",
+        "@org_skia_go_infra//task_driver/go/td",
+    ],
+)
+
+go_binary(
+    name = "bazel_build",
+    embed = [":bazel_build_lib"],
+    visibility = ["//visibility:public"],
+)

infra/bots/task_drivers/bazel_build/bazel_build.go

@@ -0,0 +1,99 @@
+// Copyright 2022 Google LLC
+//
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+//
+// This executable runs a Bazel(isk) build command for a single label using the provided
+// config (which is assumed to be in //bazel/buildrc) and any provided Bazel args.
+// This handles any setup needed to run Bazel on our CI machines before running the task, like
+// setting up logs and the Bazel cache.
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"path/filepath"
+
+	"go.skia.org/infra/go/common"
+	sk_exec "go.skia.org/infra/go/exec"
+	"go.skia.org/infra/task_driver/go/lib/bazel"
+	"go.skia.org/infra/task_driver/go/lib/os_steps"
+	"go.skia.org/infra/task_driver/go/td"
+)
+
+var (
+	// Required properties for this task.
+	bazelArgs = common.NewMultiStringFlag("bazel_arg", nil, "Additional arguments that should be forwarded directly to the Bazel invocation.")
+	cross     = flag.String("cross", "", "An identifier specifying the target platform that Bazel should build for. If empty, Bazel builds for the host platform (the machine on which this executable is run).")
+	config    = flag.String("config", "", "A custom configuration specified in //bazel/buildrc. This configuration potentially encapsulates many features and options.")
+	projectId = flag.String("project_id", "", "ID of the Google Cloud project.")
+	label     = flag.String("label", "", "An absolute label to the target that should be built.")
+	taskId    = flag.String("task_id", "", "ID of this task.")
+	taskName  = flag.String("task_name", "", "Name of the task.")
+	workdir   = flag.String("workdir", ".", "Working directory, the root directory of a full Skia checkout")
+	// Optional flags.
+	local  = flag.Bool("local", false, "True if running locally (as opposed to on the CI/CQ)")
+	output = flag.String("o", "", "If provided, dump a JSON blob of step data to the given file. Prints to stdout if '-' is given.")
+)
+
+func main() {
+	// StartRun calls flag.Parse()
+	ctx := td.StartRun(projectId, taskId, taskName, output, local)
+	defer td.EndRun(ctx)
+
+	if *label == "" || *config == "" {
+		td.Fatal(ctx, fmt.Errorf("--label and --config are required"))
+	}
+
+	wd, err := os_steps.Abs(ctx, *workdir)
+	if err != nil {
+		td.Fatal(ctx, err)
+	}
+	skiaDir := filepath.Join(wd, "skia")
+
+	opts := bazel.BazelOptions{
+		// We want the cache to be on a bigger disk than default. The root disk, where the home
+		// directory (and default Bazel cache) lives, is only 15 GB on our GCE VMs.
+		CachePath: "/mnt/pd0/bazel_cache",
+	}
+	if err := bazel.EnsureBazelRCFile(ctx, opts); err != nil {
+		td.Fatal(ctx, err)
+	}
+
+	if *cross != "" {
+		// See https://bazel.build/concepts/platforms-intro and https://bazel.build/docs/platforms
+		// when ready to support this.
+		td.Fatal(ctx, fmt.Errorf("cross compilation not yet supported"))
+	}
+
+	if err := bazelBuild(ctx, skiaDir, *label, *config, *bazelArgs...); err != nil {
+		td.Fatal(ctx, err)
+	}
+}
+
+// bazelBuild builds the target referenced by the given absolute label passing the provided
+// config and any additional args to the build command. Instead of calling Bazel directly, we use
+// Bazelisk to make sure we use the right version of Bazel, as defined in the .bazelversion file
+// at the Skia root.
+func bazelBuild(ctx context.Context, checkoutDir, label, config string, args ...string) error {
+	step := fmt.Sprintf("Build %s with config %s and %d extra flags", label, config, len(args))
+	return td.Do(ctx, td.Props(step), func(ctx context.Context) error {
+		runCmd := &sk_exec.Command{
+			Name: "bazelisk",
+			Args: append([]string{"build",
+				label,
+				"--config=" + config, // Should be defined in //bazel/buildrc
+			}, args...),
+			InheritEnv: true, // Makes sure bazelisk is on PATH
+			Dir:        checkoutDir,
+			LogStdout:  true,
+			LogStderr:  true,
+		}
+		_, err := sk_exec.RunCommand(ctx, runCmd)
+		if err != nil {
+			return err
+		}
+		return nil
+	})
+}

infra/bots/task_drivers/bazel_check_includes/bazel_check_includes.go

@@ -87,7 +87,7 @@ func bazelCheckIncludes(ctx context.Context, checkoutDir, label string, opts ...
 		runCmd := &sk_exec.Command{
 			Name: "bazelisk",
 			Args: append([]string{"build",
-				"--config=linux-rbe", // Compile using RBE
+				"--config=linux_rbe", // Compile using RBE
 				"--features=skia_enforce_iwyu",
 				"--jobs=" + strconv.Itoa(rbeJobs),
 				"--keep_going",              // Don't stop after first error

infra/bots/task_drivers/check_generated_bazel_files/check_generated_bazel_files.go

@@ -150,7 +150,7 @@ func smokeTestBuild(ctx context.Context, checkoutDir string) error {
 		runCmd := &sk_exec.Command{
 			Name: "bazelisk",
 			Args: []string{"build",
-				"--config=linux-rbe", // Compile using RBE
+				"--config=linux_rbe", // Compile using RBE
 				"--jobs=" + strconv.Itoa(rbeJobs),
 				"//example:hello_world_gl",  // This compiles and links, so is a good smoke test
 				"--remote_download_minimal", // No need to download the executable.

infra/bots/tasks.json

@@ -1,5 +1,10 @@
 {
   "jobs": {
+    "BazelBuild-modules_canvaskit_canvaskit_wasm-debug-linux_x64": {
+      "tasks": [
+        "BazelBuild-modules_canvaskit_canvaskit_wasm-debug-linux_x64"
+      ]
+    },
     "Build-Debian10-BazelClang-x86_64-Release-IWYU": {
       "tasks": [
         "Build-Debian10-BazelClang-x86_64-Release-IWYU"
@@ -3195,6 +3200,45 @@
     }
   },
   "tasks": {
+    "BazelBuild-modules_canvaskit_canvaskit_wasm-debug-linux_x64": {
+      "casSpec": "compile",
+      "cipd_packages": [
+        {
+          "name": "skia/bots/bazelisk",
+          "path": "bazelisk",
+          "version": "version:0"
+        }
+      ],
+      "command": [
+        "./bazel_build",
+        "--project_id=skia-swarming-bots",
+        "--task_id=<(TASK_ID)",
+        "--task_name=BazelBuild-modules_canvaskit_canvaskit_wasm-debug-linux_x64",
+        "--label=//modules/canvaskit:canvaskit_wasm",
+        "--config=debug",
+        "--bazel_arg=--config=linux_rbe",
+        "--bazel_arg=--jobs=100",
+        "--bazel_arg=--remote_download_minimal"
+      ],
+      "dependencies": [
+        "Housekeeper-PerCommit-BuildTaskDrivers_linux_amd64"
+      ],
+      "dimensions": [
+        "cpu:x86-64-Haswell_GCE",
+        "gpu:none",
+        "machine_type:n1-standard-16",
+        "os:Debian-10.3",
+        "pool:Skia"
+      ],
+      "env_prefixes": {
+        "PATH": [
+          "bazelisk"
+        ]
+      },
+      "idempotent": true,
+      "max_attempts": 1,
+      "service_account": "skia-external-compile-tasks@skia-swarming-bots.iam.gserviceaccount.com"
+    },
     "Build-Debian10-BazelClang-x86_64-Release-IWYU": {
       "casSpec": "compile",
       "cipd_packages": [

site/docs/dev/contrib/bazel.md

@@ -68,9 +68,18 @@ toolchain's cache. Make sure `xcode-select -p` returns a valid path.
 ## .bazelrc Tips
 You should make a [.bazelrc file](https://bazel.build/docs/bazelrc) in your home directory where
 you can specify settings that apply only to you. These can augment or replace the ones we define
-in `//.bazelrc`
+in the `//.bazelrc` configuration file.
 
-You may want some or all of the following entries in your `~/.bazelrc` file.
+Skia defines some [configs](https://bazel.build/docs/bazelrc#config), that is, group of settings
+and features in `//bazel/buildrc`. This file contains configs for builds that we use  regularly
+(for example, in our continuous integration system).
+
+If you want to define Skia-specific configs (and options which do not conflict with other Bazel
+projects), you make a file in `//bazel/user/buildrc` which will automatically be read in. This
+file is covered by a `.gitignore` rule and should not be checked in.
+
+You may want some or all of the following entries in your `~/.bazelrc` or `//bazel/user/buildrc`
+file.
 
 ### Build Skia faster locally
 Many Linux machines have a [RAM disk mounted at /dev/shm](https://www.cyberciti.biz/tips/what-is-devshm-and-its-practical-usage.html)
@@ -84,7 +93,7 @@ build:clang --sandbox_base=/dev/shm
 
 ### Authenticate to RBE on a Linux VM
 We are in the process of setting up Remote Build Execution (RBE) for Bazel. Some users have reported
-errors when trying to use RBE (via `--config=linux-rbe`) on Linux VMs such as:
+errors when trying to use RBE (via `--config=linux_rbe`) on Linux VMs such as:
 ```
 ERROR: Failed to query remote execution capabilities: 
 Error code 404 trying to get security access token from Compute Engine metadata for the default