All our bots but the iOS ones are on GN now. This cleans up a bunch of GYP and other obsolete stuff.
- Nothing's using default_flavor.py any more except as a base class.
- There are no -CMake, -Shared or -VisualBench bots anymore.
- Only the iOS bots care about GYP_DEFINES.
You'll see the PDFium bot's GYP_DEFINES change, but that doesn't matter... it's using PDFium's own (GN) build system, and it ignores GYP_DEFINES.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3567
Change-Id: I699e10f013ea77df4dcaa1cb559c51c5bf55dfdb
Reviewed-on: https://skia-review.googlesource.com/3567
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
CQ_INCLUDE_TRYBOTS=master.client.skia.compile:Build-Win-MSVC-x86-Debug-ANGLE-Trybot;master.client.skia:Test-Win-MSVC-ShuttleC-GPU-GTX960-x86_64-Release-ANGLE-Trybot
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3539
Change-Id: I076c4c5972344b6688d648a8ea5e5618e87fd88d
Reviewed-on: https://skia-review.googlesource.com/3539
Commit-Queue: Mike Klein <mtklein@chromium.org>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Not quite there yet for ANGLE. The rest should be good to go.
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3424
Change-Id: I9d6ad1de75bb0d9c1e18a9220a7ca82c84f74235
Reviewed-on: https://skia-review.googlesource.com/3424
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
CQ_INCLUDE_TRYBOTS=master.client.skia.compile:Build-Win-MSVC-x86_64-Release-Vulkan-Trybot;master.client.skia:Test-Win10-MSVC-ShuttleA-GPU-GTX660-x86_64-Debug-Vulkan-Trybot,Perf-Win10-MSVC-ShuttleA-GPU-GTX660-x86_64-Release-Vulkan-Trybot,Perf-Win10-MSVC-ShuttleA-GPU-GTX660-x86_64-Debug-Vulkan-Trybot
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3421
Change-Id: I7012719fff66778488589ab9c15fd2fa2a109cd1
Reviewed-on: https://skia-review.googlesource.com/3421
Commit-Queue: Mike Klein <mtklein@chromium.org>
Reviewed-by: Greg Daniel <egdaniel@google.com>
CQ_INCLUDE_TRYBOTS=master.client.skia.compile:Build-Win-MSVC-x86-Debug-GDI-Trybot,Build-Win-MSVC-x86-Release-GDI-Trybot;master.client.skia:Perf-Win-MSVC-GCE-CPU-AVX2-x86_64-Debug-GDI-Trybot,Test-Win-MSVC-GCE-CPU-AVX2-x86_64-Debug-GDI-Trybot,Test-Win-MSVC-GCE-CPU-AVX2-x86_64-Release-GDI-Trybot
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3324
Change-Id: I3ff2f5d16de9fff7ed77fa5127b96bef55bef957
Reviewed-on: https://skia-review.googlesource.com/3324
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Mike Klein <mtklein@chromium.org>
The default NINJA_STATUS format is "[%f/%t] ". This prepends %e to help diagnose slow builds.
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3323
Change-Id: Ied27317cedc2b07ff57c59eae61fe9e86c7fea33
Reviewed-on: https://skia-review.googlesource.com/3323
Commit-Queue: Mike Klein <mtklein@chromium.org>
Reviewed-by: Eric Boren <borenet@google.com>
This will make it temporarily 64-bit until I sort out 32-bit builds (tomorrow?) but that's an unimportant dimension for this bot... we just want to make sure we can build with /EHsc.
CQ_INCLUDE_TRYBOTS=master.client.skia.compile:Build-Win-MSVC-x86-Debug-Exceptions-Trybot
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3255
Change-Id: Ie3c604e7ce779d0b45149e0151b7ecc93f9dd523
Reviewed-on: https://skia-review.googlesource.com/3255
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
CQ_INCLUDE_TRYBOTS=master.client.skia.compile:Build-Win-MSVC-x86_64-Debug-GN-Trybot
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3246
Change-Id: Ib38d4f16788bc27ad81975a4c1b0a732e5df1de3
Reviewed-on: https://skia-review.googlesource.com/3246
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
fetch-gn is a Python script, gn is a batch script, and ninja is an exe.
Pass the win_toolchain package to GN as windk. This won't work yet.
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3245
Change-Id: Iacc59c70c06ec118fc6a45d0cdb67095ff2b951b
Reviewed-on: https://skia-review.googlesource.com/3245
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
This should make them visible to things like gn_to_cmake.py.
I'm not exactly sure what this implies about ordering and overriding.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2938
Change-Id: I0740613993fb5bbfb8363cfa126d1f59768abf60
Reviewed-on: https://skia-review.googlesource.com/2938
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Mike Klein <mtklein@chromium.org>
It's more annoying than helpful to have GCC turn mul,add into fma.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2780
CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Release-Fast-Trybot
Change-Id: I63f4615f73aed112f10f6cb516d899b820918298
Reviewed-on: https://skia-review.googlesource.com/2780
Commit-Queue: Cary Clark <caryclark@google.com>
Reviewed-by: Cary Clark <caryclark@google.com>
Angle's existing GN files only work in Chrome, so I've written a new one.
This won't work on Windows, but our GN build doesn't work on Windows anyway. So this CL is an attempt to get a ahead of that curve on ANGLE. It looks large but fairly straightforward.
Now working on Linux:
$ gn gen angle --args=skia_use_angle=true
$ ninja -C angle
$ angle/dm --config angle-gl --src gm -w dm-out
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2361983002
Review-Url: https://codereview.chromium.org/2361983002
(The failing NexusPlayer bot is a demo.)
This should make stack traces more useful, turning this sort of thing
09-20 11:29:39.536 2978 2978 F DEBUG : #00 pc 00970fd0 /data/local/tmp/dm
into something like this
09-20 11:29:39.536 2978 2978 F DEBUG : #00 pc 00970fd0 adjust_bounds_to_granularity(SkIRect*, SkIRect const&, VkExtent2D const&, int, int) at /b/work/skia/out/Build-Ubuntu-Clang-x86-Debug-GN_Android_Vulkan/Debug/../../../src/gpu/vk/GrVkGpu.cpp:1803 /data/local/tmp/dm
Some bots like the S7 already have good enough stack traces, e.g.
09-20 11:35:12.567 936 936 F DEBUG : #00 pc 00000000000bed6c /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN13QglManagedBuf14ConfirmEntriesEv+108)
09-20 11:35:12.567 936 936 F DEBUG : #01 pc 00000000000b098c /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN9QglBltLib6FillHwEPK10QglBltFillPK15QglBltColorFillPK15QglBltDepthFillPjP12QglBltStatusSB_+588)
09-20 11:35:12.567 936 936 F DEBUG : #02 pc 00000000000b23bc /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN9QglBltLib9FillImageEPK10QglBltFillPK15QglBltColorFillPK15QglBltDepthFillPjP12QglBltStatusSB_+348)
09-20 11:35:12.567 936 936 F DEBUG : #03 pc 000000000009bb00 /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN16A5xCommandBuffer22PerformConditionalFillEiiP10QglBltFillPK15QglBltColorFillP15QglBltDepthFillP12QglBltStatusS8_+256)
09-20 11:35:12.567 936 936 F DEBUG : #04 pc 000000000009c0c0 /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN16A5xCommandBuffer23HwWriteSubpassInitClearEP10QglBltFillPK15QglBltColorFillP15QglBltDepthFillP12QglBltStatusS8_+224)
09-20 11:35:12.567 936 936 F DEBUG : #05 pc 0000000000072610 /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN16QglCommandBuffer18WriteSubpassClearsEv+464)
09-20 11:35:12.567 936 936 F DEBUG : #06 pc 0000000000073ae0 /system/vendor/lib64/hw/vulkan.msm8996.so (_ZN16QglCommandBuffer12BeginSubpassEv+32)
09-20 11:35:12.567 936 936 F DEBUG : #07 pc 0000000000063120 /system/vendor/lib64/hw/vulkan.msm8996.so (vkCmdBeginRenderPass+224)
09-20 11:35:12.567 936 936 F DEBUG : #08 pc 0000000000635f60 /data/local/tmp/nanobench (_ZN24GrVkPrimaryCommandBuffer15beginRenderPassEPK7GrVkGpuPK14GrVkRenderPassjPK12VkClearValueRK16GrVkRenderTargetRK7SkIRectb+132)
or
09-20 11:42:24.557 937 937 F DEBUG : backtrace:
09-20 11:42:24.557 937 937 F DEBUG : #00 pc 0000000000069404 /system/lib64/libc.so (tgkill+8)
09-20 11:42:24.557 937 937 F DEBUG : #01 pc 0000000000066b94 /system/lib64/libc.so (pthread_kill+68)
09-20 11:42:24.557 937 937 F DEBUG : #02 pc 0000000000023a28 /system/lib64/libc.so (raise+28)
09-20 11:42:24.557 937 937 F DEBUG : #03 pc 000000000001e358 /system/lib64/libc.so (abort+60)
09-20 11:42:24.557 937 937 F DEBUG : #04 pc 000000000076430c /data/local/tmp/dm (_Z17sk_abort_no_printv+8)
These won't be affected.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2351243002
NOTREECHECKS=true
Review-Url: https://codereview.chromium.org/2351243002
Android API >= 24 implies Vulkan support, so we can have a more useful default here than 'false'. If for some reason you wanted to turn it off, you can still override skia_use_vulkan.
The defined(ndk_api) guards other users of our GN files (Fuchsia) who may not have an ndk_api argument defined in their BUILDCONFIG.gn.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2347843003
Review-Url: https://codereview.chromium.org/2347843003
These were intended to prevent GN and GYP Android bots from stomping on
each other. Turns out, they don't, even without this... they're writing
most files to completely separate paths:
- GYP puts most data under $EXTERNAL_STORAGE, generally /sdcard
and its binaries (libdm.so, libnanobench.so, libskia.so, skia_launcher)
in /data/local/tmp;
- GN puts everything under /data/local/tmp, and its binaries (dm, nanobench)
don't overlap GYP's.
So clearing /data/local/tmp was essentially just removing GN's data; GYP's
data file caching was never affected and can't conflict with GN's.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2340473002
Review-Url: https://codereview.chromium.org/2340473002
Just to take inventory of which old problems still linger and which are now moot, I've gone out of my way to have this stand alone. All of gn_android_flavor's logic is self contained, without any dependency on the platform_tools scripts.
The tricky bits turn out to be, copying directories containing symlinks---or really any copying involving more than one file---and getting the exit code back from adb shell. Luckily the ADB I've got on my desktop and my Nexus 5x seems to handle this all without the awkward workarounds you see here, so there's hope that One Day Soon the weird parts (basically, anything with inline python) can go away. Once we've got these bots landed green, I'll go see whether the fixes are due to ADB updates, Android updates, or perhaps something else like hardware.
The parts marked TEMPORARY are a nod to the fact that the devices are used by gn_android_flavor and android_flavor both today. It's mostly about not stepping on each other's toes or leaving anything laying around that might confuse each other. The marked parts can go away when bots are either gn_ or non-gn_ but not both.
I have omitted a few steps that may be important, but which are easy independent follow-ups:
- running as root
- locking clocks
- waiting on battery levels
- fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and that we're locking to the best policy. I've tried to keep this CL as trim as possible, leaving any of these vaguely optional steps for later.
As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold. Can we see trybots in Perf yet?
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002
CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot
Review-Url: https://codereview.chromium.org/2320153002
Attempt to take over all *SAN builds.
MSAN has a lot of coordination required between gn/BUILD.gn and gn_flavor.py.
I'd like to follow up to move more of this into gn/BUILD.gn, to make it easier
to use locally.
The compile steps should be much faster now. We no longer build CMake
and Clang for every run, instead using the clang_linux CIPD package. This
removes the need for all the third_party/externals/llvm/... dependencies.
Similarly, since we're using the clang_linux package, we no longer depend
on Chrome's Clang, and thus no longer need to sync chromium on these bots.
Instead of packaging up MSAN libraries and llvm-symbolizer in the compile
output, I have the test / perf bots also depend on the clang_linux package.
These do not vary from build to build.
No more need for the xsan.blacklist -include hack: Clang, GN, and Ninja
all track changes to xsan.blacklist without our help.
This has the incidental effect of upgrading the compiler used by *SAN
bots from Clang 3.8 to Clang 3.9.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2289343002
Review-Url: https://codereview.chromium.org/2289343002
I just burned 2 days debugging a confusing interaction between ccache
and the -fsanitize-blacklist argument to Clang. Let's see if we can
live without ccache (swarming affinity + Ninja seems pretty decent).
As a point of reference, the Mac bots have been looking for ccache but
failing to find it. They're proof this will be fine.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2310063003
Review-Url: https://codereview.chromium.org/2310063003
I am hoping this makes it easier to get *SAN bots going.
Today we're generating a libcompiler_rt.a that's using a
relocation type that the ld on the bots doesn't know about.
This lld is will know about anything our Clang generates.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2301273002
Review-Url: https://codereview.chromium.org/2301273002
The remaining suppression (libwebp) is already covered by the
compile-time blacklist, tools/xsan.blacklist.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2300193002
CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Ubuntu-Clang-GCE-CPU-AVX2-x86_64-Release-TSAN-Trybot,Test-Ubuntu-Clang-Golo-GPU-GT610-x86_64-Release-TSAN-Trybot
Review-Url: https://codereview.chromium.org/2300193002
