Commit Graph

64 Commits

Author SHA1 Message Date
mtklein
2b3c2a3ff9 GN: add sanitize arg
Attempt to take over all *SAN builds.

MSAN has a lot of coordination required between gn/BUILD.gn and gn_flavor.py.
I'd like to follow up to move more of this into gn/BUILD.gn, to make it easier
to use locally.

The compile steps should be much faster now.  We no longer build CMake
and Clang for every run, instead using the clang_linux CIPD package.  This
removes the need for all the third_party/externals/llvm/... dependencies.

Similarly, since we're using the clang_linux package, we no longer depend
on Chrome's Clang, and thus no longer need to sync chromium on these bots.

Instead of packaging up MSAN libraries and llvm-symbolizer in the compile
output, I have the test / perf bots also depend on the clang_linux package.
These do not vary from build to build.

No more need for the xsan.blacklist -include hack: Clang, GN, and Ninja
all track changes to xsan.blacklist without our help.

This has the incidental effect of upgrading the compiler used by *SAN
bots from Clang 3.8 to Clang 3.9.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2289343002

Review-Url: https://codereview.chromium.org/2289343002
2016-09-08 08:39:34 -07:00
mtklein
34ffb342f5 GN: do not set default GN args
It's just visual noise... this should make future gn_flavor.py
changes show fewer incidental diffs.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2324573002

Review-Url: https://codereview.chromium.org/2324573002
2016-09-08 06:49:05 -07:00
borenet
e72daf6116 Run install-sysroot script before 'gn gen' on PDFium bot
BUG=skia:5727
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2317013002

Review-Url: https://codereview.chromium.org/2317013002
2016-09-07 08:34:13 -07:00
brianosman
2c4b64e92a New gamut GM, to test gamut conversion in various code-paths
After several different strategies, this one appears to work
well. The basic test:

1) For a variety of drawing techniques, we render fixed size
   rectangles. (Solid colors via paint color, bitmap, etc...)
2) For each method in , we render to both an sRGB and
   WideGamutRGB offscreen surface. (AdobeRGB isn't wide enough
   to clearly demonstrate if things are working or not).
3) Use readPixels to fetch the raw (still in wide gamut) pixel
   data, then draw that directly to the final canvas.

So, for each pair of squares, they should look clearly
different. Currently, with the GPU backend, only the bicubic
bitmap paths have that behavior. Adding more test cases (and
fixing the ones that are already incorrect) will be the long
tail of gamut transformation.

Current output (with my other patchset, which fixes all
bitmap draws): https://screenshot.googleplex.com/wsL3x7eCtWE.png

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2293173002

Review-Url: https://codereview.chromium.org/2293173002
2016-09-07 07:04:45 -07:00
mtklein
b58fe9b899 GN: turn off ccache
I just burned 2 days debugging a confusing interaction between ccache
and the -fsanitize-blacklist argument to Clang.  Let's see if we can
live without ccache (swarming affinity + Ninja seems pretty decent).

As a point of reference, the Mac bots have been looking for ccache but
failing to find it.  They're proof this will be fine.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2310063003

Review-Url: https://codereview.chromium.org/2310063003
2016-09-07 05:12:34 -07:00
bsalomon
b2be02bb39 Update Windows build to use VULKAN_SDK rather than VK_SDK_PATH
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2318633002

Review-Url: https://codereview.chromium.org/2318633002
2016-09-06 08:40:51 -07:00
kjlubick
f54c00e817 Add device mapping for Nexus 6p (angler)
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2311253002

Review-Url: https://codereview.chromium.org/2311253002
2016-09-06 07:39:30 -07:00
jcgregorio
b55d98dda1 pdfium: Fix the pdfium builder.
BUG=skia:5715
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2300703007

Review-Url: https://codereview.chromium.org/2300703007
2016-09-02 12:36:05 -07:00
mtklein
795423198b GN: Build and link with LLD when using our Clang toolchain.
I am hoping this makes it easier to get *SAN bots going.
Today we're generating a libcompiler_rt.a that's using a
relocation type that the ld on the bots doesn't know about.
This lld is will know about anything our Clang generates.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2301273002

Review-Url: https://codereview.chromium.org/2301273002
2016-09-01 15:51:03 -07:00
mtklein
d592a4e534 The *SAN bots are now all Clang bots.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2306733002

Review-Url: https://codereview.chromium.org/2306733002
2016-09-01 15:49:44 -07:00
mtklein
67f6636b20 Remove tsan.supp.
The remaining suppression (libwebp) is already covered by the
compile-time blacklist, tools/xsan.blacklist.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2300193002

CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Ubuntu-Clang-GCE-CPU-AVX2-x86_64-Release-TSAN-Trybot,Test-Ubuntu-Clang-Golo-GPU-GT610-x86_64-Release-TSAN-Trybot

Review-Url: https://codereview.chromium.org/2300193002
2016-09-01 14:57:35 -07:00
mtklein
ae509db481 GN: expunge all environment variables that have no effect on GN
This should have no practical effect.  I just want to make sure no
environment variables beyond $PATH have any effect on GN builds.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2300173002

Review-Url: https://codereview.chromium.org/2300173002
2016-09-01 13:19:21 -07:00
mtklein
2b44853c53 preserve GN_Android in compile -> test mapper
doesn't seem anything tests this?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2306603002

Review-Url: https://codereview.chromium.org/2306603002
2016-09-01 08:47:25 -07:00
mtklein
b987afa7cc Remove empty {lsan,ubsan}.supp files.
We never used ubsan.supp,
and LSAN suppressions are now built into the test binaries.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2302803003

CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Ubuntu-Clang-GCE-CPU-AVX2-x86_64-Debug-ASAN-Trybot

Review-Url: https://codereview.chromium.org/2302803003
2016-09-01 08:08:03 -07:00
mtklein
da8f6546da Add and use a clang_linux asset.
This gives us a consistent Clang toolchain on Linux bots,
most importantly for *SAN bots.  It's ~300MB unpacked.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2294353002

Review-Url: https://codereview.chromium.org/2294353002
2016-08-31 12:35:21 -07:00
mtklein
272b311298 GN: forgot Clang+ccache fix on Android. And sort.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2298203002

No trybots for these yet...
NOTRY=true
TBR=jcgregorio@google.com

Review-Url: https://codereview.chromium.org/2298203002
2016-08-31 05:03:31 -07:00
mtklein
82023d590c add gn_android_flavor
This ought to support compiles for now.

Am I picking up my CIPD ndk packages right?

The main thing to note is that I'm passing the target_arch directly through
as target_cpu.  This means these bots will have a slightly different naming
convention than we've been using, but it'll agree with what you must type
yourself when using GN to build for Android:

  - Arm7 -> arm
  - Arm64 -> arm64
  - Mips -> mipsel
  - Mips64 -> mips64el
  - x86 -> x86 (unchanged)
  - x86_64 -> x64

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2292663002

Review-Url: https://codereview.chromium.org/2292663002
2016-08-29 13:14:55 -07:00
mtklein
cab0bb7fc6 GN: support compiler_prefix, use it for ccache on bots.
This makes it considerably easier to use ccache with the Android NDK.
You can now just set
  compiler_prefix = "ccache"
  ndk = "/path/to/ndk"
and we'll use the NDK clang, wrapped with ccache.

The name compiler_prefix is stolen from / compatible with Chrome.

If you have ccache, you can just always leave compiler_prefix="ccache" enabled.
This should make it an unusual thing for humans to have to change cc or cxx.

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2281163002

Review-Url: https://codereview.chromium.org/2281163002
2016-08-26 13:43:19 -07:00
mtklein
f0f489062a Add Mac NDK asset, and fetch NDK on Android compile bots.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2284813002

Review-Url: https://codereview.chromium.org/2284813002
2016-08-26 11:22:54 -07:00
fmalita
30afa71dd3 Remove SVG serialization suppressions
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2282883002

Review-Url: https://codereview.chromium.org/2282883002
2016-08-26 10:23:43 -07:00
fmalita
ce2fc6a045 Add imagemasksubset GM
(GM for non-N32 SkImage subsetting)

BUG=skia:5687
R=bsalomon@google.com,robertphillips@google.com,reed@google.com
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2286733002

Review-Url: https://codereview.chromium.org/2286733002
2016-08-26 10:13:40 -07:00
rmistry
3f1c9c0a72 Apply gerrit ref if it is a Gerrit patch.
Also:
* Pass through a new property 'patch_storage' to DM/Nanobench/Coverage. This will be used by the different frameworks to figure out if it is Rietveld or Gerrit issue.
* Calculate issue and patchset for Gerrit patches similar to Rietveld.

BUG=skia:5627
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2263323002

Review-Url: https://codereview.chromium.org/2263323002
2016-08-24 05:07:06 -07:00
mtklein
a61b6d4f9e compress_r11eac_blocks() required more alignment than dst has.
This shouldn't change any behavior except that the stores to dst
will no longer require 8-byte alignment.

Empirically it seems like we can use 4-byte alignment here,
but u8 (i.e. 1-byte alignment) is always safe.

BUG=skia:5637
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2264103002
CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Release-SKNX_NO_SIMD-Trybot

Review-Url: https://codereview.chromium.org/2264103002
2016-08-22 09:30:54 -07:00
mtklein
8079ba688c GN: add extra_cflags et al.
Adding flags to the end of cc or cxx is pretty useful, but these always end up
on the command line before the GN generated flags, thus setting defaults that
GN will override.

For full flexibility we want to be able to add flags after the flags GN has
added, so that custom flags can override _it_.

I've updated the Fast bots with an example here: if we said cc="clang -O3 ...",
that '-O3' would be overriden later by the default Release-mode '-Os'.  By
putting it in extra_cflags, we get the last word: our '-O3' overrides the
default '-Os'.

Another good use case is a hypothetical Actually-Shippable-Release mode.  Our
Release mode bundles in tons of debug symbols via '-g'.  libskia.a is about 10x
larger than it needs to be when built that way, but it helps us debug the bot
failures immensely.  To build a libskia.{a,so} that you'd really ship, you can
now set extra_cflags="-g0" to override '-g'.  You could set '-march' flags there
too, '-fomit-frame-pointer', etc.

There are lots of flags that won't matter where they end up in the command line.
To keep everything simple I've put them in extra_cflags with the rest.  This means
the only time we change 'cc' or 'cxx' in our recipes is to prefix 'ccache'.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2241263003

Review-Url: https://codereview.chromium.org/2241263003
2016-08-16 09:31:16 -07:00
fmalita
b5b6de9b94 Disable Nebraska-StateSeal.svg on Win8 bots
Temporary suppression, pending bug fix.

BUG=skia:5636
R=robertphillips@google.com,senorblanco@chromium.org
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2249763002

Review-Url: https://codereview.chromium.org/2249763002
2016-08-16 07:08:48 -07:00
fmalita
9ea494afd0 Fix up infra bot change
Follow up to https://codereview.chromium.org/2246943002/.

R=mtklein@google.com,robertphillips@google.com
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2247903002

Review-Url: https://codereview.chromium.org/2247903002
2016-08-15 13:38:18 -07:00
fmalita
6fb0648c35 [SVGDom] Add opacity support
Group opacity support.  Unlike the other presentation attributes we
support thus far, group opacity is not inherited.

R=robertphillips@google.com,stephana@google.com
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2246943002

Review-Url: https://codereview.chromium.org/2246943002
2016-08-15 12:45:11 -07:00
rmistry
ea246cd5fb Increase hard and io timeouts for CT builds.
BUG=skia:5641
TBR=borenet
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2249523004

Review-Url: https://codereview.chromium.org/2249523004
2016-08-15 09:54:56 -07:00
mtklein
d5d3287fe2 Skip CompressCheckerboard unit test on N6.
This is at least one of the failures making the bot red.
The particular signal indicates it's probably an alignment problem.

BUG=skia:5637
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2245063002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Test-Android-GCC-Nexus6-CPU-NEON-Arm7-Debug-Trybot

TBR=robertphillips@google.com

Review-Url: https://codereview.chromium.org/2245063002
2016-08-15 07:47:57 -07:00
jcgregorio
106a9ff313 Add extracting GM comments to the housekeeper bot.
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2237033002

Review-Url: https://codereview.chromium.org/2237033002
2016-08-12 08:50:29 -07:00
rmistry
70c43efb6e Turn on SVGs for all Test bots
BUG=skia:5628
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2240153002

Review-Url: https://codereview.chromium.org/2240153002
2016-08-12 07:25:53 -07:00
rmistry
bcba19dd02 Use SVGs CIPD package and use SVG as a DM source.
Currently only turned on for one bot. May turn it on for more after more conversations.

BUG=skia:5628

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2231943002

Review-Url: https://codereview.chromium.org/2231943002
2016-08-11 12:36:58 -07:00
mtklein
18655efb7a GN: take over some exisiting bots
As an experiment, instead of replacing these with -GN twins, take
them over in-place.  This should take over:

  -FAST
  -SKFOO

CQ_INCLUDE_TRYBOTS=master.client.skia:Perf-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Debug-SK_USE_DISCARDABLE_SCALEDIMAGECACHE-Trybot,Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Debug-SK_USE_DISCARDABLE_SCALEDIMAGECACHE-Trybot,Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Release-SKNX_NO_SIMD-Trybot,Test-Ubuntu-GCC-GCE-CPU-AVX2-x86_64-Release-Fast-Trybot

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2229463002

Review-Url: https://codereview.chromium.org/2229463002
2016-08-11 07:51:53 -07:00
rmistry
5eab99183c Move ct_skps recipe from tools repo to Skia repo.
Also move the isolate file and script from Chromium repo to the Skia repo.

BUG=skia:5620
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2221413002

Review-Url: https://codereview.chromium.org/2221413002
2016-08-09 13:46:48 -07:00
borenet
dfa55f2db4 [recipes] Smush the rest of builder_spec into appropriate places, other cleanup
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2219873003

Review-Url: https://codereview.chromium.org/2219873003
2016-08-08 04:34:01 -07:00
mtklein
03591a762c finish SkLiteDL
- makeThreadsafe()
    - fix up some alignment problems
    - drawVertices / drawPatch
    - work around drawPicture() + default paint bugs?
    - turn on tests.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2219343002

Review-Url: https://codereview.chromium.org/2219343002
2016-08-07 13:35:47 -07:00
borenet
2c3112d517 [recipes] Increase test coverage
Remove a bunch of "pragma: no cover" by removing unused code or adding test bots to cover.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2215443003

Review-Url: https://codereview.chromium.org/2215443003
2016-08-05 06:30:10 -07:00
borenet
e2251acbaa [recipes] Separate builder_spec into compile/test/perf recipes
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2205473002

Review-Url: https://codereview.chromium.org/2205473002
2016-08-05 05:40:59 -07:00
borenet
d460a3c256 [recipes] Remove build environment vars from default_env
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2209423002

Review-Url: https://codereview.chromium.org/2209423002
2016-08-05 05:18:06 -07:00
msarett
b8d1aac87a Create a separate src for colorspace tests
This will allow me to run these tests in sRGB mode, while
leaving most of the image tests disabled.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2206953006

Review-Url: https://codereview.chromium.org/2206953006
2016-08-04 12:22:06 -07:00
borenet
3f5a47599b [recipes] Fixes for Coverage bot
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2217523002

Review-Url: https://codereview.chromium.org/2217523002
2016-08-04 12:16:03 -07:00
borenet
c7233ef925 [recipes] Stop pretending that buildbot_spec is a step, simplify
Since we no longer have to call out to obtain the buildbot_spec, merge
the two functions in vars API.  As a side effect, this applies
default_env to the sync steps as well, which shouldn't have an
appreciable effect on bot behavior.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2209343002

Review-Url: https://codereview.chromium.org/2209343002
2016-08-04 11:33:02 -07:00
borenet
538d5b68e8 Move builder_spec, [dm|nanobench]_flags, builder_name_schema to recipes
- builder_name_schema becomes its own recipe module.
- builder_spec, dm, and nanobench flags move into vars module.
- recipe expectation diffs include:
  - no more buildbot_spec.py step
  - "real" dm and nanobench flags, instead of --dummy-flags
  - some inconsequential stuff in visualbench, which is removed anyway.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2215803002

Review-Url: https://codereview.chromium.org/2215803002
2016-08-04 11:07:10 -07:00
borenet
bc20a701d8 [Recipes] Move test and perf steps into test and perf recipes
This looks like another big change, but I really just moved blocks of code around.  Again, no expectations diffs because no behavior is changed.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2198973002

Review-Url: https://codereview.chromium.org/2198973002
2016-08-03 10:38:44 -07:00
borenet
9e3d3d8b36 Fix fake_specs.py, update recipe expectation
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2200763002

Review-Url: https://codereview.chromium.org/2200763002
2016-08-03 10:10:07 -07:00
borenet
1436a09e1f Re-organize Skia recipes
Break Skia recipe module into:
  - skia_vars: defines and stores variables (eg. paths)
  - skia_step: utilities for running Skia steps
  - skia_flavor: flavor-specific stuff
  - skia: top-level setup, checkout, test/perf steps, etc etc

This establishes a saner dependency structure for the recipes; skia_vars
is at the bottom level, skia_step depends on it, skia_flavor depends on
both of them, skia depends on all of the above, and the recipes
themselves may depend on any or all of them.

Next steps:
  - Merge buildbot_spec into skia_vars
  - Move test_steps and perf_steps from skia recipe_module into
    swarm_test and swarm_perf recipes
  - Cleaner checkout_steps process

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2198173002

Review-Url: https://codereview.chromium.org/2198173002
2016-08-03 08:23:10 -07:00
borenet
b6aafe66a0 Fix Chrome build for RecreateSKPs and CommandBuffer bots for GYP->GN
BUG=skia:5597
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2199273002

Review-Url: https://codereview.chromium.org/2199273002
2016-08-02 07:02:52 -07:00
borenet
ed696276fd More fixes for SwarmBucket
BUG=skia:5581
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2202053005
NOTRY=true

Review-Url: https://codereview.chromium.org/2202053005
2016-08-02 06:50:50 -07:00
borenet
d6dd44140d Roll recipe DEPS
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2205863002

Review-Url: https://codereview.chromium.org/2205863002
2016-08-02 05:11:16 -07:00
borenet
f5b675efbe Stop running nanobench on Test.*Debug bots
We'll need to add Perf.*Debug bots to achieve the same coverage.

BUG=skia:4768
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2189393002
NOTREECHECKS=true

Review-Url: https://codereview.chromium.org/2189393002
2016-08-01 05:08:52 -07:00