Commit Graph

25 Commits

Author SHA1 Message Date
Mike Klein
acb8be89d1 add retries to fm_bot
Change-Id: Ibcc6ee83cde537caaab824658721bbda300abc18
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/208273
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-15 20:41:24 +00:00
Mike Klein
bf15b66768 dump stack trace in FM on failure
In CrashHandler...
  - make CrashHandler a non-noop... don't know why it's disabled
  - fix bitrot since we last built it
  - update to demangle symbols on Linux too, not just Mac
  - catch SIGTRAP, which will catch SK_ABORT / SkASSERT,
    unless otherwise hooked (e.g. by a debugger)

In fm...
  - use CrashHandler
  - convert exit_with_failure to SK_ABORT so they'll also dump a trace
  - flush stdout after printing what's running

Change-Id: Ib20d0e4f442d73c28e193396dc6e85935fc58544
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/208151
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
2019-04-15 19:15:39 +00:00
Brian Osman
ca9465688f Make Skottie's GN setup more DLL friendly
Skottie doesn't currently build with is_component_build=true. Trying to
build a DLL with skia_enable_skottie=false failed, because we ended up
with a component("skottie") that had no sources. That led to fallback
linker behavior, but no DllMain.

To solve this, and simplify things, move the skia_enable_skottie checks
to the outer scope, and simply replace all the referenced components
with empty groups when it's not enabled. Also fix some fm code that was
assuming it was always enabled.

Change-Id: I4a47d80d882e6c557ee14b34255e22e09292cc8c
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/207302
Reviewed-by: Mike Klein <mtklein@google.com>
Reviewed-by: Florin Malita <fmalita@chromium.org>
Commit-Queue: Brian Osman <brianosman@google.com>
2019-04-10 18:35:25 +00:00
Mike Klein
a833cff194 some refactoring ideas
- streamline source initialization so that each
    kind of source only has to mention what's interesting
    to it, with defaults for some obvious fields.

  - have sources return a Result rather than a bool,
    encapsulating Ok/Skip/Fail and any failure message,
    but grab this in a middle draw() layer so each backend
    doesn't have to care about anything but Ok/Skip.

Change-Id: I715278ba5e05892e42c8875bd607160b0cc594b5
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206763
Auto-Submit: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-09 17:54:36 +00:00
Mike Klein
7b8bc53b93 --parameters -> --properties
This was a brain fart when I wrote it the first time;
the expected flag name is --properties.

    ~/skia (refactoring↑1|…) $ git grep DEFINE_string\(properties
    bench/nanobench.cpp:static DEFINE_string(properties, "",
    dm/DM.cpp:static DEFINE_string(properties, "",
    tools/fm/fm.cpp:static DEFINE_string(properties, "",

Change-Id: I9ada23e26a748bd6bc85ae5682bd03c130edc054
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206941
Commit-Queue: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-09 17:08:13 +00:00
Brian Osman
5aa11fb67a Shader serialization experiment (with Mali Offline Compiler analysis)
1) Adds a --writeShaders option to fm. When that's included, the GPU
backend is run with a persistent cache (and no binary caching).
Then we dump all of the fragment shaders (GLSL for now) that were
created to the passed-in directory, along with a JSON digest listing
the number of times each one was referenced in the cache.

2) Adds a python script that invokes the Mali Offline Compiler on a
directory generated from #1, scraping the output of each compile for
cycle counts and writing the collated information to stdout.

Change-Id: Ie4b58ddac4f62e936707c6fec44f4fe605c213fa
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206162
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
2019-04-09 13:37:55 +00:00
Mike Klein
befc7cf165 fm_bot: look for and explain too-late flags
Change-Id: I9746fb5215d449b030ecfde6051fa6a69d2e7556
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206580
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Mike Reed <reed@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
2019-04-08 15:55:10 +00:00
Mike Klein
ccdd6a14ac fix -script parsing
The ':' introduced a new scope, and we would always wait on the outside
`file`, stdin.

Change-Id: I5176a80abd40650a0167150ff5e996613ecc93e7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206581
Commit-Queue: Mike Klein <mtklein@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Reviewed-by: Mike Reed <reed@google.com>
2019-04-08 13:31:12 +00:00
Hal Canary
0f2f522246 SkMD5 returns digest
Change-Id: I9eb9095131adb5862d286b211501bf194ef8e91e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205822
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2019-04-03 17:52:22 +00:00
Mike Klein
d0389c596d split cpu+gpu work queues
Split work queues for CPU and GPU, assigning using a --backend
whitelist.  This lets us up the CPU process limit to NumCPUs,
while keeping the GPU process limit at 8.

On my desktop this speeds up this mixed workload about 2x:

cat <<EOF | go run tools/fm/fm_bot.go -quiet -script - out/fm
    b=cpu gms ct=8888 w=$WHERE/8888 legacy=true  # old and busted
    b=cpu gms ct=8888 w=$WHERE/srgb              # new hotness
    b=gl  gms ct=8888 w=$WHERE/gl
    b=mtl gms ct=8888 w=$WHERE/mtl
    b=cpu gms ct=565  w=$WHERE/565
    b=cpu gms ct=f16  w=$WHERE/f16
    b=skp gms         w=$WHERE/skp
EOF

Before:        44.37 real       303.87 user        15.40 sys
After:         21.87 real       480.05 user        27.10 sys

Change-Id: Ibea4537c8cdf278f8e4621f26ebacd942952eec3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205833
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
2019-04-03 17:12:33 +00:00
Mike Klein
1450a47409 support comments
E.g.

    cat <<EOF | go run tools/fm/fm_bot.go -processLimit 0 -quiet -script - out/fm
        b=cpu gms ct=8888 w=$WHERE/8888 legacy=true  # old and busted
        b=cpu gms ct=8888 w=$WHERE/srgb              # new hotness
    EOF

Change-Id: Ia92ba113ae95478313e1f2cc18b7270ea1aa8a42
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205831
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
2019-04-03 16:00:00 +00:00
Mike Klein
4daf637187 add --legacy to fm, legacy=true to fm_bot
E.g.

    ~/skia (make-from-memory↑2|●1…) $ cat test.sh
    #!/bin/bash

    set -e
    set -x

    WHERE=${1:-foo}

    ninja -C out fm

    rm -rf $WHERE
    mkdir -p $WHERE

    cat <<EOF | go run tools/fm/fm_bot.go -processLimit 0 -quiet -script - out/fm
        b=cpu gms ct=8888 w=$WHERE/8888 legacy=true
        b=cpu gms ct=8888 w=$WHERE/srgb
        b=cpu gms ct=565  w=$WHERE/565
        b=cpu gms ct=f16  w=$WHERE/f16
    EOF

    ~/skia (make-from-memory↑2|●1…) $ time ./test.sh
    + WHERE=foo
    + ninja -C out fm
    ninja: Entering directory `out'
    ninja: no work to do.
    + rm -rf foo
    + mkdir -p foo
    + cat
    + go run tools/fm/fm_bot.go -processLimit 0 -quiet -script - out/fm
           11.81 real       318.92 user        14.51 sys

    ~/skia (make-from-memory↑2|●1…) $ idiff foo/8888/ foo/srgb/
    671 files are identical.
    25 diffs written to diff.html

Change-Id: I01fe1ceeada971d8d18efc11961b9f753624fcf2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205830
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
2019-04-03 15:52:40 +00:00
Mike Klein
1b56cdbd9b -script - for stdin
These do the same thing:

 $ go run tools/fm_bot.go out/fm b=cpu gms
 $ echo "b=cpu gms" | go run tools/fm/fm_bot.go -s - out/fm

~/skia (make-from-memory↑2|…) $ cat test.sh

set -e
set -x

mkdir -p foo
cat <<EOF | go run tools/fm/fm_bot.go -j 0 -q -s - out/fm
    b=cpu ct=8888 gms w=foo/8888
    b=cpu ct=565  gms w=foo/565
    b=cpu ct=f16  gms w=foo/f16
EOF
~/skia (make-from-memory↑2|…) $ time ./test.sh
+ mkdir -p foo
+ cat
+ go run tools/fm/fm_bot.go -j 0 -q -s - out/fm
        9.47 real       234.71 user         9.12 sys

Change-Id: If0e8de2772f12de4257671d24f70662c423581c3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205826
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
2019-04-03 15:34:00 +00:00
Mike Klein
408c3eaada don't have fm_bot send fm empty sources to run
Running fm_bot at head I'm confronted with a puzzling error:

    2019/04/03 10:26:01
    out/fm -b cpu -w foo -s big_rrect_rect_aa_effect filterbitmap_image_mandrill_512.png                    #failed (exit status 1):
    Don't understand --source ... bailing out.

    2019/04/03 10:26:05 1 invocations of out/fm failed

And of course running that command works fine.  Puzzling!

To fix this, we must take care not to subslice sources past its end,
which fills with defaults, a bunch of extra "" sources.  And add some ''
around the unknown source to make the error message easier to parse.

Serves me right for not testing this before I landed that tweak...

Change-Id: I5c2abb35733b32a1142b0f42c4323112969ad2bb
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205825
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
2019-04-03 15:15:04 +00:00
Mike Klein
176da0bb8a tiny tweak: no need to reload lottie files
Change-Id: I759637016becb5e257c21134224ad44129c28ab0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205760
Auto-Submit: Mike Klein <mtklein@google.com>
Commit-Queue: Florin Malita <fmalita@chromium.org>
Reviewed-by: Florin Malita <fmalita@chromium.org>
2019-04-03 13:29:01 +00:00
Mike Klein
cbe93ee197 have FM run GMs in command line order
Compare running

   $ out/fm -b gl -s encode skps/desk_wikipedia.skp drawbitmaprect lotties/inattentive.json xfermodeimagefilter encode encode encode

Before:
           xfermodeimagefilter	92218904a9ae8a0a6d3eee03541b632a	    240ms
                        encode	d2d6eb57feb83662a84dc728dbd4649d	     99ms
                drawbitmaprect	d915acc291ecd0e7bbbd8b497d44cd1e	     88ms
            desk_wikipedia.skp	2bd4560dfba002395b7d5665d37a5d83	     80ms
              inattentive.json	63dfb23e06ee2557bb441523003fb4eb	     43ms

After:
                        encode	d2d6eb57feb83662a84dc728dbd4649d	    116ms
            desk_wikipedia.skp	2bd4560dfba002395b7d5665d37a5d83	     80ms
                drawbitmaprect	d915acc291ecd0e7bbbd8b497d44cd1e	     93ms
              inattentive.json	63dfb23e06ee2557bb441523003fb4eb	     42ms
           xfermodeimagefilter	92218904a9ae8a0a6d3eee03541b632a	    214ms
                        encode	d2d6eb57feb83662a84dc728dbd4649d	     90ms
                        encode	d2d6eb57feb83662a84dc728dbd4649d	     89ms
                        encode	d2d6eb57feb83662a84dc728dbd4649d	     89ms

Change-Id: I448da4c991b91a8f3e4fc0cf32af0a4ff712e19e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205591
Commit-Queue: Mike Klein <mtklein@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-03 13:01:08 +00:00
Mike Klein
832d3c5007 update some FM flags
fm itself can always be verbose.  We'll rarely call it directly
and when we do and we don't want to see its output, we can always
pipe to /dev/null.

Switch fm_bot.go over to random shuffle by default, and to verbose by
default with a -q/-quiet flag to only print failures.

Change-Id: Ibd53bf7f1d29e0fd5deb7d4689dc0ca100df648f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205587
Commit-Queue: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-02 20:09:44 +00:00
Mike Klein
2292426cb1 add skottie support to fm
Draws the same as DM.

Change-Id: I741655786c3345dfd824438f9dcaca0d5046b197
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205493
Reviewed-by: Florin Malita <fmalita@chromium.org>
2019-04-02 19:14:44 +00:00
Mike Klein
a5c27173ea allow skipped source/backend pairs in fm
GMs sometimes want to skip instead of drawing or failing.
E.g. GPU-only GMs with a non-GPU backend.

Change-Id: I5bd8acecbef1aa88b4d7e6cfb88dfde04424aa8b
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/205349
Reviewed-by: Mike Klein <mtklein@google.com>
2019-04-02 19:08:44 +00:00
Mike Klein
f2d9b865d7 Add basic fm_bot script
This handles batching and things like figuring out "skps" means "all the
files under the directory skps", "gms" means "all the GMs linked into
fm", etc.

You can set up one job on the command line, or any number of them
with a -script file, e.g.

    ~/skia (fm-bot↑1|●1…) $ cat fm.script
    gms b=cpu w=foo/8888
    gms b=cpu w=foo/565  ct=565
    skps b=gl w=foo/msaa4-skps samples=4

Change-Id: I878c623bfc8673f04b4effa247bc850c0258a89b
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/204582
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-02 15:51:04 +00:00
Mike Klein
ce90d6f088 more opinionated file loading in fm
Only try to parse .skp files as SkPictures, only .svg
files as SVG, and only try to parse any given file once.

This cleans up error logging from, e.g., trying to
parse a .jpg as an .svg.  Sometimes even asserts.

Change-Id: Ibfab76e201eb273107b96fcbd3af94d397862291
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/204882
Auto-Submit: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-04-01 14:10:13 +00:00
Mike Klein
6253d90de8 add .svg sources to fm
Change-Id: Ic364059cb277773a5d5f8011be2b68b757e889f2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/204280
Commit-Queue: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-03-27 20:42:50 +00:00
Mike Klein
c245bd9077 add --backend pdf to fm
Change-Id: I6ad822d606ecdf9a71a3626ebc3b19d524dbaf72
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/204138
Reviewed-by: Brian Osman <brianosman@google.com>
2019-03-27 20:08:57 +00:00
Mike Klein
9b46209867 add --backend skp to fm
And move --backend first among the backend flags.

Change-Id: I853ede2a8218f046c968ace2ce8c5e277421e225
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/204133
Auto-Submit: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-03-27 19:55:28 +00:00
Mike Klein
735c7ba320 FM, a dumber new testing tool
FM's a bit like DM, and a bit like the old GM tool, maybe closer to GM.

FM always does one serial run through of a list of GM, image, or .skp
sources (-s/--sources) into one drawing configuration, set at a high
level with -b/--backend, fine-tuned by flags like --ct, --at, --gamut,
--samples, etc.

FM prints all available GMs when run with no arguments, part of
how it integrates well with external parallelization like xargs:

   out/fm | time xargs -n4 -P32 out/fm -b cpu ... -s
   out/fm | time xargs -n32 -P8 out/fm -b gl ... -s
   ls *.skp | xargs ... out/fm ... -s
   ...

TODO later:
   vector backends: --backend pdf, svg, xps, skp
   --source *.svg, *.json (Skottie)
   Create/update a dm.json file

Change-Id: Idacbd2785deaf6f1d33674c66c4d984fa91dbd85
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/203520
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2019-03-27 14:00:58 +00:00