This should help with non-skia.git repos which use gen_tasks_logic.
Change-Id: I84bf101554555dd42e9349c93aec39280b7de84f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/377837
Reviewed-by: Weston Tracey <westont@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
Change-Id: Ibe4e929b7d160b3dbf0e5b6ae093b4bab419817d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375198
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2/4 of the machines upgraded their drivers, confirmed jobs work on
updated machines and upgraded 1 of the outdated machines to match.
Will upgrade the last one after this lands.
Change-Id: Ib1485767733b3907d0bac3d6c341fa8e434aa0c6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375614
Commit-Queue: Weston Tracey <westont@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
to use 10.15.7.
It seems the 2 bots we had running 10.15.1 in Golo were upgraded in the past few days.
Change-Id: Id2e200ff8d0b2fe49ca5a01c99cf38c4051ef442
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375612
Commit-Queue: Weston Tracey <westont@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
All other tests (DLL, etc) appear to work with this change, but until
we reallocate some more Golo VMs, we don't have capacity to run
more than a few.
Bug: skia:10782
Change-Id: I909dcdd4ce3a597062e6286e3391854fe7229d90
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375121
Reviewed-by: Eric Boren <borenet@google.com>
This time, push host/* instead of host/. This cuts the time it takes to
push resources from ~11m to ~8s on Android One phones.
I tried ignoring version files and blindly pushing packages, but in the
end kept the version files for compatibility with bots running at head
(or after this lands, bisecting bots), and those version files still do
save a good chunk of time. In PS12 I faked version file mismatch and it
took ~2m extra to push all the packages back.
I also tried but got nervous about --sync so I've left it for a follow
up. It's not clear that we can rely on it for packages where files will
be removed, but it's probably fine for resources/ if we want to go
faster than 8s.
Change-Id: I84de90f13177393e4d8a599ef880545538b6af81
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375076
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
Change-Id: I616d2a119886d04e2b21499aece3e042226f6f3f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375057
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This reverts commit 9de609c874.
Reason for revert: been long enough?
Original change's description:
> Temporary fix for bad data pushed to Android test devices
>
> A change to Android asset deployment left some devices with bad data
> (but correct _VERSION files). Unless we force-push assets, test runs
> will continue to fail.
>
> Once all devices have had their assets re-pushed (and any bisection
> into the offending time period is concluded), we can revert this CL.
>
> Change-Id: Ibbfe48409b99e8313186dde1dc0cc3cfb5061d4c
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372457
> Reviewed-by: Eric Boren <borenet@google.com>
> Reviewed-by: Mike Klein <mtklein@google.com>
> Commit-Queue: Mike Klein <mtklein@google.com>
> Commit-Queue: Brian Osman <brianosman@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I3a53f2da59bb947d7579a63359e0d2c349c4ce36
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/374637
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
Auto-Submit: Mike Klein <mtklein@google.com>
Also rename //modules/canvaskit/canvaskit to //modules/canvaskit/npm_build
to make it more clear the purpose of that folder (what we ship to
npm and stage our builds for local testing).
Bug: skia:11203
Change-Id: I4299ded97d14f4155c36798d60e88a660ce6fe6a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372392
Reviewed-by: Kevin Lubick <kjlubick@google.com>
A change to Android asset deployment left some devices with bad data
(but correct _VERSION files). Unless we force-push assets, test runs
will continue to fail.
Once all devices have had their assets re-pushed (and any bisection
into the offending time period is concluded), we can revert this CL.
Change-Id: Ibbfe48409b99e8313186dde1dc0cc3cfb5061d4c
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372457
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
This reverts commit 81a58cfe3b.
Reason for revert: Breaks with CIPD packages / symlinks?
Original change's description:
> Simplify pushing directories to android devices
>
> Change-Id: I5f0ce6720be821ae3825b58afc94dbeb2f50e2d8
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372217
> Reviewed-by: Mike Klein <mtklein@google.com>
> Commit-Queue: Mike Klein <mtklein@google.com>
TBR=mtklein@google.com,brianosman@google.com
Change-Id: I1c0a35a7a149292aa1fe7c3ab0452c4203a9f357
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372477
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Change-Id: I5f0ce6720be821ae3825b58afc94dbeb2f50e2d8
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/372217
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
On batch failure we're rerunning every source in the batch, while we
really only need to rerun sources that we don't know succeeded.
If for example we run sources "foo", "bar", and "baz", and foo produces
a known hash, then bar crashes, we only need to rerun bar and baz. The
batch run was enough to demonstrate foo's good.
Change-Id: I17634a6095906bcc2ad0bd33bb78eba000654b5e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/369456
Reviewed-by: Eric Boren <borenet@google.com>
Move definition of Work struct until just before it's used,
and show one of the sources as an example at kickoff-level step.
These are just cosmetic/refactors.
Change-Id: Ib23b9379683b9867e097c8d68ef8736013719cee
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/369356
Reviewed-by: Eric Boren <borenet@google.com>
Disabled on Adreno 5xx/6xx as the tests do not pass on those GPUs:
http://screen/3Dkgs9syj37cjBV
Change-Id: Ib935d01e8f06dbfe7decd5cc4e52e0688b48be08
Bug: skia:11306, skia:11308
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/368805
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Auto-Submit: John Stiles <johnstiles@google.com>
If we track how many pending batches a kickoff()
has in flight, we can endStep() it properly when
that number hits zero.
This double sync.WaitGroup trick is pretty neat.
Now we're thinking with portals...
Added some comments to prevent myself falling in
the trap of assuming we'll have runtime.NumCPU()
batches... rounding the batch size up means we'll
sometimes have fewer.
Change-Id: If50615c204485862462c240b9bbdfd4ddbad43b2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366142
Reviewed-by: Eric Boren <borenet@google.com>
It's nice to see it in the task log, and to be able to see
it's not there when we're not working with Gold (*SAN) bots.
(One trybot of each kind here.)
Change-Id: Ibb4aa20badf95ef603f3890e1c8248cad675507f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366143
Reviewed-by: Eric Boren <borenet@google.com>
Group batches from a single kickoff() into another mid-level step:
Top-Level
kickoff --some flags
batch sources...
batch (exec)
batch other sources ...
batch (exec)
rerun (exec)
rerun (exec)
batch yet other sources ...
batch (exec)
rerun (exec)
kickoff --some other flags
...
Big question: is it okay for the kickoff steps to td.EndStep() while its
kids are still running (or haven't even started) on other goroutines?
Change-Id: I77ad2274e35cea0151be0cca6c690eafc4f8983e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366140
Reviewed-by: Eric Boren <borenet@google.com>
There are bots (*SAN) that won't ever be uploading to Gold,
so *bot != "" doesn't really describe the right condition.
We could do this logic inside fm_driver.go based on *bot,
but I kind of want the flexibility to do things like upload
local ad-hoc runs or sanitizer runs if we want using --gold.
Change-Id: Id972d8b0097616c5b2802bc99c2718fdd1568fe3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366139
Reviewed-by: Mike Klein <mtklein@google.com>
Why have other bots when we can do it all here?
Change-Id: I6a3f3c2ed5d19a3b8ecf59f44cc0d2f6076bba7f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366138
Reviewed-by: Mike Klein <mtklein@google.com>
There's no need to tick wg up and down when running reruns, and as
written it's possible for the overall fm_driver program to exit before
one last call to endStep() has happened. Simply calling wg.Done() once
per item dequeued outside worker() fixes both.
Change-Id: I0fb0acc5a3f2c624dfc14f875fa094db6dd40838
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/366137
Reviewed-by: Mike Klein <mtklein@google.com>
If a batch fails, we've got to rerun everything (or at least from the
failure on), but when it's merely unknown hashes, no need to rerun
what's produced hashes we know already.
Small tweak to FM to keep all the printed source names exactly what's
passed in, keeping the whole path for skp/svg/image files. This means
zero bookkeeping needed to know what to rerun when parsing that output.
Change-Id: I1e7ed3ee51158b68a6bdd3152560f3a282109576
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/365818
Reviewed-by: Mike Klein <mtklein@google.com>
Now with ctx scoping fixed,
and steps nested just how I like them.
Change-Id: Ifa43a432faddbafaae118ab0b16f710b695b5377
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/365504
Reviewed-by: Eric Boren <borenet@google.com>
These *SAN bots should be able to replace some of the Test- bots on our
tree, and the MSVC one is just another we'll need eventually.
(Similarly we'll want GCC on Linux, but I don't know how to Docker.)
Change-Id: Ied4519626f1e13bb31fcb30f37cbd1b24133aa71
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/365597
Reviewed-by: Eric Boren <borenet@google.com>
I've been waiting to replace bots until they were uploading to Gold, but
these *SAN bots don't upload anything, so we're at parity there already.
I've just remembered the Mac ASAN/TSAN bots and the Windows ASAN bots,
so I'll be following up to replace them too.
Change-Id: Ia3edfda64091e4407a1131073829b74f22b32b71
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/365596
Reviewed-by: Eric Boren <borenet@google.com>
I swear yesterday we had ~10 unused 10.14 bots...
Anyway, as with the last move, I don't really care where these bots are
running, so long as they have enough machines to schedule reaonably
quickly. This switches their spec to the default "Mac", which is
currently 19 VMs running 10.15.7 on trashcan Mac Pros.
Similarly, Win2019 -> Win. This is a change in bot name only, just to
capture the "I'll take whatever's default" spirit. I'd use Linux or
Debian too if there were one.
Change-Id: Ifa7615735c660018a5f3f46f4d8035e0b5bf8141
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/365518
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Weston Tracey <westont@google.com>
This also marks the glorious return of td.FailStep() as the answer to my
question "now how do I find my failures in this giant list?"
Change-Id: I15f98862d77942f2e289dc626da8643789a91d48
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/364838
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
I don't really care what machine class it's running on right now, and
status.skia.org/capacity says these 10.14.6 bots are the least loaded
of any bot on the tree.
Change-Id: Ie49b2659e99d60e450235207bbbc018e565636b4
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/364716
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
