Unhealthy Test annotations for failed tests

When tests fail against your PR, the results must be triaged. You need to go through each failure, understand what happened, and decide what to do next.

Hopefully, all of the failures relate to your changes, and you can get to work figuring out how to fix each failure. However, if the test suite is occasionally unstable, things might not be so straightforward. If a test is failing on the main branch, or if a test is flaky, it might fail on your branch, too.

In this case, you need to determine whether your changes caused the failure, or if there is something else going on. This is where Launchable comes in!

As you may already know, Launchable calculates lists of Top Unhealthy Tests in your test suite. These are the Top Flaky Tests, Top Never Failing Tests, Top Always Failing Tests, and Top Longest Tests.

This information is useful for triage. For example, if a test is a Top Never Failing test, but it failed in your PR, then it's likely that your change broke this test! Similarly, if a test is a Top Always Failing test, then the opposite is probably true: your change probably didn't cause this failure.

To make this information super-easy to access, we now display Unhealthy Test annotations for failed tests on the Test Session details page where you view test results:

(Psst! You can easily reach this page from the link in your build logs.)