JUnit5 - How to get test result in AfterTestExecutionCallback

Question

I write JUnit5 Extension. But I cannot find way how to obtain test result.

Extension looks like this:

import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.TestExtensionContext;

public class TestResultExtension implements AfterTestExecutionCallback {
    @Override
    public void afterTestExecution(TestExtensionContext context) throws Exception {
        //How to get test result? SUCCESS/FAILED
    }
}

Any hints how to obtain test result?

Answer 1

As other answers point out, JUnit communicates failed tests with exceptions, so an AfterTestExecutionCallback can be used to gleam what happened. Note that this is error prone as extension running later might still fail the test.

Another way to do that is to register a custom TestExecutionListener. Both of these approaches are a little roundabout, though. There is an issue that tracks a specific extension point for reacting to test results, which would likely be the most straight-forward answer to your question. If you can provide a specific use case, it would be great if you could head over to #542 and leave a comment describing it.

Answer 2

This work for me:

public class RunnerExtension implements AfterTestExecutionCallback {

    @Override
    public void afterTestExecution(ExtensionContext context) throws Exception {
        Boolean testResult = context.getExecutionException().isPresent();
        System.out.println(testResult); //false - SUCCESS, true - FAILED
    }
}

@ExtendWith(RunnerExtension.class)
public abstract class Tests {

}

Answer 3

You can use SummaryGeneratingListener from org.junit.platform.launcher.listeners

It contains MutableTestExecutionSummary field, which implements TestExecutionSummary interface, and this way you can obtain info about containers, tests, time, failures etc.

You can create custom listener, for example:

1. Create class that extends SummaryGeneratingListener

public class ResultAnalyzer extends SummaryGeneratingListener {
    @Override
    public void testPlanExecutionFinished(TestPlan testPlan) {
        //This method is invoked after all tests in all containers is finished
        super.testPlanExecutionFinished(testPlan);
        analyzeResult();
    }

    private void analyzeResult() {
        var summary = getSummary();
        var failures = summary.getFailures();
        //Do something
    }
}

2. Register listener by creating file

src\main\resources\META-INF\services\org.junit.platform.launcher.TestExecutionListener

and specify your implementation in it

path.to.class.ResultAnalyzer

3. Enable auto-detection of extensions, set parameter

-Djunit.jupiter.extensions.autodetection.enabled=true

And that's it!

Docs

https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/SummaryGeneratingListener.html

https://junit.org/junit5/docs/5.0.0/api/org/junit/platform/launcher/listeners/TestExecutionSummary.html

https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic

Answer 4

I have only this solution:

String testResult = context.getTestException().isPresent() ? "FAILED" : "OK";

It seems that it works well. But I am not sure if it will work correctly in all situations.

Answer 5

Fails in JUnit are propagated with exceptions. There are several exceptions, which indicate various types of errors.

So an exception in TestExtensionContext#getTestException() indicates an error. The method can't manipulate actual test results, so depending on your use case you might want to implement TestExecutionExceptionHandler, which allows you to swallow exceptions, thus changing whether a test succeeded or not.

Answer 6

You're almost there.

To implement a test execution callback and get the test result for logging (or generating a report) you can do the following:

import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
import org.junit.jupiter.api.extension.ExtensionContext;

public class TestResultExtension implements AfterTestExecutionCallback
{
    @Override
    public void afterTestExecution(ExtensionContext context) throws Exception
    {
        // check the context for an exception 
        Boolean passed = context.getExecutionException().isEmpty();
        
        // if there isn't, the test passed
        String result = passed ? "PASSED" : "FAILED";

        // now that you have the result, you can do whatever you want 
        System.out.println("Test Result: " + context.getDisplayName() + " " + result);
    }
}

And then you just add the TestResultExtension using the @ExtendWith() annotation for your test cases:

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

import static org.junit.jupiter.api.Assertions.assertTrue;

@ExtendWith(TestResultExtension.class)
public class SanityTest
{
    @Test
    public void testSanity()
    {
        assertTrue(true);
    }

    @Test
    public void testInsanity()
    {
        assertTrue(false);
    }
}

It's a good idea to extend a base test that includes the extension

import org.junit.jupiter.api.extension.ExtendWith;

@ExtendWith(TestResultExtension.class)
public class BaseTest
{}

And then you don't need to include the annotation in every test:

public class SanityTest extends BaseTest
{ //... }