Using reflection to identify if the error thrown matches expected error

Question

I need to write a simple code tester program, but I got stuck comparing the given error class with the test expected class. I am supposed to use reflection in this exercise.

I have my code testing class:

public class TestRunner {

    private String result = "";

    public void runTests(List<String> testClassNames) {
        for (String testClassName : testClassNames) {
            Class<?> clazz;
            try {
                clazz = Class.forName(testClassName);
            } catch (ClassNotFoundException e) {
                throw new IllegalStateException("No such class.");
            }
            Method[] methods = clazz.getMethods();
            for (Method method : methods) {
                if (method.getAnnotation(MyTest.class) != null) {
                    if (testClassName.equals("reflection.tester.ExampleTests1")) {
                        result += method.getName() + "() - ";
                        ExampleTests1 instance = new ExampleTests1();
                        try {
                            // if null, result = OK
                            method.invoke(instance);
                            result += "OK\n";
                        } catch (IllegalArgumentException | IllegalAccessException | InvocationTargetException e) {
                            // if error is caught result = FAILED
                            result += "FAILED\n";
                        }
                    } else {
                        // the second class. should only return "OK" if the error is implemented from the exception class
                        result += method.getName() + "() - ";
                        ExampleTests2 instance = new ExampleTests2();
                        try {
                            method.invoke(instance);
                            result += "FAILED\n";
                        } catch (RuntimeException e) {
                            Throwable original = e.getCause();
                            Object expected = method.getReturnType();
                            if (original.getClass().isAssignableFrom(expected.getClass())) {
                                result += "OK\n";
                            } else {
                                result += "FAILED\n";
                            }
                        } catch (InvocationTargetException | IllegalAccessException e) {
                            result += "ERROR\n";
                        }
                    }
                }
            }
        }
    }
}

Also have two test classes. In the first one there is only one rule, if the test won't throw an exception the test should pass, and it is working. The second class is more complicated. If the thrown error class is implemented or same to the expected error class then the test should pass and OK should be added to the result. Currently my code won't catch RunTimeException at all and moves to the last catch block. How can I fix this?

I will also add the test class for more information.

public class ExampleTests2 {

    @MyTest(expected = RuntimeException.class)
    public void test3() {
        throw new IllegalStateException();
    }

    @MyTest(expected = IllegalStateException.class)
    public void test4() {
        throw new RuntimeException();
    }

    @MyTest(expected = IllegalStateException.class)
    public void test5() {
        throw new IllegalStateException();
    }

    @MyTest(expected = IllegalStateException.class)
    public void test6() {
    }

    public void helperMethod() {
    }


}

test3() and test5() should pass, test4() and test6() should fail, helperMethod() won't be checked because I only need to use the tests with @MyTest annotation.

Answer 1

JUnit has an assertThrows method that checks that an Exception is thrown. It has a method signature of

static <T extends Throwable> assertThrows?(Class<T> expectedType, Executable executable){}

Here's the documentation: https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/Assertions.html#assertThrows(java.lang.Class,org.junit.jupiter.api.function.Executable)

and here's how JUnit implements it: https://github.com/junit-team/junit5/blob/main/junit-jupiter-api/src/main/java/org/junit/jupiter/api/AssertThrows.java