'Run espresso test multiple times

Sometimes I faced with rare bug in my application. But I can't reproduce it as it's very rare. So, I decided to write simple espresso test:

@RunWith(AndroidJUnit4::class)
@LargeTest
class MainActivityTest {

    val password = "1234"

    @Rule @JvmField
    var mActivityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java)

    @Test
    fun checkNotesListNotEmpty() {
        onView(withId(R.id.password_edit_text)).perform(typeText(password))
        onView(withId(R.id.notes_recycler_view)).check { view, noMatchingViewException ->
            if (noMatchingViewException != null) throw noMatchingViewException
            assertThat((view as RecyclerView).adapter.itemCount,  Matchers.`is`(1))
        }
    }
}

How can I loop this test and stop it when matching fails?

Solution 1:^[1]

Use @Repeat annotation:

@RunWith(AndroidJUnit4.class)
public class MyTest {

    @Rule
    public RepeatRule repeatRule = new RepeatRule();

    @Test
    @Repeat(100)
    fun checkNotesListNotEmpty() {
    }

But you have to implement it yourself:

Repeat.java:

import static java.lang.annotation.ElementType.ANNOTATION_TYPE;
import static java.lang.annotation.ElementType.METHOD;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention( RetentionPolicy.RUNTIME )
@Target({ METHOD, ANNOTATION_TYPE })
public @interface Repeat {
    int value() default 1;
}

RepeatRule.java:

import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;

public class RepeatRule implements TestRule {

    private static class RepeatStatement extends Statement {
        private final Statement statement;
        private final int repeat;    

        public RepeatStatement(Statement statement, int repeat) {
            this.statement = statement;
            this.repeat = repeat;
        }

        @Override
        public void evaluate() throws Throwable {
            for (int i = 0; i < repeat; i++) {
                statement.evaluate();
            }
        }

    }

    @Override
    public Statement apply(Statement statement, Description description) {
        Statement result = statement;
        Repeat repeat = description.getAnnotation(Repeat.class);
        if (repeat != null) {
            int times = repeat.value();
            result = new RepeatStatement(statement, times);
        }
        return result;
    }
}

Solution 2:^[2]

Solution presented by @mklimek in Kotlin.

How to use:

@Rule @JvmField
var repeatRule: RepeatRule = RepeatRule()

@Test
@RepeatTest(100)
fun checkNotesListNotEmpty() {

RepeatRule.kt

class RepeatRule : TestRule {

    private class RepeatStatement(private val statement: Statement, private val repeat: Int) : Statement() {
        @Throws(Throwable::class)
        override fun evaluate() {
            for (i in 0 until repeat) {
                statement.evaluate()
            }
        }
    }

    override fun apply(statement: Statement, description: Description): Statement {
        var result = statement
        val repeat = description.getAnnotation(RepeatTest::class.java)
        if (repeat != null) {
            val times = repeat.value
            result = RepeatStatement(statement, times)
        }
        return result
    }
}

RepeatTest.kt

@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.FUNCTION, AnnotationTarget.ANNOTATION_CLASS)
annotation class RepeatTest(val value: Int = 1)

Solution 3:^[3]

Can't you have a separate test which loops that one? (if it is a one-of-its-kind situation):

@RunWith(AndroidJUnit4::class)
@LargeTest
class MainActivityTest {
    ...

    @Test fun checkNotesListNotEmpty() {...}

    @Test fun loopCheckNotesListNotEmpty() {
        while(true)
            checkNotesListNotEmpty()
    }
}

Solution 4:^[4]

Android Studio allows you to run a test N times or run until it fails. Click on "Edit configurations" for your test rule and search for the "Repeat:" setting.

Screenshot

Solution 5:^[5]

I had the same issue and as a result I've created a library to run test multiple times: https://github.com/stepstone-tech/AndroidTestXRunner

Once you run it, you can check the tests results for any failures.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Yuliia Ashomok
Solution 2	molundb
Solution 3	voddan
Solution 4	Enamul Hassan
Solution 5

'Run espresso test multiple times

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]

Solution 5:[5]