Retrying Failed Tests

Manually re-running failed tests

At the end of the test suite run, by default, testng-failed.xml is created in the output directory containing only the failed tests. We can run this test suite to rerun only the failed tests.

Programmatically re-running failed tests

To programmatically trigger the failed tests right after the tests fail, we can use IRetryAnalyzer. Implementation of the SampleRetryAnalyzer needs to be added to test methods to enable this.

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
 
public class SampleRetryAnalyzer implements IRetryAnalyzer {
 
  private int retryCount = 0;
  private static final int maxRetryCount = 2;
 
  @Override
  public boolean retry(ITestResult result) {
    if (retryCount < maxRetryCount) {
      retryCount++;
      return true;
    }
    return false;
  }
}

The above code snippet is to tell TestNG how many times the retrying of failed tests should be attempted. This implementation requires adding @Test(retryAnalyzer = SampleRetryAnalyzer.class) to every test method.

import org.testng.Assert;
import org.testng.annotations.Test;
 
public class SampleTestClass {
 
  @Test(retryAnalyzer = SampleRetryAnalyzer.class)
  public void testMethod() {
    Assert.fail();
  }
}

To avoid this repetition and make this globally applicable for all the test methods, we can make sure of IAnnotationTransformer listener of TestNG as:

public class RetryListener implements IAnnotationTransformer {

    public void transform( ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod ) {
        annotation.setRetryAnalyzer(SampleRetryAnalyzer.class);
    }
    
}

And this listener needs to be added in the test class like @Listeners(com.example.RetryListener.class) or in testng.xml as:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Sample Test Suite" parallel="tests" thread-count="5">
    <listeners>
        <listener class-name="com.example.RetryListener" />
    </listeners>
    <test name="Sample Test">
        ...
    </test>
</suite>

---

That is how the failed tests can be tried again. In the next lesson, you will learn how to make your own annotations.