TestNG: Complete Testing Guide for Developers

Introduction to TestNG

TestNG (Test Next Generation) is a powerful, open-source testing framework inspired by JUnit and NUnit, but designed with additional features that make it more flexible and suitable for modern testing needs. Created by Cédric Beust, TestNG has become the go-to framework for Java developers who need to write unit, functional, integration, and end-to-end tests with minimal boilerplate and maximum control.

Unlike traditional testing frameworks, TestNG introduces concepts like test groups, parameterized tests, dependent methods, and powerful parallel execution capabilities. These features make it especially popular in enterprise environments, Selenium-based UI automation, and API testing workflows.

Why TestNG Matters

TestNG addresses several limitations found in older testing frameworks. Understanding its advantages helps you decide when and why to adopt it in your projects.

Key Advantages

• Flexible test configuration: Annotations like @BeforeSuite, @BeforeClass, and @BeforeMethod give you fine-grained control over setup and teardown at multiple levels.
• Test grouping: You can group tests logically (e.g., "smoke", "regression") and run only the groups you need.
• Parameterized testing: Pass parameters through XML files or data providers, enabling data-driven testing without external libraries.
• Dependent methods: Define execution order and dependencies between test methods using dependsOnMethods or dependsOnGroups.
• Parallel execution: Run tests in parallel across threads or even machines, dramatically reducing execution time.
• Built-in reporting: TestNG generates detailed HTML and XML reports out of the box.

Setting Up TestNG

To start using TestNG, you need to add it as a dependency. The most common approach is through Maven.

Maven Dependency

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
</dependency>

Gradle Dependency

dependencies {
    testImplementation 'org.testng:testng:7.10.2'
}

test {
    useTestNG()
}

Once the dependency is added, your IDE (IntelliJ IDEA or Eclipse) will recognize TestNG annotations and allow you to run tests directly.

Core Annotations

TestNG provides a rich set of annotations that control the lifecycle of your tests. Understanding these is essential for writing clean, maintainable test suites.

Annotation Lifecycle

• @BeforeSuite / @AfterSuite: Run once before/after all tests in the suite.
• @BeforeTest / @AfterTest: Run before/after each <test> tag in the XML file.
• @BeforeClass / @AfterClass: Run once before/after the first/last test method in the class.
• @BeforeMethod / @AfterMethod: Run before/after every test method.
• @BeforeGroups / @AfterGroups: Run before/after a specific group of tests.
• @Test: Marks a method as a test method.

Basic Test Example

import org.testng.annotations.*;

public class CalculatorTest {

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("Before Suite: Initialize global resources");
    }

    @BeforeClass
    public void beforeClass() {
        System.out.println("Before Class: Setup test data");
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("Before Method: Reset state");
    }

    @Test
    public void testAddition() {
        int result = 2 + 3;
        assert result == 5 : "Addition failed";
        System.out.println("testAddition passed");
    }

    @Test
    public void testSubtraction() {
        int result = 10 - 4;
        assert result == 6 : "Subtraction failed";
        System.out.println("testSubtraction passed");
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("After Method: Cleanup");
    }

    @AfterClass
    public void afterClass() {
        System.out.println("After Class: Release resources");
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("After Suite: Final cleanup");
    }
}

Assertions in TestNG

TestNG provides its own assertion class org.testng.Assert with a wide range of assertion methods. These produce clear failure messages and integrate with TestNG reports.

import org.testng.Assert;
import org.testng.annotations.Test;

public class AssertionExamplesTest {

    @Test
    public void testEquals() {
        String actual = "TestNG";
        String expected = "TestNG";
        Assert.assertEquals(actual, expected, "Strings should match");
    }

    @Test
    public void testTrue() {
        boolean isActive = true;
        Assert.assertTrue(isActive, "User should be active");
    }

    @Test
    public void testFalse() {
        boolean isDeleted = false;
        Assert.assertFalse(isDeleted, "Record should not be deleted");
    }

    @Test
    public void testNotNull() {
        Object user = new Object();
        Assert.assertNotNull(user, "User object should not be null");
    }

    @Test
    public void testArrayEquals() {
        int[] expected = {1, 2, 3};
        int[] actual = {1, 2, 3};
        Assert.assertEquals(actual, expected, "Arrays should be equal");
    }
}

Test Groups

One of TestNG's most powerful features is the ability to group tests. This lets you run specific subsets of tests, such as smoke tests, regression tests, or tests tied to a particular feature.

Defining Groups

import org.testng.annotations.Test;

public class LoginTest {

    @Test(groups = {"smoke", "login"})
    public void testValidLogin() {
        System.out.println("Valid login test");
    }

    @Test(groups = {"regression", "login"})
    public void testInvalidLogin() {
        System.out.println("Invalid login test");
    }

    @Test(groups = {"smoke", "regression"})
    public void testPasswordReset() {
        System.out.println("Password reset test");
    }
}

Running Groups via XML

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="TestSuite">
    <test name="SmokeTests">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="LoginTest"/>
        </classes>
    </test>
</suite>

You can also exclude groups using <exclude name="regression"/> inside the <run> tag.

Parameterized Testing

TestNG supports two main approaches to parameterized testing: passing parameters through XML and using the @DataProvider annotation for more complex data sets.

Parameters via XML

import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class ParameterizedTest {

    @Test
    @Parameters({"username", "password"})
    public void testLogin(String username, String password) {
        System.out.println("Logging in with: " + username + " / " + password);
    }
}

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ParameterSuite">
    <test name="LoginTests">
        <parameter name="username" value="admin"/>
        <parameter name="password" value="secret123"/>
        <classes>
            <class name="ParameterizedTest"/>
        </classes>
    </test>
</suite>

Using @DataProvider

For data-driven testing with multiple rows of data, @DataProvider is the preferred approach. It returns a two-dimensional object array that TestNG feeds into the test method.

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderTest {

    @DataProvider(name = "loginData")
    public Object[][] provideLoginData() {
        return new Object[][] {
            {"admin", "admin123", true},
            {"user", "wrongpass", false},
            {"guest", "", false},
            {"manager", "manager123", true}
        };
    }

    @Test(dataProvider = "loginData")
    public void testLogin(String username, String password, boolean expected) {
        boolean actual = authenticate(username, password);
        Assert.assertEquals(actual, expected, 
            "Login result mismatch for user: " + username);
    }

    private boolean authenticate(String username, String password) {
        return "admin".equals(username) && "admin123".equals(password)
            || "manager".equals(username) && "manager123".equals(password);
    }
}

Dependent Tests

TestNG allows you to define dependencies between test methods. This is useful when one test logically depends on another, such as verifying that a user can log in before testing their profile page.

import org.testng.annotations.Test;

public class DependencyTest {

    @Test
    public void createUser() {
        System.out.println("Creating user...");
    }

    @Test(dependsOnMethods = {"createUser"})
    public void updateUser() {
        System.out.println("Updating user...");
    }

    @Test(dependsOnMethods = {"updateUser"})
    public void deleteUser() {
        System.out.println("Deleting user...");
    }

    @Test(dependsOnMethods = {"createUser"}, alwaysRun = true)
    public void logAuditTrail() {
        System.out.println("Logging audit trail regardless of upstream failures");
    }
}

The alwaysRun = true attribute ensures a test runs even if its dependencies fail, which is useful for cleanup or logging tasks.

Expected Exceptions

TestNG lets you verify that a method throws a specific exception using the expectedExceptions attribute.

import org.testng.annotations.Test;

public class ExceptionTest {

    @Test(expectedExceptions = ArithmeticException.class)
    public void testDivisionByZero() {
        int result = 10 / 0;
    }

    @Test(expectedExceptions = {IllegalArgumentException.class, NullPointerException.class})
    public void testMultipleExpectedExceptions() {
        throw new IllegalArgumentException("Invalid argument");
    }

    @Test(expectedExceptions = IllegalStateException.class, 
          expectedExceptionsMessageRegExp = ".*invalid state.*")
    public void testExceptionWithMessage() {
        throw new IllegalStateException("The system is in an invalid state");
    }
}

Parallel Execution

TestNG can run tests in parallel, which is invaluable for large test suites. You control parallelism through the XML suite file.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ParallelSuite" parallel="methods" thread-count="4">
    <test name="ParallelTests">
        <classes>
            <class name="ParallelTest"/>
        </classes>
    </test>
</suite>

The parallel attribute accepts several values:

• methods: Each test method runs in its own thread.
• classes: Each test class runs in its own thread.
• instances: Each test instance runs in its own thread.
• tests: Each <test> tag runs in its own thread.

import org.testng.annotations.Test;

public class ParallelTest {

    @Test
    public void test1() {
        System.out.println("test1 running on thread: " + Thread.currentThread().getId());
    }

    @Test
    public void test2() {
        System.out.println("test2 running on thread: " + Thread.currentThread().getId());
    }

    @Test
    public void test3() {
        System.out.println("test3 running on thread: " + Thread.currentThread().getId());
    }
}

TestNG Listeners

Listeners allow you to hook into test execution events, enabling custom logging, screenshots on failure, or integration with reporting tools. The most commonly used listener is ITestListener.

import org.testng.ITestListener;
import org.testng.ITestResult;

public class CustomListener implements ITestListener {

    @Override
    public void onTestStart(ITestResult result) {
        System.out.println("Test started: " + result.getName());
    }

    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("Test passed: " + result.getName());
    }

    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Test failed: " + result.getName());
        System.out.println("Failure reason: " + result.getThrowable().getMessage());
        // Here you could capture a screenshot for UI tests
    }

    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println("Test skipped: " + result.getName());
    }

    @Override
    public void onFinish(ITestContext context) {
        System.out.println("All tests finished. Passed: " 
            + context.getPassedTests().size() 
            + ", Failed: " + context.getFailedTests().size());
    }
}

To attach the listener, either use the @Listeners annotation on a test class or declare it in the XML suite file.

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(CustomListener.class)
public class ListenerExampleTest {

    @Test
    public void passingTest() {
        System.out.println("This test passes");
    }

    @Test
    public void failingTest() {
        throw new AssertionError("Intentional failure");
    }
}

Timeouts

You can set a maximum execution time for a test method using the timeOut attribute. If the method exceeds the limit, TestNG marks it as failed.

import org.testng.annotations.Test;

public class TimeoutTest {

    @Test(timeOut = 2000)
    public void testQuickResponse() throws InterruptedException {
        Thread.sleep(1000);
        System.out.println("Completed within timeout");
    }

    @Test(timeOut = 1000)
    public void testSlowResponse() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("This will not print");
    }
}

Best Practices

Keep Tests Independent

Avoid heavy reliance on dependsOnMethods. While dependencies can model real-world flows, overusing them creates brittle test suites where one failure cascades into many. Prefer independent tests that set up their own state.

Use Data Providers for Reusability

Instead of writing multiple similar test methods, consolidate them into a single method backed by a data provider. This reduces duplication and makes it easy to add new test cases.

Organize Tests with Groups

Tag tests with meaningful groups like smoke, regression, api, or ui. This allows CI pipelines to run fast smoke tests on every commit and full regression suites nightly.

Leverage Parallel Execution Carefully

Parallel execution can dramatically speed up test runs, but shared state and non-thread-safe resources (like a single browser instance) can cause flaky failures. Use thread-local variables or ensure each thread has its own resources.

Write Clear Assertion Messages

Always provide descriptive messages in assertions. When a test fails in CI, a clear message saves hours of debugging.

// Bad
Assert.assertEquals(actual, expected);

// Good
Assert.assertEquals(actual, expected, 
    "Expected order total to match cart sum for user with 3 items");

Use Before/After Hooks Wisely

Keep setup and teardown logic at the appropriate level. Use @BeforeMethod for per-test setup and @BeforeClass for expensive one-time initialization. Avoid putting test-specific setup in @BeforeSuite.

Integrate with Build Tools

Run TestNG through Maven Surefire or Gradle to ensure tests execute consistently in CI. Configure plugins to generate reports and fail the build on test failures.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.2.5</version>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
        </suiteXmlFiles>
    </configuration>
</plugin>

Conclusion

TestNG is a versatile and feature-rich testing framework that scales from simple unit tests to complex, data-driven, parallel test suites. Its annotation-based configuration, grouping mechanism, parameterization support, and parallel execution capabilities make it a strong choice for Java developers building robust test automation. By following best practices such as keeping tests independent, using data providers effectively, organizing tests into meaningful groups, and leveraging parallel execution carefully, you can build a maintainable test suite that provides fast, reliable feedback throughout your development lifecycle. Whether you are testing backend services, REST APIs, or Selenium-driven web applications, TestNG provides the tools you need to validate your code with confidence.