Multithreaded Tests with JUnit

http://today.java.net/pub/a/today/2003/08/06/multithreadedTests.htmlJUnit is the glue that holds many open source projects together. But JUnit has problems performing multithreaded unit tests.

sony315

1123人浏览 · 2012-01-10 04:08:15

sony315 · 2012-01-10 04:08:15 发布

http://today.java.net/pub/a/today/2003/08/06/multithreadedTests.html

JUnit is the glue that holds many open source projects together. But JUnit has problems performing multithreaded unit tests. This creates considerable difficulty for middleware developers in the open source J2EE market. This article introduces GroboUtils, a JUnit extension library designed to address this problem and enable multithreaded unit testing in JUnit. A basic understanding of JUnit and threads is recommended but not necessary for readers of this article.

Introduction

If you've worked on open source Java projects or read the multitude of books on "Extreme Programming" and other rapid-development models, then you've likely heard about JUnit. Written by Erich Gamma and Kent Beck, JUnit is an automated testing framework for Java. It allows you to define "unit tests" for your software -- programs that test whether or not the code is functioning properly, usually on a method-by-method basis.

JUnit can help your development team in many ways -- too many to cover in one article. But from one developer to another, JUnit truly excels at two things:

It forces you to use your own code. Your tests function as client code for your production code. Getting to know your software from the client's perspective can help you identify problems in the API and improve how the code will eventually get used.
It gives you confidence to make changes in your software. You'll know right away if you've broken the test cases. At the end of the day, if the light is green, the code is clean. Check it in with confidence.

But JUnit is no silver bullet. Third-party extension libraries such as HttpUnit, JWebUnit, XMLUnit, and a host of others have risen to address perceived holes in the framework by adding functionality. One of the areas JUnit doesn't cover is multithreaded unit tests.

In this article, we're going to look at a little-known extension library that solves this problem. We'll start by setting up the JUnit framework and running an example to show poor use of threads in testing. After we've identified the obstacles of threaded testing, we'll walk through an example using the GroboUtils framework.

Threads in Review

For those of you new to threads, it's all right to panic a bit at this point -- just don't overdo it. Get it out of your system. We're going to take a fifty-thousand-foot view of threads. Threads allow your software to multitask -- that is, do two things at the same time.

In their book A Programmer's Guide to Java Certification, Khalid Mugal and Rolf Rasmussen briefly describe threads as follows:

"A thread is a path of execution within a program, that is executed separately. At runtime, threads in a program have a common memory space and can therefore share data and code; i.e., they are lightweight. They also share the process running the program.

Java threads make the runtime environment asynchronous, allowing different tasks to be performed concurrently." (p.272)

In web applications, many users can send requests to your software at the same time. When writing unit tests to stress your code, you need to simulate that sort of concurrent traffic. This is especially true if you're trying to develop robust middleware components. Threaded tests would be ideal for these components.

Unfortunately, JUnit is lacking in this arena.

Problems with JUnit and Multithreaded Tests

If you want to try out the following code you need to download and install JUnit.Instructions for doing so can be found at the JUnit web site. Without delving too far into details, we're going to briefly examine how JUnit works. To write a JUnit test, you must first create a test class that extends junit.framework.TestCase, the basic unit test class in JUnit.

The main() and suite() methods are used to start the tests. From the command line or from an IDE, make sure that junit.jar is in your classpath, then compile and run the following code for the BadExampleTest class.

import junit.framework.*;

public class BadExampleTest extends TestCase {

    // For now, just verify that the test runs
    public void testExampleThread()
            throws Throwable {

        System.out.println("Hello, World");
    }

    public static void main (String[] args) {
        String[] name =
            { BadExampleTest.class.getName() };

        junit.textui.TestRunner.main(name);
    }

    public static Test suite() {
        return new TestSuite(
            BadExampleTest.class);
    }
}

Run BadExampleTest to verify that everything has been set up correctly. Once the main() method is called, the framework will automatically execute any method whose name begins with "test". Go ahead and try to run the test class. If you've done everything correctly, it should kick out the message "Hello World" in the output window.

Now, we're going to add a thread class to the program. We're going to do this by extending the java.lang.Runnable interface. Eventually, we'll switch our strategy and extend a class that automates thread creation.

Create a private inner class called DelayedHello that extends Runnable. The call to run() is implicit in the DelayedHello constructor, where we create a new Thread and call its start() method.

import junit.framework.*;

public class BadExampleTest extends TestCase {
    private Runnable runnable;

    public class DelayedHello
        implements Runnable {

        private int count;
        private Thread worker;

        private DelayedHello(int count) {
            this.count = count;
            worker = new Thread(this);
            worker.start();
        }

        public void run() {
            try {
                Thread.sleep(count);
                System.out.println(
                "Delayed Hello World");

            } catch(InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    public void testExampleThread()
            throws Throwable {

        System.out.println("Hello, World");       //1
        runnable = new DelayedHello(5000);        //2
        System.out.println("Goodbye, World");     //3
    }

    public static void main (String[] args) {
        String[] name =
            { BadExampleTest.class.getName() };

        junit.textui.TestRunner.main(name);
    }

    public static Test suite() {
        return new TestSuite(
            BadExampleTest.class);
    }
}

The method testExampleThread() isn't really much of a test method. In practice, you want the tests to be automated and don't want to ever have to check output to the console. But here, the point is to demonstrate that JUnit does not support multithreading.

Note that testExampleThread() performs three tasks:

Prints "Hello, World".
Initializes and starts a thread that is supposed to print "Delayed Hello World".
Prints "Goodbye, World".

If you run this test, you'll notice something wrong. The testHelloWorld() method runs and ends as you would expect it to. It fires off the thread without any exceptions. But you never hear back from the thread. Notice you never see "Delayed Hello World". Why? Because JUnit finishes execution while the thread is still alive. There could have been problems down the line, toward the end of that thread's execution, but your test would never reflect it.

The problem lies in JUnit's TestRunner. It isn't designed to look for Runnable instances and wait around to report on their activities. It fires them off and forgets about them. For this reason, multithreaded unit tests in JUnit have been nearly impossible to write and maintain.

Enter GroboUtils

GroboUtils is an open source project written by Matt Albrecht that aims to expand the testing possibilities of Java. GroboUtils is distributed under the MIT license, which makes it very friendly for inclusion in other open source projects.

GroboTestingJUnit Subproject

GroboUtils is broken into subprojects that experiment with similar aspects of testing. This article focuses on the GroboTestingJUnit subproject, an extension library for JUnit that introduces support for multithreaded testing. (This subproject also introduces Integration Tests and the concept of failure severity, but those features fall outside of the scope of this article.)

Within the GroboTestingJUnit subproject is GroboTestingJUnit-1.1.0-core.jar. It contains the MultiThreadedTestRunner and TestRunnable classes, both of which are necessary for extending JUnit to handle multithreaded tests.

`TestRunnable`

The TestRunnable class extends junit.framework.Assert and implements java.lang.Runnable. You should define TestRunnable objects as inner classes inside of your tests. Although traditional thread classes implemented a run() method, your nested TestRunnable classes must implement the runTest() method instead. The testRun() method will be invoked by the MultiThreadedTestRunner at runtime, so you shouldn't invoke it in the constructor.

`MultiThreadedTestRunner`

MultiThreadedTestRunner is a framework that allows for an array of threads to be run asynchronously inside of JUnit. Modeled after an article written by Andy Schneider, this class accepts an array of TestRunnable instances as parameters in its constructor. Once an instance of this class is built, its runTestRunnables() method should be invoked to begin the threaded tests.

Unlike a standard JUnit TestRunner, the MultiThreadedTestRunner will wait until all threads have terminated to exit. This forces JUnit to wait while the threads do their work, nicely solving our problem from earlier. Let's take a look at how to integrate the GroboUtils API with JUnit.

Writing the Multithreaded Test

The inner class now extends the net.sourceforge.groboutils.junit.v1.TestRunnable package that requires that we override the runTest() method.

    private class DelayedHello
            extends TestRunnable {

        private String name;

        private DelayedHello(
                String name) {

            this.name = name;
        }

        public void runTest() throws Throwable {
            long l;
            l = Math.round(2 + Math.random() * 3);

            // Sleep between 2-5 seconds
            Thread.sleep(l * 1000);
            System.out.println(
                "Delayed Hello World " + name);
        }
    }

This time, we don't create a worker thread at all. The class MultiThreadedTestRunner will do this under the hood. Instead of implementing the run() method, we override runTest(), which later gets invoked by the MultiThreadedTestRunner -- we never call it ourselves.

Once the TestRunnable is defined, we must define our new test case. In our testExampleThread() method, we instantiate several TestRunnable objects and add them to an array. After that, we instantiate the MultiThreadedTestRunner, passing the TestRunnable array in as a constructor parameter. Now that we have an instance of MultiThreadedTestRunner, we call its runTestRunnables() method.

MultiThreadedTestRunner (unlike the TestRunner in JUnit) will wait for every running thread to expire before continuing on. Also, it creates the worker threads and calls the start() methods concurrently for each TestRunnable passed in through its constructor. That means you don't have to jump through the hoops of creating your own threads -- MultiThreadedTestRunner does it for you.

Here's a final version of ExampleTest:

import junit.framework.*;
import net.sourceforge.groboutils.junit.v1.*;

public class ExampleTest extends TestCase {
    private TestRunnable testRunnable;

    private class DelayedHello
            extends TestRunnable {

        private String name;
        private DelayedHello(
                String name) {

            this.name = name;
        }

        public void runTest() throws Throwable {
            long l;
            l = Math.round(2 + Math.random() * 3);

            // Sleep between 2-5 seconds
            Thread.sleep(l * 1000);
            System.out.println(
                "Delayed Hello World " + name);
        }
    }

    /**
     * You use the MultiThreadedTestRunner in
     * your test cases.  The MTTR takes an array
     * of TestRunnable objects as parameters in
     * its constructor.
     *
     * After you have built the MTTR, you run it
     * with a call to the runTestRunnables()
     * method.
     */
    public void testExampleThread()
        throws Throwable {

        //instantiate the TestRunnable classes
        TestRunnable tr1, tr2, tr3;
        tr1 = new DelayedHello("1");
        tr2 = new DelayedHello("2");
        tr3 = new DelayedHello("3");

        //pass that instance to the MTTR
        TestRunnable[] trs = {tr1, tr2, tr3};
        MultiThreadedTestRunner mttr =
            new MultiThreadedTestRunner(trs);

        //kickstarts the MTTR & fires off threads
        mttr.runTestRunnables();
    }

    /**
     * Standard main() and suite() methods
     */
    public static void main (String[] args) {
        String[] name =
            { ExampleTest.class.getName() };

        junit.textui.TestRunner.main(name);
    }

    public static Test suite() {
        return new TestSuite(ExampleTest.class);
    }
}

Each thread will feed you back their output between two and five seconds after you fire off the test. Not only will they all show up on time, but they'll show up in a random order, proving concurrency. The unit test won't finish until they're done. With the addition of MultiThreadedTestRunner, JUnit patiently waits for the TestRunnables to complete their work before continuing on with the test cases. Optionally, you can set a maximum time allotment for the MultiThreadedTestRunner to execute (so you don't hang the test on a runaway thread).

To compile and run ExampleTest you will need both junit.jar and GroboUtils-2-core.jar in your classpath. You should see "Delayed Hello World" for each of the threads in some random order as output.

Conclusion

Writing a multithreaded unit test doesn't need to be painful or frustrating (much less impossible). The GroboUtils library provides a clear and simple API for writing multithreaded unit tests. By adding this library to your toolkit, you can extend your unit testing to simulate heavy web traffic and concurrent database transactions, and stress test your synchronized methods.

Have fun!