IBM developerWorks : Java technology
 
A Java RMI server framework
 
Contents:
Why use a framework?
The foundation
A logical walkthrough
Multipart requests
Seeing it work
Possible enhancements
Conclusion
Resources
About the author
 
Related content:
Download Logging Toolkit for Java
Using an asynchronous process manager to contain your RMI server applications

Edward Harned (ed@coopsoft.com)
Senior Developer, Cooperative Software Systems, Inc.
October 2001

The Remote Method Invocation runtime designed by the architects of the Java platform is a magnificent creation -- but it wasn't intended to serve as a full-fledged application server. By separating RMI connection activity from application processing, you will save yourself a great deal of development pain and stress. In this article, Senior Java developer Edward Harned presents a framework with which you can do just that. You can make use of the code presented here or adapt it to meet the particular needs of your applications.

Many Java developers make the mistake of assuming that they can use a Remote Method Invocation (RMI) server off the shelf as a full-fledged application server. This is a faulty assumption, and can cause extreme pain as development progresses. A better approach is to build a framework around RMI that provides a structure for just such a full-fledged application server.

The fundamental component of high-volume, recoverable, secure, and manageable software systems, first introduced with transaction servers, is asynchronous (or back-end) processing. The basic flow of asynchronous processing has the following steps:

  • Get the request into the facility by any means possible.
  • Break the request into its component parts and place those components into queues.
  • Use threads (tasks) to parallel process the queues, thereby reducing response time and overhead.
  • Place the reply into a queue for the next available output processor.

The basic building block underneath asynchronous processing is the queuing and threading structure. Add queuing and threading to an RMI server (an asynchronous entity) and you have a mission-capable server. The best part is that it's not that difficult to do.

Mention threading to most developers and they shy away; but handling threads is at the core of creating the kind of mission-capable server we want. There are so many books and articles on threading that any sensible person would feel intimidated by a multithreading project. But the truth is that a single-function thread is easy to design. The key to handling multiple threads in an application is to segregate the thread logic from the application logic and individually control each thread. This article shows you how to do that.

An RMI server runs as a separate process in the computer. Since this process and client processes can run independently of each other, the interaction is asynchronous. An asynchronous process requires a degree of management necessary to make it self-reliant -- a framework.

Road map
This article helps you understand why an asynchronous process needs management and outlines the steps necessary to eventually design your own custom asynchronous process manager.

First, we examine the queuing and threading environment of a single-component asynchronous process manager. Next, we transform this single-component environment into a request broker capable of parallel processing multiple queues.

Synchronous vs. asynchronous
Think of a GUI application as a synchronous application. You click a button and the logic is right there to process the event.

RMI is totally asynchronous. The application logic is in a different Java virtual machine (JVM) from the interface. You need to send the request and to receive the reply from that other JVM.

Prerequisites
You should be somewhat familiar with RMI, at least having taken some of the RMI tutorials mentioned in the Resources section. This is a difficult subject, and we will take it one small step at a time. Sometimes you need to see a demonstration to grasp a point; we have one in this article. And sometimes you need to see code to really figure out what's going on; we provide that, too. Naturally, you should be familiar with the differences between synchronous and asynchronous processing.

Why use a framework?
Why are EJB and GUI applications so successful? Because they run inside a container -- a framework that manages persistence, messaging, thread management, logging, the event queue, the user interface, and much more.

An RMI server doesn't have a similar application container to manage queues, threads, messaging, logging, and so on. We have to build it ourselves.

The RMI runtime
The RMI runtime is a magnificent creation of the Java architects. For most implementations, the RMI runtime instantiates a thread to handle each request from a client. Once the request finishes, the thread waits for a brief period for the next client request. In this way, the RMI connection may reuse threads. If no new request comes in, the RMI runtime destroys the thread.

What developers sometimes forget is that the RMI runtime is not and should never be a full-featured application server. Using the basic RMI runtime without application services makes development arduous. Consider these two very basic issues:

  • Timing. The client sends a request to the RMI server for information contained in a private resource. If a horde of other users are also updating that private resource, by the time the request completes, the original user has gone home. If the private resource is nonfunctioning to the point that the request cannot complete, not only has the original user gone home, but the RMI connection thread hangs forever.

  • The autonomous request, with callback. The client sends in a request and has it processed by a background thread that contacts the original client when complete. If we simply create a new application thread for every request, the create/destroy overhead and the number of application threads will put a severe strain on the server, and the JVM will eventually run out of resources.

The pragmatic solution
The practical solution to these and many more problems is to separate the RMI connection activity from the application processing. You can do this by creating an application queuing and threading structure on the server side. This is the way highly reliable, fully mission-critical software products work, and that structure can be available for any application.

Imagine using this model. For a client that needs a timed response, the RMI connection thread contacts an application thread. If the application thread does not respond within the time limit, then the RMI connection thread returns to the client with a time-out message. For an autonomous request, the RMI connection thread contacts an application thread and immediately returns to the client with an "it's been scheduled" message.

Now, the goal is to design a queuing and application threading environment so that:

  • The RMI connection threads and the application threads can talk to each other.

  • The application environment can know about client time-outs and recover.

  • A thread overload problem does not occur. Such an overload could happen when so many application threads are executing that the JVM cannot sustain any more threads, or when many application threads cause so much competition for resources that the environment effectively deadlocks.

  • The application thread create/destroy overhead does not bog down the application processing.

  • The threading environment is monitored to pinpoint stalls (that is, situations in which an autonomous request is unable to complete).

  • The entire threading environment can shut down gracefully.

We are going to examine an asynchronous process manager framework that you can run. You can download the classes for execution as well as the source code in the Resources section.

The foundation of the framework
Like any software system, this framework is made up of foundational building blocks. The overall structure may seem complex at first, but it is nothing more than components built on top of each other working together. In the sections that follow, we discuss five essential elements of the framework:

  • Logical processes -- queues and threads
  • A framework using logical processes
  • The common memory principle
  • Locating the application threads
    • Establishing a base common memory anchor
    • Chaining classes to the anchor
  • Locating calling RMI connection threads

Then we walk through a simple request from a client with the framework.

Logical processes
The operating system refers to the Java virtual machine as a process and to threads as lightweight processes. The underlying model for the asynchronous process manager we're examining is what's called a logical process. What does that look like?

Origin of the logical process
The term logical process comes from the transaction server arena. Client transactions place requests into back-end queues and let asynchronous tasks process the queues. The tasks work independently of other non-back-end tasks -- but they are not physically separate, just logically separate.

Think of a logical process like order fulfillment within Amazon.com. You go online and place an order. Since Amazon doesn't have a dedicated order picker just for you, your order goes into a queue of orders. You go on about your business. At an Amazon warehouse, somebody gets your order from the first in, first out (FIFO) order queue and fills it. If your request is for a book, it might be filled at a particular warehouse with a certain number of employees capable of filling orders. If your request is for a videocassette, it might be filled at a different warehouse with a different set of employees, perhaps larger or smaller based on demand. This order fulfillment flow is a logical process.

A managed asynchronous process environment in software behaves similarly. At Amazon, the order fillers are employees. In a software system, the order fillers are application threads.

Requests for back-end services stack up in queues. Each RMI connection thread places the client's request into a queue. Application threads fetch requests from the queue and act upon them. Developers define queues for the different types of requests (like the separate book and videocassette warehouses at Amazon) and the maximum number of application threads to service each queue. These are logical processes.

Figure 1. Logical processes
Logical processes

Overview of how our asynchronous process manager will use logical processes
The client sends a request to the server. The RMI runtime creates, or reuses, an RMI connection thread that executes a method within the implementation class. The appropriate method places the request into a prioritized wait list within a queue and uses an object-notify method to wake up an application thread to service the request. There are two kinds of requests possible here:

Wait/notify methods of class Object
wait() causes the current thread to wait until either another thread invokes the notify() method for this object or a specified amount of time has elapsed.

notify() wakes up a single thread that is waiting on this object's monitor. A thread waits on an object's monitor by calling one of the wait methods.

  • The timed request. The RMI connection thread uses the object-wait method to suspend its execution until either the application thread finishes or the time limit expires. Upon completion, the RMI connection thread returns to the client with the response from the application or a time-out message.

  • The autonomous request. The RMI connection thread returns to the client with an "it was scheduled" message. All the application processing happens asynchronously.

A framework using logical processes
A look at the various pieces of our asynchronous process manager makes the picture real. The key components are the basic things that we need to use RMI, and application-specific classes we need to implement our queue/thread model.

The basic RMI stuff
RMI requires us to set up the following things:

  • An interface that extends java.rmi.Remote
  • A concrete implementation class that carries out the interface's declaration
  • A parameter class that passes client information to the RMI server

The interface
For this framework, the remote interface (shown in Listing 1) needs at least three methods:

  • A syncRequest() method that handles the timed request
  • An asyncRequest() method that handles the autonomous request
  • A shutDown() that the client can use to gracefully shut down the RMI server

Listing 1. The FrameWork interface

public interface FrameWorkInterface
	extends Remote {
	
	public Object[] syncRequest(FrameWorkParm in)
		throws RemoteException;
		
	public Object[] asyncRequest(FrameWorkParm in) 
		throws RemoteException;
		
	public String shutRequest() 
		throws RemoteException;

The concrete implementation class
The implementation of the FrameWorkInterface is class FrameWorkImpl. Listing 2 is the instance field and part of the constructor. Because this is the class where the RMI connection thread logic lives (placing the request in a request queue, waking up an application thread, and returning the reply to the client), we examine the inner workings later in the article.

Listing 2. The FrameWork implementation class

public final class FrameWorkImpl 
       	extends UnicastRemoteObject
       	implements FrameWorkInterface {

	// instance field (base of common memory)
	private FrameWorkBase fwb;
    	// constructor
	public FrameWorkImpl (FrameWorkBase reference_to_fwb)
		throws RemoteException {
		
		// set common memory reference
		fwb = reference_to_fwb;

The parameter class
This class, shown in Listing 3, passes client information to the RMI server. The instance fields are as follows:

  • Object client_data: An optional object from the client to the application processing class.

  • String func_name: The name of the logical process; also called function later in this article.

  • int wait_time: When using a syncRequest(), the maximum number of seconds the RMI connection thread should wait for the application thread to finish (that is, the time-out interval).

  • int priority: The priority of the request. A priority 1 request should be selected before a priority 2 request, priority 2 before priority 3, and so on.

Listing 3. The FrameWorkParm class

public final class FrameWorkParm 
	implements java.io.Serializable {
	
 	 // passed data from Client 
  	private Object client_data;
      	// Function name
  	private String func_name;
      	// maximum time to wait for a reply
  	private int wait_time;
     	// priority of the request
  	private int priority;

Application-specific stuff
Beyond what we need for RMI, our application requires us to set up the following:

  • A start-up class that creates the RMI server environment
  • A queue class that defines the priority wait lists, the application threads, and a reference to the application processing class
  • A thread class that fetches requests from the wait lists and calls the application processing class
  • An application processing class to execute the application logic

The startup class
This class, shown in Listing 4, starts the RMI server. It contains logic for establishing the persistent environment.

Listing 4. The FrameWork start-up class

public final class FrameWorkServer { 

	// The base for all persistent processing 
	private static FrameWorkBase fwb = null; 
	
	// Start up Server 
	public static void main(java.lang.String[] args){ 
	
		// the base for all processing 
		fwb = new FrameWorkBase(); 
		
		// now, after initializing the other FrameWorkBase fields 
		// including the application queues and threads, 
		// do the Implementation class with a ref to FrameWorkBase
		FrameWorkImpl fwi = new FrameWorkImpl(fwb);

Application queues
Application queues contain three major elements:

  • Prioritized wait lists where requests are pending. Requests stack up in wait lists when no application threads are immediately available to act upon them. When threads finish processing a request, they look in the wait lists for the next request. This reduces machine overhead by letting each thread complete multiple requests during a thread's execution cycle.

  • Anchor points for application threads. An anchor point is simply a reference to an instantiated thread. This may be different from the thread pools with which you are more familiar. Using specific references for each thread:

    • Lets us easily find a thread when it is nonperforming so that we can either kill it or recover and restart it.

    • Lets us easily trap a thread for debugging.

    • Limits contention among threads and curtails the thread overload problem by defining the total number of threads in each queue and only instantiating a thread when it is actually necessary.

    • Making it very simple to keep statistics by having queues with solid references to each thread. Statistics form the basis for tuning.


  • The reference to the application processing class. What is usually a multipart application class (thread logic and application logic) is split into two separate classes: an application thread class (just the thread logic) and an application processing class (just the application logic).

    This separation is a little confusing at first because most developers see threading as part of the application class. Handling the thread logic in addition to the application logic requires two different thought patterns to merge as one. This framework design separates the thread logic from the application logic (what we call an application processing class) so that any application logic may easily plug in to a thread structure. (See Listing 5.)

    Still confused? Think of that browser you're using to read this. Would you like to play a tune or watch a video? Think of that poor browser developer who must put that logic into the base product. But if this additional logic is a plug-in, then not only is the base code small and easily maintained, but you can also install any logic module by any vendor at any time.

Listing 5. Queue class

public final class QueueHeader {

    private String  que_name;   // name of this queue
    private WaitLists waitlist; // pointer to wait lists
    private int nbr_waitlists;  // number of wait lists
    private int nbr_wl_entries; // number of entries in a waitlist 
    private int wait_time;      // time to wait when no work
    private int nbr_threads;    // total threads for this queue 

    private QueueDetail[] details; // detail array of thread info

    // public application logic class for use by the Queue Thread
    public DemoCall to_call; // class to call

The application thread class.
The application thread class contains the threading logic. Each thread area (class QueueDetail in Listing 6) contains the pointer to the physical thread. When an RMI connection thread wakes up the application thread, the application thread fetches the request from the queue's wait list. The application thread then calls a method on the application processing class to perform the work and sends the return object from that method back to the RMI connection thread.

Listing 6. Thread area class

public final class QueueDetail {

    private int   status;       // status of this entry
    private String tname;       // name of this thread
    private int  totl_proc;     // total requests processed
    private int  totl_new;      // total times instantiated

    private FrameWorkBase fwb;    // base storage
    private QueueHeader   AH;     // Queue this belongs to
    private QueThread     thread; // queue thread

The application processing class.
A separate application processing class contains the application logic. The application thread class we talked about previously calls methods on the application processing class. For this example, we use the DemoCall interface (in Listing 7). Any class that implements this interface is acceptable.

Listing 7. Interface DemoCall

public interface DemoCall {

    public Object doWork(Object input, FrameWorkInterface fwi)
    	throws java.lang.Throwable;

Looking at the two parameters to the application doWork() method, we see:

  • The reference to the FrameWorkParm instance from the client, passed as type Object (see Listing 3).

  • A reference to the server itself (FrameWorkImpl), passed as type FrameWorkInterface. The second reference is so the application may call the server as a client. This is recursion, one of the most useful techniques in programming and, sometimes, one of the most difficult to implement.

Common memory
The underlying principle behind this framework is the concept of common memory. For any two threads to talk to each other, they must use memory that is common between them. No one thread owns this memory; it is updatable and viewable by all threads.

How threads affect common memory
Just because we have memory that is common between threads does not mean that threads may access or modify that memory without restriction. This is true in any language.

The key to modifying variables for use in multiple threads is the synchronized statement or synchronized method modifier. In the Java language, there is shared main memory and thread memory. The JVM gives every thread a copy of the variables it needs from shared main memory and saves those variables back into shared main memory for use by other threads. This is the Java memory model. (For more information, see chapter 17 of the Java Language Specification. You can find a link in the Resources section).

Putting the access/mutate to an object's variables within a synchronized block or method accomplishes three functions:

  • Prevents other threads (synchronizing on the same object) from executing
  • Reads the current value of a variable from shared main memory to thread storage when the thread accesses that variable
  • Writes the new value of a variable from thread storage to shared main memory at the end of the block or method when the thread alters that variable

Therefore, to guarantee integrity and to make sure all threads have access to the latest value of a variable, synchronize. For the ultimate word on memory access, see "Can Double-Checked Locking Be Fixed?" by Brian Goetz. Also, for an in-depth article on multiple-CPU thread synchronization, see "Warning! Threading in a Multiprocessor World," by Allen Holub. You can find links to both articles in the Resources section.

An RMI server is persistent. This means that any objects the server creates and does not free (live references) remain for the life of the server. When the server starts, it gets a new instance of a common memory class (class FrameWorkBase in Listing 8 ) and assigns a private, static field with the reference. The RMI implementation class (class FrameWorkImpl in Listing 2) also gets a reference to the common memory class. In this way, all threads running on the server -- the RMI connection threads and the application threads -- have access to this common memory.

Figure 2 is an illustration of common memory with the threads that use it.

Figure 2. Common memory
Common memory

Yes, working with Java memory and garbage collection is somewhat difficult to comprehend. We'll demonstrate this shortly. However, before we get there, we need to talk a little about how threads find each other.

Pointers
The Java language does not support pointer arithmetic -- but that doesn't mean that there are no pointers in the Java language. In Java parlance, pointers are called references. Although you cannot increment or decrement references, you can certainly use a reference to locate other instantiated classes. This process of walking the chain is as old as the first computer. Those familiar with other languages that use pointers (such as C/C++) may recognize a common technique in the paragraphs that follow.

Using common memory to locate threads
How can an RMI connection thread find an application queue and wake up an application thread? And how can that application thread wake up the RMI connection thread? The answers lie in the structure of the environment and the use of pointers.

The anchor
The base of common memory is the class FrameWorkBase (see Listing 8). This class contains static references to other classes. For this example, the fields are public. (This is simply one way to establish common memory.)

Listing 8. The base of common memory

public final class FrameWorkBase {

    // Function Array
    public static FuncHeader func_tbl = null;

    // Remote Object myself (for recursive calls)  
    public static FrameWorkInterface fwi = null;

The RMI server start-up (class FrameWorkServer in Listing 4) instantiates the base of common memory (class FrameWorkBase) and assigns a class field to the reference of the FrameWorkBase object. Because the reference is live and the server is persistent, the JVM does not garbage collect the object. Think of the start-up class fields as anchor points.

Because the start-up class passes the FrameWorkBase reference to the constructor of the implementation class (class FrameWorkImpl in Listing 2) and the implementation class saves the FrameWorkBase reference in its instance fields, all RMI connection threads now have access to the FrameWorkBase class.

The chain
Now that we've established an anchor for our common memory, we need to chain our other classes to it. In this way, any RMI connection thread (a thread that is associated with the implementation class in Listing 2) may find an application thread.

An RMI connection thread uses its instance field reference to locate the FrameWorkBase class (see Listing 8). It then uses the FrameWorkBase instance field reference, func_tbl, to access the function array (class FuncHeader in Listing 9).

Listing 9. Function array

public final class FuncHeader {

  private int nbr_func;         // number of functions
  private FuncDetail[] details; // detail entries 

The FuncHeader details array (searched sequentially) contains an element (class FuncDetail in Listing 10) for every function supported by the framework.

Listing 10. Function detail

public final class FuncDetail { 
	private String name; // Function name 
	private long used;   // times used 
	private QueueHeader qtbl; // queue

Each FuncDetail contains:

  • The string name of the function
  • A reference to the function's queue (class QueueHeader in Listing 5)

Each queue (see Listing 5) contains:

  • The pointer to the application logic class for the queue
  • A pointer to the wait lists
  • A detail array of available threads for the queue

By searching the detail array of available threads, an RMI connection thread may find an available thread area for the request (class QueueDetail in Listing 6).

Each thread area (class QueueDetail in Listing 6) contains the pointer to the physical thread. The RMI connection thread uses the reference to the physical application thread to call a synchronized method that issues an object-notify to wake up the application thread.

Finding the application thread: A general example
We've seen a lot of classes and a lot of pointers -- it's been a lot to follow. Here it is in English with an illustration.

When the Client invokes a remote method, the RMI connection thread, having a reference to the FrameWorkBase class as an instance field (see Listing 2):

  • Uses that reference to locate the FrameWorkBase class
  • Uses the FrameWorkBase instance field reference to locate the function array
  • Searches the function array for the desired function detail element for the request
  • Uses the function detail queue reference to locate the queue for the request (Queue2, in this example)
  • Places the request into Queue2's wait list
  • Finds a waiting application thread within Queue2
  • Uses the reference to the thread (Thread2, in this example) to notify() that application thread and waits for it to complete

Note: That last bullet uses notify(). This is not a mistake. Most books on threads advise you to use notifyAll(). The notify() and notifyAll() methods are methods of class Object. The question is: for a particular Object, how many threads are there? In the case of the Queue, there is only one thread per instance:

      QueueThread qt1 = new QueueThread();
      QueueThread qt2 = new QueueThread(); 

Notify() wakes up an arbitrary thread. But the fields qt1 and qt2 are references. Because there is only one thread for each QueueThread object, notify() works. It still wakes up an arbitrary thread, but it only has one to choose from.

Figure 3 illustrates the process of walking the chain to find an application thread.

Figure 3. Finding the application thread
Finding the application thread

Finding the RMI connection thread
When the application thread finishes processing, it has to find the calling RMI connection thread to wake it up. Because there is no back chaining, how does the application thread know who called it? It doesn't. This is why the application thread must use notifyAll(), and why a little more work is necessary in the RMI connection thread.

The RMI connection thread must pass something to the application thread to uniquely identify itself. Any object the RMI connection thread creates and passes to another thread by reference is available to the other thread. After a synchronization event (remember the sidebar "How threads affect common memory"), each thread then has access to the current value of that object. For this example, the framework uses an integer array (see Listing 11.)

After the RMI connection thread finds the proper queue for the request, it places an enhanced request in the queue's wait list. This is the client's object from the parameter FrameWorkParm (see Listing 3) and several other fields in Listing 11:

Listing 11. Some fields in the enhanced request

	// requestor obj (monitor) to cancel wait
	Object requestor = this;
	
	// integer array created by RMI-Connection thread
	pnp = new int[1];
	pnp[0] = 0;
    	// passed object from the Client
	Object input_from_client = client_data;
    	// returned object from the application thread
	Object back;

  • requestor:To use the object-wait and object-notify methods, both the RMI connection thread and the application thread must have access to the implementation class Object.

  • pnp: The Java language passes arrays by reference. The RMI connection thread sets the first integer to 0. When the application finishes its work, it sets the first integer to 1.

  • input_from_client: This is the client object passed to the RMI server in the parameter (Listing 3).

  • back: The optional object returned by the application processing class.

A logical walkthrough
When the client invokes a remote method, the RMI connection thread:

  • Finds the proper queue for the request by walking the chain

  • Passes the application thread (in the queue's wait list) enough information to tell it how to indicate request completion (by assigning pnp[0] = 1) and a reference for the notifyAll() method (requestor)

  • Finds an available application thread for the request within this queue and wakes it up

  • Waits for the application thread to finish its work (see Listing 12)

  • When the wait completes, picks up the object (see Listing 11) returned from the application and passes the object back to the client

Listing 12. The RMI connection thread wait logic

// Wait for the request to complete or timeout

// get the monitor for this RMI object
synchronized (this) {

    // until work finished
    while  (pnp[0] == 0) {

        // wait for a post or timeout
        try {
            // max wait time is the time passed
            wait(time_wait);

        } catch (InterruptedException e) {}

        // When not posted
        if  (pnp[0] == 0) {

            // current time
            time_now = System.currentTimeMillis();

            // decrement wait time
            time_wait -= (time_now - start_time);

            // When no more seconds remain
            if  (time_wait < 1) {
                // get out of the loop, timed out
                break;
            }
            else {
                // new start time
                start_time = time_now;
            }
        }
    }
}

Note: The while loop at // until work finished is there because when any application thread issues a notifyAll(), the Java language wakes up all of the RMI connection threads.

The application thread
On the other side, when the processing completes, the application thread does the following (see Listing 13):

  • Obtains the monitor for the RMI connection (by synchronizing on the passed object)
  • Assigns the reference to any return object from the application processing class
  • Posts the first integer in the integer array for the proper waiting RMI connection thread
  • Issues a general wake-up for all RMI connection threads (notifyAll())

Listing 13. Application thread request completion action

// get lock on RMI obj monitor
synchronized (requestor) {
    	// the object from the application processing
    	back = data_object;
    	// set posted
    	pnp[0] = 1;
    	// wake up all RMI-Connection threads
    	requestor.notifyAll();
}

An autonomous request
The flow of events for an autonomous request is identical to the flow for a timed request, except that the autonomous request does not require the RMI connection thread to wait for completion. After waking up the application thread, the RMI connection thread returns an "it was scheduled" message to the client. The autonomous request may seem very simple as a result, but there is one catch.

What happens to the return data from the doWork() method of the application processing class (Listing 7)? It should not be the concern of an application where its return data goes. A developer must be able to use the same application logic for a timed or autonomous request. Therefore, we need an agent for the autonomous request.

An agent is simply another logical process. In other words, it is a queue with associated threads. The only difference between a normal queue and an agent queue is that the RMI connection threads access the normal queue while the application threads optionally access the agent queue.

Agents
For a timed request, the client gets the response from the application. Think of an agent as a pseudo-client on the server side: the agent gets the response from the application.

This is a little confusing, but hang in there; eventually it becomes clear. When we get into the multipart request, you'll see that you can't live without agents.

The application processing class (the doWork() method) may return an object to the application thread. For an autonomous request, when desirable, the application thread may activate an agent logical process by creating a new enhanced request (with the just-returned object), placing that enhanced request into the agent queue's wait list, and waking up an agent thread. This is basically the same procedure as the RMI connection thread.

The agent logical process completes asynchronously without any return data. The agent queue application processing class is where you may place the call back, call forward, or any other logic necessary for dealing with a request completion.

Monitoring asynchronous processes
An additional critical requirement for any asynchronous process is a way to monitor the logical processes. The synchronous requests may time out, so there needs to be a procedure for releasing their memory. The autonomous requests execute without a client waiting for their completion; when they are unable to complete -- when they stall, in other words -- detecting that stall is difficult.

Daemon threads
If you use a UNIX-type operating system, then you are familiar with the term daemon thread. Windows-type operating systems call this a terminate-and-stay thread. The thread does its work and then sleeps until the next time it is required.

One way to monitor the environment is with a daemon thread that scans the environment periodically. Calling a thread a daemon simply means that it is not part of a particular application or RMI connection. When the monitor thread finds a problem, it can correct the problem, log the problem details, send a message to a middleware message queue, or internally notify another remote object. The action depends on the application and therefore is beyond the scope of this article.

Shutting down
You need a clean way to shut down the server regardless of the requests you're processing. The RMI runtime contains threads that never end, so the only ways to end the RMI server are to programmatically use a system-exit method or to use an operating system purge. The latter is most ungraceful and requires manual intervention.

The graceful way to shut down the RMI server is with a shutdown method. However, if the shutdown method simply ends the Java Virtual Machine, the method's return message never makes it back to the client. The better way is to start a shutdown thread. This thread sleeps for about two seconds to give the method's return message a chance to clear the virtual machine, and then issues System.exit(0).

What we've covered so far
That's certainly a mouthful. Now, before trying to swallow all of this information, we need to address the three major questions necessary for every project:

  1. What good does a framework do?
  2. Does it achieve what it promises?
  3. Is there anything more to it than technology for technology's sake?

To answer the first question, this framework gives us the ability:

  • To time RMI server requests.
  • To run autonomous RMI server requests.
  • To run agents as part of any autonomous request.
  • To limit the create/destroy overhead common with application threads.
  • To curtail the thread overload problem.
  • To run recursive requests from any application.
  • To easily debug requests, especially autonomous requests. (We know the application thread and the application class where the work resides. There is no need to trap extraneous code.)
  • To use a monitor to seek out nonperforming requests.
  • To gracefully shut down the RMI server.

This framework separates the RMI threading environment from the application threading environment. Additionally, it separates the application thread logic from the actual application logic. This is the greater degree of abstraction that is so important in object-oriented design.

Does it achieve what it promises? As you will see from the code contained in the zip file that accompanies this article (download it in the Resources section), this framework performs superbly.

The answer to the third question is, "Well, it's nice, but..." The return on investment may not be worth the effort involved. An additional, critical part of the structure is error recovery. Sometimes the anomaly code far outweighs the standard code. What this framework needs is a bigger reason for living.

What if we could expand this simple framework to support multiple queues per request? When a client request involves multiple accesses to resources, if we could split the request and place each component into its own queue, then we could parallel process the request. Now the possibilities are endless. This is request brokering and it is the subject of the next section.

Processing multipart requests in parallel
Once we've set up a basic queuing environment, it soon becomes evident that some requests really contain multiple actions, or components. For instance, fulfilling a request may require accesses to two different databases. We could access each database in a linear fashion but the second access must wait for the first to complete.

Request brokering
When we introduced the logical process, we used the analogy of placing an order with Amazon.com for a book or for a videocassette. With multipart requests, we're placing a single order for both a book and a videocassette. The book part of the request goes into a book queue and the videocassette part goes into a videocassette queue. Both queues have their own employees (application threads) to process orders.

A better way to handle a multi-action request is to separate the request into its component parts and place each component into a separate queue. This enables parallel processing. It is more difficult than linear programming but the benefits far outweigh the extra work up front.

What do we need to support request brokering? We need to understand that the client request is no longer the exclusive concern of a single logical process. Therefore, we must put the client's request into a common area so that any number of logical processes may access it. Since we already have a common memory environment, we must now enhance it.

This is the easy part. The hard work is always building the first fundamental block in the structure. In the sections that follow, we enhance the single-component framework by:

  • Creating separate enhanced request classes for timed and autonomous requests
  • Adding a class in which we can place stalled autonomous requests
  • Enhancing the function description class
  • Adding the additional classes to the common memory base class

Then we walk through a simple request with the new framework.

Enhancing common memory
We need a common place to hold the request details from the client, for both synchronous and asynchronous requests. This common information includes:

  • The object from the client (this is the client_data object within the FrameWorkParm class, Listing 3)
  • The return objects from each application processing class (see Listing 5), because we no longer have a single component structure
  • All those other enhanced request fields

A simple array of objects is all that is necessary. The objects for this example are the classes SyncDetail and AsyncDetail (see Listing 14).

We also need to add both these classes to the base of common memory, the FrameWorkBase class.

Listing 14. The enhanced request classes

public final class SyncDetail {

	private Object[] output;     // output data array
    	private Object   input;      // client input data, if any
    	private int   status;        // 0=avail 1=busy
    	private int   nbr_que;       // total queue's in function
    	private int   nbr_remaining; // remaining to be processed
    	private int   wait_time;     // max wait time in seconds 
    	private Object requestor;    // requestor obj to cancel wait
    	private int[] pnp;           // 0 not posted, 1 is posted
    public final class AsyncDetail {
    	private Object input;          // client input
    	private QueueHeader out_agent; // agent queue, if any
    	private int   nbr_que;         // nbr of queues in function
    	private int   nbr_remaining;   // remaining unprocessed
    	private int   status;          // 0 = available, 1 = busy
    	private Object[] output;       // output array

The array in which each of these objects resides is a linked list. All the dynamic arrays in this framework are linked lists. Access to entries within the linked list is direct, by subscript. When a thread puts an object into any linked list, all that the thread must pass to another object is the primitive integer (subscript). Additionally, during heavy usage, it is very simple to expand a linked list by chaining on a new list to the original.

Another common place to hold information for an asynchronous request is a dynamic array of those requests that have stalled. When the synchronous request takes longer than the user can wait, the connection side of the request terminates. When an asynchronous request takes longer than is prudent, the processing may be unable to complete and the request may stall. There must be a place to put the information in order to recover from the stall. That place is the StallDetail class (see Listing 15).

We also need to add this class to the base of common memory, the FrameWorkBase class.

Listing 15. The stalled element class

public final class StallDetail {

	private long entered;   // time entered
	private int gen_name; // AsyncDetail subscript
	private int status;       // 0 = available, 1 = busy private
	int failed_reason;      // why it is here

Telling the server about the parts of a request
We talked about structuring our environment for a multicomponent request, but it is not evident how our RMI server can know what components are part of a request.

The component structure is information the developer knows from the beginning. In the basic framework, there is a single queue for each function (the name of the logical process). In the request broker framework, there is a list of queues for each function. The list of queues associated with each function is the component structure. This is the altered FuncDetail class (see Listing 16). When you code your own system, rather than using the demonstration system, you'll establish a structure for each function according to its needs.

Listing 16. The enhanced function class

public final class FuncDetail {

  	private String name;        // Function name
   	private long   used;        // times used
   	private QueueHeader agent;  // optional agent queue
   	private int    nbr_que;     // number of queues in this entry
   	private QueueHeader[] qtbl; // array of queues

Having started with a simple common memory environment, we have now enhanced that environment to support request brokering by adding several classes and altering other classes (including adding the new classes to the FrameWorkBase class). Figure 4 is the final common memory structure to support this framework.

Figure 4. Common memory referencing
Common memory referencing

Enhanced framework logical walkthrough
For any request, the RMI connection thread:

  • Creates either an AsyncDetail or SyncDetail object with the list of queues and the reference to the client_data (see Listing 3)

  • Finds the proper queues for the request by walking the chain

  • Places the integer subscript for either an AsyncDetail or SyncDetail object into the wait list of each queue in the function according to its priority

  • Wakes up an application thread on each queue

For the synchronous request, the RMI connection thread waits until all queues finish processing. The framework then concatenates the return objects from all the logical processes into a single object array and returns the object array to the client.

For the autonomous request, the RMI connection thread returns to the client with an "its been scheduled" message. The processing takes place asynchronously. When the last queue's application finishes processing, the application thread optionally concatenates the return objects from all the logical processes into a single object array and activates a new logical process, the agent. The agent application may examine those return objects and take further action in support of the request. For instance, if all the logical processes completed normally, then it will issue a commit; otherwise, it will issue a rollback.

Seeing it work
We've talked a lot about threads and queues and it's probably a little confusing. Nothing alleviates confusion like watching an asynchronous process manager in action. It's time to put it all together with a demonstration. If you haven't downloaded the zip file yet, then do so now.

This demonstration requires at a minimum version 1.1 of the Java platform. Unzip the file into a directory. The structure is as follows:

  • /Doc: Contains a single file, Doc.html, which documents all the classes and the run-time procedure
  • /Source: Contains all of the source code for this article
  • /Classes: Contains all of the class files for this article, including a policy.all file for security

Open the /Doc/Doc.html file for directions. Follow the directions for starting the RMI registry and the FrameWorkServer (in the section entitled "Run-time summary").

Follow the directions for starting a single-access client, DemoClient_3, whose function is F3, which comprises three queues (this is the section entitled "The first time").

Run-time summary
This is what happens when you start the client. The client invoked the syncRequest() method on the FrameWorkServer remote object passing a FrameWorkParm Object. The syncRequest():

  • Finds that the requested function, named F3, contains three queues, named Q1, Q2 and Q3
  • Creates a SyncDetail object and places the subscript to it into wait lists in queues Q1, Q2 and Q3
  • Finds that Q1 has no threads alive, so it instantiated a new thread *
  • Finds that Q2 has no threads alive, so it instantiated a new thread *
  • Finds that Q3 has no threads alive, so it instantiated a new thread *
  • Waits for all the logical processes to finish processing
  • When notified of the completion, picks up the return objects from each logical process, concatenates each object into an object array, and returns the object array to the client

Note: In the steps marked with *, if there had been a thread alive, the syncRequest() would only have had to notify() it.

While the syncRequest() is waiting, each application thread:

  • Searches the wait lists for the first available request
  • Picks up the SyncDetail object for the request
  • Calls the doWork() method of the application processing class to perform the logic for the queue
  • Saves the reference to the return object from the application processing class in the SyncDetail class
  • When it determines all other queue's finished processing, wakes up the waiting RMI connection thread
  • Searches the wait lists for the first available request, and, because none are found, issues a wait() until the next notify().

Load it up
The excitement comes when many clients hit on the server simultaneously. Additionally, without a visualization tool you would have no way of knowing what is going on. Within this package, there are two classes that provide just such a tool. (This is the section of the instructions entitled "Load it up.")

Follow the directions for running the visualization tool, class FrameWorkThreads.

Follow the directions for running the multiple client threads, class DemoClientMultiBegin, to put a load on the system.

Shut it down
After you are done with the server, you can shut it down gracefully with a client request, DemoClient_Shutdown.

Possible enhancements to the framework
In this brief article, we can only examine the skeleton of an asynchronous process manager. We use the word framework because the word describes a skeletal support structure. Some elements that could supplement this support are:

  • Error recovery. As we noted above, sometimes the anomaly code far outweighs the standard code. With a custom framework, error recovery depends on the application. Most detection depends on timing different aspects of the process. Catching an exception is easy; spotting a runaway thread is difficult.To know what to look for, one must know what the application does.

  • Thresholds. The question of when to instantiate or activate an application thread is paramount. The way the code example currently sits, the only time the framework instantiates or activates a new thread within a logical process is when no thread is alive in the queue or when a new request into a wait list causes an overflow into the next wait list. It is usually better to activate another thread when the load on that queue becomes greater than some user-determined value. This is threshold processing. When the RMI connection thread puts a new request into a wait list, the thread can determine the current load on that queue and, according to predetermined requirements, can start or activate another application thread.

  • Hooks and exits. How does a developer handle connection pools? How does a developer handle message queuing middleware packages? Remember, the server is persistent. You can add a start-up hook in which you build a separate memory area where you keep instantiated classes and private threads for these products. You can add a shutdown hook that gracefully shuts down the separate area.

  • Logging. Anyone who has ever worked with a background process knows how important it is to log errors. How else can anyone know what happened after a failure? Any general-purpose log will suffice. Commercial products are available today and the standard Java language will support logging in the near future. See also the open-source project Log4j, Nate Sammons's article on Syslog, and the alphaWorks Logging Toolkit for Java, all in Resources.

  • Custom versus generic. This is a custom framework. You build such a system to support a specific set of applications. When you need to support a wide range of applications or there is no time to design one yourself, then the better choice is to purchase a generic, full-featured asynchronous process manager. See Resources for a listing of generic frameworks.

Conclusion
That was a lot to cover in one article. Nobody claims that building a back-end framework is simple. Remember, the Java architects put a colossal effort into building the EJB and GUI containers. What do we now have?

We separated the RMI logic from the application logic. By doing this, we opened up the world of application queuing and threading (which is not restricted to RMI). This world enabled us to:

  • Have RMI connection threads and application threads talk to each other
  • Time requests
  • Run autonomous requests without overloading the server with application threads
  • Run agents as part of any autonomous request
  • Process multiple requests from a queue's wait lists, thereby reducing application start/stop overhead
  • Tune the server by keeping counts of every event
  • Control the create/destroy overhead inherent with threading
  • Easily plug in any application class as the subject of an application thread
  • Effortlessly trap a thread or application class for debugging
  • Run recursive requests from any application
  • Use a monitor to seek out nonperforming requests and a method to deal with them
  • Gracefully shut down the RMI server

Then we enhanced the single-process environment into a request broker capable of parallel processing. We enriched the common memory environment to:

  • Run parallel queue processing (request brokering)
  • Easily log events
  • Add almost any middleware product as an exit or hook
  • Completely customize the code for any application

From this point, the RMI server application container is no longer empty.

Resources

About the author
Since his academic introduction to queuing and subtasking, Edward Harned has been actively honing his multithreading and multiprocessing talents. He first led projects as an employee in major industries and then worked as an independent consultant. Today, Ed is a senior developer at
Cooperative Software Systems, where, for the last four years, he has used Java programming to bring asynchronous-process solutions to a wide range of tasks. Contact Ed at ed@coopsoft.com.