Monitoring and Logging

Applications deployed on Fabric3 can take advantage of its built-in injection-based monitoring framework for logging and recording application events. The monitoring framework offers a number of benefits:

• Strongly-typed monitoring interfaces and events
• High performance logging (millions of events per second) with no garbage creation for latency-sensitive appications
• Dynamic event level adjustment

Using Monitoring

To use the monitoring framework in application code entails two things, creating a monitor interface and injecting the monitor in a component.

Define a Monitor Interface.

The monitor interface is used for sending monitoring events. The interface defines operations for publishing events, monitoring levels and optional event message templates:

import org.fabric3.api.annotation.monitor.Severe;
import org.fabric3.api.annotation.monitor.Debug;

public inteface ComponentMonitor

   @Severe
   void error(String message, Throwable t);

   @Debug ("Received request {0}")
   void receivedRequest(String id);

}

The above interface defines an error event and a debug event. The debug event also specifies a formatted message that will be logged if the event is received. The following monitoring levels are supported:

• Severe - Critical errors that affect continue runtime operation
• Warning - Error conditions that do not affect continued runtime operation or a potential runtime configuration issue
• Info - Informational event
• Debug - An event useful for diagnosing a problem
• Trace - A low level event useful for diagnosing a problem

By default, Info and above are enabled at runtime.

Injecting the Monitor

A monitor is injected in a component using the org.fabric3.api.annotation.monitor.Monitor annotation:

import org.fabric3.api.annotation.monitor.Monitor;

public void TheComponent {

   @Monitor
   protected ComponentMonitor monitor;

   public void call(Message message) {
      monitor.receivedRequest(message.getId);
      try{
         validate(message));
      } catch (ValidationException e) {
         // bad message
         monitor.error("Invalid message", e)
      }

      // ...

   }

When an @Monitor annotation is encountered, the Fabric3 runtime will generate a monitor proxy and inject it based on the monitor interface. Depending on the current monitor level, events may be recorded or ignored. In the above example, if the monitor level is set to severe, the receivedRequest() event will be dropped.

Configuration

Monitor events are routed to a destination.  A destination has one or more appenders that act as sinks for monitor events. It is possible to create a custom appender (detailed below) or use one of the provided appenders such as the file or console appenders. An application (or service) can send events to a custom destination or use the default runtime destination. Sending events to a custom destinations allows traffic for a particular application, subsystem or service to be segmented from other events.  

Runtime Specific Behavior

The Fabric3 monitor framework uses different implementations depending on the host environment. For WebLogic, Maven and Ant, the native host logging framework is used. For example, on WebLogic, monitor proxies will forward events to WebLogic's logging infrastructure. Configuration is therefore specific to each host environment.  The Fabric3 Standalone and Tomcat runtimes use the native Fabric3 monitoring implementation, which is described in the rest of this section.

Configuring the Runtime Destination

The default runtime destination is where Fabric3 runtime events (and application events if not specified) are sent. The default destination can be configured in systemConfig.xml , for example, to record events to a log file instead of outputting to the console. This is done by adding appender entires under the monitor element in systemConfig.xml:

<monitor>
   <appenders>
       <appender.file file="fabric3.log"/>
       <appender.console/> 
    </appenders>
</monitor>

The above example configures both a console and file appender. The file attribute is the location of the output file, which is relative to the runtime /data directory (subdirectories are supported).

Timestamp Formatting

Timestamp formatting can be configured using the pattern attribute of the monitor element:

<monitor pattern="%d:%m:%Y %H:%i:%s.%F">
   ...
</monitor>

The following formatting directives are supported:

• %a - Abbreviated weekday name ("Sun", "Mon", "Tue")
• %b - Abbreviated month name ("Jan", "Feb", "Mar")
• %c - The Month number
• %d - The day of month in padded two digit form (01..31)
• %e - The day of month in trimmed form (1..31)
• %f - Milliseconds in trimmed form (0..999)
• %F - Milliseconds in padded three digit form (000..999)
• %H - The hour in 24-hour form (00-23)
• %h - The hour in 12-hour form (01-12)
• %i - Minutes
• %j - Day of year (001..366)
• %k - The hour in trimmed 24-hour form (0..23)
• %l - The hour in trimmed 12-hour form (1..12)
• %M - The full month name
• %m - The month number
• %p - AM or PM
• %S - The seconds in trimmed form
• %s - The seconds in padded two digit form
• %W - The full weekday name("Sunday".."Saturday")
• %w - The day of week (1 - Sunday .. 7 - Saturday)
• %Y - The four digit year
• %y - The two digit year

Configuring Custom Destinations

As mentioned previously, it is possible to setup custom destinations to segment event traffic. This is done by including a monitor entry in a composite file. The following example is taken from the monitor sample application:

<composite .... xmlns:f3="urn:fabric3.org">    
 
   <f3:monitor name="ApplicationDestination">
      <appenders>
         <appender.file file="application.log"/>
      </appenders>
   </f3:monitor>

</composite>
 

Application code can then reference the destination by specifying its name in the @Monitor annotation:

public class SomeComponent ... {
   @Monitor("ApplicationDestination") 
   protected ServiceMonitor monitor;

    public void process() {
       ...
       monitor.message("This is a message");
    }
}

Asynchronous Monitoring with Ring Buffers

By default, monitor events are routed to destinations synchronously. This may result in significant performance impact for applications that frequently emit events. The Fabric3 monitor implementation can be configured using the mode attribute to route asynchronously using a LMAX Disruptor ring buffer, as illustrated in the following excerpt:

<f3:monitor name="ApplicationDestination" mode="asynchronous">
   <appenders>
      <appender.file file="application.log"/>
   </appenders>
</f3:monitor>
 

The mode attribute is set to "asynchronous," which indicates routing should be done using a ring buffer. If you plan to use asynchronous monitoring, keep in mind several things. The first is to select a ring buffer size that is large enough to avoid wrapping events (see the LMAX Disruptor documentation for more information). The default value is set at 65536 slots. The second is how buffers for writing monitor event data are handled. To avoid object creation, Fabric3 pre-allocates byte buffers with a configurable but fixed size (3,000 bytes by default) for writing monitor events. If the byte buffer size is not large enough to handle a message, you will need to increase the pre-allocated size or data will be truncated. Setting the ring buffer slot size and byte buffer capacity are done using the ring.size and capacity attributes respectively:  

<f3:monitor name="ApplicationDestination" mode="asynchronous" ring.size="65536" capacity="3000">
   ...
</f3:monitor>
 

If you decide to use as asynchronous monitor routing, it is also recommended you use the Fabric3 bytecode generation extension (Maven coordinates {{org.codehaus.fabric3:fabric3-bytecode-proxy}} for creating monitor proxies. This is done by including the extension in the runtime /extensions directory and setting the proxy attribute to "bytecode" as follows:

<f3:monitor name="ApplicationDestination" mode="asynchronous" ring.size="65536" capacity="3000" proxy="bytecode">
   ...
</f3:monitor>
 

As a final performance recommendation, you should also consider whether to include formatted timestamp information in the log output. Formatting is a performance drain and can be avoided by using a raw nanosecond timestamp. This can be configured by setting the timestamp attribute to "unformatted" or "none":

<f3:monitor name="ApplicationDestination" mode="asynchronous" ring.size="65536" capacity="3000" proxy="bytecode" timestamp="unformatted">
   ...
</f3:monitor>

The monitor ring buffer can be fine-tuned using the following attributes. For more information, consult the LMAX Disruptor documentation:

• capacity
• ring.size
• wait.strategy
• blocking.timeout
• spin.timeout
• yield.timeout
• phased.blocking.type

Custom Appenders

The Fabric3 monitor implementation supports custom appenders. For an example, refer the sample monitor application.  

Configuration Reference

The following represents a pseudo-schema for monitor configuration in systemConfig.xml:

<monitor mode="synchronous|asynchronous" capacity="int" ring.size="power of 2" proxy="bytecode|JDK" wait.strategy="blocking|yielding|sleeping|backoff|spin|timeout"         
         blocking.timeout="nanoseconds" spin.timeout="nanoseconds" yield.timeout="nanoseconds" phased.blocking.type="lock|sleep" timestamp="formatted|unformatted|none" pattern=".." timezone="Java timezone">
  
</monitor>