EPAgent 101: Use the SDK to Send Data to APM

Document ID : KB000010082
Last Modified Date : 14/02/2018
Show Technical Document Details
Introduction:

ABOUT EPA

 

This information does not replace official documentation for EPA.  It's just a quick-start guide, so don't try to find all the answers within.

 

The CA APM EPA is a stand-alone Java application with a wide range of features for collecting data to send to the Enterprise Manager.  EPA was designed so that customers could, as easily as possible, integrate any data into CA APM – even information about the state of the garage door, lego robots, etc.

 

Introscope EPA was originally designed for customers to register script “plugins”, either in “stateful” (e.g. log reading) or “stateless” (e.g. CPU or OS stats.)  Some APM customers had needs that weren’t met by creating an EPA and running it local to an environment – the footprint of the agent and scripts were too expensive, or the process of installing an entire EPA to collect the data was inconvenient.

 

A single EPA can be configured to listen to a network port for metric data from arbitrary remote systems.  Specifically, one port can be set to listen for XML-formatted metric data, and a second can listen for data sent using HTTP requests (HTTP GET or RESTful).

  • New in version 9.7, a REST-compliant interface.  Metrics can be submitted in batches over a RESTful PUT operation.  This is the preferred interface (more efficient).
  • HTTP GET requests are super simple for quick-and-dirty problems and can often be sent from common programs like ‘wget’ or 'curl', which are often already available at the command prompt.    This is used for single-metric submission and is less efficient than the newer RESTful interface.  This interface can be quickly used via a browser to illustrate sending metrics.  This interface will be deprecated and maintained only for backwards compatibility.
  • XML format provides maximum flexibility, but is slightly more complex and requires manual management of the socket.

 

Refer to the more detailed discussion of the formats below.

Background:

WHY EPA?

 

EPA is a great way to get started because you can take your existing monitoring scripts (which I often refer to as "poor-man's monitoring") and simply wrapper them without making additional changes.  Using this technique you can continue sending current alerts to email (or wherever) and fork the data to EPA until you're ready to make a single stream.  This can be done very quickly, so you can be the hero before lunchtime.

 

When run locally, output printed to STDOUT will be sent to the EPA and output printed to STDERR will be sent to the EPA log.  So the easiest way to wrap existing script work is to create a short wrapper that takes existing script output and parses into the correct format (simple or XML; both described below) and then prints the formatted output to STDOUT (or to HTTP GET).  In this manner, existing scripts can be ported to EPA in a matter of minutes!  Libraries for formatting output are available in Perl.

 

NOTE: EPA takes string or integer data only.  If you have floating-point data, it should be converted to integer values prior to sending to EPA.

Instructions:

DETAILED TRAINING MATERIALS (INCLUDING RESTFUL-API DETAILS)

 

In addition to product documentation, CA recently provided updated training materials in support of the RESTful API update.  This training includes details on error codes, syntax highlights, etc.  Please review.  The information on this quick-start page will not be as exhaustive.

 

SETUP INSTRUCTIONS

 

Download the EPA as directed above.  Then unzip it to some fancy directory.  It can be on any machine (versions exist for Windows and UNIX but the only difference is startup files).  You will need an available JRE to run EPA (it does not ship with one embedded.)

 

Edit the config/IntroscopeEPAgent.properties

Find and uncomment the following lines:

 
  1. # Network port on which to receive simple or XML formatted data. If commented  
  2. # out or left unconfigured then EPAgent will not listen for data.  
  3. #  
  4. introscope.epagent.config.networkDataPort=8000  
  5.   
  6.   
  7. # Network port on which to listen for HTTP GET  
  8. # left unconfigured then EPAgent iwll not list  
  9. #  
  10. introscope.epagent.config.httpServerPort=8080  

 

You will probably want to reconfigure the network ports to be appropriate to your environment.  You should also edit the file for any other properties, such as the correct location of the EM to which you'll be sending data.

 

You should have a bin/ directory with a start script:

 

 
  1. [root@centos64rt2 ~]# cd epagent/bin  
  2. [root@centos64rt2 bin]# ./EPACtrl.sh start  
  3. ./EPACtrl.sh start: EPAgent started  
  4. [root@centos64rt2 bin]# ./EPACtrl.sh status  
  5. ./EPACtrl.sh status:  Agent is running  
  6. [root@centos64rt2 bin]#  

 

On Windows, install and start the service (there's an executable that will do this for you).  From a unix command-line, you can launch Introscope EPA manually with the following:

 

java -jar lib/EPAgent.jar

 

You may want to give the Introscope EPA some additional JVM memory, depending on the processing load of metrics you expect it to handle:

 

java –Xms256m –Xmx256m -jar lib/EPAgent.jar

 

You should see confirmation that the agent started successfully:

 

4/23/08 10:23:01 AM MDT [INFO] [EPAgent] Introscope EPAgent started.

 

Now the listener should be active (assuming you're going to use one of the network options and not STDOUT wrapping).

 

SENDING DATA TO EPA

 

There are three ways to send data.  Data is formatted with pipes (|) dilineating folders/resources and a colon (:) marking the metric name:

Folder1|Another Folder With A Space:Metric1

 

RESTful API (HTTP POST)

 

This is the modern, preferred method.  Once running, create a sample text file like the one below.  API Documentation can be found here.  Then pipe it into an HTTP command with the curl statement provided:

 

 
  1. michaels-macbook-pro:tmp mwalker$ cat epagent_rest_sample.json  
  2. {  
  3.   
  4.     metrics : [  
  5.   
  6.         {type: "IntCounter", name: "Sample Data|Ints:IntCounter", value : "1234"},  
  7.   
  8.         {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "3"},  
  9.         {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "4"},  
  10.         {type: "IntAverage", name: "Sample Data|Ints:IntAverage", value : "8"},  
  11.   
  12.         {type: "LongCounter", name: "Sample Data|Longs:LongCounter", value : "1234"},  
  13.   
  14.         {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "3"},  
  15.         {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "4"},  
  16.         {type: "LongAverage", name: "Sample Data|Longs:LongAverage", value : "8"},  
  17.   
  18.         {type: "StringEvent", name: "Sample Data|Strings:StringEvent", value : "The quick brown fox jumps over the lazy dog"}  
  19.     ]  
  20. }  
  21. michaels-macbook-pro:tmp mwalker$ curl -H "Content-Type: application/json" --data @epagent_rest_sample.json http://centos64rt2.ca.com:8080/apm/metricFeed  
  22. {"validCount":9}  
  23. michaels-macbook-pro:tmp mwalker$  

 

Metrics will show up in your WebView UI within 15 seconds:

 

webview.png

 

RESTful CODE SAMPLES

We have a number of samples to assist you in actual coding.

d     

 

HTTP GET

As an HTTP GET request (deprecated), EPA metrics take the following form:

 

http://localhost:8080/?metricType=IntCounter&metricName=LarryA|MoeA:CurlyB&metricValue=60

 

You will need to change the host and port as appropriate to your environment.  You can test by putting that into a browser because it's a simple GET request (not a POST).  After pasting this URL into a browser, your WebView/Workstation UI will show the new metric within 15 seconds (as shown above).

 

Metric names with spaces should be quoted when using command-line tools, but do not need quoting when testing with a live browser session.

 

‘wget’ is an excellent, free command-line tool for generating such requests from shell scripts and similar.  NOTE: When using wget, you may need to quote the URL for correct operation; this will also allow the use of spaces in metric names.

 

Send XML Data Directly to the Socket

The XML format of output is much more robust and allows for the sending of errors and transaction traces in addition to time-series metrics.  Connect to the XML network port, report a snippet of XML like what follows, and disconnect from the port (or leave the port open synchronously).

 

 
  1. <metric type="LongCounter" name="Resource Usage|File IO:diskWrites" value="37" />  
  2. <metric type="StringEvent" name="Apache Errors:LastErrorString" value="ERROR: Apache shutdown unexpectedly" />  

 

NOTE: The entire XML Schema is documented in the official documentation.

 

METRIC TYPES

 

When sending data use the following metric types:

 

  1. PerIntervalCounter (useful for rate metrics, like “miles per hour” or “errors per interval”; resets to zero at each new interval)
  2. IntCounter (useful for tally metrics, like “msgs in queue”, and does not change until a new value is reported)
  3. IntAverage (useful for response times, like “average time in seconds”; you don’t calculate the average yourself; just report all the applicable metrics (like in a loop) and the calculation will be performed automatically at the end of the interval)
  4. LongCounter  (same as above, but for very large numbers)
  5. LongAverage (same as above, but for very large numbers
  6. StringEvent (use to report string values, like “startup command-line” or a log entry).  NOTE: StringEvents are not stored historically; only current values are used.

 

 

That's all you need to get started.  Now go script something cool!