How do I activate enhanced logging for app-access logs using Apache Tomcat 7?

Document ID : KB000030674
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

How do I activate enhanced logging for application access (app-access) log files using Apache Tomcat 7?

Solution:

This is useful for both out-of-memory and performance issues:

Access logs are produced by the front-end web server that is connected to your application server. By default this exists for Tomcat but not for Websphere and Weblogic. Front-end web servers need to be setup to be able to produce app-access.logs. The information below is targeted for Tomcat.

By default PPM sets up Tomcat to use the "common" access log valve in Apache.

The class in use is org.apache.cataline.valves.FastCommonAccessLogValve.

The FastCommonAccessLogValve produces some useful data not enough data to make the access.log useful as a troubleshooting tool. This is especially true when an environment is behind a load balancer (LB). When behind a load balancer only the IP address of the LB is pushed through to the log and the independent client IP is not logged. There is no way to know "who" the requester was. There is also a limitation to finding out when the request was triggered. The time stamp in the log is when the request was completed, not when it started.

This code is set in:

%CLARITY_HOME%/.setup/scripts/commondeploy.xml

<xpath file="@{server.xml.file.name}">
  <apply parent="/Server/Service/Engine/Host" select="Valve"> 
    <element name="Valve" select="Valve[@prefix = '@{app.id}-access-' or @className = 'org.apache.catalina.valves.FastCommonAccessLogValve' or @className = 'org.apache.catalina.valves.AccessLogValve']">
      <attribute name="className" value="${app.access.valve.class.name}"/>
      <attribute name="pattern" value="${app.access.valve.pattern}"/>
      <attribute name="resolveHosts" value="false"/>
      <attribute name="suffix" value=".log"/>
      <attribute name="directory" value="${install.dir}/logs"/>
      <attribute name="prefix" value="@{app.id}-access-"/> 
      <attribute name="resolveHosts" value="false"/>
    </element>
  </apply>
</xpath> 

This logging is useful but does not include elapsed time, is not easily imported into a spreadsheet and does not contain session data.

Solution:

A new enhanced logging format, which uses the AccessLogValve class (instead of the FastCommonAccessLogValve) has been created to provide elapsed time, a delimited format for accessing the data and session id's to help identify users when load balancers are used.

Access the file, %CLARITY_HOME%/.setup/scripts/commondeploy.xml, and change the valve class block.

<xpath file="@{server.xml.file.name}">
  <apply parent="/Server/Service/Engine/Host" select="Valve"> 
    <element name="Valve" select="Valve[@prefix = '@{app.id}-access-' or @className = 'org.apache.catalina.valves.FastCommonAccessLogValve' or @className = 'org.apache.catalina.valves.AccessLogValve']">
      <attribute name="className" value="${app.access.valve.class.name}"/>
      <attribute name="pattern" value="%h|%t|%r|%s|%b|%D|%{sessionId}c"/>
      <attribute name="resolveHosts" value="false"/>
      <attribute name="suffix" value=".log"/>
      <attribute name="directory" value="${install.dir}/logs"/>
      <attribute name="prefix" value="@{app.id}-access-"/> 
      <attribute name="resolveHosts" value="false"/>
    </element>
  </apply>
</xpath>

Save the file, then run:

service stop remove app
service add deploy start app

(Repeat for all apps in a cluster).

The access log will now produce the following in a "|" (pipe) separated format:

IP address of requestor|
Date/Time of the completed request|
URL request|
HTTP Status Code|
Bytes sent back to the client (excluding the header)|

Elapsed time of the request (in ms, can be subtracted from the DateTime to get the request startTime)| Clarity sessionId cookie (a distinct way to group requests, can be tied back to the sessions tables in Clarity to find the user).

For more information on pattern values, go to the following URL http://tomcat.apache.org/tomcat-7.0-doc/config/valve