Troubleshooting an application slowdown that may be Agent-related.

Document ID : KB000009866
Last Modified Date : 14/02/2018
Show Technical Document Details
Introduction:

 A production application can run slowly with the Agent monitoring it.  This depends on what the application exposes to the Agent. Therefore, troubleshooting this can take some time in determining which metric(s) are causing the issue at hand.

Background:

 Depending on what your application exposes, certain metrics may cause the application to slow down.  

Environment:
All supported APM environments.
Instructions:

 There are two ways to handle this:

 

Approach #1: Check the Investigator to see where the overhead is coming from.   

1.)  Go to the Investigator Tree

2.)  Select your Agent

3.)  Select the “Search” Tab

4.)  Type “Responses Per Interval”

5.)  Sort by largest values first

 

Approach #2: If the above doesn't show a culprit, then proceed with the following:

 

1.) To fully ensure if it indeed is the APM Agent causing the slowdown, modify the IntroscopeAgent.profile and set the following property as shown below.  A restart of the application is required.

introscope.autoprobe.enable=false

 If your application runs faster after a restart, then the issue is with the Agent instrumentation.  If there is no improvement after the restart, then this indicates that the issue is in your application. Because with instrumentation disabled, there is very little interaction between the application and the Agent.  Proceed to the next steps if you did see an improvement in faster speed.

 2.) Check your hotdeploy directory and your directives line in the Agent profile (introscope.autoprobe.directivesFile) for any custom pbd/pbl.  If there are, temporarily remove them.  Note: this may result in your custom metrics from showing up but it is a good test to verify that anything custom is not causing the problem.

 3.) If after applying step 2 does not improve the speed, consider disabling the following"

  Note that each change below will require a restart of the application.  Best practice would be to eliminate one item at a time and restart, but if you are facing time constraints, I would recommend grouping some of them in each test and from there, disable one by one if finding that disabling a group of them speeds up the application.

    a.) Socket tracing - in toggles typical or full pbd

    b.) UDP tracing (Network) - in toggles typical or full pbd

    c.) Filesystem tracing - in toggles typical or full pbd

    d.) SQLAgent tracing - comment out sqlagent.pbd

    e.) Deep inheritance - set introscope.autoprobe.deepinheritance.enabled=false in the IntroscopeAgent.profile

     f.) Leakhunter - set introscope.agent.leakhunter.enable=false in the IntroscopeAgent.profile

    g.) ChangeDetector -  set introscope.changeDetector.enable=false in the IntroscopeAgent.profile