SRM AIM Receives 403 HTTP Status From Web Server

Document ID : KB000007323
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

After adding a HTTP(S) URL to be monitored by the SRM AIM, it would fail with a 403 Forbidden HTTP status code. The same URL may respond properly from a Web Browser on the SRM AIM host.

SRM jcollector.log entries for the problem URL may contain messages like the following:

[LOG_DEBUG1][{DATE} {TIME}][Thread:Thread-1337][Pass #101]: Sending request...
[LOG_DEBUG][{DATE} {TIME}][Thread:Thread-1337][Pass #101]: java.io.IOException: Server returned HTTP response code: 403 for URL: https://www.ca.com/
[LOG_DEBUG][{DATE} {TIME}][Thread:Thread-1337][Pass #101]: Trying to get error stream...
[LOG_DEBUG][{DATE} {TIME}][Thread:Thread-1337][Pass #101]: read header: HTTP/1.1 403 Forbidden
[LOG_CRITICAL][{DATE} {TIME}][Thread:Thread-1337][Pass #101]: [#42] ERRSRC:https ERRCODE:403 INDEX:42 NAME: TESTDESC:www_ca_com ERROR: jcollector.SATestException: Error downloading object: jcollector.SATestException: https://www.ca.com/ : jcollector.SATestException: HTTP server indicated error 403: HTTP/1.1 403 Forbidden
jcollector.SATestException: Error downloading object: jcollector.SATestException: https://www.ca.com/ : jcollector.SATestException: HTTP server indicated error 403: HTTP/1.1 403 Forbidden

 

Cause:

While 4XX HTTP status errors are generally seen as "client errors", in the case of the 403 Forbidden the server is actively refusing to process the request for one reason or another. Whereas a 404 Not Found can be caused by a bad URL from the client, or a 401 Unauthorized could be caused by bad credentials from the client.

Resolution:

The only option that that the SRM AIM has control over that might cause a Web Server to return the 403 Forbidden error is the "User Agent String". As the SRM AIM is java based, the default User Agent String for SysEDGE/SRM 5.9 is "Java/1.7.0_72". It is possible to block requests based on the User Agent.

 

To set the SRM's User Agent to something different, edit the \port(161|1691)\plugins\svcrsp\svcrsp.cf file on the SRM AIM system and uncomment the following line by removing the "#":

# Uncomment to override the default UserAgent string used in HTTP tests
#
#useragent="Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 7.1; Trident/5.0)"

should then become:

useragent="Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 7.1; Trident/5.0)"

If the sample User Agent that is provided in the svcrsp.cf is still not acceptable, this can be modified to a different value. For example (Firefox 55 64 Bit, running on Windows 10 64 Bit):

useragent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0"

Once the file has been modified, save it and restart the SystemEDGE agent for the new value to be picked up.

Note: If this does not resolve the problem, you may want to check with the web server admin to confirm why the 403 is being returned and/or open a support case with CA Support.

 

Additional Information:

The "wget" command can be used to troubleshoot this type of condition. Use the "-U" option to change the User Agent accordingly. See TEC1847151 - Troubleshooting SRM AIM HTTP/HTTPS Connection Problems for additional details/examples.

 

Example wget output with a 403 error:

--{DATE} {TIME}--  https://www.ca.com/
Resolving www.ca.com (www.ca.com)... {IPADDRESS}
Connecting to www.ca.com (www.ca.com)|{IPADDRESS}|:443... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 403 Forbidden
  Date: {DATE} {TIME}
  Content-Type: text/html; charset=UTF-8
  Transfer-Encoding: chunked
  Connection: keep-alive
  Set-Cookie: __cookie=4d652077616e7420434f4f4b49452120434f4f4b494521; expires={DATE} {TIME}; path=/; domain=.ca.com; HttpOnly
  Cache-Control: max-age=10
  Expires: {DATE} {TIME}
{DATE} {TIME} ERROR 403: Forbidden.

 

Example wget with a 200 OK response after changing the User Agent:

--{DATE} {TIME}--  https://www.ca.com/
Resolving www.ca.com (www.ca.com)... {IPADDRESS}
Connecting to www.ca.com (www.ca.com)|{IPADDRESS}|:443... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Date: {DATE} {TIME}
  Content-Type: text/html
  Transfer-Encoding: chunked
  Connection: keep-alive
  Set-Cookie: __cfduid=4d652077616e7420434f4f4b49452120434f4f4b494521; expires={DATE} {TIME}; path=/; domain=.ca.com; HttpOnly
  Last-Modified: {DATE} {TIME}
  X-Frame-Options: SAMEORIGIN
Length: unspecified [text/html]
Saving to: ‘index.html’

     0K .......... .......... .......... .......... .....       186K=0.2s

{DATE} {TIME} (186 KB/s) - ‘index.html’ saved [46726]

 

This Google query should show your current user agent.

useragentstring.com will show your current user agent and also has lists of many other user agents.