Getting Certificate not verified issue
Document ID : KB000107561
Last Modified Date : 23/07/2018
 Show Technical Document Details
Question:
We are facing an below issue when tried to call application. But it work's from only one node. Can you please advise on this issue?

Error:
Problem routing to https://backendserver.com.au:443/path. 

Error msg: 

Unable to obtain HTTP response from https://backendserver.com.au:443/path: 
Certificate not verified. Caused by: Certificate [c=...,o=...,,cn=...] path validation and/or revocation checking failed
 
Environment:
API Gateway. 8.3 
(Soln for Gateway 8.4 +  also shown).
Answer:

The error your getting on forward request to the backend is : 
---
Problem routing to https://backend.com.au/... 

Certificate not verified. Caused by: Certificate key usage or extended key usage disallowed by key usage enforcement policy for activity: sslServerRemote 
---

Looking up the error we find solution : 

https://communities.ca.com/thread/241760126-certificate-key-usage-or-extended-key-usage-disallowed

Which gives, different soln for 8.4 (simpler one) and more complex usage for 8.3 

For Gateway 8.3 we followed the soln : 

https://docops.ca.com/ca-api-gateway/8-3/en/publish-services-and-configure-policies/working-with-policies/key-usage-enforcement-policy

Where we create an cluster wide property : 
 
pkix.keyUsagePolicy

with the value : 
 <keyusagepolicy xmlns="http://www.layer7tech.com/ws/keyusage">
   <!-- A permit rule without an action applies to every action -->
   <permit><req>anyExtendedKeyUsage</req></permit><!-- A permit rule without requirements always succeeds -->
   <permit action="signXml"/> <permit action="decryptXml"/>
   <!-- Multiple permit rules may be specified for an activity.At least one permit rule must succeed. -->
   <permit action="verifyXml"><req>digitalSignature</req></permit>
   <permit action="verifyXml"><req>nonRepudiation</req></permit>
   <!-- Multiple requirements may be specified for a permit rule.
   All requirements must match for that permit rule to succeed. -->
   <permit action="sslClientRemote"><req>digitalSignature</req><req>keyEncipherment</req></permit>
   <permit action="sslClientRemote"><req>nonRepudiation</req><req>keyEncipherment</req></permit>
   <!-- An ext. key usage requirement may use a dotted-decimal OID. -->
   <permit action="sslClientRemote"><req>1.3.6.1.5.5.7.3.2</req></permit>
   <permit action="encryptXml"><req>keyEncipherment</req></permit>
   <permit action="sslServerRemote"></permit>
   <permit action="verifyClientCert"><req>keyCertSign</req></permit>
   <permit action="verifyCrl"><req>cRLSign</req></permit>
   </keyusagepolicy>


Note:
 we remove the <req> (requirement) section from the sslServerRemote : 
so : 
   <permit action="sslServerRemote"><req>keyEncipherment</req></permit>
   <permit action="sslServerRemote"><req>keyAgreement</req></permit>
   <permit action="sslServerRemote"><req>id-kp-serverAuth</req></permit>
becomes : 
   <permit action="sslServerRemote"></permit>

This means that we do not need those special usage attributes set in the cert for it to be used for sslServerRemote authentication (all other requirements, trusted path etc still apply).

Save and restart the ssg service and then the certificate was trusted. 
Additional Information:
It was odd how the problem started, since you did not have the issue, and then the issue occurs on one gateway server, and not the other (at least in the test environment).   The PROD environment has the same issue.

But we would expect the key usage to have either been a problem all along, or a problem due to some update of the certificate with different key usage - so cause like that.   But this seemed to occur without some previous change.   It is possible that the condition occurs on restart, so perhaps update of a patch and delay of restart may be other way the issue could occur. 

Nevertheless we identified the error as keyusage. 

Note2:  
Debugging was a little complicated, since the backend http route was using a proxy, and openssl version (pre 1.1 ) does not accept -proxy setting.


Note3: 
We looked at enabling java SSL tracing - but that too would be more complex - so were luck the above solved it : 
https://docops.ca.com/ca-api-gateway/8-3/en/configure-security/tasks-menu-security-options/manage-log-audit-sinks/working-with-log-sinks-and-debug-logs