Kubernetes agent showing error messages in agent log

Document ID : KB000077019
Last Modified Date : 10/04/2018
Show Technical Document Details
Issue:
Customer has enabled the Kubernetes monitoring agent for APM Saas environment. The Kubernetes monitoring agent is running ok, and agent is also reporting data to APM Saas environment. But the agent is encountering many error messages which is showing in Kubernetes agent log.

[INFO] [IntroscopeAgent.OpenShiftMonitor] Trying to connect https://xxxx.xxxx.xxxx.azure.com using API Token
io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:57)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getRootPaths(BaseOperation.java:190)
    at io.fabric8.kubernetes.client.BaseClient.rootPaths(BaseClient.java:150)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShift(OpenshiftAdapterSupport.java:51)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isAdaptable(OpenshiftAdapterSupport.java:35)
    at io.fabric8.kubernetes.client.AutoAdaptableKubernetesClient.adapt(AutoAdaptableKubernetesClient.java:107)
    at io.fabric8.kubernetes.client.AutoAdaptableKubernetesClient.<init>(AutoAdaptableKubernetesClient.java:96)
    at com.ca.apm.openshift.OpenShiftMonitorService.IAgentService_startService(OpenShiftMonitorService.java:156)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.startService(ServiceAdministrator.java:604)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.startServiceSafe(ServiceAdministrator.java:618)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.IExecutableItem_execute(ServiceAdministrator.java:482)
    at com.wily.util.task.AsynchExecutionQueue$WrapperItem.IExecutableItem_execute(AsynchExecutionQueue.java:414)
    at com.wily.util.task.AsynchExecutionQueue.doTask(AsynchExecutionQueue.java:205)
    at com.wily.util.task.ATask$CoreTask.run(ATask.java:132)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: xxxx.xxxx.xxxx.azure.com
    at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
    at java.net.InetAddress.getAllByName(InetAddress.java:1192)
    at java.net.InetAddress.getAllByName(InetAddress.java:1126)
    at okhttp3.Dns$1.lookup(Dns.java:39)
    at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:172)
    at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:138)
    at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:80)
    at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:178)
    at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:129)
    at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:98)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:109)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:124)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at io.fabric8.kubernetes.client.utils.HttpClientUtils$3.intercept(HttpClientUtils.java:102)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:170)
    at okhttp3.RealCall.execute(RealCall.java:60)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:235)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:230)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getRootPaths(BaseOperation.java:183)
    ... 13 more

[ERROR] [IntroscopeAgent.OpenShiftMonitor] Failed to connect to https://xxxx.xxxx.xxxx.azure.com

[ERROR] [IntroscopeAgent.InfrastructureAgent Agent] Failed to re-register IAgentRemoteConfigurationService on Agent name change event.

 
Environment:
APM Kubernetes monitoring agent 10.7
APM Saas
Cause:
It looks like host name is not being resolved properly. 
This can be because of the way Kubernetes network is setup from the container..
Resolution:
We suggest you to use the IP address for host xxxx.xxxx.xxxx.azure.com  which is showing up in above error message.