IM Data Aggregator rest call does not return items

Document ID : KB000029128
Last Modified Date : 14/02/2018
Show Technical Document Details

IM Data Aggregator rest call does not return items when there is too large a number of items to return.

Error in the <IMDataAggregator Instaldir>/apache-karaf-<#.#.#>/data/log/karaf.log:

WARN  | tp1547271220-374 | <date-time> | WebApplicationExceptionMapper | pl.WebApplicationExceptionMapper   71 | xf.cxf-rt-frontend-jaxrs |

      | java.lang.IllegalArgumentException:  The estimated count of 350123 for endpoint devices exceeds the maximumn allowed by the mananger of 200000. To avoid risking the integrity of the server, we are unable  to return the request XML.


Currently, the maximum number of items able to be returned is 200000.


Get tunneling can be used to filter the list.


What is Get Tunneling?

Get Tunneling is where we "Post" some XML to the WebServices on the Data Aggregator which defines a "query". This query specifically includes or excludes data to ensure only the data you want returned is actually returned by the data aggregator web services. This can be useful when you have a large number of items and want to restrict the data being returned by the system to a specific subset to both speed up the query AND simplify the data response.

What web service end point does this apply too?
This applies to ALL web services under to <DA_HOST>:8581/rest endpoint.

What is the URL that I would send XML to? What is the HTML method?
You must use the POST method to send XML to the following endpoint: <DA_HOST>:8581/rest/<specific_endpoint>/filtered
The format of the XML sent is very specific.

What is the format of the XML that we send?
The GetTunneling XML that you post has two distinct sections.

The first section, called the "Filter" section, is used to filter or restrict the number of ITEMs returned from the web service call. You can either specify the IDs of the items you want returned directly, OR, specify a dynamic list by querying specific attributes and comparing them to desired values.

The second section, called the "Select" section, is used to filter or restrict the attributes returned for ALL desired items, allowing your returned XML to have simple detail such as an ID, or as much detail as possible including all attributes.

In both cases, it is important to know that the more items, and the more attributes you include, the longer it will take to render and the more the DA will be impact. To ensure that the DA is not overloaded, there is a restriction on the number of items/attributes that can be returned.

Is there a sample filter XML available?

The following XML selects all communication profiles (i.e. SNMP profiles) which have "SNMP" in the name OR ( are ranked 3rd or lower AND have an SNMP port of 161). Once found, only the CommunityName, and the generic CommunicationProfile details are returned.

<FilterSelect xmlns:xsi="" xsi:noNamespaceSchemaLocation="filter.xsd">
            <CommunicationProfile.ProfileName type="CONTAINS">SNMP</CommunicationProfile.ProfileName>
               <CommunicationProfile.Rank type="LESS_OR_EQUAL">3</CommunicationProfile.Rank>
               <SNMPv1Profile.PortNumber type="EQUAL">161</SNMPv1Profile.PortNumber>
    <Select use="exclude" isa="exclude">
       <SNMPv1Profile use="exclude">
          <CommunityName use="include"/>
       <CommunicationProfile use="include" />