How to Calculate the Nodes in Your XOG XML Input File and Batch Process Your Requests

Document ID : KB000104078
Last Modified Date : 03/07/2018
Show Technical Document Details
Introduction:
Each CA PPM environment has an upper bound limit on the maximum number of XML Nodes allowed in any XML Open Gateway (XOG) XML input file used for a XOG read or write request. This article explains how to calculate the number of nodes in your input file so that you can break it down into chunks digestible by XOG and batch process your requests.
Background:
XOG Governor Limit: 

When you import or export many instances using XOG, you can encounter Out of Memory (OOM) or time-out exceptions. To handle these issues, you can configure the Maximum XML Nodes option available under the Application tab in the CSA. The Maximum XML Nodes specifies the number of XML nodes that can be imported or exported at any given time. The default value is 150,000 for both the On-Demand and On-Premise environments. 

Note: The Maximum XML Nodes is referred to as nodes limit in this document. 
A node is a single unit of Maximum XML Nodes and is the sum of an element and its attributes. 
Environment:
CA PPM 15.1+ SaaS and On Premise environments
Instructions:
Nodes are not rows, nodes are every element and attribute value. To see how many nodes are in your file search it for the count of ‘=’ for a rough approximation and then add some. Notepad++ or the ‘find’ command used at a command prompt in Windows can be used to quickly obtain such counts. 

You can count in your file. 

Count of ‘=’ (attribute assignments): 
Count of ‘/>’ (empty tags): 
Count of ‘</’ (closing tags): 

We strongly recommend that you design your interface to batch updates to stay under the node limitation to produce a scalable and stable interface. You would need to change how you are pulling/pushing the data. 

Although the system may allow you to access the whole set of data, it will be at the expense of a higher memory footprint, leading eventually to OOM issues. If/When your data grows to the point where 150,000 won't work, you'll be in the same boat once again. 
The file can be divided up into a smaller files (a task for the customer or consultant helping them) and then each file can be XOG’d in individually until all the pieces of the batch are complete. The file should be divided into smaller pieces to XOG in successfully.