API gateway: Optimizing for Large Files and Messages

Document ID : KB000042880
Last Modified Date : 10/09/2018
Show Technical Document Details
Introduction:

Background

The default configuration of the CA API Gateway is optimized for handling messages in the range of 1 to 50 kilobytes. There are a several specific changes that are required for tuning the Gateway appliance to handle larger messages.

Implementation

Several settings and configuration files must be changed in order to meet this use case:

  1. The amount of system memory allocated to Java application.
  2. Cluster-wide properties that negotiate various maximum data sizes.
  3. Certain policy changes to limit message sizes.

Java memory allocation

  1. Log in to the API Gateway appliance as the ssgconfig user.
  2. Select Option #3: Use a privileged shell (root).
  3. Open /opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh in a text editor.
  4. Modify the file to appear as below (starting at line #29 as of version 8.0.0 of the Gateway):
#if [ `expr $java_ram \> 2074412` == 1 ] ; then
# ? ?if [ $usebigmemory -eq 1 ]; then?
# ? ? ? ? ? ?# we have more ram than java can use?
# ? ? ? ? ? ?# FIXME: when we start running 64 bit JVM?
# ? ? ? ? ? ?java_ram=2074412;
# ? ? ? ? ? ?# CAP at 2 gigs
# ? ?else ? ?
# ? ? ? ?java_ram=4194304;
# ? ?fi
#fi
  1. Save the file and exit the text editor
  2. Restart the Gateway application

Cluster-Wide Properties

Log in to the CA API Gateway Policy Manager as an administrative user.
Open the Manage Cluster-Wide Properties task
Add or edit the following properties:

  1. io.signedPartMaxBytes = 0
  2. io.xmlPartMaxBytes = 0
  3. request.compress.gzip.allow = true
  4. response.compress.gzip.allow = true
There are also three additional cluster-wide properties that need to be set with customized values:
  1. template.partBodyMaxSize
  2. attachment.diskThreshold
  3. io.httpParamsMaxFormPostBytes

template.partBodyMaxSize specifies the maximum amount of memory to use for storing message parts. The value of this property should meet or exceed the maximum expected size of an individual message part. Setting this property to a value that exceeds the total size of the message should be sufficient.

attachment.diskThreshold specifies the maximum size of a message attachment to store in memory before stashing it to disk. If a message attachment exceeds the size of this value then the attachment will be stashed to disk. This value should meet or exceed the maximum total size of any individual message attachments for MTOM or Soap With Attachments.

io.httpParamsMaxFormPostBytes maximum number of bytes to buffer when processing an HTTP form post.  Technical Note:  The io.httpParamsMaxFormPostBytes cluster property replaces the former com.l7tech.message.httpParamsMaxFormPost system property.  However if the system property is set, it overrides this cluster property.

Policy Changes

In some circumstances, it may be necessary to change one or more service policies or global policy fragments to minimize the risk of globally increasing the maximum message and attachment thresholds.?There are a few factors that are at play with respect to this behavior and what should be changed. The following items must be taken into consideration:

  1. The current value of io.xmlMaxPartBytes?
  2. The current use of the Limit Message Size assertion?
  3. Whether or not Perform WS-Security Processing is enabled for a particular service?
  4. Whether or not the service policy acts upon the request or response message?


If the service policy acts upon a request or response message that exceeds io.xmlMaxPartBytes then the request will fail. Acts upon includes but is not limited to: Any threat protection assertion, XPath evaluation, regular expression evaluation, message transformation. This limitation is overridden by using Limit Message Size assertion. The Gateway will enforce the message size on a per-service basis as long as that assertion is used. The assertion is overridden by enabling "Perform WS-Security Processing..." within the Service Properties dialog.  If this is enabled then io.xmlMaxPartBytes overrides the value of Limit Message Size assertion. This occurs because the WS-Security processing specified above occurs before the service policy itself (and before Limit Message Size would take over as the authority).?

Below are some examples to illustrate the interaction of these changes:

  1. If a service policy streams the request or response and does not act upon the message then the Gateway does not limit the size of a message. Assertions?are only necessary to shield a protected service from attack relying upon message size.
  2. If a service policy performs message transformation or evaluation then the policy should?use the Limit Message Size assertion to set the maximum size on a per-service basis.?
  3. If a Gateway will always handle large messages and inspect, evaluate, or transform the request or response--regardless of the published service being consumed--then?set io.xmlMaxPartBytes to an adequately large number.?
  4. If a service on the Gateway requires WS-Security processing then set io.xmlMaxPartBytes to an adequately large number.?
  5. If multiple services on the Gateway?require WS-Security processing then publish a "pre-security" global policy fragment and add the Limit Message Size assertion.?
Instructions:
Please Update This Required Field