How to perform Offline Installation for Logstash and APM Output Plugin for Digital Experience Collector

Document ID : KB000010276
Last Modified Date : 14/02/2018
Show Technical Document Details
Introduction:

CA APM monitors end-user business services and transactions. CA Digital Experience Collector is a data collector for the end-user monitoring. The Browser Agent posts the collected end-user monitoring data to Digital Experience Collector. The collector receives and aggregates the data based on the accumulators and sends to the Enterprise Manager.

CA Digital Experience Collector consists of three components:

  • Digital Experience Collector Server
  • Apache Kafka
  • Logstash and CA APM Output Plugin

Logstash is an open source tool that is used to ingest data from multiple sources. Logstash is a data processing pipeline that pulls data from Kafka,  transforms the data, and sends to CA APM. The CA APM Output plugin transforms the data from JSON format to APM metrics using JSON to APM Transformer. The APM Agent is the core agent that sends the transformed APM metric data to CA APM.

Background:

There may be a situation where the Digital Experience Collector needs to be installed in a restricted environment, for example, on a server with no Internet connection. In this case, the APM Output plugin installation which requires Internet access to download the latest gem package would fail with the following error:

 

[root@axatraining logstash-2.3.4 bin]$ ./logstash-plugin install DigitalExperienceCollector-Release-APM-10.5.1-DxC-1.4-SNAPSHOT-2017_01_14_13_24/logstash/apm-logstash-plugin/logstash-output-apm-Release-APM-10.5.1-DxC-1.4-20170114.212431-8.gem

Validating DigitalExperienceCollector-Release-APM-10.5.1-DxC-1.4-SNAPSHOT-2017_01_14_13_24/logstash/apm-logstash-plugin/logstash-output-apm-Release-APM-10.5.1-DxC-1.4-20170114.212431-8.gem

Unable to download data from https://rubygems.org/ - Errno::ECONNREFUSED: Connection refused - Connection refused (https://api.rubygems.org/latest_specs.4.8.gz)

ERROR: Installation aborted, verification failed for DigitalExperienceCollector-Release-APM-10.5.1-DxC-1.4-SNAPSHOT-2017_01_14_13_24/logstash/apm-logstash-plugin/logstash-output-apm-Release-APM-10.5.1-DxC-1.4-20170114.212431-8.gem

 

Environment:
CA Application Performance Management 10.5
Instructions:

To perform offline installation of the Logstash and APM Output Plugin on a machine with no Internet access, please follow the steps outlined below:

 

1. On a machine connected to the Internet, run the following command from the <dxc_logstash_home>\bin folder to install the "logstash-filter-translate" gem:

logstash-plugin install logstash-filter-translate  

 

 For example:

[root@axatraining logstash-2.3.4]# ./logstash-plugin install logstash-filter-translate
Ignoring ffi-1.9.13 because its extensions are not built. Try: gem pristine ffi --version 1.9.13
Validating logstash-filter-translate
Installing logstash-filter-translate
Installation successful


2. Pack all the plugins for offline installation with the following command:

logstash-plugin pack

 

For example:

[root@axatraining logstash-2.3.4]# ./logstash-plugin pack
Ignoring ffi-1.9.13 because its extensions are not built. Try: gem pristine ffi --version 1.9.13
Packaging plugins for offline usage

Generated at /opt/CA/DxC/logstash-2.3.4/plugins_package.tar.gz

 

3. Move the generated plugins_package.tar.gz to the machine with no Internet connection (or wherever your DxC install resides).

 

4. Unpack the file with the following command:

logstash-plugin unpack  "<path_to_the_tar_file>"

 

For example:

[root@axatraining logstash-2.3.4]# bin/logstash-plugin unpack plugins_package.tar.gz
Unpacking plugins_package.tar.gz
Unpacked at /opt/CA/DxC/logstash-2.3.4/vendor/cache

 

5. The unpacked plugins can now be installed in a local-only mode using the following command:

logstash-plugin install --local logstash-filter-translate

 

For example:

[root@axatraining logstash-2.3.4]# bin/logstash-plugin install --local logstash-filter-translate
Installing logstash-filter-translate
Installation successful

 

 6. Now you can proceed to install the logstash APM output plugin in offline mode with the following command:

logstash-plugin install --local --no-verify  "provide logstash-apm-plugin" <gem_file_ location>

 

For example:

cd /opt/CA/DxC/DigitalExperienceCollector-Release-APM-10.5.1-DxC-1.4-SNAPSHOT-2017_01_14_13_24/logstash/apm-logstash-plugin

[root@axatraining apm-logstash-plugin]# /opt/CA/DxC/logstash-2.3.4/bin/logstash-plugin install --local --no-verify logstash-output-apm-Release-APM-10.5.1-DxC-1.4-20170114.212431-8.gem
Installing logstash-output-apm
Installation successful

 

7. Verify that the plugins had been installed successfully by running the following command:

list | grep apm 

 

For example:

[root@axatraining apm-logstash-plugin]# /opt/CA/DxC/logstash-2.3.4/bin/logstash-plugin list | grep apm
logstash-output-apm

Additional Information: