QOS_Definition from PVS causes data_engine queue to stop processing

Document ID : KB000047855
Last Modified Date : 14/02/2018
Show Technical Document Details
Symptoms:

1. Data engine queuing, queue buildup alarm possible
2a. "query failed: call spn_de_CreateQosDefinition" in the data_engine log
? OR?
2b. "Data too long for column 'pQosDescription' at row" in data_engine log
? OR
2c. (errno:1406) in data_engine logs


Resolution:

Steps are two-fold:

1. To relieve the queue, use Dr. Nimbus to pop two messages off the data_engine queue: the QOS_DEFINITION in question, and likely the next QOS_MESSAGE.

See article,?https://na4.salesforce.com/kAH6000000000px?srPos=1&srKp=kaH for similar screenshots

2. Look at the message to find the probe sending the invalid data. (RCA) In my case there was a static monitor in the PVS probe for QOS_PVSDISK_WRITECACHE_TYPE where the description from the device was too long. We disabled the static monitor.



Data_engine Log Snapshot:
Apr ?9 11:55:53:790 [140293580179200] de: qos_check - subscriber attached to queue: OK (bulk size=999) (queue: data_engine)
Apr ?9 11:55:53:811 [140293580179200] de: HandleMessage: Insert Data - nimid: IV92947679-00112 ? ? (2015-04-08 21:34:39) table_id: ? ? 169522 value= ? ? ? ? 0.00 time: 2015-04-08 21:34:38?? ??? ? qos=QOS_PVSDISK_ACTIVE ? source=EDC::Reverse_Image_Locations target=NOSERVER::TWD_XA65_W2K8_12_002 ??
Apr ?9 11:55:53:813 [140293580179200] de: [QoSData] thread_id: 158211, query failed: call spn_de_CreateQosDefinition ('QOS_PVSDISK_WRITECACHE_TYPE','QOS_MACHINE','0 (Private), (other values are standard image) 1 (Cache on Server), 2 (Cache on Server Encrypted), 3 (Cache in Device RAM), 4 (Cache on Device Hard Drive), 5 (Cache on Device Hard Drive Encrypted), 6 (Device RAM Disk), or 7 (Cache on Server Persistent). Min=0, Max=8, Default=0','Integer','int',0,0,0,@pQosDefId,0,@pErrorMessage,@pReturnCode);, took 1 ms
Apr ?9 11:55:53:813 [140293580179200] de: data_engine [QoS] [QoSData] - Execute1(query) - Error: [QoSData] thread_id: 158211, error: Execute1(query): Data too long for column 'pQosDescription' at row 206332 (errno:1406)
Apr ?9 11:55:53:813 [140293580179200] de: InsertNewDefinition - failed for QOS_PVSDISK_WRITECACHE_TYPE (0 (Private), (other values are standard image) 1 (Cache on Server), 2 (Cache on Server Encrypted), 3 (Cache in Device RAM), 4 (Cache on Device Hard Drive), 5 (Cache on Device Hard Drive Encrypted), 6 (Device RAM Disk), or 7 (Cache on Server Persistent). Min=0, Max=8, Default=0)
Apr ?9 11:55:53:813 [140293580179200] de: data_engine [QoS] [QoSData] - InsertNewDefinition - Error: [QoSData] thread_id: 158211, error: InsertNewDefinition: Data too long for column 'pQosDescription' at row 206332 (errno:1406)
Apr ?9 11:55:53:813 [140293580179200] de: InsertDefinition - QOS_PVSDISK_WRITECACHE_TYPE failed to create definition and tables, rc: error (%rc)
Apr ?9 11:55:53:813 [140293580179200] de: DataCallback - HandleData returned 1 (error)
Apr ?9 11:55:53:813 [140293580179200] de: DataCallback - processed 2 messages, sending reply: error (1)
Apr ?9 11:55:53:906 [140293496108800] de: [LSV] thread_id: 158209, commit, exec bulk query...
Apr ?9 11:55:53:906 [140293496108800] de: infile_init_bigbuffer(data), total: 5001, buffer size: 37
Apr ?9 11:55:53:906 [140293496108800] de: infile_read_bigbuffer, bufsize: 8192, tot: 37, read: 37, left: 0
Apr ?9 11:55:53:906 [140293496108800] de: infile_end_bigbuffer
Apr ?9 11:55:53:929 [140293580179200] de: HandleMessage: Insert Data - nimid: IV92947679-00112 ? ? (2015-04-08 21:34:39) table_id: ? ? 169522 value= ? ? ? ? 0.00 time: 2015-04-08 21:34:38?? ??? ? qos=QOS_PVSDISK_ACTIVE ? source=EDC::Reverse_Image_Locations target=NOSERVER::TWD_XA65_W2K8_12_002 ??
Apr ?9 11:55:53:930 [140293580179200] de: [QoSData] thread_id: 158211, query failed: call spn_de_CreateQosDefinition ('QOS_PVSDISK_WRITECACHE_TYPE','QOS_MACHINE','0 (Private), (other values are standard image) 1 (Cache on Server), 2 (Cache on Server Encrypted), 3 (Cache in Device RAM), 4 (Cache on Device Hard Drive), 5 (Cache on Device Hard Drive Encrypted), 6 (Device RAM Disk), or 7 (Cache on Server Persistent). Min=0, Max=8, Default=0','Integer','int',0,0,0,@pQosDefId,0,@pErrorMessage,@pReturnCode);, took 0 ms


keywords: pvs, qos definition, queue, queuing, backed up, alarm queue size exceeds, slm, drnimbus, dr.nimbus, doctor nimbus, sniffer, description, limit, citrix