SpectroSERVER keeps crashing with librtms module in the stack trace due to IP SLA Test model

Document ID : KB000048548
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

After a Read-Only discovery of pre-configured IP SLA Test on a Cisco router, SpectroSERVER keeps crashing. The $SPECROOT/SS/VNM.OUT file shows the following when SS crashed.

terminate called after throwing an instance of 'std::out_of_range'
what(): basic_string::substr

Stack trace of generated core dump or dmp file is consistently showing stacks similar to the following:


 	(gdb) where 	
 	#0 0xffffe405 in __kernel_vsyscall () 	
 	#1 0x0076adf0 in raise () from /lib/libc.so.6 	
 	#2 0x0076c701 in abort () from /lib/libc.so.6 	
 	#3 0xf5f63b50 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/libstdc++.so.6 	
 	#4 0xf5f61555 in ?? () from /usr/lib/libstdc++.so.6 	
 	#5 0xf5f61592 in std::terminate() () from /usr/lib/libstdc++.so.6 	
 	#6 0xf5f616ca in __cxa_throw () from /usr/lib/libstdc++.so.6 	
 	#7 0xf5ef80df in std::__throw_out_of_range(char const*) () from /usr/lib/libstdc++.so.6 	
 	#8 0xf0cc3f69 in RTM_Address::extractIP(RTMDomain::TestType_e) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#9 0xf0d29a7d in RTM_TestParameters::getIPAddress(RTMDomain::RTM_EventTarget_e) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#10 0xf0d29af2 in RTM_TestParameters::getAddressMH(RTMDomain::RTM_EventTarget_e) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#11 0xf0d22a6b in RTM_Test::correlateTestAlarms(int) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#12 0xf0cfd8df in RTM_If::resultEventToTestModel(RTM_Test*, RTMDomain::TestType_e const&, int, long, CsAttrValList*)  
 	 	() from appl/spectrum/SS/Modules/librtms.so.1 	
 	#13 0xf0cfdc06 in RTM_If::getResultsFromDevice(CsModelHandle const&, CsModelHandle const&, CsObjectID const&, CsObjectID const&, 	
 	 	RTMDomain::TestType_e const&, unsigned short, unsigned short*) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#14 0xf0d19ef5 in RTM_Test_Host::getResults(CsModelHandle const&, CsObjectID const&, CsObjectID const&, RTMDomain::TestType_e, unsigned short, 	
 	 	unsigned short*) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#15 0xf0d2491e in RTM_Test::runTest() () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#16 0xf0d13da7 in RTM_Scheduler::processWorkNode(RTM_WorkNode*) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#17 0xf0d13f97 in RTM_Scheduler::do_periodic_work(CsPeriodicWorkNode const*) () from /appl/spectrum/SS/Modules/librtms.so.1 	
 	#18 0xf6d83392 in CsPeriodicWorkScheduler::process_work_node(CsWorkNode*) () from /opt/SPECTRUM/lib/libwkmgr.so.1 	
 	#19 0xf6d843ed in CsWorkScheduler::do_work() () from /opt/SPECTRUM/lib/libwkmgr.so.1 	
 	#20 0xf7d8edfb in Thread::call_thread_function() () from /opt/SPECTRUM/lib/libmoot.so.1 	
 	#21 0xf7d905a7 in thread_start () from /opt/SPECTRUM/lib/libmoot.so.1 	
 	#22 0x0077ab7b in makecontext () from /lib/libc.so.6 	
 	#23 0x0e6b3b60 in ?? () 	
   #24 0x00000000 in ?? ()

Solution:

This issue has been identified as a known bug through work item 217676.

The permanent fix will be included in hotfix patch H12 for Spectrum 9.2.3.

There are available debug patches for Spectrum 9.2.3 (H10) and Spectrum 9.2.3 H11, they are Spectrum_09.02.03.D597 and Spectrum_09.02.03.D625 respectively. Please contact Spectrum Technical Support if you wish to apply the debug patch instead of waiting for hotfix patch H12.

The same solution will be included in hotfix patch H01 for the 9.3.0 release of Spectrum through work item 220866.