SpectroSERVER crashing due to conflicting HashNodes while loading the Evpn and Vpn modules.

Document ID : KB000007406
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

The SpectroSERVER crashed while loading the Enterprise VPN Module.

We are seeing a lot of the following in the VNM.OUT:

 mts free: heap -1 unallocated address 0x7f70c1d02c5d on page 0x7f70245c8000 (size >= 60K objects)

The stack from the generate core files contained:

/lib64/libc.so.6[0x3778e75dee]
/lib64/libc.so.6[0x3778e78c3d]
/opt/SPECTRUM/lib/../SS/libmdlsvint.so.1(_ZN18GlobalCollectionIH16reconcile_searchERK13CsModelHandleP15CsModelDescList+0x578)

[0x7fd3595a64e8]
/opt/SPECTRUM/lib/../SS/libmdlsvint.so.1(_ZN18GlobalCollectionIH10run_searchERK13CsModelHandleNS_17SearchInitiator_eE+0x138)

[0x7fd3595a74b8]
/opt/SPECTRUM/lib/../SS/libsskrnl.so.1(_ZN12CsIHTimerMgr16do_periodic_workEPK18CsPeriodicWorkNode+0x574)[0x7fd35b449cc4]
/opt/SPECTRUM/lib/libwkmgr.so.1(_ZN23CsPeriodicWorkScheduler17process_work_nodeEP10CsWorkNode+0x81)[0x7fd3568f4651]
/opt/SPECTRUM/lib/libwkmgr.so.1(_ZN15CsWorkScheduler7do_workEv+0xda)[0x7fd3568f53aa]
/opt/SPECTRUM/lib/libmoot.so.1(moot_thread_start+0x23)[0x7fd35b134843]
/lib64/libc.so.6[0x3778e43760]

Cause:

The instability was caused by duplicate definitions of HashNode in two different generics - Evpn, and Vpn. These two generics don't reference or link with each other, so one might think this isn't an issue. 

However, both these modules get loaded into the same process, and these are 'weak' symbols, and quietly collide. Sustaining Engineering's guess is that generally, by luck, the larger object (one is 12 bytes, while the other one is 8 bytes) wins, and this also was really a bug exposed by the x64 project as now pointers are bigger.

Resolution:

Spectrum_10.2.0.D230 changes the names of HashNode in each generic, to be unique, and also takes an additional step of changing how we load Modules so that this isn't a problem in the future. 

 

 

This will be included in Spectrum 10.2.2 and Spectrum 10.3.