Problem
- Product: VisiBroker
- Product Version: 7.0 & later
Server application crashes while shutting down.
Running pflags command on the generated core indicates the following to be the offending thread:
/6: flags = DETACH
sigmask = 0xffffbefc,0x0000ffff cursig = SIGSEGV
Running pstack command on the generated core shows the following stack trace for thread #6:
----------------- lwp# 6 / thread# 6 --------------------
fec75b73 __1cYVISGIOPProtocolConnectorJreconnect6MX_v_ (824bfa8, 0, 0) 33
fec61fa7 __1cNVISIIOPBidderDBidMgetConnector6MX_pnOProtocolEngineRProtocolConnector__ (824fc08, 0, 0) 19f
fec631bb __1cMVISPortfolioMgetConnector6MX_pnOProtocolEngineRProtocolConnector__ (fcd51040, 0, 0) 20f
fec653ab __1cSVISProtocolManagerMgetConnector6MpnMCORBA_Object_pnDIOPIIORValue_X_pnOProtocolEngineRProtocolConnector__ (8235348, 824e418, 824fa20, 0, 0) 9b
fec6ad66 __1cLVISDelegateF_bind6MpnMCORBA_Object_CC_v_ (824f168, 824e418, 0, 0) 6ae
fec6ee33 __1cLVISDelegateS_verify_connection6MpnMCORBA_Object_C_v_ (824f168, 824e418, 1) 713
fec6b7a3 __1cLVISDelegateHrequest6MpnMCORBA_Object_pkcC_pnWCORBA_MarshalOutBuffer__ (824f168, 824e418, fedf7576, 1) 43
fec6fdb3 __1cLVISDelegateEis_a6MpnMCORBA_Object_pkc_C_ (824f168, 824e418, 81bb250) 97
fed6c77b __1cMCORBA_ObjectM_safe_narrow6kMrknOCORBA_TypeInfo__pv_ (824e418, fe993ef8) 4f
fe96eec4 __1cJCosNamingQNamingContextExtH_narrow6FpnMCORBA_Object__p1_ (824e418) 28
fe7412d8 __1cbETSS_Auth_Cache_get_auth_server6Frpc_pnDTSSJAuthority__ (fcd51dec) 558
fe741b07 __1cOTSS_Auth_CacheKIs_changed6Frpc_i_ (fcd51fa8) a7
fe853638 __1cWEAC_Auth_Cache_SessionDRun6M_v_ (82338f8) f8
fe847058 __1cTEAC_Session_ManagerMThread_start6Fpv_1_ (82338f8) 78
fda08ffb _thr_setup (fd052200) 4e
fda09300 _lwp_start (fd052200, 0, 0, fcd51ff8, fda09300, fd052200)
The following thread #1 actually shows that the application was exiting and the ORB was in the process of shutting down.
----------------- lwp# 1 / thread# 1 --------------------
fda0c545 _read (b, 8018e24, 4) 15
fe9c3156 __1cHVISPipeQread_from_writer6MpcL_L_ (823c810, 8018e24, 4) 26
fd1c9812 __1cGDSUserGresume6M_v_ (823c758) be
fd1c74fb __1cGDSUserFclose6M_v_ (823c758) 437
fd1c669f __1cGDSUserIshutdown6F_v_ (fd1fd174, 0, 8018f38, fe070c30, 8018f38, fd1dedfa) 12f
fd1c6754 __1cGDSUserHdestroy6F_v_ (824feb8, 35, fee589f0, fee589f0, 8018f38, fec49bdf) 18
fd1dedfa __1cPVISLocationImplMORB_shutdown6M_v_ (8246780) 62
fec13017 __1cOVISInitManagerIshutdown6M_v_ (8193700) 15b
fec88d49 __1cKVISManagerR_complete_cleanup6M_v_ (821fcd0) 135
fec88afe __1cKVISManagerHcleanup6MCC_v_ (821fcd0, 1, 0) 2ee
fec8878d __1cKVISManagerPatexit_callback6F_v_ (fd233fc0, fda75f18, fda70000, fda75c80, 80191bc, feef4178) 3d
fec45b3a __1cHVISUtilIshutdown6F_v_ (801a6e0, 20000, fda70000, fda75f00, 8019edc, fd9856e2) ee
fd99313d _exithandle (81338a8, 8073a63, 0, 801a6e0, 8019ec8, feffa910) 4e
fd9856e2 exit (3, 8019f0c, 8019f1c) 12
0807096a _start (3, 801a750, 801a77e, 801a781, 0, 801a790) 7a
Resolution
The crash is caused due to a request in progress while a non-graceful shutdown was invoked.
This crash normally occurs when an application is acting as both a server and client at the same time. A server thread within the application has invoked ORB::shutdown() but a client thread within the same application is invoking a CORBA method on another server at the same time without knowing that the ORB used is already in the process of shutting down itself. Since the ORB has no knowledge of client-side processing, it has started deleting and cleaning up. This results in a core dump when the client thread is about to send the request.
To avoid the core dump, the application should co-ordinate between the server and client threads using condition variables and mutexes. The client thread should check if the ORB is shutting down before invoking any CORBA requests. Likewise, the server thread should also not issue ORB::shutdown() when any client thread is in the process of sending a CORBA request or waiting for reply.
#VisiBroker
#VISDelegateS_verify_connection
#Security
#VISProtocolManager--getConnector
