Hi All,

I am facing a issue with Artix/C web service. We have one web service module that is written in C .

Module simply receive a request and send to some other interface. and receive the response and send back to corresponding request.

When we are trying one request at a time, it is working fine.. we can say that it is working fine with sequential requests. 

But when we try multiple request simultaneously then core is coming..  the strange thing is that core is coming on differnet place every time and pointing to some artix like things when we try debug using dbx.

please see log below for multiple core debug using dbx tool.

x: warning: cannot demangle '__1c9uX__unnamed_9QdDQdDQdDKAdNHJ0IXIErrorMap2F6Mi_rnGIT_BusOFaultException__'

Reading libit_xslt_util_sc53.so.5

Reading libit_xalan_sc53.so.1

Reading libit_http_api_sc53.so.5

Reading libCstd.so.1

Reading libgen.so.1

Reading libit_xalanmsg_sc53.so.1

Reading libCstd_isa.so.1

Reading libit_http_sc53.so.5

Reading libit_messaging_port_sc53.so.5

Reading libit_giop_snoop_sc53.so.5

Reading libit_soap_sc53.so.5

dbx: warning: cannot demangle '__1c9uX__unnamed_9QdDQdDQdDKn1zOJ06kSwrite_header_parts6FpknGIT_BusRWSDLBindingFacade_pkn0BbAWSDLBindingOperationFacade_rn0BPWritableMessage_rn0BQSoapHeaderWriter_bb_v_'

dbx: warning: cannot demangle '__1c9uX__unnamed_9QdDQdDQdDKn1zOJ06kbCget_security_must_understand6FpnJIT_ConfigNConfiguration__b_'

dbx: warning: cannot demangle '__1c9uX__unnamed_9QdDQdDQdDKn1zOJ06kbRget_security_client_inbound_must_understand6FpnJIT_ConfigNConfiguration__b_'

Reading libit_soap_common_sc53.so.5

Reading libit_coordinator_stub_sc53.so.5

Reading libit_wsat_protocol_stub_sc53.so.5

detected a multithreaded program

t@17 (l@9) terminated by signal BUS (invalid address alignment)

Current function is CORBA::string_dup

  118       return CORBA_string_dup(str);

(/opt/SUNWspro/bin/../WS6U1/bin/sparcv9/dbx) file

basic_types.h

(/opt/SUNWspro/bin/../WS6U1/bin/sparcv9/dbx) list

  118       return CORBA_string_dup(str);

  119   }

  120

  121   IT_NAMESPACE_STATIC_INLINE void

  122   string_free(

  123       char* str

  124   )

  125   {

  126       CORBA_string_free(str);

  127   }

(/opt/SUNWspro/bin/../WS6U1/bin/sparcv9/dbx) where

current thread: t@17

  [1] _smalloc(0x8, 0xfea427a8, 0x0, 0x8, 0x0, 0x0), at 0xfe9c1c68

  [2] malloc(0x1, 0xe68056, 0x0, 0x0, 0x0, 0x1597390), at 0xfe9c1ca8

  [3] CORBA_string_dup(0x1, 0x1558350, 0x38d080, 0xb01600, 0x0, 0xe68055), at 0xfdbca5c8

=>[4] CORBA::string_dup(str = 0xe68055 ""), line 118 in "basic_types.h"

  [5] los_global_namespace::po_vitCsImpl::sendCustomerServiceRecord(this = 0xffbeb758, WSHeader = CLASS, constCustomerServiceRecordRequest = CLASS, WSResponseHeader = CLASS, CustomerServiceRecordResponse = CLASS), line 4158 in "po_vitCsImpl.cpp"

  los_global_namespace::PreOrderServer::smf_dispatch_sendCustomerServiceRecord(0xfa30d2f0, 0xfa30d2f0, 0xfa30d314, 0xfa30d314, 0xfa30d2f0, 0xfa30d2f0), at 0xa8ce14

  [7] los_global_namespace::PreOrderServer::dispatch(0xffbeb758, 0x13b82e8, 0xf4ca74, 0x0, 0x274708, 0xfe665710), at 0xa8eee4

  IT_Bus::ServantWrapper::dispatch(0x125f730, 0xfa30d2cc, 0xfedf460c, 0xfa30cf68, 0xfa30cf64, 0xff085b78), at 0xfedd21d0

  [9] IT_Bus::SoapInterceptor::intercept_around_dispatch(0x1295438, 0xfa30d2cc, 0xf7531028, 0x143fbc0, 0xf6b67ef8, 0xf757bcc4), at 0xf6b65614

  [10] IT_Bus::ServerEndpointHandlerImpl::message_received(0x12c8380, 0x1493458, 0x13b82e4, 0x1466d48, 0xf757ca50, 0xfa30d374), at 0xf753e6d0

  [11] IT_Bus::ServerEndpointImpl::message_received(0x125ecf0, 0xfa30d960, 0x143fb70, 0xf757ca50, 0xf753e28c, 0xf757e1b4), at 0xf7539154

  [12] IT_Bus::DispatchContextImpl::execute(0x143fb70, 0xfa30d960, 0xf7578d70, 0xf7564a88, 0xfa30d82c, 0x125ecf0), at 0xf754034c

  [13] IT_Bus::MessagingTransportCallbackImpl::dispatch(0x13a6110, 0xfa30d960, 0x143fb70, 0x143fb70, 0xfb6538, 0x1), at 0xf75450d4

  [14] IT_HTTP::ArtixHTTPService::do_post(0x13a64e8, 0xf6bd2a20, 0xf8f94134, 0xfa30d960, 0x13a653c, 0xfb6538), at 0xf8f4524c

  [15] IT_HTTP::ArtixHTTPService::service_request(0x13a64e8, 0x142ff88, 0x14304a8, 0x3, 0xf8d8d8a4, 0xf8f495e4), at 0xf8f49608

also we tried multiple approach to resolve but still core is coming at uncertain place every time.

we tried following approach - 

1- tried to see if there is any glocal variable and that is getting corrupt for multiple request.

2- most important thing is that we put in mutex block to make sure that only one thread is using at a time. bbut till we were getting core for multiple request.

3- we put extensive logging in code but there is no pointer provided by dbx or logging.

4- sometime we are getting core in 10 simultaneous transaction and sometime we are getting in 15 simultaneous transaction.

can anyone please point or suggest anything for this situation?