Tracing C exceptions on Solaris 10 with DTrace 

 

 

The following applies to Solaris 10 systems with C runtime patches 119963-13 and up (for SPARC) and 119964-13 and up (for x86). Use "showrev -p | grep <patch #>" to check for patch revisions.

 

This is a how-to on tracing the activities of the C exception runtime on Solaris 10 with the help of DTrace. As an example, let's consider the complaint that the Orbix Locator throws an exception that violates an exception specification and crashes.

 

The question that needs to be answered is where is that exception thrown from? It turns out that the C runtime on Solaris 10 is instrumented with several DTrace probes that make it possible to find the answer.

 

1. Check to See if the user running the locator needs DTrace privileges.

Run

 

>dtrace -l  

 

as the user in question and if a ton of output (a list of DTrace probes) starts scrolling by, skip to step 2.

 

Otherwise, enable the privileges like

 

<login_name>::::defaultpriv=basic,dtrace_proc,dtrace_user,dtrace_kernel  

 

, where <login_name> is the login name of the user in question, needs to be added to the /etc/user_attr file. would require root access for this. After that, the user will need to to re-login. After logging back in, repeat the "dtrace -l" test.

 

2. Start the locator and make a note of the locator's PID
 

3. Run the following incantation:
 

>dtrace -n 'libCrun$target:::exception_thrown { @[ustack()] = count(); } END {printa(@);}' -p <locator PID> | tee locator.d.out  

 

Should see

 

dtrace: description 'libCrun$target:::exception_thrown ' matched 2 probes  

 

and the dtrace process will block for as long as the locator is running.

 

4. Run the app and reproduce the exception we considered as an example

Once the locator has crashed, the dtrace process started in step 3 will return and produce something like the following on the standard output:

 

dtrace: pid 13257 has exited  

CPU     ID                    FUNCTION:NAME  

  3      2                             :END  

             libCrun.so.1`_1cGCrunIex_throw6Fpvpkn0AQstatic_type_info_pF1_v_v 0x58            l
libit_locator_svr_sc57.so.5`_1cbDIT_LOC_RandomLoadBalancerImplMroute_client6MpnQIT_ClientRoutingNClientRequest_pn0BMReplicaGrouppn0BHReplica_ 0x49            libit_locator_svr_sc57.so.5`_1cbBIT_LOC_EndpointReplicaGroupVchoose_target_replica6MpnWIT_LOC_RequestInfoBasepnWIT_LOC_EndpointReplica_ 0xce            libit_locator_svr_sc57.so.5`_1cbBIT_LOC_EndpointReplicaGroupOhandle_request6MrnKIT_AutoPtr4nWIT_LOC_RequestInfoBase__v 0x4a5              libit_locator_svr_sc57.so.5`_1cbBIT_LOC_EndpointReplicaGroupRRequestDispatcherHexecute6M_nMIT_WorkQdDueueOWorkItemStatus_ 0x7e 

              libit_art_sc57.so.5`_1cUIT_Work_WorkerThreadDrun6M_pv 0x111 

              libit_ifc_sc57.so.5`IT_ThreadFactory_call_thread_body 0x11 

              libc.so.1`_thr_setup 0x4e 

              libc.so.1`_lwp_start  

                1 

 

Since this output is saved in the locator.d.out file, filter it through c filt and see where the offending exception came from:

 

>cat locator.d.out | c filt  

CPU     ID                    FUNCTION:NAME  

  3      2                             :END  

 

             libCrun.so.1`void __Crun::ex_throw(void*,const __Crun::static_type_info*,void(*)(void*)) 0x58               libit_locator_svr_sc57.so.5`IT_ClientRouting::Replica*IT_LOC_RandomLoadBalancerImpl::route_client(IT_ClientRouting::ClientRequest*,IT_ClientRouting::ReplicaGroup*) 0x49             libit_locator_svr_sc57.so.5`IT_LOC_EndpointReplica*IT_LOC_EndpointReplicaGroup::choose_target_replica(IT_LOC_RequestInfoBase*) 0xce 

              libit_locator_svr_sc57.so.5`void IT_LOC_EndpointReplicaGroup::handle_request(IT_AutoPtr<IT_LOC_RequestInfoBase>&) 0x4a5 

              libit_locator_svr_sc57.so.5`IT_WorkQueue::WorkItemStatus IT_LOC_EndpointReplicaGroup::RequestDispatcher::execute() 0x7e 

              libit_art_sc57.so.5`void*IT_Work_WorkerThread::run() 0x111 

              libit_ifc_sc57.so.5`IT_ThreadFactory_call_thread_body 0x11 

              libc.so.1`_thr_setup 0x4e 

              libc.so.1`_lwp_start  

                1