Problem:

  • Product Name: VisiBroker C
  • Product Version: 8.x
  • Component: VisiNotify
  • Platform: AIX5.3

Considering a simple deployment diagram, the consumer will have an IOR with both LIOP and IIOP on UNIX environment.



On the first establishment between VisiNotify and consumer, the connection bids and uses LIOP (using UNIX's local IPC) since both processes are in the same host machine. When the consumer is uexpectedly terminated and restarted, the VisiNotify will re-bid and re-establish on LIOP then on IIOP. However, if the consumer's IOR is cached on VisiNotify (by default, it is cached), it has an LIOP profile of the previously terminated consumer process.

The VisiNotify will not be successful in its bid to LIOP and will attempt to use IIOP. Then onwards, VisiNotify and consumer are connected via IIOP. All GIOP packets will be sent through the network stack. If there are too many packets sent over the network stack and machine configuration couldn't handle the traffic, then Nagle's algorithm on TCP-no-delay will be in effect. You will observe that the rate of notification from VisiNotify to consumer will be lowered after the consumer was restarted.

Resolution:

Below is sample VisiNotify log snippet to know if the LIOP failed and the IIOP succeeded:
 
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 861457ms Tid# 4370 Src# v_vnpxsup Msg# 4105:resuming
  ::
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876135ms Tid# 4884 Src# connection Msg# Submitted an IIOP bid
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876203ms Tid# 4884 Src# connection Msg# Submitted an LIOP bid
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876231ms Tid# 4884 Src# connection Msg# Creating a protocl connector for LIOP
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876300ms Tid# 4884 Src# connection Msg# Waiting for connection lock
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876326ms Tid# 4884 Src# connection Msg# Got the  connection lock
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876353ms Tid# 4884 Src# connection Msg# Creating a  LIOP connection
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876906ms Tid# 4884 Src# connection Msg# Destroying the protocol connector
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876957ms Tid# 4884 Src# client Msg# VISPortfolio::getConnector: caught TRANSIENT, probably stale IOR, so throwing as OBJECT_NOT_EXIST
  ::
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 876992ms Tid# 4884 Src# connection Msg# Creating a new protocol connector
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 877025ms Tid# 3856 Src# connection Msg# Read 12 bytes, 0 bytes remaining. fd=27
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 877046ms Tid# 4884 Src# connection Msg# Waiting for connection lock
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 877077ms Tid# 4884 Src# connection Msg# Got the  connection lock
Pid# 446520 Tim# Thu Jul  1 07:54:10 2010 877103ms Tid# 4884 Src# connection Msg# Creating a new TCP connection
 
One of the ways to improve the performance is to set to TRUE the vbroker.ce.iiop.connection.tcpNoDelay on VisiNotify and supplier processes. This article focuses on AIX5.3 machine. However, if you notice that other platforms may also benefit from this, you may do so.

Another option is to ensure that the LIOP is not cached and is always updated everytime the VisiNotify re-binds to the consumer, by using vbroker.orb.cacheDSQuery=false. However, this property is only applicable when using osagent.
 
Incident #2457116 / #2457119