Recently I had to get involved in a particularly challenging problem that started happening to a process that has been rock-solid for many years.
Context:
UV 11.2.5 on
RHEL 7We have a process that performs numerous transaction-bracketed updates.
It is capable of being run as a phantom.
Under the right conditions the phantom process would just stop.
We have a COMO recording active while the phantom process is running, but it captured nothing of any value.
The last line in the COMO was always
Message[001467]
For those that are interested, this is an item in the SYS.MESSAGE file
>CT SYS.MESSAGE 001467
001467
0001 Spooler Entry #%i
0002
>
The program that is active at the time of the abort is performing BASIC WRITE to a file whilst an TRANSACTION is active.
After using
strace
to monitor the phantom pid I came to understand that when the phantom would die it was after updating a
cacheXXXXX
file in the UVTEMP directory.
These are dynamic files, and are used as disk overflow of the in-memory transaction cache.
Digging around for information on these files lead me to the release notes for UV 11.1.4, which included this note:
"UNV-5474 – Under a specific set of conditions, UniVerse might core dump. The conditions involved updating a file which contained a virtual field index, transactions were being used (i.e. this would be true if using triggers), the size of the transaction exceeded the in memory transaction cache buffer as defined by the uvconfig parameter TXMEM, and the virtual field index returns the null (i.e. empty string) as the result. The workaround at this release is to increase TXMEM so the transaction remains in memory. This issue has been corrected in the 11.1.5 release of UniVerse."
The file being written to did have a new index added to it recently. It is a virtual field index (it uses the SUBR() function to call a subroutine, which in turn is locking and updating other records). The virtual field returns null, and the index is created with the NO.NULLS option.
We had not changed the TXMEM setting of UniVerse, so it was still set to 32.
So the issue UNV-5474 is still alive in UV 11.2.5.
For us it was triggered in our code when we added a new index that calls a subroutine that added extra file io to the transaction cache.
It seems that when UV 11.2.5 has to use the disk cache during the transaction it can trigger a corruption that aborts the process.
The fix is to adjust the TXMEM setting. We have adjusted this up to 256 and the update process is no longer aborting.
#txmem #UNV-5474 #UniVerse #11.2.5 #rhel
------------------------------
Gregor Scott
gregors
------------------------------