testing for scenarios that cause "Fwrite failed at DBwrite"

Brian,

• Exceeding the maximum file size as determined by ulimit or 32Gb if a 32-bit file - whichever is first encountered.
• Permissions also include ACLs from UniVerse 12 onward, though ACL support could appear in UniVerse 11.x at some point.
• File system type - I am aware that EMC (now Dell) OneFS supports 64-bit block addressing for example that is an issue, though it can be resolved by using a 'legacy mode'.
• Insufficient space on device.

I am sure there are many more, but if you have a specific error you  can reasonably reproduce then you can use truss, strace, tusc  (all the same command really)  to track a process to the specific point of error. Reference materials and examples are readily available in syntax and usage, and I suggest with trapping error conditions specifically on file operations, noting that not all failures are actual errors as a common way to check for a file's existence is to try and open it.

Regards

JJ

Thank you John for the response. I've witnessed a scenario where a basic program (process A) is writing records to a type 18 file, and a separate process (process B) is deleting records from the same file at the same time. Process B is using a list of keys it gets from an alternate source. Some type of collision occurred, causing process A to abort with "Fwrite failed at DBwrite". Unable to reproduce the behavior, and without insight into how the underlying OS api code works, I am left to theorize about how this could happen. I assume that in order to service the optional ON ERROR and ELSE clauses for the WRITE statement, an immediate read occurs just after Fwrite to confirm the commit before passing back a result to the interpreter. My theory is that the delete from process B was able to remove the record just after the Fwrite but before the confirmation read at a low level. I have no idea whether this is plausible or even possible. I am also theorizing that the native low-level implementation of the DELETE statement respects a bit / flag managed by the WRITE apparatus with the intention of preventing such a thing. The kicker is that process B is a .NET service using the toolkit to perform the delete via UniRPC. This brings us to the final part of my theory which is that the implementation of the delete function via UniRPC is different code that does not respect the commit completion bit. Is there anybody that can shed any light on this from a Rocket product engineering standpoint? What could possibly be going on here?

Brian,

Were the 2 programs using record locking so the same record id was not being processed for writing and deleting at the same time? Also, the example indicates that process B which is deleting the records is the program which aborted. Did it abort with the "Fwrite failed at DBwrite" message? If so, was there an indication of the program line number and did it map to the delete operation? Curious as to the error message if the action was a delete rather than a write. 

Neil

Hi Neil. No record locking involved. My belief is that the remote routine issuing the delete via UniRPC (process B) did so on the same key process A was writing. To clarify, process A aborted with "Fwrite failed at DBwrite".

Thanks Brian. Without any record locking, I suspect that you are correct that a concurrency/timing issue occurred with both processes attempting the write and delete at the same time. With that said, I'm thinking a Group "WR" lock on the group set by either one of the processes would have given that process exclusive access.  Is there a reason record locking was not in play? Performance of the batch job?

Thanks,

Neil

Thank you for the input. In this case, I think introducing locks in an attempt to ensure the record is written completely before the delete can occur doesn't get at the root of the issue and it will certainly introduce too much latency. It is a very obscure issue - it does recur in a production environment in the context of normal / routine activity. However, recreating the issue directly with isolated test processes is unsuccessful. If my suspicion is correct, the cure will be for the .NET application to call a UV Basic subroutine that performs the delete (natively) instead of the Delete method issued directly via UniRPC. I'd really love it if someone who has access to the underlying code can explain if/how the CRUD operations via UniRPC might differ from what occurs at a low level via a basic program running as a phantom or in an interactive session. This might make conjuring a solution less of a blind experiment. Thanks again fo the discussion.

Thanks Brian. The suggestion of using record locking was just a best practice suggestion to ensure multiple processes were not attempting "update" operations on the same record at the same time. Also, I'm not at all familiar with the .NET functionality and how it might differ from a standard BASIC program. Hopefully someone else can comment on that behavior and shed some light on the theory that the failure is caused by a WRITE and DELETE operation happening simultaneously on the same record. 

Out of curiousity, I have a question related to this topic. With the possibility that the two independent processes can be updating the same record simultaneously, which operation is expected to happen first? If the WRITE happens first, then the record will be gone after the DELETE. But if the DELETE happens first, then the record will still exist. 

Thanks,

Neil