Path Length Allowed in UD VOC

We had an unusual issue where it APPEARS Attb 2 of a DIR VOC type cannot exceed a certain length.

LIST / SELECT both work as expected.

However, AE using an Active SELECT list or just 'raw' would return "New Item" and anything Filed would issue a WRITE error (Status =1) with a 'check for triggers' message -- which cannot be true on UD DIR types.

After a while of not figuring out the issue, we moved the folder to make Attb2 of the VOC shorter and viola - the issue resolved.

Does anyone know the character count allowed in attribute 2 of a DIR VOC type?  And more weirdly, why is only AE was impacted?  We might have found the issue earlier if SELECT and LIST did not work either -- as it was, we were running down to see if there were some weird permissions issues - and was about 30 minutes in when we realized the path was much longer than usual do to where the folder had been nested.

We had an unusual issue where it APPEARS Attb 2 of a DIR VOC type cannot exceed a certain length.

LIST / SELECT both work as expected.

However, AE using an Active SELECT list or just 'raw' would return "New Item" and anything Filed would issue a WRITE error (Status =1) with a 'check for triggers' message -- which cannot be true on UD DIR types.

After a while of not figuring out the issue, we moved the folder to make Attb2 of the VOC shorter and viola - the issue resolved.

Does anyone know the character count allowed in attribute 2 of a DIR VOC type?  And more weirdly, why is only AE was impacted?  We might have found the issue earlier if SELECT and LIST did not work either -- as it was, we were running down to see if there were some weird permissions issues - and was about 30 minutes in when we realized the path was much longer than usual do to where the folder had been nested.

I did think about U_MAXFNAME in limits however I'm pretty sure this covers something else, so I tested it anyway and I could have a directory path greater than U_MAXFNAME  without a problem.

How long was the PATH that was being used as I suspect a READ in UniBasic will show the same issue, as AE is written in UniBasic. I would like to explore this deeper in case it is a UniData bug.

Thanks,

The Path we had was 131 characters long (lots and lots of nested folders with version specific names!)

We shorted it to 101 characters and it worked fine.

And yes -- we saw the AE would not read (showed 'new item' on items that existed) or write (AE filename HELLO .. and attempt FI would fail.)

We are on UD. 8.2.4 Build 3001

Psst, Hey David. Any chance the editor is running PRC programs first? There are subroutines in PRC that do their own path resolution. Could the problem be in there?

It fails running OS.AE and in folders that are not monitored by PRC - so you're off the hook on this one!  :-)

Whew!  I'm interested to learn what this turns out to be.

@Susan Joslyn @David Wolverton 

As Susan said, she would be interested to know the answer, well ... what's below will make for interesting reading, although no definitive answer yet and  I  found a secondary problem that's worse while testing this and I have a strong suspension of what AE is doing (just need some free cycles to review the AE source).

I did some testing on UniData 8.2.4 and running on Windows, my idea was to reproduce your problem and then run then test through the debugger here to see what was happening. I started with a directory path length of 126. Why 126 it happens to be the maximum length of a record id and I wanted to eliminate a thought I had with what AE was doing in the background before running some debugger testing. What I suspect it is now doing is even stranger.

My initial pointer was.
001: DIR
002: C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\
SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9
003: D_BP

This worked fine
AE LONGDIRTEST JDS
Top of New "JDS" in "LONGDIRTEST".
*--: I
001= NEW RECORD
*--: FI
Filed "JDS" in file "LONGDIRTEST".

I checked the file afterwards with a LIST and doing a DIR at the command prompt. Everything was fine. I then deleted the JDS record and made sure DIR and LIST were clear.

My next step was to increase the length past 126 so made the final directory SOFTVERSION900 and I can reproduce the exact problem seen with AE failing and LIST working etc.

AE LONGDIRTEST JDS
Top of New "JDS" in "LONGDIRTEST".
*--: I
001= NEW RECORD
*--: FI
In C:\\u2\\ud\\sys\\CTLG\\a\\AE_AE at line 2214 WRITE item JDS to file LONGDIRTEST err
or.
[AE] UniBasic WRITE failed, STATUS=1, check triggers.
Quit "JDS" in file "LONGDIRTEST" not created.

I then decided to do one more test and changed SOFTVERSION900 back to SOFTVERSION9 and I added another subdirectory to take it 140 and this is where things produced an unexpected result.

:AE VOC LONGDIRTEST
Top of "LONGDIRTEST" in "VOC", 3 lines, 149 characters.
001: DIR
002: C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\
SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9\\SOFTVERSION10
003: D_BP
Bottom.

To my surprise AE appeared to work

:AE LONGDIRTEST JDS2
Top of New "JDS2" in "LONGDIRTEST".
*--: I
001= NEW RECORD 2
*--: FI
Filed "JDS2" in file "LONGDIRTEST".

However, a LIST LONGDIRTEST showed nothing, I scratched my head for a moment
:LIST LONDIRTEST

No records listed.

I tried AE again and it pulled the record back. I then checked what was happening at the OS level .. nothing is SOFTVERSION10

C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9\\SOFTVERSION10>dir
 Volume in drive C is Windows
 Volume Serial Number is 7838-4C29

Directory of C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9\\SOFTVERSION10

14/04/2023  09:47    <DIR>          .
14/04/2023  09:47    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  295,873,118,208 bytes free

Head scratch moment, what had AE done with the record, so I went back up one directory
C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9>dir
 Volume in drive C is Windows
 Volume Serial Number is 7838-4C29

Directory of C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOFTVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION9

14/04/2023  09:59    <DIR>          .
14/04/2023  09:59    <DIR>          ..
14/04/2023  09:59                12 JDS2
14/04/2023  09:47    <DIR>          SOFTVERSION10
               1 File(s)             12 bytes
               3 Dir(s)  295,876,792,320 bytes free

JDS2 is being stored in SOFTVERSION9 

I then added a new level of SOFTVERSION11, AE worked but the record was again stored in SOFTVERSION9.

There are two problems ... the first is the WRITE failure as first reported and the second the storing of the record in the incorrect location which you must be very unlucky to hit. I expect the problem is going to come down to some hard coded cut at 126 characters in AE (backing up my first thoughts). The reason I suspect my second test using SOFTVERSION10 produced the results it did is because the 127th character is a \\, hence the 126 characters making up a pointer to SOFTVERSION9 is a valid directory, where in the first case SOFTVERSION900 becomes truncated to SOFTVERSION9 and does not exist.

When I get some free cycles I will review the AE source and see if I can find the problem and fix it and then look to getting it rolled out in a UniData release.

WOW -- I bet you are 100% correct Jonathan!  AE is doing some 'fixed length' directory process.  What a crazy find!

We did not test if we could OPEN / READ /WRITE to the file from BASIC, but that would be a natural next step as well.

I hope you find a cycle to review -- this is getting interesting!!   (Grabbing popcorn)

DW

@Susan Joslyn @David Wolverton  ... So I can confirm the problem is not limited to AE and I have managed to produce the problem outside of AE and it is a strange one and I will be raising a bug ticket for it.

The description of the problem can be broken down to :-

If a directory file variable exceeds the maximum item id length and that file variable is used in any UniBasic operation that requires a lock check to be done (so WRITEU, MATWRITEU and READ are not affected as no lock operation is required) the file variable will become truncated and then cannot be used a second time or may write to an incorrect location if the truncation still points to a location.

In the case of AE , the READU of the record works correctly the first time but truncates the file variable and when it hits the WRITE statement it fails. I got lucky where the truncation happens on a "\\".  

The following simple program shows the problem. After the debugger call at line 12, note the file variable has been truncated.

001: OPEN "LONGDIRTEST" TO F.FILE ELSE STOP
002: TD = TIMEDATE()
003: R.FILE = TD
004: DEBUG
005: WRITEU R.FILE ON F.FILE,"JDS"
006: DEBUG
007: READ R.FILE FROM F.FILE,"JDS" ELSE NULL
008: IF R.FILE = TD THEN CRT "SAME" ELSE CRT "DIFFERENT"
009: DEBUG
010: READU R.FILE FROM F.FILE,"JDS" ELSE NULL
011: IF R.FILE = TD THEN CRT "SAME" ELSE CRT "DIFFERENT"
012: DEBUG
013: CRT "DONE"
014: END

:RUN BP TEST45
***DEBUGGER called at line 4 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
***DEBUGGER called at line 6 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
SAME
***DEBUGGER called at line 9 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
DIFFERENT
***DEBUGGER called at line 12 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99
!G
DONE
:

I am also going to see if this only applies to directory type files and the UniData bug I raise will cover that if required as well.

Thanks for this find Jonathan!   And really interesting this is just showing up after 20+ years!  The best bugs are well hidden!

I will watch release notes with interest!!

David W.

@Susan Joslyn @David Wolverton  ... So I can confirm the problem is not limited to AE and I have managed to produce the problem outside of AE and it is a strange one and I will be raising a bug ticket for it.

The description of the problem can be broken down to :-

If a directory file variable exceeds the maximum item id length and that file variable is used in any UniBasic operation that requires a lock check to be done (so WRITEU, MATWRITEU and READ are not affected as no lock operation is required) the file variable will become truncated and then cannot be used a second time or may write to an incorrect location if the truncation still points to a location.

In the case of AE , the READU of the record works correctly the first time but truncates the file variable and when it hits the WRITE statement it fails. I got lucky where the truncation happens on a "\\".  

The following simple program shows the problem. After the debugger call at line 12, note the file variable has been truncated.

001: OPEN "LONGDIRTEST" TO F.FILE ELSE STOP
002: TD = TIMEDATE()
003: R.FILE = TD
004: DEBUG
005: WRITEU R.FILE ON F.FILE,"JDS"
006: DEBUG
007: READ R.FILE FROM F.FILE,"JDS" ELSE NULL
008: IF R.FILE = TD THEN CRT "SAME" ELSE CRT "DIFFERENT"
009: DEBUG
010: READU R.FILE FROM F.FILE,"JDS" ELSE NULL
011: IF R.FILE = TD THEN CRT "SAME" ELSE CRT "DIFFERENT"
012: DEBUG
013: CRT "DONE"
014: END

:RUN BP TEST45
***DEBUGGER called at line 4 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
***DEBUGGER called at line 6 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
SAME
***DEBUGGER called at line 9 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99999
!G
DIFFERENT
***DEBUGGER called at line 12 of program BP\\_TEST45
!\\F.FILE
an unidata file; filename = C:\\MASTER\\SOFTVERSION1\\SOFTVERSION2\\SOFTVERSION3\\SOF
TVERSION4\\SOFTVERSION5\\SOFTVERSION6\\SOFTVERSION7\\SOFTVERSION8\\SOFTVERSION99
!G
DONE
:

I am also going to see if this only applies to directory type files and the UniData bug I raise will cover that if required as well.