General Discussion & Announcements

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

Hi Dimuthu,

These three command function differently. 

The READ command loads the contents of the read into the dynamic variable you specify. 

The READV command extracts the attribute you specify from the record and stores in the dynamic variable you specify.

The MATREAD command loads the content of the read into a dimensioned variable you initialized using the DIM command.

With today's modern systems, there isn't much of a performance difference. The system allocates memory dynamically based on the size of the record. The constraint on dimensioned variables is that they will have a maximum number of attributes as defined by the DIM command. If you go over the bounds of the dimension, the system will balk at you. 

I often use dimensioned arrays however with an INCLUDE to handle the DIM command. The only issue with this is that if you need to expand the record size, you need to recompile all programs with the INCLUDE so the code is not as easily extensible. I use a lot of INCLUDES for dimensioned variables that contain equates (EQU) for the names of the fields so anyone else using the includes uses the same variable names. Keeps things consistent. 

------------------------------
CURT VALENTINE
SOFTWARE ENGINEER III
Rocket Forum Shared Account
------------------------------

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

READ loads the entire record into a dynamic array.
READV loads a single field (which could be multi-valued) into a variable.
MATREAD loads an entire record into a dimensioned array.

READ is the most commonly used but can have minimal (very minimal) performance issues it there are a lot of fields.  If you want to access field 100, the U2 program must parse through the prior 99 fields to get to it.
READV is best used if you know you only want to read 1 value from a record.  If you want more than one, use a READ.
MATREAD takes the most coding, but is that fastest if you are going to be dealing with multiple fields in the same record.  Unlike a READ, it you want to access field 100, the U2 program does NOT need to parse through the prior 99 fields.

Note that the parsing is transparent to the user - it is built into the U2 programming language.  The bigger a record is and the more fields you wish to access, the slower the program will run due to the parsing.  The more data it has to parse through to get the desired field, the slower it is.  Again, this parsing tends to be fast.  I rarely use MATREADs unless I know I am dealing with large records.  That being said, if you have a program that is running slowly and deals with a lot of data, replacing the READs with MATREADs can have staggering performance improvements.  It really comes down to: you need to know your data to make the best decision.

------------------------------
John Isael
Rocket Forum Shared Account
------------------------------

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

@John Isael 

This is a good level of response to this question to Dimuthu Dissanayake's question. We can overdo the detail for what sounds like a general introductory question. We can continue the discussion as Dimuthu requests more detail.

One additional note that obfuscates the performance aspect of your response is field-level caching that was introduced in UniVerse at 7.3.1 and in current versions of UniData. Field level cache stores field number and the offset of the last accessed field. This alters the performance profile of many workloads. Scanning fields sequentially (FOR NDX = 1 TO N; A = ARRAY<NDX>; NEXT NDX) becomes almost painless, and only about twice the time of using REMOVE in the same loop. While this suggests we should access fields in field number sequence to take advantage of field-level caching, I lean towards code clarity.

Decomposing each field into an array element wih MATREAD, the caching moves the tradeoff towards more field accesses in a record before needing the benefit of MATREAD. You seem to also lean in this direction. When curious, measure.

Further, note that caching, accessing sequential values in a field exhibits the same performance characteristics of accessing fields prior to the implementation of the cache. (FOR NDX = 1 TO N; A = ARRAY<X,NDX>; NEXT NDX) . In these cases, the RAISE() and LOWER() functions provide access to the values at the field level, and enable use of the caching mechanism.

------------------------------
Mark A Baldridge
Principal Consultant
Thought Mirror
Nacogdoches, Texas United States
------------------------------

Original Message:
Sent: 03-23-2023 08:38
From: John Isael
Subject: READ READV MATREAD

READ loads the entire record into a dynamic array.
READV loads a single field (which could be multi-valued) into a variable.
MATREAD loads an entire record into a dimensioned array.

READ is the most commonly used but can have minimal (very minimal) performance issues it there are a lot of fields.  If you want to access field 100, the U2 program must parse through the prior 99 fields to get to it.
READV is best used if you know you only want to read 1 value from a record.  If you want more than one, use a READ.
MATREAD takes the most coding, but is that fastest if you are going to be dealing with multiple fields in the same record.  Unlike a READ, it you want to access field 100, the U2 program does NOT need to parse through the prior 99 fields.

Note that the parsing is transparent to the user - it is built into the U2 programming language.  The bigger a record is and the more fields you wish to access, the slower the program will run due to the parsing.  The more data it has to parse through to get the desired field, the slower it is.  Again, this parsing tends to be fast.  I rarely use MATREADs unless I know I am dealing with large records.  That being said, if you have a program that is running slowly and deals with a lot of data, replacing the READs with MATREADs can have staggering performance improvements.  It really comes down to: you need to know your data to make the best decision.

------------------------------
John Isael
Rocket Forum Shared Account

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

Dimuthu,

My information may be a bit dated, but here is what I was taught. The READ will read the string from the file. MATREAD will perform a READ and then a MATPARSE statement. The READV will READ and then extract the value desired.

As far as the memory allocation and performance that is difficult for me to say. As I said my knowledge is dated. That being said the READ is done in any case, whether it is cached or not for multiple READV operations I do not know. If not then each READV will read the record and perform an extract. For multiple READVs on the same record I would think a single read with multiple extracts would be more efficient.

The MATREAD needs to do a READ and a MATPARSE statement. Unless a Matrix is needed it would seem to be extra overhead.

Sorry for the caveat. I'm sure as the product has gone through many revisions and some of the inefficiencies have been addressed. I would think that a UniVerse Software Engineer could answer your question better.

Jon

------------------------------
Jon Kristofferson
Pick Programmer
Snap-on Credit LLC
Libertyville IL US
------------------------------

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

Dimuthu,

Here's a summary of the READ, READV and MATREAD...

1. READ will read in your data and place the data in a dynamic array.
2. READV is similar to READ, but you are only reading in one attribute.
3. MATREAD is reading in the data but placing your data in a dimensioned array.

As for the memory and performance, I really don't notice much difference to be honest.   Each one has their advantages and disadvantages.    The one advantage that MATREAD has is the dimensioned array is addressable in memory.  But, if you don't size your array correctly, it will cause an error that the array is too small.

With READ, you don't really have to worry about this, since the data is loaded into a dynamic array.

READV is really nice, if you are reading in just one attribute.   But I will not advise having multiple READV statements if it is reading the same file.   In this case, you are better off using READ.

I have used all 3 in my programs, so in my case, it is dependent on what I'm trying to accomplish.

Hope this helps!!

Sincerely,

------------------------------
Grant Boice
Principal Software Developer
Matheson Tri-Gas Inc.
Warren, NJ 07059
------------------------------

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------

Let us assume that you have created a Dimensioned array as 100 cells

DIM MyArray(100)

When you do a MATREAD of a record that is actually less than 100 attributes

MATREAD MyArray from Customer, "215413" ELSE ...

The system will still try to fill that 100 cell array until it runs out of attributes

Each cell of the array takes 10 bytes

If an attribute overflows the 10 bytes, that cell will instead contain a pointer to where the full-sized string of that cell actually now resides somewhere else in dynamic overflow

If it does not fill all 100 cells, the remaining cells still take up 10 bytes but are empty

The reason for a static array is so when you reference say MyArray(52) the run time code can jump directly to that cell without scanning the whole array

So the run time engine calculates the location where this cell should be and jumps to it

------------------------------
Will Johnson
Systems Analyst
Rocket Forum Shared Account
------------------------------

Original Message:
Sent: 03-22-2023 01:39
From: Dimuthu Dissanayake
Subject: READ READV MATREAD

what is the different between these commands READ READV and MATREAD and how the memory allocation and performance work in these commands.

------------------------------
Dimuthu Dissanayake
Software Engineer
Rocket Forum Shared Account
------------------------------