Hi all
I've been doing some experiments accessing UniVerse data via Python for some time and I had my own library before the official Rocket uopy utilities were available to me.
One thing I implemented is my own custom representation of a UniVerse record as a Python dictionary and I was wondering if it would be useful to anyone. Maybe I can add it to the Python github.
Anyway, I'll explain the logic in my implementation and why I still use it in some places instead of the "natural" DynArrays.
My implementation stems from the structure of a lot of the data we use here, as a lot of records are sparsely populated (I use to call my own records "sparse records" to differentiate them from the DynArray)
Example: A register with 90 values, which only 4 contain real data. One of them is a multivalue that also contains several "empty" fields.
1. "Value1"
2. "Value2"
3. ""
4. ""
...
60. "a|b|c|d||||||||m|n"
...
90. "Value90"
Using a DynArray results in a mostly empty list, with the footprint that this involves, maily using thousands of huge sparsely populated records.
It will look like this
['Value1', 'Value2', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ['a', 'b', 'c', 'd', '', '', '', '', '', '', '', '', 'm', 'n', '', '', '', '', '', '', '', '', '', '', ''], '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', 'Value90']
In my solution I'm using a defaultdict. This is a variation of the usual Python dictionary that returns a default value when a key is not present.
class UniverseDict(defaultdict):
def __init__(self):
super().__init__(lambda: None)
In this case the value is None, as sometimes I need to tell if a value is not present versus if a value is just an empty string. But it can return an empty string if that is more convenient.
The previous example would look like this in my sparse dictionary:
{
0: 90,
1: {
0: 1,
1: 'Value1'
},
2: {
0: 1,
1: 'Value2'
},
60: {
0: 14,
1: 'a',
2: 'b',
3: 'c',
4: 'd',
13: 'm',
14: 'n'
},
90: {
0: 1,
1: 'Value90'
}
}
The keys are integers that are equivalent to the indexes used in a DynArray. record[1] = 'Value1', record[90] = 'Value90', etc
The key 0 is special as its value is the number of values in the register or in a value. In this case, record[0] is 90, as there are 90 values. record[60][0] is 14 as there are 14 values. The default value for this entry is 1, to represent single-valued entries (such as 'Value1' or 'Value2').
My trouble arises when using this dictionaries along with uopy. I have made a custom class derived from uopy.File so I can create my own records. It's just a copy of uopy.File with a call to my constructor instead of a call to the constructor of DynArray.
class SparseFile(uopy.File):
def __init__(self, name, dict_flag=0, session=None):
super().__init__(name, dict_flag, session)
def read_uv_register(self, record_id, lock_flag=0):
_logger.debug("Enter", record_id, lock_flag)
self._check_opened()
with self._lock:
# some code
if resp_code != 0:
# other code
else:
record = UVRegister(record_id, datos=in_packet.read(2), session=self._session)
_logger.debug("Exit", record)
return record
This is a house-made hack and I was wondering if something like this could be integrated into uopy. Maybe not the SparseRecord class per se but allowing uopy.File to have another parameter in the read_uv_register method that allows to pass a factory method or object so that custom representations of UniVerse records can be built on the fly.
Also, if this approach may be useful to somebody I would be glad to add the whole code to the github repository, as it's quite lengthy.
------------------------------
Hector Cortiguera
Rocket Forum Shared Account
------------------------------