Rocket U2 | UniVerse & UniData

 View Only
  • 1.  Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 29 days ago

    I have a statistics program that builds an array with <1,POS>= sku and <2,POS>=qty.  The output would often have 8-9k SKUs.

    When I dump it out to a sequential file, I want to output the data as  a sinlge row for each sku in the format of SKU:<delimiter>:QTY

    I know I can do this programatically using a for...next loop, but wondered if ther was a quicker/built-in way to do this?

    Nelson



    ------------------------------
    Nelson Schroth
    president
    C3CompleteShop LLC
    Harrison OH US
    ------------------------------


  • 2.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 29 days ago

    Hi Nelson,

    It can be done but probably not as simply as a for next loop.  There's a CATS function that will concatenate to dynamic arrays  and return a new dynamic array with the results.  It's not that simple though if you want to concatenate a new delimiter.  Here's an example that does it two ways. Take a look and see which one you think is easier to understand and maintain.

          ARRAY = 'A1':@VM:'A2':@VM:'A3':@VM:'A4'
          ARRAY<2> = 'B1':@VM:'B2':@VM:'B3':@VM:'B4'

          LN = DCOUNT(ARRAY<1>, @VM)

          * PUT THEM TOGETHER WITH A FOR NEXT LOOP
          LOOP.RESULT = ''
          FOR LINE.NO = 1 TO LN
             LOOP.RESULT<1,LINE.NO> = ARRAY<1,LINE.NO>:'-':ARRAY<2,LINE.NO>
          NEXT

          PRINT LOOP.RESULT

          * DO THE SAME THING WITH THE CATS FUNCTION

          * CATS CONACTENATES THE VALUES FROM ONE ARRAY TO ANOTHER ARRAY AND RETURNS THE RESULTING ARRAY
          * IN ORDER TO USE IT, WE NEED TO HAVE A CORRECTLY SIZED ARRAY OF THE DELMITERS SO THAT WE
          * CAN CONACTENATE IT WITH THE SECOND ARRAY
          DELIMS = STR('-':@VM, LN)

          * WE'RE ALWAYS GOING TO END UP WITH AN EXTRA ELEMENT AT THE END SO WE NEED TO STRIP THAT OFF
          DELIMS = FIELD(DELIMS, @VM, 1, LN)

          * NOW WE CAN BUILD A NEW ARRAY BY CONCACTENATING OUR DELIMITERS WITH ARRAY<2>
          DELIM.ARRAY = CATS(DELIMS, ARRAY<2>)

          * FINALLY WE CAN BUILD THE NEW ARRAY WITH ARRAY<1> AND DELIM.ARRAY
          RESULT = CATS(ARRAY<1>, DELIM.ARRAY)
          
          PRINT RESULT

          STOP

    If you're trying to get the best performance, CATS is faster. To loop through each of those sections 999,999 times took the first method 2,510 milliseconds. The second took 960 milliseconds.  For most real world cases it's better to go with understandable.



    ------------------------------
    Joe Goldthwaite
    Consultant
    Phoenix AZ US
    ------------------------------



  • 3.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 29 days ago

    Oops, I forgot to show the output.

    A1-B1ýA2-B2ýA3-B3ýA4-B4
    A1-B1ýA2-B2ýA3-B3ýA4-B4



    ------------------------------
    Joe Goldthwaite
    Consultant
    Phoenix AZ US
    ------------------------------



  • 4.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 29 days ago

    Well $%@#@,

    You ca use the SPLICE function:

          PRINT SPLICE(ARRAY<1>, '-', ARRAY<2>)

    It's a bit simpler.



    ------------------------------
    Joe Goldthwaite
    Consultant
    Phoenix AZ US
    ------------------------------



  • 5.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 28 days ago
    Edited by Joseph von Arx 28 days ago

     I created a subroutine to work like Excel to transpose an array.  I've attached it for reference.  My first pass I used Splice but it was slow on larger data sets.  I hope this helps.

    Edit:

    I don't know why uploading did not seem to attach the file, so I've added it here.

    SUBROUTINE TRANSPOSE.ARRAY(ITEM)
    * ------------------------------------------------------------------------
    *  Name        : TRANSPOSE.ARRAY
    *  Description : Transpose an array
    *  Programmer  : Joseph von Arx
    *  Date        : 7/20/2020
    *  ------------------------------------------------------------------------
    *
    *  Notes
    *  -----
    *
    *  Modification History
    *  Date        Who  Description
    *  ----------  ---  -----------------------
    *  03/20/2023  JLVA Rewrote code for optimization and efficiency for larger arrays
    *
    * ------------------------------------------------------------------------
        $OPTIONS DEFAULT
        FMAX = COUNT(ITEM, @AM)+1
        DIM WORK(FMAX)
        MAT WORK = ''
        MATPARSE WORK FROM ITEM USING @AM
    !***
    !* MAKE SURE ALL FIELDS HAVE SAME NUMBER OF MULTIVALUES
    !***
        VMAX = COUNT(WORK(1), @VM)+1
        FOR FMC = 2 TO FMAX
            VMC = COUNT(WORK(FMC), @VM)+1
            IF VMC GT VMAX THEN VMAX = VMC
        NEXT FMC
    !***
    !* Extract the elements and load them into an array.
    !* This should be faster than <> extraction on large Items.
    !***
        DIM ARRAY(VMAX)
        MAT ARRAY = ''
        DIM VALARRAY(VMAX)
        FOR II = 1 TO FMAX
            LINE = WORK(II)
            MAT VALARRAY = ''
            MATPARSE VALARRAY FROM LINE USING @VM
            FOR JJ = 1 TO VMAX
                IF II # 1 THEN ARRAY(JJ) := @VM
                VAL = VALARRAY(JJ)
                ARRAY(JJ) := VAL
            NEXT JJ
        NEXT II
        MAT VALARRAY = ''
        MAT WORK = ''
    !***
    !* Done.
    !***
        MATBUILD ITEM FROM ARRAY USING @AM
        ICNT = DCOUNT(ITEM, @AM)
        IF ICNT LT VMAX THEN
            ITEM := STR(@AM, (VMAX - ICNT))
        END
    *
        RETURN
    END
    



    ------------------------------
    Joseph von Arx
    Software Developer
    Data Management Associates Inc DMA
    Cincinnati OH US
    ------------------------------



  • 6.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 27 days ago

    Coming to this late but always worth remembering for UniVerse:

    MATPARSE is quick but so is iterating through fields (not values) in a dynamic array because of the hint pointer in a dynamic array.

    Often just rewriting:

    For i = 1 To NumberOfValues

        ... MyArray<1, i>

        ... MyArray<2,i>

    Next

    to promote the values:

    MyField1 = Convert(@VM,@FM, MyArray<1>)
    MyField2 = Convert(@VM, @FM, MyArray<2>)

    and then iterating at the field level (so it doesn't go back to the start to position the extract) can be so much quicker.

    Also, time taken to build the array in the first place - you said you have a statistics program that builds the array with two sets of values. 

    One option is to use UDOs, yes they are pig-ugly to code (!) but I've found can be a lot quicker than using large arrays in Basic, which makes sense when you think about the memory allocation underneath. Not knowing the context of what you are doing to create the arrays in the first place, that might be quicker still to build and render out if each sku is a UDO property added to an array. Same for a python dictionary of course.



    ------------------------------
    Brian Leach
    Director
    Brian Leach Consulting
    Chipping Norton GB
    ------------------------------



  • 7.  RE: Is there a UniVerse BASIC command like Excel TRANSPOSE?

    Posted 25 days ago
    Edited by Stuart Boydell 23 days ago

    You can also use python to convert and write your csv (or xlsx if that's the end game). I haven't tested the transform.

    #  Write to "PP/ConvertToCSV.py"
    import pandas as pd
    def write_csv(mvList,fileName):
       UVAM = '\uf8fe'; UVVM = '\uf8fd'
       pd.read_csv(mvList.replace(UVAM,'\n'),UVVM)
       pd.T.to_csv(fileName, header=False, index=False) # T to transform

    * write to BP CSV.TEST, compile and run 
    program CSV.TEST

       mvList      = 'name':@vm:'address':@vm:'notes':@vm:'numbers'
       mvList<2>   = 'Miss "January" Jones':@vm:'666 Mockingjay way, Parramatta NSW 2200'
       mvList<2,3> = 'These are some notes':char(10):'with, arbitrarily, a line break.':char(10):'Third line.'
       mvList<2,4> = 123.45
       moduleName  = 'ConvertToCSV'
       funcName    = 'write_csv'
       outName     = 'output.csv'

       pyresult = PyCallFunction(moduleName, funcName, mvList, outName)

       crt 'Done ':@pyexceptionmsg
    end



    ------------------------------
    Stuart Boydell
    Technical Specialist
    Meier Business Systems PTY LTD
    Carnegie Vic AU
    ------------------------------