Open-source Languages & Tools for z/OS

 View Only
  • 1.  Still trying to get an install --- Weeks of frustration is building

    Posted 01-20-2022 09:41
    I am working with z/OS 2.4

    I was using the manifest file in order to get all dependencies resolved in one pass.

    You have given me some ideas
    A) a newer manifest file. I think there is a newer one from Rocket
    B) individual or small group installs of key products only.

    Unfortunately, I have no clue which package is generating the bash character (unknown command) problem.

    I have a vague recollection from earlier threads that if the 'wrong' level of python was in your PATH, bad things could happen. That is something else that will eat up my working days. What should be at most a two day project has been a burden to me since I started it over 3 months ago


    ------------------------------
    Tom Longfellow
    Senior Systems Programmer
    Maryland Judicial Information Systems (JIS)
    Annapolis MD US
    ------------------------------


  • 2.  RE: Still trying to get an install --- Weeks of frustration is building

    Posted 01-20-2022 14:38
      |   view attached
    Another day with no working results.
    Ended up reinstalling from scratch.  miniconda install worked (eventually --- but I am getting better at this)
    Environment create says it works.
    conda activate of the new environment worked.

    conda install -c https://conda.anaconda.org/zoss-appdev zip unzip
    Fails
    with 
    bash: $'\025': command not found

    Again - no clue as to what part of conda is producing the bash error or how I could find it or correct it.
    Another damned brick wall  


    ------------------------------
    Tom Longfellow
    Senior Systems Programmer
    Maryland Judicial Information Systems (JIS)
    Annapolis MD US
    ------------------------------

    Attachment(s)

    txt
    conda install 2.txt   1 KB 1 version


  • 3.  RE: Still trying to get an install --- Weeks of frustration is building

    ROCKETEER
    Posted 01-20-2022 15:40
    Hello Tom,

    Looking at your environment variables in one of the attached files, I can tell you're definitely missing some of the important ones.

    First, there's no such thing as _BPX_AUTOCVT - the correct name is _BPXK_AUTOCVT (yes, both BPX and BPXK prefixes exist on z/OS, and are used for different parts of USS; they're not interchangeable of course).

    Then, conda requires more than just _BPXK_AUTOCVT... The PDF doc provided with Miniconda (zOS Miniconda Documentation 1.2.1.pdf) explicitly states that:

    Each end user who needs access to Miniconda should have following settings in a Bash startup file:

    export _BPXK_AUTOCVT=ON
    export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
    . <miniconda_installation_path>/etc/profile.d/conda.sh

    So you need the _CEE_RUNOPTS, too. Without these two variables, most of our Ported Tools aren't going to work properly.

    --
    Regards,
    Vladimir

    ------------------------------
    Vladimir Ein
    Software Engineer
    Rocket Software
    ------------------------------



  • 4.  RE: Still trying to get an install --- Weeks of frustration is building

    Posted 01-21-2022 07:49
    I have noticed that bash on the mainframe outputs non-displayable characters in oct. \025 in oct is hex 15, which in EBCDIC 037/1047 is the new line character. I think bash wants input in American ascii (or maybe iso8859-1). So sending an EBCDIC newline to bash results in bash interpreting it as a command, which can't be found on the path, which gives the error.

    Setting up auto-conversion, as Vladimir suggests, will probably fix the problem.

    ------------------------------
    Adam Martin Britt
    IT-Architect
    BEC
    Randers SV DK
    ------------------------------



  • 5.  RE: Still trying to get an install --- Weeks of frustration is building

    Posted 01-24-2022 12:37
      |   view attached
    Much closer but still no go for installing the entire bundle.   This time, after the correct AUTOCVT was specified, we get
    # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
        Traceback (most recent call last):
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/exceptions.py", line 819, in __call__
            return func(*args, **kwargs)
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/cli/main.py", line 78, in _main
            exit_code = do_call(args, p)
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 77, in do_call
            exit_code = getattr(module, func_name)(args, parser)
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/cli/main_install.py", line 11, in execute​
            install(args, parser, 'install')
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/cli/install.py", line 182, in install
            specs.extend(common.specs_from_url(fpath, json=context.json))
          File "/usr/local/Rocket/miniconda/lib/python3.7/site-packages/conda/cli/common.py", line 128, in specs_from_url
            for line in open(path):
          File "/usr/local/Rocket/miniconda/lib/python3.7/codecs.py", line 323, in decode
            (result, consumed) = self._buffer_decode(data, self.errors, final)
        UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 5: invalid start byte
    `$ /usr/local/Rocket/miniconda/bin/conda install -c https://conda.anaconda.org/zoss-appdev --file /u/tech999/appdev_manifest_1.2.1.txt`
     
    Any ideas??
    I do see that UTF-8 has raised its ugly head.   ISO8859 must not be good enough.  chtag -tc 859 was used during miniconda installation
     
    The entire install output is attached


    ------------------------------
    Tom Longfellow
    Senior Systems Programmer
    Maryland Judicial Information Systems (JIS)
    Annapolis MD US
    ------------------------------

    Attachment(s)

    txt
    conda install 3.txt   4 KB 1 version


  • 6.  RE: Still trying to get an install --- Weeks of frustration is building

    Posted 01-24-2022 16:52
    Finally --- SUCCESS ---

    The change to the BPXK AUTOCVT  variable took me further.   BUT a recommendation to others.   Avoid installing the manifest files as downloaded.   Something in there triggered Utf-8 character conversions when trying to install  'everything'

    I pared back to the ones I needed and the install completed.    Next will be some testing.

    Thanks to everyone for their help.

    ------------------------------
    Tom Longfellow
    Senior Systems Programmer
    Maryland Judicial Information Systems (JIS)
    Annapolis MD US
    ------------------------------



  • 7.  RE: Still trying to get an install --- Weeks of frustration is building

    ROCKETEER
    Posted 01-25-2022 13:05
    Hello Tom,

    First of all I would like to clarify a couple of things.

    The manifest files are treated by conda as UTF-8 files. However, our manifests only contain single-byte UTF-8 characters - these characters form a subset that is actually called ASCII, and this set of characters is identical in both UTF-8 and ISO8859-1 (and many other encodings). So, in a certain sense, our manifest files are valid in both UTF-8 and ISO8859-1. There's nothing in there that 'triggers UTF-8 character conversions', because UTF-8 conversion (or, rather, decoding) for manifest files happens always.

    That said, as long as conda reads manifest files as valid ASCII text, there will be no UTF-8 validation issues - valid ASCII text is always valid in UTF-8, too. For this to happen on USS:

    (1) the file must be tagged according to the encoding of its contents, and
    (2) the encoding itself must be eligible for auto-conversion - which limits us to ISO8859-1 and IBM-1047.


    Looking at your output with the UTF-8 error, I can guess that (sorry, only a guess):
    • The manifest file was copied to mainframe over FTP in text mode, which effectively converts the file from ISO8859-1 to IBM-1047. Binary mode is preferable since it retains the file's original encoding.

    • You then changed the file tag to the code page 859, which is an alias for ISO8859-15, not ISO8859-1. Not sure where 859 came from; the Miniconda doc mentions 819, which is an alias for ISO8859-1. Now, ISO8859-15 is not eligible for automatic conversion, and so the manifest file was read 'as is', in its current IBM-1047 encoding. In this encoding, it's not a valid UTF-8, neither it is a valid ASCII, - hence the UnicodeDecodeError.


    Now let's get back to your last message. I would like to reiterate what Tatiana already said in the other thread: we do test our products on a live system ('QA' in her message stands for 'quality assurance'), and this includes the installation process. The manifest files published on the Rocket Community portal can and should be used 'as is'. Please verify your assumptions before giving other users bad advice like that.

    Regards,
    Vladimir

    ------------------------------
    Vladimir Ein
    Software Engineer
    Rocket Software
    ------------------------------



  • 8.  RE: Still trying to get an install --- Weeks of frustration is building

    Posted 01-25-2022 14:13
    Thanks for the education.   I was aware that character conversions are an unavoidable thing in the Linux/Unix/PC/EBCDIC  world.
    So much so, that I have developed bad habits over the years. 
     I usually use the PC ISO8859 value between mainframes and 'server boxes'.   These bad habits have been at the root of all my problems so far.   I used BPX  (and not BPXK) out of the habit of working with CEE and USS  on the mainframe.   I used -tc 859 because the echo of ISO8859 was in my head.  Also, my FTP client on my PC is geared to treat *.txt  as an ISO8859 PC file that needs to be translated to 037/1047 on the mainframe.  Overrides are of course possible during the FTP process but it requires monitoring and special ftp client commands.

    So, the final result has been a long process, but knowledge has been created for future installs.

    Thanks again for helping me through this interesting world of cross system development and Open Source

    ------------------------------
    Tom Longfellow
    Senior Systems Programmer
    Maryland Judicial Information Systems (JIS)
    Annapolis MD US
    ------------------------------