z/OS Tools & Language

Expand all | Collapse all

Python environment initialization problem

  • 1.  Python environment initialization problem

    Posted 10-25-2018 11:33

    Hello All,

    We have Python 3.6 installed in our z/OS 2.2 and it works fine in interactive mode.
    However when I tried to write my own application and embed Python there I ran into problem with call to Py_Initialize().

    It receives control and then terminates abnormally with Fatal Error apparently while reading file …lib/python3.6/encodings/init.py
    I also receive a message that looks like complete garbage, but when I redirected it to a file and tagged it as ASCII i was able to seen this:

    File “/usr/lpp/python36/lib/python3.6/encodings/init.py”, line 1
    ⌂⌂⌂@⣁�� .
    … several ines of the same garbage…
    SyntaxError: invalid syntax
    initstdio: failed to import module encodings.utf_8
    Fatal Python error: Py_Initialize: can’t initialize sys standard streams

    Environment variable _BPXK_AUTOCVT set to ON;
    The env variable PYTHON_HOME is set correctly, so my C program can see Python DLLs. and all I do in my main() is a call to Py_Initialize ();
    It seems to me the Py_Initialize () is reading this file using incorrect encoding.

    What am I doing wrong?

    Here is my small repro of the problem:
    #include <stdio.h>

    #include <Python.h>

    int main (int argc, char* argv[])
    int i, j, done = 0; /* variables I will need later */

    Py_Initialize ();


    The compile command:
    c++ -c -W"c,lp64,FLOAT(HEX)" -W"c,DFP,TUNE(7),ARCH(7)" -+ -W"c,LANG(EXTENDED),XPLINK(STOREARGS),XPLINK(BACKCHAIN)" -W"c,FEDBG(-qxflag=EnableDeclToTypeLinkagePropagation)" -W c,expo,ros -2 -W c,NOANS,gonumber -I/usr/lpp/python36/include/python3.6m -I/usr/lpp/python36/lib/python3.6/site-packages/numpy/core/include -o ptst.o ptst.c

    Link script:
    export _CXX_WORK_SPACE="(32000,(300,150))"
    export _CXX_WORK_UNIT=“VIO”
    c++ -Wl,lp64 -o ptst.out ptst.o /usr/lpp/python36/lib/libpython3.6m.x -lm -lc

  • 2.  RE: Python environment initialization problem

    Posted 11-01-2018 07:42

    I haven’t had a chance to look at this in detail, but you might try adding -qascii to the compile and link commands. Rocket tools use IBM’s enhanced ASCII support, so the programs use ASCII internally.

  • 3.  RE: Python environment initialization problem

    Posted 11-01-2018 17:20

    Thanks! Seems to work. Now gonna make it work in mixed environment - main compiled and linked as EBCDIC calls Python dll.s that are ASCII.

  • 4.  RE: Python environment initialization problem

    Posted 11-02-2018 06:40

    I strongly advise against that; I think you find debugging to be especially painful. Is there a reason not to compile the entire program as an ASCII-mode program?

  • 5.  RE: Python environment initialization problem

    Posted 11-02-2018 10:37

    Yes, there is a big reason to keep our code in EBCDIC. Our app is a huge portable application server that works on all platforms from Windows to mainframe and calls multiple third party APIs – several millions lines of code. So nobody is going to port it to ASCII in z/OS.
    After some debugging of LE environment I was finally able to initialize Python environment successfully from my EBCDIC routine. I would really appreciate if you could be a little more specific and explain your concerns.

  • 6.  RE: Python environment initialization problem

    Posted 11-06-2018 12:47

    If your application is running on non-z/OS platforms, I would think porting it as an ASCII program would actually be easier. This has been our experience porting open source tools to z/OS.

    My concern about mixing EBCDIC and ASCII libraries is that any given thread is in a single “mode” (ASCII or EBCDIC) at a given time, and I would think that when you cross library boundaries you would have to change the mode. Otherwise the automatic code translation enabled by _BPXK_AUTOCVT is going to be confused and do the wrong thing.

    My comment about debugging is that, if you step through code in dbx, you will have to keep changing the character encoding mode of dbx depending where you are in the stack (via the $asciichars and $asciistrings variables). Also, if you debug by adding printf calls, if EBCDIC and ASCII portions of the code write to the same file descriptor, the encoding will be wrong for one or the other.