Open-source Languages & Tools for z/OS

Hello All,

We have Python 3.6 installed in our z/OS 2.2 and it works fine in interactive mode.
However when I tried to write my own application and embed Python there I ran into problem with call to Py_Initialize().

It receives control and then terminates abnormally with Fatal Error apparently while reading file …lib/python3.6/encodings/init.py
I also receive a message that looks like complete garbage, but when I redirected it to a file and tagged it as ASCII i was able to seen this:

File “/usr/lpp/python36/lib/python3.6/encodings/init.py”, line 1
⌂⌂⌂@⣁�� .
… several ines of the same garbage…
@��
^
SyntaxError: invalid syntax
initstdio: failed to import module encodings.utf_8
Fatal Python error: Py_Initialize: can’t initialize sys standard streams

Environment variable _BPXK_AUTOCVT set to ON;
The env variable PYTHON_HOME is set correctly, so my C program can see Python DLLs. and all I do in my main() is a call to Py_Initialize ();
It seems to me the Py_Initialize () is reading this file using incorrect encoding.

What am I doing wrong?

Here is my small repro of the problem:
#include <stdio.h>

#include <Python.h>

int main (int argc, char* argv[])
{
int i, j, done = 0; /* variables I will need later */

Py_Initialize ();

return(0);
}

The compile command:
c++ -c -W"c,lp64,FLOAT(HEX)" -W"c,DFP,TUNE(7),ARCH(7)" -+ -W"c,LANG(EXTENDED),XPLINK(STOREARGS),XPLINK(BACKCHAIN)" -W"c,FEDBG(-qxflag=EnableDeclToTypeLinkagePropagation)" -W c,expo,ros -2 -W c,NOANS,gonumber -I/usr/lpp/python36/include/python3.6m -I/usr/lpp/python36/lib/python3.6/site-packages/numpy/core/include -o ptst.o ptst.c

Link script:
export _CXX_WORK_SPACE="(32000,(300,150))"
export _CXX_WORK_UNIT=“VIO”
c++ -Wl,lp64 -o ptst.out ptst.o /usr/lpp/python36/lib/libpython3.6m.x -lm -lc

I haven’t had a chance to look at this in detail, but you might try adding -qascii to the compile and link commands. Rocket tools use IBM’s enhanced ASCII support, so the programs use ASCII internally.

Thanks! Seems to work. Now gonna make it work in mixed environment - main compiled and linked as EBCDIC calls Python dll.s that are ASCII.

I strongly advise against that; I think you find debugging to be especially painful. Is there a reason not to compile the entire program as an ASCII-mode program?

Yes, there is a big reason to keep our code in EBCDIC. Our app is a huge portable application server that works on all platforms from Windows to mainframe and calls multiple third party APIs – several millions lines of code. So nobody is going to port it to ASCII in z/OS.
After some debugging of LE environment I was finally able to initialize Python environment successfully from my EBCDIC routine. I would really appreciate if you could be a little more specific and explain your concerns.

If your application is running on non-z/OS platforms, I would think porting it as an ASCII program would actually be easier. This has been our experience porting open source tools to z/OS.

My concern about mixing EBCDIC and ASCII libraries is that any given thread is in a single “mode” (ASCII or EBCDIC) at a given time, and I would think that when you cross library boundaries you would have to change the mode. Otherwise the automatic code translation enabled by _BPXK_AUTOCVT is going to be confused and do the wrong thing.

My comment about debugging is that, if you step through code in dbx, you will have to keep changing the character encoding mode of dbx depending where you are in the stack (via the $asciichars and $asciistrings variables). Also, if you debug by adding printf calls, if EBCDIC and ASCII portions of the code write to the same file descriptor, the encoding will be wrong for one or the other.