Open-source Languages & Tools for z/OS

 View Only

  • 1.  Git for z/os - Chinese Character code page 935 translation problem

    PARTNER
    Posted 09-14-2021 06:35

    hi dear Rocket supporter,

    We are testing Git for z/os to try to migrate our mainframe source code from ClearCase to Git. As we are a banking in China, so we need to make sure Git for z/os support Chinese character. We tried the below configuration:


    Then, I update the txt file on git server,  add a new line with "中国银行" which are four Chinese characters, as below

    Then, I checkout the update on z/os uss, I got the wrong code page translation, the ending '0f' is lost.

    fyi, you can see "中国" is correctly displayed, because I added the "中国" from the z/os uss end, then push to git server. But if I do the update from the git server end and then checkout the update from the z/os uss end, the error occurred. 

    Please could you help to take a look at this problem? Thanks a lot!

    Xiaohang Wang
    IBM z software presales



    ------------------------------
    Doria Wang
    client support
    IBM China Company Limited
    BEI JING China
    ------------------------------


  • 2.  RE: Git for z/os - Chinese Character code page 935 translation problem

    ROCKETEER
    Posted 09-14-2021 12:10

    Hi Doria,

    Thank you for reporting the issue. I've tried to reproduce it, but it wasn't successfully yet.
    Could you say which version and build of git you are using and send the output of 'env | sort' command if possible?

    Thanks,
    Sergey



    ------------------------------
    Sergey Rezepin
    Rocket Software
    ------------------------------

    Original Message


  • 3.  RE: Git for z/os - Chinese Character code page 935 translation problem

    PARTNER
    Posted 09-14-2021 22:38
    hi Sergey,

    Thanks for your prompt response, please check the below info, thanks so much!
    (gitenv) -bash-4.3¥ git version
    git version 2.26.2-78

    (gitenv) -bash-4.3¥ env|sort
    ANT_HOME=/u/rocket/miniconda/envs/gitenv
    ANT_HOME_CONDA_BACKUP=
    ASCII_TERMINFO=/u/rocket/miniconda/envs/gitenv/share/terminfo
    COLUMNS=160
    CONDA_DEFAULT_ENV=gitenv
    CONDA_EXE=/u/rocket/miniconda/bin/conda
    CONDA_PREFIX=/u/rocket/miniconda/envs/gitenv
    CONDA_PROMPT_MODIFIER=(gitenv)
    CONDA_PYTHON_EXE=/u/rocket/miniconda/bin/python
    CONDA_SHLVL=1
    CURL_CA_BUNDLE=/u/rocket/miniconda/envs/gitenv/etc/ssl/cacert.pem
    FFI_LIB=/u/rocket/miniconda/envs/gitenv/lib/ffi
    GIT_EXEC_PATH=/u/rocket/miniconda/envs/gitenv/libexec/git-core
    GIT_SHELL=/u/rocket/miniconda/envs/gitenv/bin/bash
    GIT_TEMPLATE_DIR=/u/rocket/miniconda/envs/gitenv/share/git-core/templates
    HOME=/u/tmp/tso03
    JAVA_HOME=/usr/lpp/java/J8.0_64
    LANG=C
    LIBPATH=/u/rocket/miniconda/envs/gitenv/lib:/u/rocket/miniconda/lib:/lib:/usr/lib:.:/u/rocket/miniconda/envs/gitenv/lib/perl5/5.24.4/os390/CORE
    LINES=58
    LOGNAME=TSO03
    MAIL=/usr/mail/TSO03
    MANPATH=/u/rocket/miniconda/envs/gitenv/share/man:/u/rocket/miniconda/envs/gitenv/man:/usr/man/%L
    NLSPATH=/usr/lib/nls/msg/%L/%N
    OLDPWD=/u/tmp/tso03
    OPENSSL_CONF=/u/rocket/miniconda/envs/gitenv/ssl/openssl.cnf
    PATH=/u/rocket/miniconda/envs/gitenv/bin:/usr/lpp/java/J8.0_64/bin:/bin
    PERL5LIB=/u/rocket/miniconda/envs/gitenv/lib/perl5:/u/rocket/miniconda/envs/gitenv/lib/perl5/site_perl
    PWD=/u/tmp/tso03/test001
    PYTHON_DVERSION=python3.7
    PYTHON_VERSION=python37
    PYV=py37
    SHELL=/u/rocket/miniconda/bin/bash
    SHLVL=1
    SSL_CERT_DIR=/u/rocket/miniconda/envs/gitenv/ssl/certs
    SSL_CERT_FILE=/u/rocket/miniconda/envs/gitenv/ssl/cert.pem
    STEPLIB=none
    TERM=dumb
    TZ=TAIST-8
    _=/bin/env
    _BPXK_AUTOCVT=ON
    _BPX_TERMPATH=OMVS
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _TAG_REDIR_ERR=TXT
    _TAG_REDIR_IN=TXT
    _TAG_REDIR_OUT=TXT


    ------------------------------
    Doria Wang
    client support
    IBM China Company Limited
    BEI JING China
    ------------------------------

    Original Message


  • 4.  RE: Git for z/os - Chinese Character code page 935 translation problem

    PARTNER
    Posted 09-14-2021 22:47
    hi Sergey,

    The problem looks same as this apar https://www.ibm.com/support/pages/apar/IV07477, which for java.
    You can reference this apar to check whether we have same problem in python code. Thanks so much!

    ------------------------------
    Doria Wang
    client support
    IBM China Company Limited
    BEI JING China
    ------------------------------

    Original Message


  • 5.  RE: Git for z/os - Chinese Character code page 935 translation problem

    ROCKETEER
    Posted 09-15-2021 12:02
    Hi Doria,

    Thank you for the detail information, I reproduced the behavior with 'git merge' locally, without using a git server. Git truncates the last SHIFT_IN (\x0f) character when merging for adding the new line with the double byte characters.

    It looks, the closing SHIFT_IN in the end of file is not necessary required for the conversion from IBM-935 to UTF-8 back properly, because the file converts well, but the editor mentioned in your post can't read the last sentence correctly, because it adds single byte \x40 characters after that. In this case, SHIFT_IN is required to split double bytes and single bytes characters for correct reading. Actually, the original file have no \x40 characters at the end of file.

    Anyway, we need to research this issue. Now, I can only recommend to add a single byte character at the end of the file as a workaround. For example, add 'new line' character at the end of file after adding a last line with double byte characters. It will help to avoid the issue.

    Thanks,
    Sergey

    ------------------------------
    Sergey Rezepin
    Rocket Software
    ------------------------------

    Original Message