Open-source Languages & Tools for z/OS

 View Only
  • 1.  Git for z/OS - Convert UTF-8 with BOM to EBCDIC

    Posted 08-15-2020 11:17

    I am using Git repositories whose sources are encoded in UTF-8 with a BOM (3 bytes EF BB BF at the start of the file).
    When I clone the repository with Git for z / OS, and a “* git-encoding=utf-8 zos-working-tree-encoding=ibm-1147” option declared in a “.gitattributes” file at the root of the repository, the sources are well converted to EBCDIC IBM-1147, but there is a parasitic x3F character at the beginning of the file.
    Is this problem known and there are plans to resolve it?

    Note: the “.gitattributes” file must imperatively be encoded in ISO-8859-1 in the “server” Git repository otherwise it is not recognized by Git for z / OS.

    UTF-8 with BOM:

    EBCDIC IBM-1147:

  • 2.  RE: Git for z/OS - Convert UTF-8 with BOM to EBCDIC

    Posted 08-18-2020 01:06

    Hi Denis,
    Git uses the iconv library to convert files. And this issue is a result of iconv conversion:

    echo “EFBBBF”| xxd -r -p | iconv -f utf-8 -t ibm-1147 > f_1147
    od -h f_1147

    You can see that f_1147 file has the x3F character. Iconv replaces all characters, which don’t have corresponding characters in the destination encoding, with the substitute character. On Git side there is no way to modify the iconv behavior. You shouldn’t use in the UTF-8 files characters, that don’t exist in the destination character set.