Open-source Languages & Tools for z/OS

 View Only

  • 1.  Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    Posted 10-18-2022 05:41
    Hi,

    We use in our Jenkins pipelines Git commands triggered by a Groovy script of the form:

    echo sh(script:'git status', returnStdout: true)

    When this command is triggered directly from the JenkinsFile file, the response is indeed retrieved in clear plain text:


    On the other hand, when this command is triggered from a Jenkins SharedLib, then the response is unintelligible, probably because of an EBCDIC / ASCCI-UTF-8 conversion problem:



    We do not understand where the problem comes from and therefore how to solve it...

    The issue is specific to Git for z/OS, as the other commands (vim in the example) show correct responses.

    We also tested saving the git response to a file on the Jenkins workspace:

    git status >output.txt

    The output.txt file is correctly readable, and created on z/OS-USS with an ISO8859-1 ccsid.


    User profile environment variables:

    export _BPXK_AUTOCVT=ON
    export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
    export _TAG_REDIR_ERR=txt
    export _TAG_REDIR_IN=txt
    export _TAG_REDIR_OUT=txt

    export PATH=/global/SYS9/OAD/bin:$PATH
    export LIBPATH=/global/SYS9/OAD/lib:$LIBPATH
    export MANPATH=/global/SYS9/OAD/share/man:$MANPATH

    git config --global http.sslVerify false
    git config --global core.pager ''

    export GIT_SHELL=/global/SYS9/OAD/bin/bash
    export GIT_EXEC_PATH=/global/SYS9/OAD/libexec/git-core
    export GIT_TEMPLATE_DIR=/global/SYS9/OAD/share/git-core/templates

    export IBM_JAVA_ENABLE_ASCII_FILETAG=ON

    export DBB_HOME=/global/SYS9/DBB
    export DBB_CONF=/var/DBB/conf
    export GROOVY_HOME=$DBB_HOME/groovy-2.4.12
    export CLASSPATH=$CLASSPATH:$DBB_CONF
    export PATH=$GROOVY_HOME/bin:$DBB_HOME/bin:$JAVA_HOME/bin:$PATH

    Jenkins agent configuration:

    IBM_JAVA_COMMAND_LINE=java -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/u/JENKINS/tmpdir -Xnoargsconversion -Dorg.jenkinsci.plugins.gitclient.CliGitAPIImpl.user.name.file.encoding=IBM-1047 -Dorg.jenkinsci.plugins.gitclient.CliGitAPIImpl.user.password.file.encoding=IBM-1047 -Djavax.net.ssl.trustStore=/u/JENKINS/articacerts -Djavax.net.ssl.trustStorePassword=changeit -jar remoting.jar -workDir /u/JENKINS -jar-cache /u/JENKINS/remoting/jarCache -text

    ------------------------------
    Fallai Denis
    M
    Self Registered
    L'ISLE-ADAM FR
    ------------------------------


  • 2.  RE: Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    ROCKETEER
    Posted 10-18-2022 06:04
    Hi Fallai,
    Just as an idea add 'env | sort' call to the script and compare the variables for both cases.
    Thanks,
    Alex

    ------------------------------
    Alexander Klochkov
    QA
    Rocket Internal - All Brands
    Vilnius LT
    ------------------------------

    Original Message


  • 3.  RE: Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    Posted 10-18-2022 06:29

    Hi,

    Thanks for your answer.

    We did the comparison of environment variables in the Jenkinsfile case and the Sharedlib case: same values as far as the USS context is concerned.



    ------------------------------
    Fallai Denis
    M
    Self Registered
    L'ISLE-ADAM FR
    ------------------------------

    Original Message


  • 4.  RE: Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    Posted 10-18-2022 09:12

    Hi,

    We may have identified the context that creates the problem (which a priori only concerns scripts in SharedLib): the sysout must have already been created before the Git command to write something intelligible in this sysout.

    For example, if we precede the Git command with another "echo" command which writes in sysout, then the result of the Git command is usable!

    echo sh(script: 'git status', returnStdout: true)

    + git status
    [Pipeline] echo
    çá à€ÀÁÈ/ÄÇÁÀ€/È€'ÃÃÄŽäÇ/>ÅÁË€>?È€ËÈ/ÅÁÀ€Ã?Ê€Ä?__ÑÈšŽ€€ˆÍËÁ€‚ÅÑÈ€/ÀÀ€ÃÑ%Áž‚€È?€ÍøÀ/ÈÁ€ÏÇ/È€ÏÑ%%€ÂÁ€Ä?__ÑÈÈÁÀ‰Ž€€ˆÍËÁ€‚ÅÑÈ€ÊÁËÈ?ÊÁ€ÃÑ%Áž‚€È?€ÀÑËÄ/ÊÀ€ÄÇ/>ÅÁË€Ñ>€Ï?Ê,Ñ>Å€ÀÑÊÁÄÈ?Ê`‰Ž_?ÀÑÃÑÁÀš€€€ÊÁ/À_Á_ÀŽŽí>ÈÊ/Ä,ÁÀ€ÃÑ%ÁËšŽ€€ˆÍËÁ€‚ÅÑÈ€/ÀÀ€ÃÑ%Áž‚€È?€Ñ>Ä%ÍÀÁ€Ñ>€ÏÇ/È€ÏÑ%%€ÂÁ€Ä?__ÑÈÈÁÀ‰ŽÅÑÈàáâíå%?ÅŽÈÁËÈ%?ÅŽŽ>?€ÄÇ/>ÅÁË€/ÀÀÁÀ€È?€Ä?__ÑÈ€ˆÍËÁ€‚ÅÑÈ€/ÀÀ‚€/>À?Ê€‚ÅÑÈ€Ä?__ÑÈ€/‚‰Ž


    echo sh(script: 'echo ""; git status', returnStdout: true)

    + echo
    + git status
    [Pipeline] echo

    HEAD detached at 701f2fc
    Changes not staged for commit:
    (use "git add <file>..." to update what will be committed)
    (use "git restore <file>..." to discard changes in working directory)
    modified: readme.md

    Untracked files:
    (use "git add <file>..." to include in what will be committed)
    gitDEBUG.log
    test.log

    no changes added to commit (use "git add" and/or "git commit -a")


    Strange ?



    ------------------------------
    Fallai Denis
    M
    Self Registered
    L'ISLE-ADAM FR
    ------------------------------

    Original Message


  • 5.  RE: Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    Posted 10-19-2022 05:06

    Corrigendum

    Hi,

    After detailed analysis of our direct Jenkinsfile scripts, we realized that Git commands which produce output are systematically preceded by another non-Git command which also produces output and which initializes the file which is retrieved by the " returnStdout:yes".

    So the problem raised concerns both Git commands placed at the Jenkinsfile level and orders placed at the Sharedlib level.

    Sorry for the wrong information given at the start.



    ------------------------------
    Fallai Denis
    M
    Self Registered
    L'ISLE-ADAM FR
    ------------------------------

    Original Message


  • 6.  RE: Git for z/OS - Problem retrieving responses to Git commands if launched from a Jenkins SharedLib

    ROCKETEER
    Posted 10-26-2022 05:56

    Hi Fallai,

    I tried to use echo sh(script:'git status', returnStdout: true) for my  tst1 pipeline and have the same problem, but
    sh 'git status'
    or 
    echo sh(script:'git status', returnStdout: false)
    are both ok and have no garbled output. Any reason to use returnStdout: true?

    I did some research with returnStdout: true:
    When you run echo sh(script:'someprogramhere', returnStdout: true) Jenkins creates output.txt in tmp folder of your pipeline (in my case it was workspace/tst1@tmp folder). The output .txt file is created in the same encoding in which the program was compiled. In the case of git it was 't ISO8859-1 T=on output.txt' in the case of echo or some other EBCDIC program, it was 't IBM-1047 T=on output.txt'.
    This file is readable in both cases and has the correct content and tag. So the problem here is not in the git, it's Jenkins's problem, it cannot correctly print the file to the console output.

    Thanks,
    Alexander



    ------------------------------
    Alexander Klochkov
    QA
    Rocket Internal - All Brands
    Vilnius LT
    ------------------------------

    Original Message