I have written a REXX exec to inface to GIT.

Specifically I use BPXWUNIX to issue my commands. The issue I have is the output from the command is always in ASCII (ISO-8859-1). Forcing me the perform a conversion before I can read the results.

This doesn’t happen if I issue the command in OMVS.

I’ve tried using BPXWUNIX using redirection

eg    x=BPXWUNIX('git status >output.txt',,,,,1)  

and coding the stdout in the function call

eg   x=BPXWUNIX('git status',,'output.',,,1) 

I have my profile setup with

export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _BPXK_AUTOCVT=ON
export _TAG_REDIR_ERR=TXT
export _TAG_REDIR_IN=TXT
export _TAG_REDIR_OUT=TXT

Does anyone know how I can get the system/git to perform the conversion to EBCDIC before I get the output ?

Regards,
Gary

I am facing the same kind of issues. Not even when just interfacing from REXX, but also straight from the command line. The failing command on my end is “git pull”.

It returns the data in ASCII and not EBCDIC, causing havoc in parsing output from commands like

git rev-parse --is-bare-repository

I made some changes in ‘git-sh-setup’ that make my git pull work, yet it does not feel like a ‘solid’ fix to me…

Changes are:

cd_to_toplevel () {                                                        
 cdup=$(git rev-parse --show-toplevel | iconv -f iso8859-1 -t ibm1047) &&  
 cd "$cdup" || {                                                           
  echo >&2 "Cannot chdir to $cdup, the toplevel of the working tree"       
  exit 1                                                                   
 }                                                                         
}                                                                          

require_work_tree_exists () {                                                                
 if test "z$(git rev-parse --is-bare-repository | iconv -f iso8859-1 -t ibm1047)" != zfalse  
 then                                                                                        
  die "fatal: $0 cannot be used without a working tree."                                     
 fi                                                                                          
}                                                                                            

git_dir_init () {                                                                
 GIT_DIR=$(git rev-parse --git-dir | iconv -f iso8859-1 -t ibm1047) || exit      
 if [ -z "$SUBDIRECTORY_OK" ]                                                    
 then                                                                            
  test -z "$(git rev-parse --show-cdup)" || {                                    
   exit=$?                                                                       
   echo >&2 "You need to run this command from the toplevel of the working tree."
   exit $exit                                                                    
  }                                                                              
 fi                                                                              

i works, but makes me feel kidna stupid :)…

Any other ideas>??

:facepalm:

Just make sure you use the correect bash and live will be good (without the iconvs)

How does one “use the correct bash” ? Could it be that GIT hasn’t been installed correctly ?

I’ve been having the same issue. I don’t know what “the correct bash” means either. I have PROGRAM=/rsusr/ported/bin/bash in my OMVS segment, and that’s what shows up when I do echo $SHELL on a command line.

I spent a lot of time trying to understand this stuff over the weekend. I still have no solution but I have a little better understanding.

It seems rocket has supplied two versions of bash. The “production” version is bash 4.2. I’ve found the following:

  /_PRDS/PTZ1/bash/share/doc/bash/README.ZOS

    Tool - bash-4.2                      
    Works only with the encoding EBCDIC  

Plus there is a version delivered in the git directories

 /_PRDS/PTZ1/git/bash/share/doc/bash/README.ZOS

    Tool - bash-4.3  
    Works with the encoding ISO-8859-1 and IBM-1047 

My conclusion the BASH delivered with git should work with both ASCII and EBCDIC files while the “production”/no-git version only support EBCDIC.

If I type “bash --version” in OMVS I get

$ bash --version
bash: FSUM7351 not found
$

This is because neither bash is in my path.

Here is another example: if I type the following on OMVS

echo 'Start GIT Status';  cd ~/zGIT-Repository/EBCDIC;  git status;  echo 'End GIT Status'; 

I receive the following:

Start GIT Status                                                                
On branch master                                                                
                                                                            
Initial commit                                                                  
                                                                            
Untracked files:                                                                
  (use "git add <file>..." to include in what will be committed)                
                                                                            
        .gitattributes                                                          
        EXEC/                                                                   
                                                                            
nothing added to commit but untracked files present (use "git add" to track)    
End GIT Status                                                                  

If I run the following REXX

command="echo 'Start GIT Status';cd :/zGIT-Repository/EBCDIC;git status;echo 'End GIT Status';"
rcode=bpxwunix(command,,'stdout.','stderr.','env.',1) 
do i=1 to stdout.0                                    
   say stdout.i                                       
end                                                   

I receive the following:

Start GIT Status                                                                                                                                               
|>:::/>:::_/:::::::>:::/%::?__:::::>::/:,:::::%::::::::::::::::/::::::%::::::::?::>:%:::::>:::/::::%%:::::?__:::::::::::::
/::::::::::::::::::>?:::>::/::::::?::?__::::::::>::/:,:::::%::::::::>::::::::::::/:::::?:::/:,::End GIT Status      

The start and end messages are in EBCDIC the output from GIT is in ASCII. Which makes it impossible to do anything with.

Regards,
Gary

Hi,
There is a file “Release notes for Git 2.3.5 for zOS.pdf” on our customer portal. It includes information about git dependecies:
“Git for z/OS requires the latest Rocket z/OS ports of: bash 4.3…”.
You CAN’T use git 2.3.5 with bash 4.2 or /bin/sh, because git 2.3.5 has enhanced ASCII support and bash 4.2 doesn’t.
So right steps should be:

1. Install bash-4.3 according to README.ZOS
2. Install git-2.3.5 according to README.ZOS
3. Be sure, that next settings are in your config (~/.profile or ~/.bashrc):
   export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
   export _BPXK_AUTOCVT=ON
   export _TAG_REDIR_ERR=TXT
   export _TAG_REDIR_IN=TXT
   export _TAG_REDIR_OUT=TXT
4. Run bash-4.3
5. Run git-2.3.5 over bash-4.3

yep. All done. Although I believe the “TXT” should be in lower case.

Its using bash 4.3 as supplied in the git directories

We have a wrapper for git that looks like this

#!/bin/sh

name=$( basename $0 )
dir=$( dirname $0 )

ptz_dir="$dir/…/…"

#export _BPXK_AUTOCVT=ON

. $dir/_gitenv.sh $ptz_dir

if [[ -x $GIT_BIN/$name ]]; then
."$GIT_BIN/$name" "$@"
elif [[ -x $GIT_EXEC/$name ]]; then
.$GIT_EXEC/$name "$@"
else
echo "unsupported call to $name"
fi

and _gitenv.sh looks like this

#! /bin/sh

if [[ $# > 0 ]]; then
TGTDIR="$1"
else
TGTDIR="/_PRDS/PTZ"
fi

ZOS_GIT_DIR=$TGTDIR/git
GIT_BIN=$ZOS_GIT_DIR/bin

#for git use: source in bash, perl and unzip environment
. $TGTDIR/etc/profile.d/unzip.sh $TGTDIR

#. $TGTDIR/etc/profile.d/perl.sh $TGTDIR
#. $TGTDIR/etc/profile.d/bash.sh $TGTDIR
#NOTE: for the time being (i.e. as of April 2017) we deliver bash and perl inside git
#as both are not yet ready for general use
export PATH="$ZOS_GIT_DIR/bash/bin:$ZOS_GIT_DIR/perl/bin:$PATH"
export LIBPATH="${TGTDIR}/perl/lib/perl5/5.22.0/os390/CORE:$LIBPATH"
export PERL5LIB="${TGTDIR}/perl/lib/perl5:$PERL5LIB"

#. $TGTDIR/etc/profile.d/bash.sh
#for the time being bash 4.3 isn’t usable outside of git
#there we cannot use the usual sourcing mechanism
export PATH=$ZOS_GIT_DIR/bash/bin:$PATH

export GIT_SHELL=$TGTDIR/bash/bin/bash

#-z returns true if the string is empty
if [[ -z ${GIT_TEMPLATE_DIR:-} ]]; then
export GIT_TEMPLATE_DIR=$ZOS_GIT_DIR/share/git-core/templates
fi

export GIT_EXEC_PATH=$ZOS_GIT_DIR/libexec/git-core

#make our wrapper scripts available
export PATH=$PATH:$ZOS_GIT_DIR

#MANPATH=$MANPATH:$ZOS_GIT_DIR/man
#perl related
export PERL5LIB=$PERL5LIB:$ZOS_GIT_DIR/lib/perl5

#ASCII support
export _BPXK_AUTOCVT=ON
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt

Regards,
Gary

I have created a sample program that I’m hoping someone will run on their system and publish the results here.

/* REXX */
env.0=1
env.1='HOME=/u/u700215’
command=“echo ‘Start GIT Status’;”,
“cd ~/zGIT-Repository/COPYX;”,
“git status;”,
"echo ‘End GIT Status’;"
rcode=bpxwunix(command,‘stdout.’,‘stderr.’,‘env.’,1)
if rcode>0 then say 'Rcode from bpxwunix='rcode
say
say copies(‘STDOUT–’,5)
do i=1 to stdout.0
say stdout.i
end
say copies(‘STDERR–’,5)
do i=1 to stderr.0
say stderr.i
end

Line 3 will need to be undated with your home directory,
Line 5 should be changed so the “cd” points to a git repository.

My results are:

STDOUT–STDOUT–STDOUT–STDOUT–STDOUT–
Start GIT Status
|>:::/>:::_/:::::>?:::>:::?::?__:::::?:,:>::::::::?::::%:/>:End GIT Status
STDERR–STDERR–STDERR–STDERR–STDERR–

What do you get ???

Regards,
Gary

Hi Gary,

git works over bash-4.3. When you run git via REXX this way, bash-4.3 is not started and these variables are not exported:

export _BPXK_AUTOCVT=ON
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt

Hi Tatyana,

Thanks for the feedback. So what do you suggest ?

While I’m no Unix expert, I believe our wrapper code does setup bash 4.3. How can I tell ?

If bash 4.3 is not being used via this REXX, what do I need to do get it working ?

Regards,
Gary

I tried to figure out what’s going wrong.

So I created ~/zGIT-Repository/COPYX and there I did a git init.

Then I wrote a script setting all the variables as given in git/share/doc/git/2.3.5/README.ZOS

#! /bin/sh
...
git status

When calling it from UNIX shell output is:

On branch master

Initial commit

nothing to commit (create/copy files and use “git add” to track)

So all seems good.

Now I called above script using Gary’s REXX, and I get ASCII garbage as output from the git status command.

Now comes the even stranger part:

When I insert a curl command like this

curl -s http://www.yahoo.com

just before the git command then the REXX runs fine, i.e. no garbage at all.

I have no idea why something like this happens.

Hi Manfred,

So what you are saying is if you pass the following string to to BPXWUNIX

echo ‘Start GIT Status’ ; cd ~/zGIT-Repository/COPYX ; git status ; echo ‘End GIT Status’

The output from the git command comes back in ASCII, but if you pass the string

echo ‘Start GIT Status’ ; cd ~/zGIT-Repository/COPYX ;curl -s http://www.yahoo.com ; git status ; echo ‘End GIT Status’

The output from the git command comes back in EBCDIC

Is that correct ?

Gary

Hi Gary,
Admittedly confusing but this is what I observed. Actually, I have no consistent theory what’s going on here.

Manfred

Hi Tatyana,
You could test like follows:

1. Have git, bash, perl, unzip and curl all in a certain directory tree. In my case it is /u/manfred/rocket.

2. In /u/manfred/rocket I have a script git_env which contains:

ROCKET=/u/manfred/rocket
export PATH=$ROCKET/bin:$PATH
export _BPXK_AUTOCVT=ON
export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt

and use this to source in the environment, i.e. . /u/manfred/rocket/rocket_env

3. Go to a directory with a git repository and run
   git status

This should worke fine.

4. In a batch job run this REXX:

/* REXX */
env.0=1
env.1=‘HOME=/u/manfred’

/* This is a test case to show Rocket that something is wrong
*/

BASE = “/u/manfred"
ROCKET_BASE = BASE||”/rocket"

command=". “ROCKET_BASE”/rocket_env;",
‘curl’,
‘http://www.yahoo.com;’,
“cd /u/manfred/zGIT-Repository/COPYX;”,
“git status;”,
“echo ‘End GIT Status’;”

rcode=bpxwunix(command,‘stdout.’,‘stderr.’,‘env.’,1)
if rcode>0 then say 'Rcode from bpxwunix='rcode
say
say copies(‘STDOUT-’,5)
do i=1 to stdout.0
say stdout.i
end
say copies(‘STDERR-’,5)
do i=1 to stderr.0
say stderr.i
end

This should run fine as well. Note that there is an invocation of curl before doing git status

5. Delete the two curl related lines and run the job again.
   Now git output is ASCII gibberish.

If you need a sample of a complete batch job just tell me. I could post it here or send it do you via mail. Whatever you like.

Manfred

Just WOW on the activity and good replies here

Still a bit ‘untangible’ as to where it fails and why. I have no different results with bash 4.2 or 4.3. So for now the iconv-fix is making it work…

If you want a ‘working’ experience too, I’ve git-gisted my solution here : https://gist.github.com/wizardofzos/c588e203aa14b80576712d664406df11

The internal issue is opened for this case.

A colleague of mine had a very good idea. He said: Check what curl and git do to STDOUT and STDERR in terms of tagging.

I did and found that curl changes tagging to

STDOUT: 1047 8000
STDERR: 1047 8000

This is the reason that git produces readable output when calling curl first.

When calling git without prior invocation of curl no tagging takes place that is before and after calling git we have:

STDOUT: 0000 0000
STDERR: 0000 0000

Excellent Manfred.

Is there anything I can do to change the tagging for GIT or is this an internal thing that Rocket has to do to the GIT product internally?

Regards,
Gary

Hi Gary,

This problem occurs in git version, which was released more than in 2016. cURL uses new internal library, so this problem doesn’t exist in cURL. We are working on porting new git version (2.14), it will be released soon. The best way is to wait for new version.

Thanks,
Tatyana

For the time being, as a workaround I told Gary to do a

curl --version

before invoking git. This seems to be sufficient to get a readable output from git.

I should add that simply tagging stdout and stderr as 1047 and text before invoking git isn’t enough to get readable output from git. Therefore, the workaround above.

Hi Manfred,

Thanks for your participation on the forum =)

Any update on this “internal issue” ?

Hi Gary,

The fix will be available for 2.14 git version.

Thanks,
Sergey