z/OS Tools & Language

Expand all | Collapse all

git anomaly on test lpar

  • 1.  git anomaly on test lpar

    Posted 03-01-2021 21:13

    I'm having an issue on a test lpar with git for z/OS – both version 2.14 and the latest 2.26 ports. 

    I've looked at this as have several others without success. 

    When I attempt a git clone I get this error: 

    fatal: protocol error: bad line length character: ---Á  

    When I do a ssh git@github.com git-receive-pack lbdyck/qtab.git I get this:

    --ÄÁ---Ä--Ã----Ã-À--ÄÂÂ--Á-/ÄÁ-------À----/--ÊÁÃËÇÁ/ÀË_/ËÈÁÊÊÁø?ÊÈ-ËÈ/ÈÍË-ÊÁø?ÊÈ-ËÈ/ÈÍË-Î--ÀÁ%ÁÈÁ-ÊÁÃË-ËÑÀÁ-Â/>À---,-ÉÍÑÁÈ-/È?_ÑÄ-?Ã

    Ë-ÀÁ%È/-øÍËÇ-?øÈÑ?>Ë-?¦ÁÄÈ-Ã?Ê_/È-ËÇ/--/ÅÁ>È-ÅÑÈÅÑÈÇÍÂ-Å-ÃÃ-Ä/À----------                          

     

    If I do the same ssh git@github.com git-receive-pack lbdyck/qtab.git on one of my other systems it looks like this:

    00ce342c27f0363f6d24cbb68e8ace7454218d4828a2 refs/heads/masterreport-status report-status-v2 delete-refs side-band-64k quiet atomic

    ofs-delta push-options object-format=sha1 agent=git/github-g2ff1cad44179   

    I have these in the environment:

    _BPXK_AUTOCVT=ON
    _CEE_RUNOPTS=FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)
    _TAG_REDIR_IN=txt    
    _TAG_REDIR_OUT=txt   
    _TAG_REDIR_ERR=txt    

    We've looked at the TCP/IP setup, the SSH setup, and the Git setup with no joy.  We've also confirmed that the ssh public key has been properly added to github.

     

    Can anyone provide any suggestions that will help resolve this?



    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------


  • 2.  RE: git anomaly on test lpar

    Posted 03-02-2021 04:30
    Hi Lionel,

    Disclaimer: I am just a regular git for z/OS user.

    I have the same environment variables set as you and does not have any issues (I am using gitlab not github, though).

    Do you have the same version of ssh on both LPARs? (I assume "my other systems" refer to other z/OS systems?)

    I've previously used
    export GIT_SSH_COMMAND="ssh -v -v"
    to debug ssh related issues.



    ------------------------------
    Joern Thyssen
    Rocket Software
    ------------------------------



  • 3.  RE: git anomaly on test lpar

    Posted 03-02-2021 08:09
    I have the same SSH on all systems.

    I tried what you suggested and I get this on the bad lpar - strange.

    debug1: identity file /u/lbdyck/.ssh/id_rsa type 0
    debug1: key_load_public: EDC5129I No such file or directory. (errno2=0x05620062)
    debug1: identity file /u/lbdyck/.ssh/id_rsa-cert type -1
    debug1: key_load_public: EDC5129I No such file or directory. (errno2=0x05620062)
    debug1: identity file /u/lbdyck/.ssh/id_dsa type -1
    debug1: key_load_public: EDC5129I No such file or directory. (errno2=0x05620062)

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 4.  RE: git anomaly on test lpar

    Posted 03-02-2021 08:10
    Then later I see

    debug2: key: /u/lbdyck/.ssh/id_rsa (B17EAA8)
    debug2: key: /u/lbdyck/.ssh/id_dsa (0)
    debug2: key: /u/lbdyck/.ssh/id_ecdsa (0)
    debug2: key: /u/lbdyck/.ssh/id_ed25519 (0)

    strange.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 5.  RE: git anomaly on test lpar

    Posted 03-02-2021 08:12
    And further down this:

    debug1: Authentication succeeded (publickey).
    Authenticated to github.com ([140.82.113.4]:22).

    Looks good - no anomalies here.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 6.  RE: git anomaly on test lpar

    Posted 03-03-2021 21:54
    We're stumped by this! Is the problem LPAR in a sysplex with the LPARs that work? Can you dump the BPXPRMnn parameter member?

    ------------------------------
    David Crayford
    Senior Software Engineer
    Rocket Software
    ------------------------------



  • 7.  RE: git anomaly on test lpar

    Posted 03-04-2021 06:33
    How about a D OMVS,O

    OMVS 000F ACTIVE OMVS=(00,FS,F2,GT)
    CURRENT UNIX CONFIGURATION SETTINGS:
    MAXPROCSYS = 900 MAXPROCUSER = 25
    MAXFILEPROC = 64000 MAXFILESIZE = NOLIMIT
    MAXCPUTIME = 1000 MAXUIDS = 200
    MAXPTYS = 800 MAXIOBUFUSER = 2048
    MAXMMAPAREA = 40960 MAXASSIZE = 209715200
    MAXTHREADS = 200 MAXTHREADTASKS = 1000
    MAXCORESIZE = 4194304 MAXSHAREPAGES = 131072
    IPCMSGQBYTES = 2147483647 IPCMSGQMNUM = 10000
    IPCMSGNIDS = 500 IPCSEMNIDS = 500
    IPCSEMNOPS = 25 IPCSEMNSEMS = 1000
    IPCSHMMPAGES = 25600 IPCSHMNIDS = 500
    IPCSHMNSEGS = 500 IPCSHMSPAGES = 262144
    SUPERUSER = BPXROOT FORKCOPY = COPY
    STEPLIBLIST =
    USERIDALIASTABLE=
    PRIORITYPG VALUES: NONE
    PRIORITYGOAL VALUES: NONE
    MAXQUEUEDSIGS = 1000 SHRLIBRGNSIZE = 67108864
    SHRLIBMAXPAGES = 4096 VERSION = / ,N
    SYSCALL COUNTS = NO TTYGROUP = TTY
    SYSPLEX = NO BRLM SERVER = N/A
    LIMMSG = NONE AUTOCVT = OFF
    RESOLVER PROC = RESOLVER LOSTMSG = ON
    AUTHPGMLIST = NONE
    SWA = BELOW NONEMPTYMOUNTPT = NOWARN
    SERV_LINKLIB =
    SERV_LPALIB =
    MAXUSERMOUNTSYS = 0 MAXUSERMOUNTUSER= 0
    MAXPIPEUSER = 8730 PWT = ENV
    KERNELSTACKS = ABOVE UMASK = NONE
    SC_EXITTABLE =

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 8.  RE: git anomaly on test lpar

    Posted 03-05-2021 15:20
    Hi Lionel,

    This looks like a problem with SSH configuration, probably unrelated to Git altogether. In your command 'ssh git@github.com git-receive-pack lbdyck/qtab.git', z/OS SSH client (/bin/ssh) runs the command git-receive-pack remotely on GitHub (that is, GitHub's own version of git-receive-pack), and for some reason interprets the output in a wrong encoding.

    Can you please check a couple more things on the 'bad' LPAR? 

    1. Try 'git clone' with a different server, not github.com

    2. If you have a Linux machine, try to SSH to it from the mainframe. On the bad LPAR, issue the following command:

    ssh linux-user@your-linux-machine-name env

    This should run the 'env' command on Linux; even though Linux produces ASCII output, ssh should display it correctly in the terminal. Unfortunately this is not going to work in TSO OMVS; ssh cannot input passwords from 3270 terminals. You'll need to connect to mainframe via Putty or a similar terminal.

    Regards,
    Vladimir


    ------------------------------
    Vladimir Ein
    Rocket Software
    ------------------------------



  • 9.  RE: git anomaly on test lpar

    Posted 03-05-2021 16:50
    Unfortunately this LPAR in question is behind a firewall and SSH is not allowed from outside :(

    I will try to setup a different git server and try over the weekend.  I have no linux machine that I have access to from this LPAR.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 10.  RE: git anomaly on test lpar

    Posted 03-05-2021 17:14
    I was able to ssh from the bad lpar to a different z/OS lpar using the ssh userid@xxx.xxx.xxx.xx env

    and the results came back as ascii.

    I saved the results in a file and looked at it as both ebcdic and as ascii - and the data was definitely ascii.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 11.  RE: git anomaly on test lpar

    Posted 03-07-2021 13:47
    Found the issue - on this system (z/OS 2.4) the zos_ssh_config had

    ChannelConvert shell

    It should have had 

    ChannelConvert shell,exec

    Making this change and then doing this solved the problem
    kill -s HUP $(cat /var/run/sshd.pid)


    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 12.  RE: git anomaly on test lpar

    Posted 03-08-2021 00:08

    Thanks for letting us know Lionel.Did you need to specify exec to get it to work with bpxwunix()?

    Can you also post your resolution to the IBMMAIN thread for the archives?

    Thanks



    ------------------------------
    David Crayford
    Senior Software Engineer
    Rocket Software
    ------------------------------



  • 13.  RE: git anomaly on test lpar

    Posted 03-08-2021 08:39
    One other interesting fact on this.  On another z/OS 2.4 LPAR there is no problem and there is no zos_ssh_config file.

    Curious....

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 14.  RE: git anomaly on test lpar

    Posted 03-08-2021 09:04
    Hi Lionel,

    The default is shell,exec according to
    https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.4.0/com.ibm.zos.v2r4.foto100/fotz1141.htm

    ------------------------------
    Joern Thyssen
    Rocket Software
    ------------------------------



  • 15.  RE: git anomaly on test lpar

    Posted 03-08-2021 09:20
    That would explain why it works on the system without the zos_ssh_config.  Why it was created on the system I had the issue on is a question that probably won't get an answer.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 16.  RE: git anomaly on test lpar

    Posted 03-08-2021 09:28
    Edited by Joern Thyssen 03-08-2021 09:30
    What if the sysprog used git to version control the config files and parmlib members... :) Infrastructure-as-code...

    ------------------------------
    Joern Thyssen
    Rocket Software
    ------------------------------



  • 17.  RE: git anomaly on test lpar

    Posted 03-08-2021 09:35
    that would be an interesting exercise.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------



  • 18.  RE: git anomaly on test lpar

    Posted 03-08-2021 10:54
    I think long term it is a necessary exercise.

    The process should really be:
    1) Sysprog clones git repo 
    2) Sysprog changes ssh config file; git commit + git push
    3) Sysprog starts pipeline (maybe ansible playbook) to deploy updated config (in this case it should deploy to /etc/ssh and issue the kill to refresh the daemon)

    This is business-as-usual for developers and the sysprog is really just a infrastructure-developer :)

    ------------------------------
    Joern Thyssen
    Rocket Software
    ------------------------------



  • 19.  RE: git anomaly on test lpar

    Posted 03-08-2021 14:45
    I agree it would be worthy exercise. I would strong encourage the git clone repo to be a unique directory and not the production location for those files to prevent overlaying the production files ​with an update from another system that may not work on the local system.  I would like to see a 'read-only' repository which could reference the production ssh configs and other key files (perhaps most of /etc) that would add/commmit/push the updates but would not allow a pull.

    Just me being very cautious.

    ------------------------------
    Lionel Dyck
    lbdyck
    ------------------------------