Upgrading to Git 2.26

I'm a bit confused about the instructions for upgrading Git from 2.14 to 2.26.

There are some notes:
Once you change a repository with Git for z/OS 2.26.2, this repository won't be compatible with previous versions of Git for z/OS. It is vital that entire team will move to the new version of Git
and
When you clone the repository with Git for z/OS 2.26.2, you need to run git status command. You might see that Git marks some files with the attribute as modified, even if you didn't modify them.
...
We recommend to update the files in the index (commands: git add ; git commit).

Typically different people will be working with separate clones of the repository and different branches, so at what point does the incompatibility occur?

Presumably you can still clone a repository and check out old commits with the old version?

Does the commit for files that 2.26 marked as modified interfere with merging branches, either on z/OS or via pull request?
What happens with e.g. git bisect?

I'm not sure what "It is vital that entire team will move to the new version of Git" actually means in practice. I fear that it means all branches need to be converted to the new version. Or worse, perhaps branches crossing the release can't be easily merged, so all existing branches need to be merged first.

Can you clarify, and hopefully set my mind at ease?

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

I have the same question. We have a LOT of git 2.14 level repositories and I would hate to see them corrupted by having some git users using 2.14 and others 2.26.

------------------------------
Lionel Dyck
lbdyck
------------------------------

I have the same question. We have a LOT of git 2.14 level repositories and I would hate to see them corrupted by having some git users using 2.14 and others 2.26.

------------------------------
Lionel Dyck
lbdyck
------------------------------

Hi Andrew,

As you may know, previous versions of ported Git supported only 2 encodings: ISO8859-1 and IBM-1047. We had high demand from customers to add UTF-8 support. That feature was implemented in z/OS Git 2.26.2.

What does UTF-8 support actually mean? It indicates the encoding of the repository and index themselves. Any commit with test files made with ported 2.26.2 will be stored in UTF-8 in the repository. In the previous versions the repository encoding was ISO8859-1. Git 2.14.4 doesn't have forward compatibility, so in some cases it will not be able to checkout UTF-8 commits properly. That's why we ask to migrate to the new version simultaneously.

If you move to Git 2.26.2, you will be able to checkout existing repositories. But once you push a commit, your colleagues, who are using previous versions, probably won't be able to checkout new commits. I say "probably" because the actual result is very dependent on the content of repository (which files are there, text or binary, whether it has .gitattributes, what the attributes are, etc).

The migration strategy also depends on the contents of repository. We are going to prepare a guide with examples for different initial states.

------------------------------
Tatiana Balaburkina
Rocket Software
------------------------------

Hi Andrew,

As you may know, previous versions of ported Git supported only 2 encodings: ISO8859-1 and IBM-1047. We had high demand from customers to add UTF-8 support. That feature was implemented in z/OS Git 2.26.2.

What does UTF-8 support actually mean? It indicates the encoding of the repository and index themselves. Any commit with test files made with ported 2.26.2 will be stored in UTF-8 in the repository. In the previous versions the repository encoding was ISO8859-1. Git 2.14.4 doesn't have forward compatibility, so in some cases it will not be able to checkout UTF-8 commits properly. That's why we ask to migrate to the new version simultaneously.

If you move to Git 2.26.2, you will be able to checkout existing repositories. But once you push a commit, your colleagues, who are using previous versions, probably won't be able to checkout new commits. I say "probably" because the actual result is very dependent on the content of repository (which files are there, text or binary, whether it has .gitattributes, what the attributes are, etc).

The migration strategy also depends on the contents of repository. We are going to prepare a guide with examples for different initial states.

------------------------------
Tatiana Balaburkina
Rocket Software
------------------------------

Migrating simultaneously is not a realistic proposition.

A git repository may be shared (cloned) by many different organizations. Also, an organization may have clones of repositories from many other organizations. So any requirement for a simultaneous migration cascades through the entire ecosystem.

More importantly, how does it work with branches?

If I have BranchA, BranchB, BranchC created with git 2.14.4 and later commit to all of them using 2.26.2 can I then merge them? Or do the separate commits to store UTF8 in each branch - which will be substantially but not entirely the same changes - create merge conflicts?

It would be better if repository encoding of UTF8 was explicitly selectable - then you could plan the change, start it in the main branch and migrate it to other branches via merge as for any other change.

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

Hi Andrew,

As you may know, previous versions of ported Git supported only 2 encodings: ISO8859-1 and IBM-1047. We had high demand from customers to add UTF-8 support. That feature was implemented in z/OS Git 2.26.2.

What does UTF-8 support actually mean? It indicates the encoding of the repository and index themselves. Any commit with test files made with ported 2.26.2 will be stored in UTF-8 in the repository. In the previous versions the repository encoding was ISO8859-1. Git 2.14.4 doesn't have forward compatibility, so in some cases it will not be able to checkout UTF-8 commits properly. That's why we ask to migrate to the new version simultaneously.

If you move to Git 2.26.2, you will be able to checkout existing repositories. But once you push a commit, your colleagues, who are using previous versions, probably won't be able to checkout new commits. I say "probably" because the actual result is very dependent on the content of repository (which files are there, text or binary, whether it has .gitattributes, what the attributes are, etc).

The migration strategy also depends on the contents of repository. We are going to prepare a guide with examples for different initial states.

------------------------------
Tatiana Balaburkina
Rocket Software
------------------------------

I was just wondering whether there was any additional information available on this subject yet, e.g. the effect of upgrading from 2.14.4 to 2.26.2 for a repository with multiple branches. Does it cause issues at merge time?

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

I was just wondering whether there was any additional information available on this subject yet, e.g. the effect of upgrading from 2.14.4 to 2.26.2 for a repository with multiple branches. Does it cause issues at merge time?

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

I also have this question.  This morning I was testing 2.26 and cloned from github a repo that was created using 2.14 and a PDS member containing an ISPF panel showed up as modified. I did a diff and here is a sample:


This is NOT good as it worked cleanly with 2.14 and apparently 2.26 has 'hosed' the data.

Please advise.

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

I also have this question.  This morning I was testing 2.26 and cloned from github a repo that was created using 2.14 and a PDS member containing an ISPF panel showed up as modified. I did a diff and here is a sample:


This is NOT good as it worked cleanly with 2.14 and apparently 2.26 has 'hosed' the data.

Please advise.

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

Here is another example of how messed up things are:

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

Here is another example of how messed up things are:

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

I think ISPF panels are a particular challenge with Git.

They are mostly text so it is very tempting to manage them as text files with the merge etc. advantages that brings, but they also contain binary data. The binary data means that treating them as text files is risky. While it might work most of the time, there are likely to be problems from time to time.

In this case the Git upgrade and change to the encoding seems to have triggered the corruption, but there are other things that can cause similar problems. For example, depending on the byte sequences a panel contains, checking out on Windows or Linux might also corrupt the panel due to line ending translation.

I think ISPF panels probably need to be treated as binary data in Git - because they do contain non-text bytes.

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

I think ISPF panels are a particular challenge with Git.

They are mostly text so it is very tempting to manage them as text files with the merge etc. advantages that brings, but they also contain binary data. The binary data means that treating them as text files is risky. While it might work most of the time, there are likely to be problems from time to time.

In this case the Git upgrade and change to the encoding seems to have triggered the corruption, but there are other things that can cause similar problems. For example, depending on the byte sequences a panel contains, checking out on Windows or Linux might also corrupt the panel due to line ending translation.

I think ISPF panels probably need to be treated as binary data in Git - because they do contain non-text bytes.

------------------------------
Andrew Rowley
Black Hill Software
------------------------------

It seems only a few ISPF panels had this issue but it wasn't apparent in 2.14 but is now.

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

Hey Lionel,

 

I'm assuming you have some hex non-display character as attributes between each of your pull-downs (File, Edit, Edit_Settings, etc), and I'm assuming its X'15' which is similar to panel ISREDDE2 where IBM uses X'15'  which basically means this panel cannot be managed by Git because X'15' is EBCDIC's NL (New Line) character.  It goes into Git OK but when retrieved its interrupted as a new line, like you are seeing.

 

I'm still using Git 2.14 and my .attribute file is

 

#                                                                               
#  Default .gitattributes file specifiing that all files are in EBCDIC          
#                                                                               
                                                                                
*                    zos-working-tree-encoding=ibm-1047 git-encoding=iso8859-1  
                                                                                
.gitattributes       zos-working-tree-encoding=iso8859-1 git-encoding=iso8859-1

 

With this combination the only problem I have is with the X'15' character.  All other characters go into and out of Git unchanged.

 

 

Regards / Ci Zhi Jing Li / Mit freundlichen Grüßen / Saudações
Gary Freestone

Senior z/OS Technical Specialist
Software Management Group Leader
GSD Platform Services
Integrated Technology Delivery
IBM Global Services Australia and New Zealand
Email: garyf@au1.ibm.com

Tie line: 67097083
Phone: +61-3-533-97083

This is the .gitattributes that ZIGI creates and which I'm using:

.gitattributes git-encoding=iso8859-1 zos-working-tree-encoding=iso8859-1

Had no issues with 2.14 - this issue showed up with 2.26 :(

Thus I will have to review all of the affected panels in my repositories and find alternate characters.

------------------------------
LionelDyck <><
https://github.com/zigi
https://github.com/lbdyck
------------------------------

Hey Lionel,

 

I'm assuming you have some hex non-display character as attributes between each of your pull-downs (File, Edit, Edit_Settings, etc), and I'm assuming its X'15' which is similar to panel ISREDDE2 where IBM uses X'15'  which basically means this panel cannot be managed by Git because X'15' is EBCDIC's NL (New Line) character.  It goes into Git OK but when retrieved its interrupted as a new line, like you are seeing.

 

I'm still using Git 2.14 and my .attribute file is

 

#                                                                               
#  Default .gitattributes file specifiing that all files are in EBCDIC          
#                                                                               
                                                                                
*                    zos-working-tree-encoding=ibm-1047 git-encoding=iso8859-1  
                                                                                
.gitattributes       zos-working-tree-encoding=iso8859-1 git-encoding=iso8859-1

 

With this combination the only problem I have is with the X'15' character.  All other characters go into and out of Git unchanged.

 

 

Regards / Ci Zhi Jing Li / Mit freundlichen Grüßen / Saudações
Gary Freestone

Senior z/OS Technical Specialist
Software Management Group Leader
GSD Platform Services
Integrated Technology Delivery
IBM Global Services Australia and New Zealand
Email: garyf@au1.ibm.com

Tie line: 67097083
Phone: +61-3-533-97083

Hi Gary,
I would like to know how do you cope with this problem of the X'15' character.
I am still using GIT 2.14 like you.

Best regards
Cris

Hi Gary,
I would like to know how do you cope with this problem of the X'15' character.
I am still using GIT 2.14 like you.

Best regards
Cris

Cristiano,

Unfortunately I haven't found a way around the X'15' problem :-(

The best I can do is, perform a test when a user attempts to save a member/dataset in ISPF edit display a message informing them of the problem. As seen here:

------------------------------
Gary Freestone
IBM Australia
------------------------------

Cristiano,

Unfortunately I haven't found a way around the X'15' problem :-(

The best I can do is, perform a test when a user attempts to save a member/dataset in ISPF edit display a message informing them of the problem. As seen here:

------------------------------
Gary Freestone
IBM Australia
------------------------------

Thank you Gary.
I think I'll follow your lead.

Cris

Thank you Gary.
I think I'll follow your lead.

Cris

@Tatiana Balaburkina

In your post dated 2021-03-25 you stated  "We are going to prepare a guide with examples for different initial states."   Has this been done, if so, where can I get a copy from ?

Gary Freestone  

------------------------------
Gary Freestone
Systems Programmer
Kyndryl Inc
Mt Helen Australia
------------------------------

@Tatiana Balaburkina

In your post dated 2021-03-25 you stated  "We are going to prepare a guide with examples for different initial states."   Has this been done, if so, where can I get a copy from ?

Gary Freestone  

------------------------------
Gary Freestone
Systems Programmer
Kyndryl Inc
Mt Helen Australia
------------------------------

Hi Gary,
We are working on the guide at the moment. We will announce as soon as it is ready. Thanks for your patience.

------------------------------
Tatiana Balaburkina
Rocket Internal - All Brands
------------------------------