Free Open Mainframe Summit Event Tickets - share your z/OS Open Source Story

Next Wednesday and Thursday (September 22 & 23) is the Linux Foundation Open Mainframe Summit, and our own @Tatiana Balaburkina and @Vladimir Kudriakov are presenting! If you're attending don't miss their session, but if you are not we are offering free tickets ($50 value).​​ Now here's the catch: we have at least 10 tickets but in order to receive one we need you to reply to this thread and tell us how you are using open-source tools on the mainframe.

What do we want to know? As much as you want to share. At a minimum tell us:

• What operating system you use the Rocket open source tools on
• Which tools you are using, and
• What processes/activities have you enabled. If you are still experimenting tell us how far you've gotten and where you want to get to (maybe we can help)
• Bonus points for sharing how this solution is benefitting your organization

It's that easy. We'll send you a promo code that can be immediately be used for registration.  



------------------------------
Dave Andrews
Head of Customer Engagement
Rocket Software
South Salem NY United States
------------------------------

Hello,

We at BEC in Denmark are using Rocket open source tools on USS. 
Our organisation is using bash and git. Personally, I am using vim.
We are currently migrating our source code to git. We have migrated to around 60 repositories and will migrate to around 350 repositories at project completion. This is a multi-year project, where much of the work is integrating existing systems to read from git. The biggest problems are reading from the unix filesystem instead of the MVS filesystem and codepage issues.

Best regards,

Adam Britt

------------------------------
Adam Martin Britt
IT-Architect
BEC
Randers SV Denmark
------------------------------

Hello,

We at BEC in Denmark are using Rocket open source tools on USS. 
Our organisation is using bash and git. Personally, I am using vim.
We are currently migrating our source code to git. We have migrated to around 60 repositories and will migrate to around 350 repositories at project completion. This is a multi-year project, where much of the work is integrating existing systems to read from git. The biggest problems are reading from the unix filesystem instead of the MVS filesystem and codepage issues.

Best regards,

Adam Britt

------------------------------
Adam Martin Britt
IT-Architect
BEC
Randers SV Denmark
------------------------------

Hello Adam, glad to know that you are embracing future through Rocket ported tools. Can you please elaborate more on the issues that you are facing with respect to codepage and "reading from unix file system"? Are you facing issues with the .gitattributes file for ASCII/EBCDIC encoding? Here is a great post from the forum on .gitattributes and the env variables necessary to set up git for your profile https://community.rocketsoftware.com/forums/forum-home/digestviewer/viewthread?GroupId=79&MID=2509&CommunityKey=1e694975-142d-4f2d-9b52-0e37e225db41&tab=digestviewer

------------------------------
Ajith Thankappan
Rocket Internal - All Brands
------------------------------

Hello,

We at BEC in Denmark are using Rocket open source tools on USS. 
Our organisation is using bash and git. Personally, I am using vim.
We are currently migrating our source code to git. We have migrated to around 60 repositories and will migrate to around 350 repositories at project completion. This is a multi-year project, where much of the work is integrating existing systems to read from git. The biggest problems are reading from the unix filesystem instead of the MVS filesystem and codepage issues.

Best regards,

Adam Britt

------------------------------
Adam Martin Britt
IT-Architect
BEC
Randers SV Denmark
------------------------------

Adam.
What release of git are you using? 
Rocket Software git for z/OS version 2.14.4 assumes the remote repository is encoded in ISO-8859-1, where-as git on other platforms assumes it is encoded in UTF-8.  However, the current version of Rocket Software git for z/OS version 2.26.2, installed using MiniConda from repository https://anaconda.org/zoss-appdev/repo?type=conda&label=main, assumes the remote repository is encoded in UTF-8, the same as other platforms. 

Moving from git 2.14.4 to git 2.26.2 with an existing repository has to be done carefully due to the encoding change.  You must perform steps to re-encode any items in existing repositories that contain code points that encode differently in ISO-8859-1 vs UTF-8, and once you have done so you must never use git 2.14.4 against the repository.

------------------------------
Greg Dyck
Internal - Rocket
Denver CO United States
------------------------------

Adam.
What release of git are you using? 
Rocket Software git for z/OS version 2.14.4 assumes the remote repository is encoded in ISO-8859-1, where-as git on other platforms assumes it is encoded in UTF-8.  However, the current version of Rocket Software git for z/OS version 2.26.2, installed using MiniConda from repository https://anaconda.org/zoss-appdev/repo?type=conda&label=main, assumes the remote repository is encoded in UTF-8, the same as other platforms. 

Moving from git 2.14.4 to git 2.26.2 with an existing repository has to be done carefully due to the encoding change.  You must perform steps to re-encode any items in existing repositories that contain code points that encode differently in ISO-8859-1 vs UTF-8, and once you have done so you must never use git 2.14.4 against the repository.

------------------------------
Greg Dyck
Internal - Rocket
Denver CO United States
------------------------------

Was there ever an answer to this question about the git upgrade?
Open-source Languages & Tools for z/OS

What does "re-encode any items in existing repositories" and "never use git 2.14.4 against the repository" mean for a repository with multiple clones and multiple branches?

------------------------------
Andrew Rowley
Self Registered
Ballarat Australia
------------------------------

Was there ever an answer to this question about the git upgrade?
Open-source Languages & Tools for z/OS

What does "re-encode any items in existing repositories" and "never use git 2.14.4 against the repository" mean for a repository with multiple clones and multiple branches?

------------------------------
Andrew Rowley
Self Registered
Ballarat Australia
------------------------------

I would like a ticket to the summit as we are working on extensively on extending our modernizing the Mainframe.

• We are running the Rocket software on a z/OS
• Primarily git - for now
• Migrating git from another source version control system
• Consolidate our source accross platforms 

------------------------------
Rene Søndergaard
Tooling Architect
BEC
4000 Denmark
------------------------------

Hello Adam, glad to know that you are embracing future through Rocket ported tools. Can you please elaborate more on the issues that you are facing with respect to codepage and "reading from unix file system"? Are you facing issues with the .gitattributes file for ASCII/EBCDIC encoding? Here is a great post from the forum on .gitattributes and the env variables necessary to set up git for your profile https://community.rocketsoftware.com/forums/forum-home/digestviewer/viewthread?GroupId=79&MID=2509&CommunityKey=1e694975-142d-4f2d-9b52-0e37e225db41&tab=digestviewer

------------------------------
Ajith Thankappan
Rocket Internal - All Brands
------------------------------

Hi Ajith,

Codepage issues is due to this awful idea IBM had, where it tries to magically convert codepages under the hood. When I use an ssh client to connect to USS, output to the shell is converted, so everything I see in the shell is not what was output.  I have the same problem with bpxwunix in rexx. Using the syscalls spawn and pipe give me more control but working it all out is a pain. I have learned not to trust what I see, but to read the hex values. I also had to learn that the unix command xxd will also iconv before displaying the hex values if the file is tagged, but the unix command "od -x" doesn't read tags at all. Codepages in USS is a mess if you are using non-american codepages.

The codepage issue was made worse by Rockets decision to use ISO8859-1 as the default codepage in git 2.14 and to use utf-8 in 2.26. 

The "reading files from unix" issue is most of our programs are designed to read from datasets, not unix files. Unix files don't have a record length, they have a newline character. Our current SCM has a subsystem, where we in JCL use the SUBSYS parameter and can supply a version number. The program using the file thinks the file is a normal dataset. As far as I know, there isn't a similar functionality for git. To solve the problem, our programs need to learn to read unix files, or we need to synchronize unix files to datasets and use the datasets. We are using both solutions depending on the context, but that is the most time consuming part of migrating to git - making changes to 20+ systems.

------------------------------
Adam Martin Britt
IT-Architect
BEC
Randers SV Denmark
------------------------------

Adam.
What release of git are you using? 
Rocket Software git for z/OS version 2.14.4 assumes the remote repository is encoded in ISO-8859-1, where-as git on other platforms assumes it is encoded in UTF-8.  However, the current version of Rocket Software git for z/OS version 2.26.2, installed using MiniConda from repository https://anaconda.org/zoss-appdev/repo?type=conda&label=main, assumes the remote repository is encoded in UTF-8, the same as other platforms. 

Moving from git 2.14.4 to git 2.26.2 with an existing repository has to be done carefully due to the encoding change.  You must perform steps to re-encode any items in existing repositories that contain code points that encode differently in ISO-8859-1 vs UTF-8, and once you have done so you must never use git 2.14.4 against the repository.

------------------------------
Greg Dyck
Internal - Rocket
Denver CO United States
------------------------------

Hi Greg, 

We just updated git from 2.14 to 2.26 this past weekend, after a failed upgrade in August. We were burned by the situation you described. I ended up writing a program that converts all commits from ISO8859-1 to UTF-8 using git fast-export and git fast-import and doing the conversion on the fly. We did this because we needed to be able to get historical versions and know what codepage it was encoded in. It's also cleaner when the whole repo is in the same codepage.

Converting the git repos has the consequence that all (or almost all) of the commit hashes have changed. We store the git hash used to compile in a CSECT in our load modules, so we also needed to relink all of our programs, so they have the correct hash.

Everything seems to be working as expected for us now. I just hope, that Rocket doesn't make backwards incompatible changes like this again.

------------------------------
Adam Martin Britt
IT-Architect
BEC
Randers SV Denmark
------------------------------

Was there ever an answer to this question about the git upgrade?
Open-source Languages & Tools for z/OS

What does "re-encode any items in existing repositories" and "never use git 2.14.4 against the repository" mean for a repository with multiple clones and multiple branches?

------------------------------
Andrew Rowley
Self Registered
Ballarat Australia
------------------------------

>What does "re-encode any items in existing repositories" and "never use git 2.14.4 against the
>repository" mean for a repository with multiple clones and multiple branches?

There are multiple ways to "re-encode" the data in the repository, all of them painful in some way.  For our team there were two areas to deal with-
1) The copyright character, which is encoded differently
2) Tool generated UTF-8 HTML, CSS, JS, etc data which was not properly identified in .gitattributes.

One of the steps in the process is to update the .gitattributes file so that all items get encoded appropriately in the local repository on checkout.  There also were attributes in our original file that were not recognized/supported by the new git which had to be changed.

I would have liked to have done the same solution as Adam stated he did, exporting, converting, and importing all commits, as that leaves the repository totally consistent in the end.  However the team decision was to identify the affected items, check them out using git 2.14, which re-encoded them from ISO-8859-1 to IBM-1047, and then forcing the affected items to be seen as updated and committed using the newer version of git (this was done using an earlier release 2.26.2) , which re-encoded them from IBM-1047 to UTF-8 in the repository.  This had to be done for every branch in the repository that had not already been merged (but not yet deleted).

If I had to do it again, I would press for doing an export, convert, import as the way we did it sometimes causes issues during rebasing.

Once the corrected repository has been pushed to a remote server it is important that only a new version of git be used against all replicas which will be fetching/pulling/pushing commits to the repository so that data gets encoded as expected, both locally and on the server.

------------------------------
Greg Dyck
Internal - Rocket
Denver CO United States
------------------------------

Next Wednesday and Thursday (September 22 & 23) is the Linux Foundation Open Mainframe Summit, and our own @Tatiana Balaburkina and @Vladimir Kudriakov are presenting! If you're attending don't miss their session, but if you are not we are offering free tickets ($50 value).​​ Now here's the catch: we have at least 10 tickets but in order to receive one we need you to reply to this thread and tell us how you are using open-source tools on the mainframe.

What do we want to know? As much as you want to share. At a minimum tell us:

• What operating system you use the Rocket open source tools on
• Which tools you are using, and
• What processes/activities have you enabled. If you are still experimenting tell us how far you've gotten and where you want to get to (maybe we can help)
• Bonus points for sharing how this solution is benefitting your organization

It's that easy. We'll send you a promo code that can be immediately be used for registration.  



------------------------------
Dave Andrews
Head of Customer Engagement
Rocket Software
South Salem NY United States
------------------------------

Hello,

Me and my colleagues from BEC would love to join the the summit today and tomorrow.

As you can see from my colleagues our primary focus right now is Git, but next step will be exploit the full potential of Zowe creating a full-blown CI/CD pipeline.

If I also could have a ticket it would be great.

Thanks in advance.

------------------------------
Henrik Jensen
IT architect
Self Registered
Roskilde Denmark
------------------------------