Rocket DevOps (formerly Aldon)

 View Only
Expand all | Collapse all

How do I get Source Code from LMi into Github or BitBucket?

  • 1.  How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 03-17-2021 07:33

    Many of our customers are choosing to store a copy of their IBMi source code from LMi, within a centrally accessible online repository such as Github or BitBucket, for code analysis purposes, or centralisation with distributed systems.

    If you wish to do this, here is what you need to do…..

    1.
    Start by checking your product compatibility. The LMi Git integration commands were introduced in Rocket Lifecycle Manager for IBMi Edition (LMi) Version 8.3. If you are on a version prior to this, contact us to discuss upgrading options.

    2. Go to https://docs.rocketsoftware.com and download the Rocket Lifecycle Manager for IBMi Edition Daily Operations Reference Version 8.x (where x is the version of LMi). Navigate to the Appendix section within the document and open the 'Configuring the LMi Git Integration' topic. Check the Prerequisites section to ensure you have the correct licensed programs and credentials required. The GIT licensed program must be installed on the IBMi server.

    3. Follow the instructions in the guide to complete the tasks listed below.

    • Use the LMIGITBASE command to define the local Git working directory.
    • Use the LMIGITSYNC command to populate the local git repository with LMi source files from within a single Group/Application/Release or choose to add ALL source files within LMi to the Git repository in a single operation.
    • Use the LMIGITEXIT program to configure the LMi exit programs, so that the local Git repository is updated automatically during an LMi Promotion action.
    • Configure the LMIGITNEP monitor job.

    Once the configuration is complete, test that your local Git repository is updated by checking out an object and perform a promotion action as normal. Navigate to the path where the Git working directory was created using the WRKLNK command and locate the source file that was changed.

    4. This next section references how to push the changes from the local Git working directory upto Github or BitBucket. This is not described within the Rocket documentation guide, as it uses standard Git commands. The configuration actions that should be considered are;

    • Link the local Git working directory to your GitHub or BitBucket repository using the 'git remote add origin URL' command (where URL is the address of the GitHub or BitBucket repository).
    • Create SSH keys if this is required by your organisation. The SSH keys can be generated on the IBMi server using the ssh-keygen command, and then added to the GitHub or BitBucket repository list of keys for authorised users. We would recommend searching the ibm.com website for further information on this command if you are not already familiar with it.

     Once the configuration is complete, to send the changes to the attached GitHub or BitBucket repository, the push is performed using the standard Git commands 'git add, git commit, git push', which can be done in several ways;

    • Using an SSH terminal session such as PuTTY and navigating to the Git working directory using the 'cd' command.
    • Using STRQSH on the IBMi server and navigating to the Git working directory.
    • Writing a CL program that navigates to the appropriate path and executes the above Git commands.

    So, as you can see, it is a simple job to get all source files within LMi into a centralised repository such as GitHub or BitBucket and have them updated when future changes occur. If you have any other questions on the LMi Git Integration, just ask!


     



    ------------------------------
    Andy Finley
    Rocket Software
    ------------------------------


  • 2.  RE: How do I get Source Code from LMi into Github or BitBucket?

    PARTNER
    Posted 05-07-2021 10:19
    Hello Andy,

    This post is really fantastic, I have adjusted these instructions too many customers with success.

    Do you have an example about  "
    Writing a CL program that navigates to the appropriate path and executes the above Git commands" 

    I would like to run the git push command automatically from IBMi

    Best regards and congratulations for your great post !
    Miguel

    ------------------------------
    Miguel Garcia - Partner
    IT Specialist
    CDS Americas
    Quito - Pichincha - Ecuador
    ------------------------------



  • 3.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 05-14-2021 05:02
    Edited by Andrew Finley 03-23-2022 12:15
    Hello Miguel,

    If you wish to automate the 'push' from the local GIT repository to the hosted GitHub or Bitbucket repository, you can do the following;

    1. Navigate to your local GIT repository in the IFS.
    2. Link your local GIT repository to your hosted GitHub or Bitbucket account using the standard GIT commands.
    3. Create and save a 'filename.sh' script, and insert the following code (changing the highlighted text with the appropriate values);

    ************Beginning of data**************
    #!/bin/sh

    LOG="/tmp/filename.txt"
    REPO_DIR="/path/to/your/local/GIT Repository"
    cd $REPO_DIR > $LOG 2>&1
    if ^^ -n $(git status -s) ]]; then
    git add --all . >> $LOG 2>&1
    git commit -m "Commit from IFS_systemname" >> $LOG 2>&1
    git push -u origin master >> $LOG 2>&1
    fi
    ************End of Data********************

    4. To automate the execution of the above script upon an LMi promotion, you could choose to update the EXIT program to include the following code;

    QSH CMD('/QOpenSys/usr/bin/sh -c "/path/to/your .sh script.sh"') /* command to push local Git in IFS into GitHub or Bitbucket Server */
    /**/

    OR

    Have a batch job running that loops around and executes the above command periodically, by creating a CL program as shown below;

    PGM
    DCL VAR(&COUNT) TYPE(*DEC) LEN(3 0) VALUE(1)
    /* */
    /* SUBMIT A JOB TO RUN EVERY 30 SECS FOR ONE HOUR */
    /* 30SECS X 240 = 7200 SECS / 60 = 120 MINS */
    DOUNTIL COND(&COUNT = 240) /* TOTAL OF 1 HOUR RUNNING JOB
    QSH CMD('/QOpenSys/usr/bin/sh -c +
    "/path/to/your .sh script.sh"') /* command to push local +
    Git in IFS into Github or Bitbucket Server */
    /**/
    CHGVAR VAR(&COUNT) VALUE(&COUNT + 1)
    DLYJOB DLY(30)
    ENDDO
    ENDPGM

    *Please note that you may be required to use SSH keys to authenticate from the local GIT repository to the online GitHub or Bitbucket repository.
    A key can be generated on the IBMi by accessing QSH, and using the 'ssh-keygen' command. Once the key is generated, open the file using the 'cat' command, and copy the full key, which will start with 'ssh-rsa' followed by the unique key, and add this to the online GitHub or Bitbucket repository for this user.

    I hope this helps, 

    Many thanks, Andy.

    ------------------------------
    Andy Finley
    Rocket Software
    ------------------------------



  • 4.  RE: How do I get Source Code from LMi into Github or BitBucket?

    PARTNER
    Posted 05-18-2021 10:14
    Hello, Andy

    Thank you so much for your great answer, this was what I really needed, a guide to develop and practice. 

    I am going to apply these important instructions. If I have a little doubt , could I write you again ?

    Have a nice day!!!!

    Miguel

    ------------------------------
    Miguel Garcia - Partner
    IT Specialist
    CDS Americas
    Quito - Pichincha - Ecuador
    ------------------------------



  • 5.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 05-20-2021 10:22
    Hi Miguel, yes sure, absolutely no problem at all!

    Thanks, Andy.

    ------------------------------
    Andy Finley
    Rocket Software
    ------------------------------



  • 6.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 07-29-2021 16:57
    Hola, Miguel.
    If you have more specific questions on the LMIGIT stuff, please open a case with support and Newton or I can take a look.  This will help us track questions and fill gaps in the documentation or add new functionality.
    Best, 
    d

    ------------------------------
    Dale Asher
    Rocket Internal - All Brands
    ------------------------------



  • 7.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 03-23-2022 12:14
    Hello Miguel,

    If you wish to automate the 'push' from the local GIT repository in the IFS, to the hosted Github or BitBucket repository, you can do the following;

    1. Navigate to your local GIT repository in the IFS.
    2. Link your local GIT repository to your hosted Github or BitBucket account using the standard GIT commands.
    3. Create and save a  'filename.sh' script, and insert this code (changing the highlighted text with the appropriate values);

    ************Beginning of data**************
    #!/bin/sh

    LOG="/tmp/filename.txt
    REPO_DIR="/path/of/your/local/GIT Repository"
    cd $REPO_DIR > $LOG 2>&1
    if ^^ -n $(git status -s) ]]; then
    git add --all . >> $LOG 2>&1
    git commit -m "Commit from IFS_systemname" >> $LOG 2>&1
    git push -u origin master >> $LOG 2>&1
    fi
    ************End of Data********************

    4. To automate the calling of the above script upon promotion, you could choose to modify your LMi Exit program by adding this code;

    QSH CMD('/QOpenSys/usr/bin/sh -c +
    "/SEGITWKDIR/GitBuild1/BANKING/CORE/gitpush+
    banking.sh"') /* command to push local +
    Git in IFS into Bitbucket Server */

    ------------------------------
    Andy Finley
    Rocket Software
    ------------------------------



  • 8.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 03-28-2022 10:07
    NOTE -
    1 - Regarding the '^' in your example for the loop - these should be '[' brackets.
    #!/bin/sh
    
    LOG="/var/log/aldon/LMI/git_auto_push.log"
    REPO_DIR="/<path_to_git_root>/"
    cd $REPO_DIR > $LOG 2>&1
    if [[ -n $(git status -s) ]]; then
    git add --all . >> $LOG 2>&1
    git commit -m "Commit from IFS `hostname`" >> $LOG 2>&1
    git push -u origin master >> $LOG 2>&1
    fi

    2 - I actually removed the if..fi loop (as 'git status -s' was not cooperating with output in all cases to trigger the push).

    Something like this:
    #!/bin/sh
    
    LOG="/var/log/rocket/git_auto_push.log"
    touch $LOG 
    REPO_DIR="/<path_to_local_IFS_repo/"
    cd $REPO_DIR >> $LOG 2>&1
    echo `date` >> $LOG 2>&1
    echo "add..." >> $LOG 2>&1
            git add --all . >> $LOG 2>&1
    echo "commit..." >> $LOG 2>&1
            git commit -m "Commit from IFS `hostname`" >> $LOG 2>&1
    echo "push..." >> $LOG 2>&1
            git push -u origin master >> $LOG 2>&1


    ------------------------------
    Dale
    L3 Support for the enterprise bits, portals and plugins...
    Rocket|Aldon
    Somewhere on the Oregon coast...
    ------------------------------



  • 9.  RE: How do I get Source Code from LMi into Github or BitBucket?

    Posted 05-13-2021 13:15
    Andy,

    Has anyone confirmed that your approach for Git integration with LM(i) is compatible with the MS Azure DevOps Git server/repository?

    In our experience at SMA Technologies, Microsoft has updated their command line interface (CLI) multiple times in recent years (especially as Azure replaced Visual Studio Team Services, Visual Studio Online and Team Foundation Service Preview), and the current Azure CLI supports some Git commands that vary from the open source standard.  Either that, or the Azure DevOps Git server has unique parameters or parameter requirements that are different from the open source Git repositories commonly used by the general public on the internet.

    During my experiments with connecting a Git repository local copy that resides in the IFS '/' root file system of my IBM i partition, I have figured out how to capture, store and engage my secure identity token from Azure DevOps, so that I can log into our development repository.  But not all of the commands that are required to complete an entire Git work cycle are working, and I am having difficulty diagnosing the errors.

    Thanks,
    George Loose
    Technical Product Manager - IBM i
    SMA Technologies
    Houston, Texas, USA

    ------------------------------
    George Loose
    Technical Product Manager - IBM i
    SMA Technologies
    ------------------------------



  • 10.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 05-14-2021 05:10
    Hello George,

    I have not tried to connect my local GIT repository in the IFS to an MS Azure DevOps Git server/repository - I have only tried GitHub and BitBucket. However, I will make an effort to test this.
    You mentioned that 'not all commands that are required to complete an entire Git work cycle' are working correctly - are you referring to the 'git add', 'git commit' and 'git push' commands?

    Thanks, Andy.


    ------------------------------
    Andy Finley
    Rocket Software
    ------------------------------



  • 11.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 03-28-2022 10:09
    George.
    Please open a case with Support if you have specific details/questions on issues with use of git with LMi and we'll take a look.
    Best, 
    d

    ------------------------------
    Dale
    L3 Support for the enterprise bits, portals and plugins...
    Rocket|Aldon
    Somewhere on the Oregon coast...
    ------------------------------



  • 12.  RE: How do I get Source Code from LMi into Github or BitBucket?

    Posted 04-22-2022 10:02
    Hi Andy,  I am a complete novice at this whole GIT/Code for IBMi/Open Source stuff, so please forgive me if this is a pointless question.  As I read the Operations Reference for LMi 8.5A, it sounds like I can use LMi to move source to the IFS, make mods in VS Code for IBMi, but it will not move the code back to the IBMi when finished.  Is that correct, or am I missing something?  
    Thanks, 
    Kevin Monahan 
    DestinationXL

    ------------------------------
    kevin monahan
    systems analyst
    Casual Male Retail Group Inc
    Canton MA US
    ------------------------------



  • 13.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 04-22-2022 10:39
    Hi Kevin, 
    The VS Code extension that I'm using (Code for IBM i)  does work from either IFS or native libs for review and update.  The current implementation of git with LMi is pushing the dvp library updates during promote to the IFS.  So, to keep things in sync, LMi is still used to drive the IFS and git repo commit and push.
    For development, I'm using the Code for IBM i extension to point to my dev library for LMi and open/edit/save the source file members directly.  Then use LMi promote for the promote (and our LMIGITNEP to perform the git commit).
    We are working on a VS Code plugin to streamline development and would be interested in your thoughts / comments on the implementation.  Initially, would go after actions supported via Plugin for RDi (Checkout, Compile, Promote) and then move on to others (Task create/update/promote, source compare/merge, dependency details...).
    Best, 
    d
    VS Code with Code for IBM i and LMi used via tn5250j emulation on i


    ------------------------------
    Dale
    L3 Support for the enterprise bits, portals and plugins...
    Rocket|Aldon
    Somewhere on the Oregon coast...
    ------------------------------



  • 14.  RE: How do I get Source Code from LMi into Github or BitBucket?

    Posted 04-22-2022 11:33
    Thanks Dale,  I guess I need to work with this for a little while before I can make any intelligent/coherent comments.  I'll keep following the discussion thread as it moves along.

    ------------------------------
    Kevin Monahan
    Senior Solutions Developer
    DestinationXL Group
    Canton MA US
    ------------------------------



  • 15.  RE: How do I get Source Code from LMi into Github or BitBucket?

    Posted 04-27-2022 10:01
    Hi Dale, one question.  Is there anything in Aldon LMi that prevents someone from using Code for IBMi to access source directly on the IBMi but from something other than the DVP environment?  In other words, something to stop me from grabbing/updating source in the ITG level and bypassing DVP?

    ------------------------------
    kevin monahan
    systems analyst
    Casual Male Retail Group Inc
    Canton MA US
    ------------------------------



  • 16.  RE: How do I get Source Code from LMi into Github or BitBucket?

    ROCKETEER
    Posted 04-27-2022 10:32
    Kevin,
    Thanks for the great question.
    We (LMi/RDOi apps) support host IBM i system authority options by user or group profiles for *lib and objects. 
    If your non-DVP libs are restricted (our PDN libs are when we manage with LMi), the Code for IBM i would have browse capability (save would be prohibited - same as PDM or RDi access).
    If you have users with *allobj, then that leaves a back door that should be addressed as part of your security review for the i.  If you have developers using a Group profile with limited access to non-dvp libs (or require another group or *allobj for access to those lib/objects), you can limit ability to edit outside of dev libs.
    There is a great deal of info available in our LMi docs for 'Security'.  We can be quite granular. The security reference info covers use of 'reference objects' to automatically mirror or model authority during promote, deployment or data set updates.
    We can open a case with the support team if you have specific questions or need a quick review of your existing setup.
    d

    ------------------------------
    Dale
    L3 Support for the enterprise bits, portals and plugins...
    Rocket|Aldon
    Somewhere on the Oregon coast...
    ------------------------------