Rocket Terminal Emulator

 View Only
Expand all | Collapse all

Anyone Running RTE in z/OS zCX container?

  • 1.  Anyone Running RTE in z/OS zCX container?

    Posted 05-30-2023 10:49

    Hi,

    I am hoping someone on here has played with RTE using z/OS zCX containers.    I currently have RTE WE 10.1.5.0065 running successfully under z/OS V2.5 in USS with IBM supplied NODE.JS version 18.14.2.

    My question is about running RTE in a z/OS ZCX container as documented in the administrators guide. For the zCX container, I have s390x/node.js V20 docker image downloaded s390x/node - Docker Image | Docker Hub here. However, what is not clear in the admin guide is whether or not I need to use the extracted download image for z/OS USS which is the larger .pax file on your website, or do I use the extracted .tar file instead? 

    I think the Linux .tar download is what I need, and have been using to create the docker image.

    I actually had the docker build run successfully with the .tar version, but had issues because the first s390x/node.js package was the "slim" version, which i hadnt noticed, and doesnt contain everything needed. RTE startup had errors, and i couldnt connect to it, so I assumed it was the wrong version of node.js. Once I downloaded the full version, and imported that, is when the problems below started.

    I ask because the docker build is failing with:

    dockerfile:

    FROM s390x/node:latest

    # Create app directory

    RUN mkdir -p /home/rte

    # Bundle app source

    COPY ./bzwapps /home/rte/bzwapps

    # Expose the HTTP port

    EXPOSE 8543

    # Start the application

    WORKDIR /home/rte/bzwapps/bin

    CMD [ "sh", "nodeServer-docker.sh" ]


    Fails with

    admin@ctsatest:/tmp/RocketRTEinstall$ docker build . -t bluezone:latest

    Sending build context to Docker daemon 369.2MB

    Step 1/6 : FROM s390x/node:latest

     ---> 9350c47e2c80

    Step 2/6 : RUN mkdir -p /home/rte

     ---> Running in 22e5e482b372

    failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------


  • 2.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 05-30-2023 22:24
    Edited by Limei Wu 05-30-2023 23:48

    Hi Dave, you are correct. zCX container use the .tar version. RTE Web deployment in zCX works for me. It seems the working directory is incorrect. Please check if this path is correct: WORKDIR /home/rte/bzwapps/bin.

    WORKDIR /home/rte/bzwapps/bzwapps/bin?


    Thanks,
    Limei



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 3.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 05-31-2023 07:37

    Limei,

    thanks for the clarification on which version of code to use.   I found my problems.   First, I will say that zCX is new territory for me, so learning as I go.  When i installed the required node.js, I did a docker import of the .tar file, instead of a docker load.   Once I got past that, the docker build worked successfully.    The second hurdle where I couldnt connect to the RTE server, was because I did not start the container with the -p option to publish the port.   Once I did that I am able to connect.

    Thanks again!

    Dave



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 4.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 05-31-2023 09:12

    Dave, great to hear that you have figure this out.



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 5.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 06-01-2023 08:17

    Limei,

    If you dont mind, another question.    When a ZCX instance is shutdown, do the containers running inside it come down nicely?  Is there a method to have the RTE shutdown.sh script run automatically at shutdown?    I shutdown ZCX container, and restarted, and it seems as though data was lost or corrupted, because I can no longer logon to the RTE instance, and am having to start over it seems?

    In the same thought, is there an automated method to start the container as well?

    Thanks in advance.



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 6.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 06-01-2023 22:48

    Hi Dave, when you shutdown the zCX container, the application inside it will not be shutdown but cannot access too. When you start the container again, you can access the application inside it again. If you just shutdown/stop the container and then start it again, the data should not be lost.  How do you shutdown and start the container?

    docker stop xx

    docker start xx

    If you use the docker run command to start the container, it will create a new container and then start it. The data will be lost since it is a new container. To keep the data, you need to mount the data out, for example docker run -p 8543:8543 -v /home/rte/bzwapps/: /opt/bzwapps --name container-name image-name. I believe there are several persistence storage solution for container data, like using volume .etc.



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 7.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 06-02-2023 09:41

    Thank-you Limei,

    Your pointers helped a lot.  I did create a volume, and started new container with:

    docker run -d -p 8543:8543 --volume bluezone_data:/home/rte/bzwapps --name bluezone --restart unless-stopped bluezone:10.1.5

    One other question if you do not mind, but in reading the RTE administrator guide when doing upgrades, i see references for copying the ./deploy folder to ./migrate, and then upgrade.    How would this be accomplished in a docker environment?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 8.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 06-05-2023 02:41

    Hi Dave, I think you can go to the volumn/mounted data storage, and copy the ./deploy folder to ./migrate and then move to the RTE bzwapps folder. Then start the new version's container with a new volumn. After the upgrading test successfully, you can remove the old volumn data.



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 9.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-06-2023 11:45

    Hi Limei,

    I'm hoping you can assist me again.   I have an open case I hope you can take a peek at.  It is 00929750.    In summary, I have RTE running in ZCX containers on two different hosts, and am trying to get clustering working.    So far no luck.   I suspect it is a zCX docker configuration issue.   Ive rebuilt the container to expose the http, https, and the cluster listening ports, and started a new container in both locations(same run command, but different IP/host names)

    docker run -d --ip 192.168.34.37 -p 8543:8543 -p 8544:8544 -p 8643:8643 -h 192.168.34.37 -v bluezone_deploy:/home/rte/bzwapps/deploy -v bluezone_migrate:/home/rte/bzwapps/migrate  --name bluezone --restart unless-stopped bluezone:10.2.0.1

    trying to add the peer from the master fails with: 

    Failed to add peer

    Failed to get peer status

    Would you be willing to give me some pointers on this subject?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 10.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 11-07-2023 01:40

    Hi Dave, could you check the ipv4 showing in the peer metadata? You can copy it out or click the eye icon to view it. By default, I think it is the container's ip.  So that is why it is failed to add the peer since it cannot access another container's ip. How to communicate between different host's containers? One possible solution is using the overlay network driver.

    Overlay networks are distributed networks that span multiple Docker hosts. The network allows all the containers running on any of the hosts to communicate with each other without requiring OS-level routing support.

    Here is a reference for this:  https://spacelift.io/blog/docker-networking.



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 11.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-07-2023 06:03

    Thanks for the pointers Limei.    Another docker item to learn.  I guess I dont understand why the two containers cannot communicate via the same IP's we use in the browser to reach RTE?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 12.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-07-2023 07:20

    Another clarification?   The RTE instances are in different ZCX instances on different hosts.  Is that going to make a difference?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 13.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-07-2023 09:46

    MOre questions....  I looked at the doc you referenced.   Can I just create a new SWARM overlay?  I know its somewhat outside the scope of RTE, but you support clustering, its just the ZCX docker portion that is somewhat difficult.   are you able to provide me with any examples/samples to get me going?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 14.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 11-08-2023 02:52

    Hi Dave, the peer metadata is generated when the RTE Web server starts up. It is getting the container's ip. I am not sure if it can get the host ip. The peers nodes communicate with each other through p2p network to share and access resources directly. I think the docker networking driver can solve this. We also need to investigate and try this. I will talk with our team to see if we can provide some examples.

    Thanks,

    Limei



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 15.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-08-2023 07:17

    Thank-you Limei.   It would be great if your team were to get this working in your labs.  I also wanted to let you know that I have an open ticket with IBM ZCX support.   The article you passed along looked promising, but when we I got to the part for customizing the gwbridge interface, ZCX doesnt allow that,  IBM is confirming that, and am waiting for their response. 

    Customize the docker_gwbridge interface

    The docker_gwbridge is a virtual bridge that connects the overlay networks (including the ingress network) to an individual Docker daemon's physical network. Docker creates it automatically when you initialize a swarm or join a Docker host to a swarm, but it is not a Docker device. It exists in the kernel of the Docker host. If you need to customize its settings, you must do so before joining the Docker host to the swarm, or after temporarily removing the host from the swarm.

    1. Stop Docker.

    2. Delete the existing docker_gwbridge interface.

      $ sudo ip link set docker_gwbridge down
      
      $ sudo ip link del dev docker_gwbridge


    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 16.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-08-2023 11:01

    Limei,

    I just wanted to pass along that I think I have had some success.    I was able to add the node with the meta-data simply by overtyping the 172.18 IP's in the metadata with the host IP's when I added the machine to the cluster.

    {"id":"12D3KooWK4CXRvdvzhnnhJ9SAcfw7jgruSrjsHpSsqQcEVmT4dHE","peerId":{"id":"12D3KooWK4CXRvdvzhnnhJ9SAcfw7jgruSrjsHpSsqQcEVmT4dHE","pubKey":"CAESIIlEaO9OBJBLf7pNGFsnZexVqlYKfHzoIFSyrUjHt+bV"},"port":8643,"version":{"major":2,"minor":0,"patch":3},"multiaddrs":["/ip4/127.0.0.1/tcp/8643/p2p/12D3KooWK4CXRvdvzhnnhJ9SAcfw7jgruSrjsHpSsqQcEVmT4dHE","/ip4/172.18.0.3/tcp/8643/p2p/12D3KooWK4CXRvdvzhnnhJ9SAcfw7jgruSrjsHpSsqQcEVmT4dHE"],"serverURL":"http://192.168.34.37:8543","localIp":"172.18.0.3","timestamp":1699027365293}



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 17.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 11-09-2023 22:52

    Wow, that's great, Dave. Could you see if the data syncronization works between clustering nodes? If yes, replacing with host ip is a simple solution too. If any further issue, please let us know.

    Thanks,

    Limei



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------



  • 18.  RE: Anyone Running RTE in z/OS zCX container?

    Posted 11-10-2023 07:09

    Yes, everything sync'd right up.   It would be great if I didnt have to do that, though.  Could this be a considered a "bug" in the RTE code?   Pulling local IP works everwhere except containers, whether on zcx or x86 I presume?



    ------------------------------
    Dave Jousma
    Principal Systems Programmer
    Fifth Third Bank, N.A.
    Grand Rapids MI US
    ------------------------------



  • 19.  RE: Anyone Running RTE in z/OS zCX container?

    ROCKETEER
    Posted 11-12-2023 21:01

    Hi Dave, I don't think this is a bug in the RTE code. The container is an isolated env and cannot get the host ip directly. Yes, it seems we can support to edit the peer metadata to change the ip manually in the old version. In our new version(10.2.1), the ip will be overwritten to container ip when reatarting because we need to support another scenario in the kubenates. However, we support to configure the Env to change the ip and port of the peer metadata in 10.2.1. We have the senario to use the proxy ip between the containers. In your senario, the host ip behaves as similar as the proxy.

    So you can think this is an enhancement. 



    ------------------------------
    Limei Wu
    Rocket Internal - All Brands
    Dalian CN
    ------------------------------