Open-source Languages & Tools for z/OS

 View Only
Expand all | Collapse all

Calling an Apigee endpoint URL from a JCL using CURL

  • 1.  Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-09-2023 08:38
    Hi,

    I've been working to call an API via JCL using CURL by passing public & private keys of the certificate. I was able to successfully make a call to the PCF endpoint URL using below JCL where I got the request as expected by the API developer, however, due to security reasons now I need to make a call to an Apigee endpoint instead of PCF cloud directly. But for that I've been asked by our API team to pass below 3 parameters while making a CURL call. 

    1.) Client-meta data
    2.) API-key
    3.) requestUniqueID

    Below is the JCL.

    //ABCJ4API JOB (XXXX,P),'CURL',MSGCLASS=T,SCHENV=SYSPLEX,USER=XXX 
    //*
    //S5QCUR1 EXEC PGM=BPXBATCH,
    // REGION=0M
    //STDPARM DD *
    SH cd /appshare/XXXX/road4z/bin;
    curl --get -v URL
    https://apigw-int.devxx.xxx.com/xxxx/pdg/v1
    --key '/appshare/API/key.pem'
    --cacert '/appshare/API/ca.pem'
    /*
    //STDOUT DD SYSOUT=*
    //STDERR DD SYSOUT=*

    Please let me know how to pass the above 3 parameters using CURL and if this is not the correct forum, please let me know where I can post and get some help.

    Thanks in advance.

    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------


  • 2.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    ROCKETEER
    Posted 01-19-2023 21:08

    Hi Santosh Reddy Chintagari,

    As I understood right you need to send some kind of POST request via curl. In that case, it doesn't matter how to do it in the console or from the JCL, so this question is more to the curl community. 

    the request probably should be like

    curl "https://apigee.googleapis.com/v1/organizations?parent=projects/$PROJECT_ID"  \
      -H "$AUTH" \
      -X POST \
      -H "Content-Type:application/json" \
      -d '{
        "name":"'"$PROJECT_ID"'",
        "analyticsRegion":"'"$ANALYTICS_REGION"'",
        "runtimeType":"CLOUD",
        "billingType":"'"$BILLING_TYPE"'",
        "authorizedNetwork":"'"$NETWORK_NAME"'",
        "runtimeDatabaseEncryptionKeyName":"'"$RUNTIMEDB_KEY_ID"'"
      }'

    you can try to adapt it to your situation

    info from https://cloud.google.com/apigee/docs/api-platform/get-started/install-cli


    Thanks,
    Alexander



    ------------------------------
    Alexander Klochkov
    QA
    Rocket Internal - All Brands
    Vilnius LT
    ------------------------------



  • 3.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-20-2023 16:15
    Hi Alexander,

    Thank you for your reply. I tried few ways using the request you gave and also gone through the link, but it wasn't much helpful. 
    I'll try posting the query in CURL community and see if someone can provide details from Mainframe perspective. 

    Thanks,
    Santosh.


    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 4.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-23-2023 10:22
    Hi Santosh,

    Does the curl command work as expected from a USS shell, e.g., using OMVS or SSH into USS? Or maybe even try from your own workstation.

    My recommendation would be to get it working in USS or from your workstation first. Once it is working from the command line you can try to wrestle it into a JCL with BPXBATCH.

    ------------------------------
    Jorn Thyssen
    Solutions Advisor
    Rocket Internal - All Brands
    Waltham MA US
    ------------------------------



  • 5.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-25-2023 12:31
    Hi Jorn,

    Thank you for taking time to reply to my request.

    I did try to execute the commands via OMVS and it's not working from their either. I'm getting below error which I've reported to the developer who built the API. 

    Meanwhile is there anything that you can suggest me to look into fix this error please?


    HTTP/1.1 431 Request Header Fields Too Large

    {"fault":{"faultstring":"request headers size exceeding 25,600","detail":{"errorcode":"protocol.http.TooBigHeaders"}}}$            


    I really appreciate if there is anything that you can suggest me in fixing this error please. 

    Thanks,
    Santosh.



    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 6.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-26-2023 02:19
    Sorry, I am not an expert in REST APIs nor curl usage. HTTP 431 suggests you're sending a very large HTTP header.
    As Alexander suggested you can try asking for help in the curl community as this is not a z/OS specific issue.

    ------------------------------
    Jorn Thyssen
    Solutions Advisor
    Rocket Internal - All Brands
    Waltham MA US
    ------------------------------



  • 7.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 01-31-2023 08:48
    No problem Jorn, I've reached out to our API team and they are looking into it. Looks like the it's coming from Apigee.
    Thanks again for taking time on my query.

    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 8.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-09-2023 16:59

    Hi Alexander,

    In continuation to my testing this piece of calling an API from a JCL using cURL, I need some assistance please.

    I've tried below script from Windows dos command and it's working fine, I get a successful response, whereas when I try the same using the JCL it isn't.

    @echo off 
    set clientmetadata={\"appOrg\":\"com.xxx.fg\",\"appCode\":\"XXXX\",\"appVersion\":\"v1.2.3\", \"operatorId\":\"123454321\"}
    echo %clientmetadata%

    curl -X GET https://apigw-int.dev.xxx.com/XB80/audit/v1/ca/auditChanges  ^
    -H "Accept: application/json" ^
    -H "client-metadata: %clientmetadata%" ^
    -H "apikey: XXXXXXLATGB42AVTfzUmN47N2nY7iEE1" ^
    -H "requestUniqueId: E265685E-XXXX-XXXX-9C8A-XXXXE02DC1155C"

    Using JCL:

    //S5QCUR1  EXEC PGM=BPXBATCH,                                                  
    //             REGION=0M                                                       
    //STDPARM  DD  *                                                               
    SH cd /appshare/4j00/road4z/bin;                                               
    set clientmetadata={\"appOrg\":\"com.xxx.fg\",\"appCode\":\"XXXX\",            
        \"appVersion\":\"v1.2.3\", \"operatorId\":\"123454321\"}                   
    echo %clientmetadata%                                                          
    curl --get -v https://apigw-int.dev.xxx.com/XB80/audit/v1/ca/auditChanges
    -H "Accept: application/json"                                                  
    -H "client-metadata: %clientmetadata%"                                         
    -H "apikey: XXXXXXLATGB42AVTfzUmN47N2nY7iEE1"                                  
    -H "requestUniqueId: E265685E-XXXX-XXXX-9C8A-XXXXE02DC1155C"                                        
     --key '/dev/API/key.pem'                                        
     --cacert '/dev/API/ca.pem'                                      
    /*                                                                             
    //STDOUT   DD  SYSOUT=*                                                       
    //STDERR   DD  SYSOUT=*                                                        

    @Echo is a DOS command, but not sure the equivalent in Mainframe.

    Any thoughts please.

    Regards,

    Santosh.



    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 9.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-10-2023 03:52
    Edited by Tatiana Balaburkina 03-10-2023 03:52

    Hi Santosh,

    Could you share the JCL output? Since your company is on support, would you like to open a support case via customer portal https://my.rocketsoftware.com/?



    ------------------------------
    Tatiana Balaburkina
    Engineering Manager
    Rocket Internal - All Brands
    ------------------------------



  • 10.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-10-2023 10:03
    Edited by Santosh Reddy Chintagari 03-10-2023 10:03

    Hi Tatiana,

    I've already asked our Mainframe team open a support case as we developers are not allowed to do so. It's been close to 2 months and I don't see any reply yet.

    Coming to the question you have asked.

    I don't see any output when I run the JCL so I'm unable to figure out what's the issue. 

    Yesterday when I tried with a different script and I got the below response. The JCL took sometime to complete.

    000003 //S5QCUR1  EXEC PGM=BPXBATCH,                                            
    000004 //             REGION=0M                                                 
    000005 //STDPARM  DD  *                                                         
    000006 SH cd /appshare/4j00/road4z/bin;                                         
    000007 curl --get -v URL                                                        
    000008   https://apigw-int.dev.com/XB80/audit/v1/ca/auditChanges      
    000009   -H "Content-Type:application/json"                                     
    000010   -H "client-metadata:{"appOrg": "com.xxx.fg", "appCode": "XXXX",        
    000011     "appVersion": "v1.2.3", "locationId": "04538", "assetId": "PROD", 
    000012     "operatorId": "999996541", "legacyId": "234123411",                 
    000013     "ipAddress": "127.0.0.1"}"                                           
    000014   -H "Content-Length: 411"                                               
    000015   -H "apigw-int.devfg.rbc.com"                                           
    000016   -H "apikey: xWcFqByCZvFv1uFYoNuCqLmrQy9JdXdO"                          
    000017   -H "requestUniqueId: 24345678941"                                
    000018  --key '/dev/API/key.pem'                                  
    000019  --cacert '/dev/API/ca.pem'                                
    000020 /*                                                                       
    000021 //*STDOUT   DD  SYSOUT=*                                                 
    000022 //STDOUT   DD  DSN=BB0024.PDGAUDIT.RESP.LOGS,                            
    000023 //             DISP=OLD                                                  
    000024 //STDERR   DD  SYSOUT=*                                                  

     

    Output:

    curl: (52) Empty reply from server

    I tried another way as below and I got the error, somehow the client meta data is not getting accepted. Without the clientmetadata, Apikey & RequestUniqueId the call is going perfect, however for security reasons we need to pass these parameters for a secure call.

    HTTP-CODE: 400 BAD_REQUEST _error_info_catched.","additionalStatus":Ý{"statusCode":"XB80-10001","severity":"Error","statusDesc":"client-metadata error: field: appOrg can't be empty. field: appCode can't be empty. field: appVersion can't be empty. field: operatorId can't be empty. "}¨}}<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">

    //S5QCUR1  EXEC PGM=BPXBATCH,                                                   
    //             REGION=0M                                                        
    //STDPARM  DD  *                                                                
    SH cd /appshare/4j00/road4z/bin;                                                
    curl --get -v https://apigw-int.devfg.rbc.com/XB80/pdg-audit/v1/ca/auditChanges 
    -H "Accept: application/json"                                                   
    -H "client-metadata" : {                                                        
          "appOrg : com.xxx.fg"                                                     
          "appCode : XXXX"                                                          
          "appVersion : v1.2.3"                                                     
          "operatorId : 999996541"                                                 
      -H "apikey: xWcFqByCZvFv1uFYoNuCqLmrQy9JdXdO"                                 
      -H "requestUniqueId: 24345678941"                                       
     --key '/dev/API/key.pem'                                         
     --cacert '/dev/API/ca.pem'                                       
    /*                                                                              
    //*STDOUT   DD  SYSOUT=*                                                        
    //STDOUT   DD  DSN=BB0024.PDGAUDIT.RESP.LOGS,                                   
    //             DISP=OLD                                                         
    //STDERR   DD  SYSOUT=*                                                         

    Thanks,

    Santosh.



    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 11.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    ROCKETEER
    Posted 03-13-2023 08:44

    Hi Santosh,

    If your line with clientmetadata is too long for JCL you can also use file with headers:

    e.g.:
    curl -H @/path/to/file/with/headers.txt https://postman-echo.com/headers
    content of headers.txt should be like:
    client-metadata: {"appOrg":"com.xxx.fg","appCode":"XXXX","appVersion":"v1.2.3", "operatorId":"123454321"}
    test: 123
    make sure headers.txt is readable (cat headers.txt is ok) and has the correct tag.

    Thanks,
    Alexander



    ------------------------------
    Alexander Klochkov
    QA
    Rocket Internal - All Brands
    Vilnius LT
    ------------------------------



  • 12.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-21-2023 12:16

    Hi Alexander,

    Sorry for the late reply as I'm back to work this week. 

    I'll try it out by creating headers.txt and test the JCL and get back to you. Thank you.

    Regards,

    Santosh.



    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------



  • 13.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-10-2023 14:04

    SH is the IBM unix shell which has a very different syntax than the windows platform, see https://www.ibm.com/docs/en/zos/2.5.0?topic=descriptions-sh-invoke-shell

    Try

    export clientmetadata="{\"appOrg\":\"com.xxx.fg\",\"appCode\":\"XXXX\",            \"appVersion\":\"v1.2.3\", \"operatorId\":\"123454321\"}"

    and

    echo $clientmetadata

    and

    -H "client-metadata: $clientmetadata" 

    As mentioned earlier in this thread, I strongly recommend debugging the curl command in the USS shell before you try to wrestle with BPXBATCH 

    ssh into your mainframe, test the curl commands, and then try to create a BPXBATCH job afterwards. 



    ------------------------------
    Jorn Thyssen
    Solutions Advisor
    Rocket Internal - All Brands
    Waltham MA US
    ------------------------------



  • 14.  RE: Calling an Apigee endpoint URL from a JCL using CURL

    Posted 03-21-2023 12:47

    Hi Jorn,

    Sorry for the late reply as I'm back to work this week. 

    Thanks for your feedback. I tried testing the JCL using the "export" variable before clientmetadata but it's causing further issues in recognizing the CURL call command as part of the JCL.

    export: FSUM7354 "--get" is not an identifier

    export: FSUM7354 "-v" is not an identifier

    FSUM7354   "string" is not an identifier                                                 
                                                                                             
     Explanation                                                                             
     You attempted to use an nonalphanumeric name as an identifier.                          
                                                                                             
     User response                                                                           
     Use only alphanumeric names for identifiers.

    I'm trying to understand how can I debug using the USS shell, could you please add more details as how can test this. Thank you.

    Regards,

    Santosh.



    ------------------------------
    Santosh Reddy Chintagari
    Technical Systems Analyst
    Rocket Forum Shared Account
    ------------------------------