Open-source Languages & Tools for z/OS

I’m calling REST API services with cURL and get a JSON payload back. Using cURL 7.16.1 the response is in ASCII, however using cURL Version: 7.42.1 Build Number: 0013 the response comes back in EBCDIC. Additionally to the EBCDIC output there is error message curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding.This all could be seen in the trace files.
As we run all that in z/OS 2.2 I would prefer the EBCDIC response, but how to react to the error message?
Any help is appreciated.
Thanks, Werner

by the way, I tried the --use-ascii option mentioned in another thread to no avail.
Werner

We are also using the 7.42.1 build of curl and are experiencing a similar issue with JSON REST results. About half the time the results are returned in EBCDIC, and the other half it comes back in ASCII. We’ve controlled for this by running the output through the iconv utility when it comes back in ASCII.

We are having an issue in certain cases where the codepage switches in the middle of the output, and half the string is in ASCII, half in EBCDIC. If anyone from rocket is monitoring this post - is there any explanation for why this would occur?

–use-ascii had no effect for us as well

Thanks Matt for sharing your experience. Oh no, your case is much worse than ours, if the coding changes within a string …
Anyway, good to know I’m not alone

To clarify, we are also on build 13. I tried re-extracting the package over again, in case I had missed anything and still had the same results. For now, a retry on the failure message is a simpler coding challenge than trying to handle mixed output, but if this is something that can be addressed with a new build, I’d be happy to provide samples if it will help.

cURL 7.52.1 was released.

Thanks,
Sergey

Sergey,

Thanks, I did see that announcement, and I have begun testing with the new version and it does consistently return an ASCII response, at least so far. With autoconvert, I do see it translated correctly to EBCDIC when I execute the query.

I do have a related problem, but moreso with Enchanced ASCII in general, in that when cURL is executed from within a Perl script, autoconversion does not occur even though the environment variables appear to be set up correctly.

Thanks,
Matt

Matt,
where do you specify autoconvert to get EBCDIC?

Thanks,
Werner

Sergey.
thanks a lot for the new build. Using it shows some changes:

• the cURL trace file is now ASCII (7.42.1 was EBCDIC)
• the cURL response is now ASCII (7.42.1 was EBCDIC)
• the JSON input file for the REST services have to be in ASCII now. As I call cURL from REXX (z/OS) the JSON file is created in EBCDIC. Now I have to convert it to ASCII before calling cURL. This is different to build 7.42.1, where the input file could be EBCDIC. Any chance/parm to change that?
• the error message (56) Illegal or missing hexadecimal sequence in chunked-encoding disappeared. It ends now without any error messages.

What you mean by “be sure your files are tagged”???
Thanks,
Werner

Werner,

What you mean by “be sure your files are tagged”???

To see the current tags:

ls -T

To tag EBCDIC file:

chtag -tc 1047 file

To tag ASCII file:

chtag -tc 819 file

It may be helpful to avoid the behaviour when untagged files reads as ASCII.

It’s recommended to tag files according to their encoding.
And, in that case, cURL must be working correctly with the input tagged files independent of their encoding.

All new files created by cURL have ASCII encoding and tagged as ASCII - it’s normal behaviour.

Thanks,
Sergey

Werner,

To set autoconvert on, there are a few USS environment variables to set. See the following post regarding unzip with Enhanced ASCII:

And:
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ceea300/filtag.htm

Regards,
Matt

Thanks, Sergey, the chtag command was unknown to me.
So I tagged the input JSON file as EBCDIC, but I still get the error message “unexpected character”, because the data are required to be in ASCII (HTTP annd REST) and no conversion is done internally (at least my interpretation). I also tried the autoconvert options Matt supposed, The only effect was that the trace file returned is EBCDIC.
Sorry to bother you, but I’m not very skilled with UNIX.
Thanks,
Werner

Hi Werner,

The following values specified for environment variables for the latest versions of bash in README.ZOS:

export _CEE_RUNOPTS="FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
export _BPXK_AUTOCVT=ON
export _TAG_REDIR_ERR=txt
export _TAG_REDIR_IN=txt
export _TAG_REDIR_OUT=txt

But they are used not only for bash. Probably, we need to add this information to README.ZOS for cURL too.
In that case, all new files create in ASCII encoding and tag as ISO8859-1 by cURL.
The existent files read according to their tags.

Could you please to write some examples of commands you are using for work with JSON and REST, especially what key/options are using for work with the files?

Thanks,
Sergey

Hi Sergey,
I’d like to send you data about my tests, content of JSON file, REST commands, comparison of test results with cURL 7.42.1 and 7.52.1, because they behave differently. However, this is too much for this forum. Is there a way to send this to you directly (if you are interested, of course)?
Thanks,
Werner

Hi Werner,

it is possible to attach a file(s) to your reply here. The 7th button on replying form is “Upload”. Could you try this way?

Thanks,
Tatyana

Ok, here’s the first cURL_test_results.doc (28.5 KB)

And the second… cURL cmd_JSONfile.doc (22.5 KB)

Please note there will be a delay in delivering a new build that supports new environment variables previously mentioned. The engineering team has been diverted to a more urgent task.
Regards,
-Peter

Hi Werner,

Now, cURL is supporting only IBM-1047 and ISO8859-1 encodings for existent tagged files.
And all new files created by cURL is being saved as ASCII and tagged as ISO8859-1 (the trace files are included).

In next build, there is a planning to add the following environment variables:
_ENCODE_FILE_NEW - to set encoding and tag for the new files.
It’s allow to choose output encoding for files.
_ENCODE_FILE_EXISTING - to control reading/writing untagged files properly.
They could be set to the following values: ISO8859-1, IBM-1047, UNTAGGED, BINARY.

Unfortunately, if you have files in another encoding (for example, IBM-1141) you need to convert them to ISO8859-1 or IBM-1047 and tag them in according with their encoding before using by cURL.
iconv -f IBM-1141 -t IBM-1047 file1 > file2
chtag -tc IBM-1047 file2
or
iconv -f IBM-1141 -t ISO8859-1 file1 > file2
chtag -tc ISO8859-1 file2
I’m studying your case and will discuss with team is there a possibility to use another EBCDIC code pages.

Thanks,
Sergey