Skip to main content

Curl through proxy creates mixed EBCDIC/ASCII data

  • December 7, 2016
  • 5 replies
  • 0 views
A
  • Anonymous

Using curl 7.42.1 (i370-ibm-openedition)
through a firewall with an HTTP proxy like:
http_proxy=http://redacted.com:80
ALL_PROXY="$http_proxy" export http_proxy ALL_PROXY

trying to fetch:
tp://service.boulder.ibm.com/s390/holddata/month.txt

I get a strange behavior such that the first few thousand bytes
of the file when translated from EBCDIC to ASCII (that’s
right!) appear as EBCDIC. The rest of the file is proper EBCDIC.

curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=I’
curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=A’
produce identical bad results.

– gil

    5 replies

    A
    • Anonymous
    • 0 replies
    • December 9, 2016

    Using curl 7.42.1 (i370-ibm-openedition)
    through a firewall with an HTTP proxy like:
    http_proxy=http://redacted.com:80
    ALL_PROXY="$http_proxy" export http_proxy ALL_PROXY

    trying to fetch:
    tp://service.boulder.ibm.com/s390/holddata/month.txt

    I get a strange behavior such that the first few thousand bytes
    of the file when translated from EBCDIC to ASCII (that’s
    right!) appear as EBCDIC. The rest of the file is proper EBCDIC.

    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=I’
    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=A’
    produce identical bad results.

    – gil

    Thanks for posting. We’ll take a look.

    A
    • Anonymous
    • 0 replies
    • December 12, 2016

    Using curl 7.42.1 (i370-ibm-openedition)
    through a firewall with an HTTP proxy like:
    http_proxy=http://redacted.com:80
    ALL_PROXY="$http_proxy" export http_proxy ALL_PROXY

    trying to fetch:
    tp://service.boulder.ibm.com/s390/holddata/month.txt

    I get a strange behavior such that the first few thousand bytes
    of the file when translated from EBCDIC to ASCII (that’s
    right!) appear as EBCDIC. The rest of the file is proper EBCDIC.

    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=I’
    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=A’
    produce identical bad results.

    – gil

    Hi Paul,

    1. What build of cURL do you have?
      You may find this information in file: curl-7.42.1/share/doc/curl/VERSION.ZOS.

    2. Did you tryied to use -B or --use-ascii (Use ASCII/text transfer) keywords?
      If not, please try to use and share results to us.

    Regards,
    Andrey

    A
    • Anonymous
    • 0 replies
    • December 12, 2016

    Hi Paul,

    1. What build of cURL do you have?
      You may find this information in file: curl-7.42.1/share/doc/curl/VERSION.ZOS.

    2. Did you tryied to use -B or --use-ascii (Use ASCII/text transfer) keywords?
      If not, please try to use and share results to us.

    Regards,
    Andrey

    Hi, Andrey,

    Thanks for your followup. It appears that the version number is embedded
    in the path, so I need to know the version number to find the version.

    And I doubt that our admin ever did “make install”, but just unpacket the
    tarball and left it at that.

    Does any of this help:

    user@OS/390.25.00: ( cd /team/rocket && head $( find . -name VERSION.* ) )
    ==> ./cURL/share/doc/curl/VERSION.ZOS <==
    Tool: curl
    Version: 7.42.1
    Build Number: 0006

    ==> ./cURL/share/doc/openssl/VERSION.ZOS <==
    Tool: openssl
    Version: 1.0.2c
    Build Number: 0002

    ==> ./cURL/share/doc/zlib/VERSION.ZOS <==
    Tool: zlib
    Version: 1.2.8
    Build Number: 0003
    user@OS/390.25.00:
    user@OS/390.25.00: $( whence curl ) --version
    curl 7.42.1 (i370-ibm-openedition) libcurl/7.42.1 OpenSSL/1.0.2c zlib/1.2.8 iconv libssh2/1.4.3 nghttp2/1.0.1
    Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
    Features: IPv6 Largefile NTLM NTLM_WB SSL libz CharConv TLS-SRP HTTP2 UnixSockets Metalink
    user@OS/390.25.00:

    I could supply a trace; perhaps too big to post on a forum. It appears that the
    first packet is translated from ASCII to EBCDIC twice; remaining packets only
    once. I doubt that I can blame the proxy; I’d expect it to be EBCDIC-unaware.

    Thanks,
    Paul

    A
    • Anonymous
    • 0 replies
    • December 13, 2016

    Using curl 7.42.1 (i370-ibm-openedition)
    through a firewall with an HTTP proxy like:
    http_proxy=http://redacted.com:80
    ALL_PROXY="$http_proxy" export http_proxy ALL_PROXY

    trying to fetch:
    tp://service.boulder.ibm.com/s390/holddata/month.txt

    I get a strange behavior such that the first few thousand bytes
    of the file when translated from EBCDIC to ASCII (that’s
    right!) appear as EBCDIC. The rest of the file is proper EBCDIC.

    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=I’
    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=A’
    produce identical bad results.

    – gil

    Hi Paul,

    1. Please update the cURL. You use very old version. Build 12 of cURL is available on download page:
      http://www.rocketsoftware.com/ported-tools/download

    2. Did you use cURL keyword -B/–use-ascii?
      e.g.
      curl -q -B ftp://service.boulder.ibm.com/s390/holddata/month.txt
      curl -q --use-ascii ftp://service.boulder.ibm.com/s390/holddata/month.txt
      Please update cURL and run this commands. We would like to know your results.

    Thanks,
    Andrey

    A
    • Anonymous
    • 0 replies
    • December 15, 2016

    Using curl 7.42.1 (i370-ibm-openedition)
    through a firewall with an HTTP proxy like:
    http_proxy=http://redacted.com:80
    ALL_PROXY="$http_proxy" export http_proxy ALL_PROXY

    trying to fetch:
    tp://service.boulder.ibm.com/s390/holddata/month.txt

    I get a strange behavior such that the first few thousand bytes
    of the file when translated from EBCDIC to ASCII (that’s
    right!) appear as EBCDIC. The rest of the file is proper EBCDIC.

    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=I’
    curl -q 'ftp://service.boulder.ibm.com/s390/holddata/month.txt;type=A’
    produce identical bad results.

    – gil

    This problem is fixed in build 13. Please download from our public download page.


    Terms & ConditionsAccessibility statement