Skip to main content

upgrade from NE 5.1.6 to NE 5.1.12, diacritic problem

  • August 17, 2016
  • 11 replies
  • 0 views
D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

11 replies

Chris Glazier
Forum|alt.badge.img+2

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

There should be no difference in behavior between 5.1.6 and 5.1.12 in this area. I looked at the reported incident and it appears like the problem is actually related to how PIC X data is being returned from an Oracle database when using Pro*COBOL, is that correct? This is more likely to be an Oracle configuration option than anything you can set in COBOL.

There is not enough information provided in the incident to make a determination of the cause of this difference. We have asked for a cutdown demo so that we can reproduce the behavior in-house so we can troubleshoot the issue, but all we have been provided with are two source programs without their required  copybooks and no indication as to how the Oracle table is defined or which statement is returning data incorrectly.

We can probably attempt to recreate the issue if you provide us with a sample SELECT statement along with host variable definitions and the DDL for the table in question that would populate rows with the data required to run the SELECT and examine the results.

A single row of data would be enough to test this I believe.

If you could isolate this to a simple SELECT that returns the column in question incorrectly then that would be very helpful to us but we must be able to test this in-house so we can compare 5.1.6 with 5.1.12.

Is this something that you can provide us with?

One other question, did this difference occur after only upgrading 5.1.5 to 5.1.12 or did other variables change as well such as Oracle version, Pro*COBOL version, new computer, etc.?

Thanks

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

We only upgrading from 5.1.6 to 5.1.12 . I do not use ORACLE in this program.

It is an XML-file with UTF8 .

In the lister we see two bytes for a diacritic.

In 5.1.6 the debugger shows as   2 bytes in the field.

In 5.1.12 the debugger shows one character in the field and no longer two bytes.

I have one Pc with 5.1.6 and run the program with debug. In this version it is still working.

Now a take the same xml-file and run it with 5.1.12. one character is shown, so not working anymore.

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

We only upgrading from 5.1.6 to 5.1.12 . I do not use ORACLE in this program.

It is an XML-file with UTF8 .

In the lister we see two bytes for a diacritic.

In 5.1.6 the debugger shows as   2 bytes in the field.

In 5.1.12 the debugger shows one character in the field and no longer two bytes.

I have one Pc with 5.1.6 and run the program with debug. In this version it is still working.

Now a take the same xml-file and run it with 5.1.12. one character is shown, so not working anymore.

Chris Glazier
Forum|alt.badge.img+2

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

OK, so you are using the XML-IO preprocessor to read in an xml file. I now understand this. We still need a small test program that we can compile and run here so that we can reproduce the problem. You do not have to send us your actual xml data as I understand that your company does not allow this but we do require a dummy xml file containing a couple of entries and a program that will read this file and demonstrate the problem to us.

If the only change that you made was to upgrade the wrappack levels of NX and you are still using the same set of directives to compile on the same computer then chances are that this difference in behavior may be due to a product bug that will need to be addressed by development and not any compiler directive or setting that you can make on your part.

In order to address this we need to first be able to reproduce the behavior.

You should work this through the open support incident instead of this forum and the engineer can possibly do a screen share to have you demonstrate the problem if it is not possible for you to send an example.

Thanks.

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

My problems are not the same, but similar.

I solved the case by opening the XML file first as simple line sequential file. Then analyze the contents one byte after another. If I find a possible UTF character: > X"7F" I look for the next character, which must be larger than > X"7F". If not, I reject the whole XML file and do not even try to open/read it.  This is a “selfmade” solution to a problem, that NETEXPRESS should detect, but does not.

My report (nov.2014) was:

We receive a lot of invoices in XML format from our suppliers.  All have the header  

<?xml version="1.0" encoding="UTF-8"?>

and should be in that format.  Unfortunately some do not fulfill this format and have some fields filled with ASCII characters (not UTF-8 as it should be), including accented vowels á é í ó ú.   X”F3”  X”E9”  X”E1”  X”FA” etc.

The Internet Explorer does not open/show the file due to this wrong codification.

My program has the

ENVIRONMENT DIVISION.

FILE-CONTROL.

select factura-XML assign to ID-ARCHIVO-FAC

                    file status is file-status-spec.

The program generally works very fine if the XML is encoded right and reads all fields. But if the READ  statement tries to read such a malformed file, the status does not report any error code, but aborts the program with a 114 error code.

What must I do to receive an error code instead of aborting the program?

I have no chance to examine the    file-status-spec and inform the operator, because the program aborts immediately with the message

Execution error: file LEC-XML5

error CODE 114, PC=0, CALL=1, SEG=0

114 Attempt to access item beyond bounds of memory (Signal 11)

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

In adition to my previous report, here a more detailed description of the problem.

The wrong coded XML that I received starts with the following:

<?xml version="1.0" encoding="UTF-8"?>

<cfdi:Comprobante xmlns:xsi="www.w3.org/.../XMLSchema-instance&quot; xsi:schemaLocation="http://www.sat.gob.mx/cfd/3 www.sat.gob.mx/.../cfdv32.xsd&quot; version="3.2" folio="857" fecha="2014-11-04T11:31:18" formaDePago="Pago en una sola exhibición"

The Word “exhibición”  is wrong coded.

“exhibición”  x”65 78 68 69 62 69 63 69 F3 6E         the accented o  is not coded in UTF-8

                         e   x   h    i    b   i   c   i    o

It should be

“exhibición”  x”65 78 68 69 62 69 63 69 C3 B3 6E    the accented o   =  X”C3B3”

                         e   x   h    i   b    i   c    i    o

Our problem is, that the NETEXPRESS does not return any error code, but just aborts the program with

Execution error: file LEC-XML5

ERROR CODE: 114, PC=0, CALL=1, SEG=0

114   Attempt to access item beyond bounds of memory (Signal 11)

As I wrote in my previous report, I solved the case by analyzing the XML as plain text file.

If any byte > than X”7F”  (in this case X”F3” ) is not followed by a byte also > than X”7F” (in the wrong case X”6E”)  I reject the whole file, and don’t even open or read the XML.  

If the file is coded right as UTF-8   X"C3"  is followed by X"B3"

Perhaps an unusual way to solve the problem, but it works. Of course it would be better if Micro Focus corrects the compiler. I would not need the analyzing, but just open the XML, read it and look at the status-code.


#UTF-8XML
Chris Glazier
Forum|alt.badge.img+2

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

Rolf,

We should be able to handle this error condition in a better manner than a RTS 114 error that would alert you to the condition but allow the program to continue.

Have you reported this to Customer Care in a support incident? What was the number?

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

Yes, I reported it 11/03/2015   with Report  2802674 .   As I found a way around the Problem they asigned a medium priority.

Chris Glazier
Forum|alt.badge.img+2

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

I just tested the problem here and it does still exist in the latest wrappack of NX 5.1 but it appears to already be fixed in later products like Studio 6.0 and Visual COBOL 2.3. In these products an xml status code of -9 is returned on the read instead of an RTS 114.

If you would like this to be fixed in 5.1 then please update the incident with that request and the engineer will assign a higher priority to it.

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

I have upload a program and a file, where you can test the diacritic problem. It is a zip file.

I hope it can be solved quickly (incidentnumber  2870879)

D

We have upgraded (Windows) NE 5.1.6 to NE 5.1.12.

After this update there is an automatic conversion from 2 bytes diacritic to one character diacritic
Now some my programs are not working any more .

Example.
My program converted Indonesië to Indonesie .

translation table:

# Kolommen : Cod   = Unicode
#            S     = Soort   (X = exact of V = vervangend of E = Escapeteken)
#            F     = Formaat (H = Hexadecimaal of A = Alfanumeriek)
#            Teken = Teken volgens tekenset
#ade
#Cod S F Teken      Omschrijving
#--- - - ---------- ----------------------------
00E9 X H C3A9 LATIN SMALL LETTER E WITH ACUTE

The 5.1.6 program showes the 'ë' as two bytes and translated it to 'e'

Now 5.1.12 program, noting changed in program code, see it as 'ë' (while debugging) one character

How can we change it to the 2 bytes like the 5.1.6 version ?

Code-set or settings or cfg or something else ?

The incidentnumber is 2870879.

It is an urgent problem......


#cobolnetexpress

I have upload a program and a file, where you can test the diacritic problem. It is a zip file.

I hope it can be solved quickly (incidentnumber  2870879)

Recent badge winners

Explore all badges
Terms & ConditionsAccessibility statement