Skip to main content

Writing large XML files

  • February 15, 2013
  • 0 replies
  • 0 views

Problem:

Writing large XML files using XML-I-O is time consuming.

How can this be improved?

The XML-File was declared as

select myfile-frame assign "MyFile.XML"

    organization XML                            

    document-type "MyNamespace"                 

    file status XML-STAT.        

       XD myfile-content.

       01 Support identified "Support".

          02 rec  identified "Person".

             03 Names pic x(30).

             03 Job   pic x(30) IDENTIFIED "Job" ATTRIBUTE.

             03 Colleagues IDENTIFIED "Colleague"

                           COUNT in Colleague-Available.

                05 aName pic x(30).

Writing 10,000 "rec"s cost 3 min 30 seconds.

Resolution:

1st Improvement.

Writing the XML-I-O into memory instead to a file is a significant improvement.

Split the file into 2 parts.

       select myfile-frame assign address in-memory

           organization XML

           document-type "MyNamespace"

           file status XML-STAT.

       select myfile-content assign address in-memory

           organization XML

           file status XML-STAT.

       01 Support identified "Support"

          pic x.

       XD myfile-content.

      *01 Support identified "Support".

       01 rec  identified "Person".

             03 Names pic x(30).

             03 Job   pic x(30) IDENTIFIED "Job" ATTRIBUTE.

             03 Colleagues IDENTIFIED "Colleague"

                           COUNT in Colleague-Available.

                05 aName pic x(30).

With a definition

       01 in-memory        pic x(32000)

and write the parts directly by CBL_WRITE_FILE runtime routine reduces the time of 10,000 write operations to less than a second.

####################################

For further improvement 1,000,000 "rec" writes were used.

2nd Improvement

When the CBL_WRITE_FILE runtime routine gets a buffer of 1 Meg bytes which is only written when it is full, the time is reduces from 33 seconds to 27 seconds.

3rd Improvement

Correcting the length of the data block where the XML-I-O writes to, here from

    01 in-memory        pic x(32000).

to

    01 in-memory        pic x(3200).

accelerates the demo to run in 17 seconds.

This was checked using a Windows XP operating system.

Using a different operating system may give varying results.

Attachments:

XML6-bigest.zip

Old KB# 1584

0 replies

Be the first to reply!