Skip to main content
Question

Convert to UTF8 and replace original file

  • November 1, 2019
  • 4 replies
  • 14 views

AnuskaWR
Forum|alt.badge.img+1

I am trying to convert source file to UTF-8 format and save in the same directory (replacing original file) or to a new directory (saving file with the same "original" name ). Can someone help? Please note that source file does not have a static name. I can save the source file to a static name (as in the code below), but I need to retain the value of the original source file.
iconv -f ISO88599 -t utf-8 input_*.txt > /UTF8/input.txt
Your thoughts are appreciated.
-Anna

4 replies

mosheg
Forum|alt.badge.img+2
  • Participating Frequently
  • November 2, 2019

The following script example can handle none empty files.

#!/bin/bash
for orig_file_name in $(ls ./SOURCE_DIR); do
   iconv -f UTF-16 -t UTF-8 ./SOURCE_DIR/${orig_file_name} -o ./TARGET_DIR/${orig_file_name}
done

AnuskaWR
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • November 2, 2019

Mosheg, thanks! Can I incorporate this code in bat file? How do I make it execute on demand when called from an ETL process?


mosheg
Forum|alt.badge.img+2
  • Participating Frequently
  • November 3, 2019

In general yes, but the syntax mentioned is for Linux.
You can load files from Windows client, however, for better performance to have parallel load you need to put your files on shared storage where each node on the cluster can access the files.


AnuskaWR
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • November 4, 2019

Thanks mosheg. I think this example can give me direction on how to resolve the issue. The source directory can have multiple files and I need to convert one specific file.