Skip to main content

Vertica load procedure

  • December 26, 2015
  • 4 replies
  • 20 views

Kazuki

I would like to know How Vertica load data.

 

In documentations, 

COPY :

FILE/STDIN -> WOS -> Moveout(Sort & Compress) -> ROS -> Mergeout(Bond)

COPY DIRECT :

FILE/STDIN -> ROS -> Mergeout(Sort & Compress & Bond)

(Do i understand correctly?)

 

However, i can't find when segmentation or partitioning occur.

Does anyone know that?

 

4 replies

eli_revach
Forum|alt.badge.img+2
  • Participating Frequently
  • December 30, 2015

Hi

Small correction for the ROS load :

FILE/STDIN -> ROS (Sort & Compress )  -> Mergeout(Sort & Compress & Bond) 

 

About partition and segmentation 

Segmentation  WOS & ROS - as part of the data load the internal loader  applay the hasing function to the segmentaion keys and send it to the related executer node .

 

Partition is only applay to ROS data , so it will be applay as part of MoveOut for WOS data and by internal loader for ROS 

 

Thanks 


Kazuki
  • Author
  • New Participant
  • December 31, 2015

Hello eli_revach

Thank you for reply

 

  • initiator node
    1. copy data from file/stdin
    2. segment data by segmentation keys and distribute it  to executer node
  • executer node
    1. sort data
    2. compress data using RLE  <= WOS creation completed
    3. Moveout(sort, compress, partitioning) <= ROS created
    4. Mergeout(sort, compress) 

If data is loaded like above, is segmented data distributed as raw data(not compressed)?

 


eli_revach
Forum|alt.badge.img+2
  • Participating Frequently
  • December 31, 2015

Yes ,

initiator is responsable for loading the data from input file parse it  and segment it and send it to each one of the nodes as a raw , each one of the executer nodes (including initiator ) sorting & compress  are localy take place .

Thanks 


Kazuki
  • Author
  • New Participant
  • January 1, 2016

Thank you very much for your help!