Skip to main content

Vertica File Format?

  • February 25, 2016
  • 6 replies
  • 17 views

sjyu
  • Participating Frequently

Are there any resources out there that explain how a Vertica ROS container is structured?  Something like this resource about how ORC files are structured in HDFS.

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC

Thanks!

6 replies

Chris_Daly_HPE
Forum|alt.badge.img
  • Participating Frequently
  • March 10, 2016

Hi!

      Can I ask what the intention is this information?  Accessing data directly from the ROS containers is really not advisable.  Let me see if I can find you a resource.

 

-Chris


sjyu
  • Author
  • Participating Frequently
  • March 10, 2016

Really just for understanding and curiousity.  Every system I've worked with I learned about how writes happen at the file/block level and thought it'd be interesting to know.

 

Thanks for the help!


FiliN
Forum|alt.badge.img
  • Participating Frequently
  • March 11, 2016

Try to find "Materialization Strategies in the Vertica Analytic Database: Lessons Learned"

It explains a lot. The Best Vertica archictural document I ever read!


sjyu
  • Author
  • Participating Frequently
  • March 16, 2016

Wow, thanks for that!  It was a very good read and very informational.

 

I have a follow up question about container storage.  So I know data is stored in pairs of files per column, in multiple containers. But what determines the maximum number of tuples per container?  


FiliN
Forum|alt.badge.img
  • Participating Frequently
  • March 16, 2016

Have you read about TP strata algorithm?


eli_revach
Forum|alt.badge.img+2
  • Participating Frequently
  • March 17, 2016

Hi ,
Their is no such setting for max tuples per contener.

 

The setting is on disk storage size level , meaning when container reach certain size threshold ,  vertica will no longer try to merge to it new data. at  this point vertica will create new container.


This concept is implemented inside their internal strata algorithm
I hope you will find it useful


Thanks