Skip to main content

file handle count calculation

  • February 23, 2016
  • 1 reply
  • 7 views
Navin_C
Forum|alt.badge.img+2

Hi All,

 

I have been struggling to find a relation between Open file handle count and Vertica process on each node.

How can we determine, how many files does vertica open for a query.

I know we can use resource acquisitions table, but I need to understand the logic behind these numbers.

 

Logically.

A ROS container can have many files, one file for each column of that container.

 

If a select query with only one column is executed, then it should open only the index file and file containing the column values in that ROs container as a result of partition pruning, but this is not observed in simple cases.

 

I tried querying the single column of a partitioned table with 20 values. The data was distributed on all nodes of a 3 node cluster, but the query shows file handle count as 

Node 1 : 7 (initiator)

Node 2 : 4

Node 3 : 4

 

I query two columns and the result changes

Node 1 : 9 (initiator)

Node 2 : 6

Node 3 : 6

 

Any expalnation to logic would be appreciated.

 

Thanks

    1 reply

    eli_revach
    Forum|alt.badge.img+2
    • eli_revach
    • Participating Frequently
    • February 23, 2016

    Hi ,

    Looks like from 7.X and above you no longer have pidx and fdb files , vertica do not maintain the index file in seperate file , for now on , you will see file with extention _0.gt .

     

    Now for your specific use case  i see few options :

    1. Mutipule ROS files per specific columns 
    2. Mutipule ROS files per specific columns + temp files (Sort_Temp_<id>.dat) , the temp files are create and delete at the end of the query 

     

    Very easy way to monitor which files are open is using lsof utility , you can corellect the name of the files you get from the lsof utility (try to grep the results by data location you have on the file system )  with the sal_storage_id column exists in the storage_containers table .

     

    Hope you will find it useful

    Thanks .

     

     

     

     

     

     

    Still scrolling?
    Ask a question
    Terms & ConditionsAccessibility statement