Storage locations and space

My guess is this presents an opportunity for an addition to our documentation, a blog, or some other kind of communication.

I don't know the default algorithm but from some simple tests it appears it alternates columns first on projection sort order and then on other column positions. As you indicated I don't think it looks at storage usage at all so in your case it probably just started alternating on new data. Assuming the storage is the same performance and you don't want to set performance priorities you may just need to use the ALTER_LOCATION_LABEL metafunction to assign a label to your locations and then use SET_OBJECT_STORAGE_POLICY to move some of your table projections to get a balance first. I have only tested that in the past with moving some data to HDFS and then back to ROS but it should work the same.

Thanks!

Good point, DaveT. Yes, it seems like it started to apply on new data to alternate between both storage locations, but what it is confusing is that you cannot load or purge more data if one of the storage locations is full. If one of them is full, then it should continue to write on the other, right?

So, could we rebalance data from the existing projections between both storage locations? If we do SET_OBJECT_STORAGE_POLICY we are telling a table to use one specific storage location by default, so I am not sure that we are rebalancing existing data in the disks. And in the case that the new storage location comes from new disks installations (so in a new partition), we are not taking advantages of every disks available in the node in terms of raid performance.

Again, I don't think the algorithms consider space usage so you are probably stuck until you either move some data to a different location or expand the raid device/filesystem and recover. If you want to rebalance everything with a new location (no node down time) you can slowly unload/truncate/reload. Alternatively, you can stop node, rebuild the device/filesystem and recover, and in this case you wouldn't need to add another location.

I worked with a customer where we rebuilt/expanded raid device/filesystem and recovered on one node at a time over a period of a few weeks but you could recover multiple nodes at a time as long as you prepare and choose the right nodes in each grouping. Either way you need to ensure you have appropriate backups and I would recommend testing on a small test system first.

Thanks DaveT. Yes, I think they would need to rebalance. They only have one big table with few projections and it does not make any sense for them to store some projections in different storage locations.
As you said, we always have the option of rebuild the filesystem partition with a backup/restore strategy.

Anyway, I think that we should do something about our documentation when telling "adding disk space to a node", because users can be confused about all the impact of doing in that way because of new storage location management.

Thanks!