We're working a POC where customer is attempting to materialize a 1.6TB Parquet file as a ROS table where the DATA location is MapR-FS by doing CTAS. They say the query appears to hang, even appears in MC, but it's clear from the log that the plan was abandoned. How can we diagnose the issue? The Parquet file can be queried as an external table so we don't think permissions. Other smaller ROS tables created successfully. Can we set debug logging?
I see other posts use select set_debug_log('SAL',"ALL"); will that work here?
Here is the relevant vertica.log for the txn:
2018-04-23 10:11:42.946 Init Session:7f04a6508700-b000000001fb30 [Txn] Begin Txn: b000000001fb30 'insert into ros.mer_ctgy_cd_grp_int select * from public.mer_ctgy_cd_grp_int;'
2018-04-23 10:11:42.955 Init Session:7f04a6508700-b000000001fb30 @v_vertica_node0001: 00000/6610: 0 percent of data matched to local nodes (min 0, max 0 percent per node). 0 percent of data can be loaded within same racks
2018-04-23 10:11:42.956 Init Session:7f04a6508700-b000000001fb30 [Command] [AutoProj] creating default projection query string: SELECT * FROM ros.mer_ctgy_cd_grp_int
2018-04-23 10:11:42.956 Init Session:7f04a6508700-b000000001fb30 [Command] [AutoProj] creating default projection order by string: ORDER BY iso_mer_ctgy_cd,mcc_grp_id,mcc_grp_nm
2018-04-23 10:11:42.956 Init Session:7f04a6508700-b000000001fb30 [Command] [AutoProj] creating lazy projection: CREATE PROJECTION ros.mer_ctgy_cd_grp_int AS SELECT * FROM ros.mer_ctgy_cd_grp_int ORDER BY iso_mer_ctgy_cd,mcc_grp_id,mcc_grp_nm SEGMENTED BY hash( mcc_grp_id,iso_mer_ctgy_cd,mcc_grp_nm) ALL NODES KSAFE;
2018-04-23 10:11:43.016 Init Session:7f04a6508700-b000000001fb30 [Session] [AutoProj] rerun exec_simple_query("insert into ros.mer_ctgy_cd_grp_int select * from public.mer_ctgy_cd_grp_int;", 0)
2018-04-23 10:11:43.021 Init Session:7f04a6508700-b000000001fb30 @v_vertica_node0001: 00000/6610: 0 percent of data matched to local nodes (min 0, max 0 percent per node). 0 percent of data can be loaded within same racks
2018-04-23 10:11:43.023 Init Session:7f04a6508700-b000000001fb30 [Opt] Dump SendRecvNodes Map :
2018-04-23 10:11:43.024 Init Session:7f04a6508700-b000000001fb30 [Opt] Load is using default ExprEval, not LoadExprEval
2018-04-23 10:11:43.081 Init Session:7f04a6508700-b000000001fb30 [VMPI] [abandon plan] - New synced txn catalog version : 10 (Old : 0)
2018-04-23 10:11:49.957 Init Session:7f04a6508700-b000000001fb30 [Session] [Query] TX:b000000001fb30(v_vertica_node0001-54019:0x9a6fe) select * from ros.mer_ctgy_cd_grp_int;
2018-04-23 10:11:59.069 Init Session:7f04a6508700-b000000001fb30 [Session] [Query] TX:b000000001fb30(v_vertica_node0001-54019:0x9a6fe) select distinct sc.node_name, ps.anchor_table_schema, ps.anchor_table_name, sc.projection_name, sc.location_label, ps.row_count, ps.ros_used_bytes
2018-04-23 10:11:59.086 Init Session:7f04a6508700-b000000001fb30 [Opt] JoinInput Local Operation was Combine, but the data was already combined
2018-04-23 10:11:59.086 Init Session:7f04a6508700-b000000001fb30 [Opt] JoinInput Local Operation was Combine, but the data was already combined
2018-04-23 10:11:59.121 Init Session:7f04a6508700-b000000001fb30 [VMPI] [prepare plan] - New synced txn catalog version : 148 (Old : 10)
2018-04-23 10:11:59.122 Init Session:7f04a6508700-b000000001fb30 [Catalog] Loading storage_containers table by scanning the catalog.
2018-04-23 10:11:59.167 Init Session:7f04a6508700-b000000001fb30 [VMPI] [compile plan] - New synced txn catalog version : 190 (Old : 148)
2018-04-23 10:11:59.170 Init Session:7f04a6508700-b000000001fb30 [EE] SortManager found maxMerges 7 too small(0 MB Assigned).
2018-04-23 10:11:59.170 Init Session:7f04a6508700-b000000001fb30 [EE] After disabling optimization, maxMerges becomes 7.
2018-04-23 10:11:59.170 Init Session:7f04a6508700-b000000001fb30 [EE] SortManager found maxMerges 7 too small(0 MB Assigned).
2018-04-23 10:11:59.170 Init Session:7f04a6508700-b000000001fb30 [EE] After disabling optimization, maxMerges becomes 7.
2018-04-23 10:11:59.175 Init Session:7f04a6508700-b000000001fb30 [EE] SortManager found maxMerges 7 too small(0 MB Assigned).
2018-04-23 10:11:59.175 Init Session:7f04a6508700-b000000001fb30 [EE] After disabling optimization, maxMerges becomes 7.
2018-04-23 10:11:59.185 Init Session:7f04a6508700-b000000001fb30 [VMPI] [execute plan] - New synced txn catalog version : 194 (Old : 190)
2018-04-23 10:12:46.164 Init Session:7f04a6508700-b000000001fb30 [Session] [Query] TX:b000000001fb30(v_vertica_node0001-54019:0x9a6fe) select export_objects('','ros.mer_ctgy_cd_grp_int');
2018-04-23 10:12:46.174 Init Session:7f04a6508700-b000000001fb30 [Txn] Starting Commit: Txn: b000000001fb30 'insert into ros.mer_ctgy_cd_grp_int select * from public.mer_ctgy_cd_grp_int;' 774
2018-04-23 10:12:46.174 Init Session:7f04a6508700-b000000001fb30 [TM] updateCPE: Updating projection mer_ctgy_cd_grp_int_b1 CPE to 0x14 from 0xffffffffffffffff for wos 0xb0000000127dfe
2018-04-23 10:12:46.174 Init Session:7f04a6508700-b000000001fb30 [TM] updateCPE: Updating projection mer_ctgy_cd_grp_int_b0 CPE to 0x14 from 0xffffffffffffffff for wos 0xb0000000127df0
