Skip to main content
Question

PopulateVirtualProjection?

  • August 18, 2019
  • 1 reply
  • 4 views

mosheg
Forum|alt.badge.img+2
  • Participating Frequently

From the known Query Phase Duration, most of the time spent on PopulateVirtualProjection.
What is PopulateVirtualProjection and how it can be controlled?
Does the following accurate?
Where querying on system tables, there are two execution steps that affect the most: Preplan and PopulateVirtualProjection.
PopulateVirtualProjection step is executed on each node, but Preplan is only executed on the initiator node.
Thus, non-initator nodes have to wait the initiator node finishes Preplan step, and then every node starts the PopulateVirtualProjection step.
PopulateVirtualProjection is the bottleneck, and is proportional to the number of ROSes.
select date_trunc('hour',time) hour_date, execution_step, sum(timestampdiff('millisecond',time,completion_time)) sum_elapsed_ms from dc_query_executions where time>sysdate-1 group by 1,2 order by 1,3 desc;
hour_date | execution_step | sum_elapsed_ms
------------------------+-------------------------------+----------------
2019-08-17 11:00:00+00 | ExecutePlan | 20989586
2019-08-17 11:00:00+00 | PopulateVirtualProjection | 9181641
2019-08-17 11:00:00+00 | PreparePlan:LocalPlan | 4029496
2019-08-17 11:00:00+00 | CompilePlan:EEpreexecute | 3623722
2019-08-17 11:00:00+00 | PreparePlan | 1390455
2019-08-17 11:00:00+00 | PreparePlan:DeserializePlan | 1387411
2019-08-17 11:00:00+00 | PreparePlan:EEcompile | 1218123
2019-08-17 11:00:00+00 | BeforePlan | 648197
2019-08-17 11:00:00+00 | Plan | 451724
2019-08-17 11:00:00+00 | CompilePlan | 442172
2019-08-17 11:00:00+00 | CompilePlan:ReserveResources | 431791
2019-08-17 11:00:00+00 | BeforePlan:AnalyzeRewrite | 292145
2019-08-17 11:00:00+00 | PreparePlan:TunePlanResources | 271059
2019-08-17 11:00:00+00 | AbandonPlan | 156407
2019-08-17 11:00:00+00 | InitPlan | 110731
2019-08-17 11:00:00+00 | PreparePlan:DistPlanner | 94158
2019-08-17 11:00:00+00 | PreparePlan:TakeTableLocks | 63701
2019-08-17 11:00:00+00 | SerializePlan | 38715
2019-08-17 11:00:00+00 | BeforePlan:Parser | 1613
2019-08-17 11:00:00+00 | Remote Execute Plan | 352
2019-08-17 11:00:00+00 | ExecutePlan:RemoteInitiator | 3
2019-08-17 11:00:00+00 | UpdateOneExecutorPlan | 3
2019-08-17 12:00:00+00 | ExecutePlan | 24478671
2019-08-17 12:00:00+00 | PopulateVirtualProjection | 10275414
2019-08-17 12:00:00+00 | PreparePlan:LocalPlan | 4850834
2019-08-17 12:00:00+00 | CompilePlan:EEpreexecute | 3928011
2019-08-17 12:00:00+00 | PreparePlan | 1654913
2019-08-17 12:00:00+00 | PreparePlan:DeserializePlan | 1586926
2019-08-17 12:00:00+00 | PreparePlan:EEcompile | 1328729
2019-08-17 12:00:00+00 | BeforePlan | 674690
2019-08-17 12:00:00+00 | CompilePlan | 528653
2019-08-17 12:00:00+00 | Plan | 445316
2019-08-17 12:00:00+00 | PreparePlan:TunePlanResources | 307809
2019-08-17 12:00:00+00 | BeforePlan:AnalyzeRewrite | 303754
2019-08-17 12:00:00+00 | AbandonPlan | 185929
2019-08-17 12:00:00+00 | CompilePlan:ReserveResources | 182448
2019-08-17 12:00:00+00 | InitPlan | 119301
2019-08-17 12:00:00+00 | PreparePlan:DistPlanner | 102881
2019-08-17 12:00:00+00 | PreparePlan:TakeTableLocks | 63809
2019-08-17 12:00:00+00 | SerializePlan | 44257
2019-08-17 12:00:00+00 | BeforePlan:Parser | 1237
2019-08-17 12:00:00+00 | Remote Execute Plan | 417
2019-08-17 12:00:00+00 | ExecutePlan:RemoteInitiator | 1
2019-08-17 12:00:00+00 | UpdateOneExecutorPlan | 1

1 reply

mosheg
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • August 18, 2019

Thank you Eugenia :)
It was Bentzi queries on DC he asked about