Hi Experts,

I have performance issue while doing duplicate check before saving the data. Below is my analysis.

The logic to check duplicate records does the following 

remember_occ = $curocc(entity_name)
  setocc "entity_name", 1
while ($status > 0)

getitem/id junk, local_list, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
    
    if ($status > 0)
     ; raise duplicate error message
    endif

putitem local_list, -1, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
setocc "entity_name", $curocc(entity_name)+1
    endwhile
    setocc "entity_name",remember_occ

If the logic above is commented then data changes are saved within a minute else it takes more than 10minutes for 17thousand records.

also this is a legacy logic written in earlier version(s) of uniface hence there might be issues in it. Current version of uniface that we are working is Uniface 10.

please suggest!

thanks,

imran.

Hi Experts,

I have performance issue while doing duplicate check before saving the data. Below is my analysis.

The logic to check duplicate records does the following 

remember_occ = $curocc(entity_name)
  setocc "entity_name", 1
while ($status > 0)

getitem/id junk, local_list, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
    
    if ($status > 0)
     ; raise duplicate error message
    endif

putitem local_list, -1, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
setocc "entity_name", $curocc(entity_name)+1
    endwhile
    setocc "entity_name",remember_occ

If the logic above is commented then data changes are saved within a minute else it takes more than 10minutes for 17thousand records.

also this is a legacy logic written in earlier version(s) of uniface hence there might be issues in it. Current version of uniface that we are working is Uniface 10.

please suggest!

thanks,

imran.

Hi Experts,

I have performance issue while doing duplicate check before saving the data. Below is my analysis.

The logic to check duplicate records does the following 

remember_occ = $curocc(entity_name)
  setocc "entity_name", 1
while ($status > 0)

getitem/id junk, local_list, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
    
    if ($status > 0)
     ; raise duplicate error message
    endif

putitem local_list, -1, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
setocc "entity_name", $curocc(entity_name)+1
    endwhile
    setocc "entity_name",remember_occ

If the logic above is commented then data changes are saved within a minute else it takes more than 10minutes for 17thousand records.

also this is a legacy logic written in earlier version(s) of uniface hence there might be issues in it. Current version of uniface that we are working is Uniface 10.

please suggest!

thanks,

imran.

Hi,

AFAIK the problem is, that you combine idlist and index-list - so you force the Uniface to rebuild an index with every statement... the fix is very easy, use idlist only...

I mean... do not use putitem to insert a new item into an idlist. You should use putitem/id instead.

So - keep the line getitem/id junk, local_list, "xxx" as is,

but change the putitem to: putitem/id local_list, "xxx", ""

This should fix your problem.

Zdeněk

HI,

Are you able to add a candidate key to the field combination? If you could, findkey would be able to look for duplicates.

Kind regards,

Mike

So, I think the issue is that all 'list' handling in uniface is actually string searching, and there are no indexes, even for indexed lists, and the string searching slows right down for 'large' string values. 

You could replace the string with a struct pretty easily. 

variables
	struct v_done
	string v_id
endvariables

	remember_occ = $curocc("entity_name")
	v_done = $newstruct
   	forentity "entity_name"
	
		v_id = "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
      	if (v_done->"%%v_id%%%" != "")
     		; raise duplicate error message
		endif
		v_done->"%%v_id%%%" = "T"
    endfor
	v_done = $newstruct ; Clear memory
	setocc "entity_name", remember_occ

I think this will handle WAY faster than any kind of string handling (either putitem or putitem/id)

Iain

HI,

Are you able to add a candidate key to the field combination? If you could, findkey would be able to look for duplicates.

Kind regards,

Mike