Uniface User Forum

Hi Experts,

I have performance issue while doing duplicate check before saving the data. Below is my analysis.

The logic to check duplicate records does the following 

remember_occ = $curocc(entity_name)
  setocc "entity_name", 1
while ($status > 0)

getitem/id junk, local_list, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
    
    if ($status > 0)
     ; raise duplicate error message
    endif

putitem local_list, -1, "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
setocc "entity_name", $curocc(entity_name)+1
    endwhile
    setocc "entity_name",remember_occ

If the logic above is commented then data changes are saved within a minute else it takes more than 10minutes for 17thousand records.

also this is a legacy logic written in earlier version(s) of uniface hence there might be issues in it. Current version of uniface that we are working is Uniface 10.

please suggest!

thanks,

imran.

Hi,

AFAIK the problem is, that you combine idlist and index-list - so you force the Uniface to rebuild an index with every statement... the fix is very easy, use idlist only...

I mean... do not use putitem to insert a new item into an idlist. You should use putitem/id instead.

So - keep the line getitem/id junk, local_list, "xxx" as is,

but change the putitem to: putitem/id local_list, "xxx", ""

This should fix your problem.

Zdeněk

the suggested change worked. Thanks!

HI,

Are you able to add a candidate key to the field combination? If you could, findkey would be able to look for duplicates.

Kind regards,

Mike

Hi Imran,

just an addendum to Mike answer: if the combination of your three field (

"%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"

) is the primary key of your table an instruction retrieve/o too is searching either on dbms and in memory; when found in memory it is repositioning automatically on the existing record.

Regards,
Gianni

Thank you all for the inputs. These should be useful going forward.

currently this issue is closed and customer is happy with fix . I  will be trying them if required.

Regards,

Imran.

So, I think the issue is that all 'list' handling in uniface is actually string searching, and there are no indexes, even for indexed lists, and the string searching slows right down for 'large' string values. 

You could replace the string with a struct pretty easily. 

variables
	struct v_done
	string v_id
endvariables

	remember_occ = $curocc("entity_name")
	v_done = $newstruct
   	forentity "entity_name"
	
		v_id = "%%field1.entity_name|%%field2.entity_name|%%field3.entity_name"
      	if (v_done->"%%v_id%%%" != "")
     		; raise duplicate error message
		endif
		v_done->"%%v_id%%%" = "T"
    endfor
	v_done = $newstruct ; Clear memory
	setocc "entity_name", remember_occ

I think this will handle WAY faster than any kind of string handling (either putitem or putitem/id)

Iain

Thanks Iain for the inputs.

the customer is happy with performance after replacing putitem with putitem/id. I will try out your suggestion if required. 

Thanks again!