Data Express - Distributed sampling validation

Problem:

  Distributed sampling allows two types of execution:

1. Direct – starts sampling, by pressing the “Start” button in the Distributed Sampling menu. It exports comma separated side files (sampling.dat, method.rc) in the “config” subfolder of the Data Express extension installation path, produces sampling results and imports them into the client knowledge base.
2. Manual – the user exports the comma separated side files, starts sampling manually from the console and navigates to “Load results from file” in Data Builder to load the produced sampling results file into the client knowledge base.

The insert into the knowledge base is driven by IDs (Machine ID and Company, Data store, Data element…). When the process is started directly, no validation is needed, because all phases of sampling are done simultaneously. When the process is performed manually, however, the user could attempt to insert a wrong sampling results file into the knowledge base (having wrong IDs). In this case, Data Builder could show SQL error messages or introduce inconsistent results.

Resolution:

A new component “Sampling data file” is added to the “Load results from file” tab in the “Distributed Sampler” functionality. It should contain the sampling.dat file that has been exported in the “config” subfolder prior to the sampling execution. Its purpose is to make sure that the results, that are about to be inserted in the knowledge base, match the data stores that will hold them. This component is not mandatory, leaving it empty will discard the validation and the results will be inserted in the data stores, indicated in the sampling results file.