Hello,
We are testing some of the machine learning functions in Vertica and we noticed an unexpected result when using the function Balance. We have highly umbalanced dataset, and we wanted to test under_sampling, over_sampling and hybrid_sampling.
dbadmin=> select class,count(*) from creditcard group by class;
class | count
-------+--------
0 | 284314
1 | 492
(2 rows)
SELECT balance ('cc_balanced_hybrid', 'creditcard', 'class', 'hybrid_sampling');
SELECT balance ('cc_balanced_over', 'creditcard', 'class', 'over_sampling');
SELECT balance ('cc_balanced_under', 'creditcard', 'class', 'under_sampling');
We noticed that the view created by hybrid_sampling returns sometimes data from one class only:
dbadmin=> select class, count(class) from cc_balanced_hybrid group by class;
class | count
-------+--------
1 | 142188
(1 row)
dbadmin=> select class, count(class) from cc_balanced_hybrid group by class;
class | count
-------+--------
0 | 284314
1 | 142680
(2 rows)
The view created with under sampling returns only rows from the minority class, regardless of the ratio used:
dbadmin=> select class, count(class) from cc_balanced_under group by class;
class | count
-------+-------
1 | 492
(1 row)
Can any one help us understand this output?
dbadmin=> select version();
version
------------------------------------
Vertica Analytic Database v8.1.1-4
Many thanks
Chaima BERRACHDI