Loading and Removing Primary Key Duplicates

Well, the WITH clause is a part of SELECT statements, and nothing else. That's how the SQL standard is supposed to be.

And, remember: Vertica is an MPP database; if your table is big enough to be segmented, you have no guarantee to obtain your data in any order you might expect, unless you give it an additional column that you can order by.
The first of the many nodes that manages to deliver its subset of rows will be the first to deliver its rows to the requesting, single-thread, initiator.

If you're lucky, you get a sequence number or a timestamp that you can order by.

I'd load a test_stg table to contain the dupes:
CREATE LOCAL TEMPORARY TABLE test_stg(col1,col2,seq)
ON COMMIT PRESERVE ROWS AS
SELECT 1, 20,1
UNION ALL SELECT 1, 10,2
UNION ALL SELECT 2, 10,3
UNION ALL SELECT 2, 30,4
);

Then, work with this target table:
CREATE TABLE test (
col1 INT NOT NULL PRIMARY KEY
, col2 INT
);

And then use the Analytic LIMIT clause that uses both the pk column and the sequence column:
INSERT INTO test
SELECT
col1
, col2
FROM test_stg
LIMIT 1 OVER(PARTITION BY col1 ORDER BY seq)
;

Thanks Marco. I tried to use a CTE as the requirement was to do this under NO COMMIT mode.

Something like the following might help?

dbadmin=> select * from test;
 col1 | col2
------+------
    1 |   10
    1 |   20
    2 |   10
    2 |   30
(4 rows)

dbadmin=> delete from test where hash(col1 || col2) in (select h from (select hash(col1 || col2) h, row_number() over (partition by col1) rn from test) foo where rn > 1);
 OUTPUT
--------
      2
(1 row)

dbadmin=> select * from test;
 col1 | col2
------+------
    1 |   10
    2 |   10
(2 rows)

But as Marco pointed out, there is no guarantee on the order the records will be returned. That's why you should always have a column called CREATE_TIMESTAMP that stores the date and time the record was created. Then you can order by it ...