Skip to main content

fjsonparser() - json file size limit???

  • June 19, 2017
  • 4 replies
  • 15 views

MaciejPaliwoda
Forum|alt.badge.img+2

Hi All,
When I try to load properly formed json file but in size that is ober 300kb fjson parser does not import it into flex table. When I reduce it size (by removing some entries in gateways[] structure - it is loading ok.

files are being generated by Kafka. The problem occurs only for "bigger" json files.

Any hints?

Thx in advance
Maciej

4 replies

Car1os
Forum|alt.badge.img
  • Participating Frequently
  • June 19, 2017

I haven't had issues with file size, I think the problem is on the length of the column in the flex table, you can change it using ALTER TABLE


Ariel_Cary
Forum|alt.badge.img
  • Participating Frequently
  • June 19, 2017

Hi Maciej.

What is the size of the flex __raw__ column? It's possible that the resulting VMap for those large JSON records cannot fit in that column. vertica.log or the COPY error logs should contain indication of that. If that is the case, increasing the size of __raw__ will help you.


MaciejPaliwoda
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • June 19, 2017

works thanks :)
M.


MaciejPaliwoda
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • June 19, 2017
Size was long varbinary(130k) altering to 400k solved the issue.