-Max Size of Hive String Type
Peter Chu 2013-05-14, 17:08
The results of my hive jobs is inserted in a MySql db.
One of the tables in MySql has a field of text type, which corresponds to the String type in Hive's external table. We now have some rows (though very small amount) that exceeds the limit of text type (65k chars).
We are thinking of changing the type to mediumtext, I am wondering on the Hive side, for String type, is there a limit of number of characters. I have been searching and can only come up with a limit of 2GB on Java String, if so, this would be big enough, can someone please shed some light if this applies to Hive?
How will this hit performance, when it is very large string size, say 100k chars.