Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - UDF problem: Java Heap space

Copy link to this message
UDF problem: Java Heap space
Aniket Mokashi 2011-02-24, 03:49
I ve written a simple UDF that parses a chararray (which looks like
...[a].....[b]...[a]...) to capture stuff inside brackets and return them
as String a=2;b=1; and so on. The input chararray are rarely more than
1000 characters and are not more than 100000 (I ve added log.warn in my
udf to ensure this). But, I still see java heap error while running this
udf (even in local mode, the job simply fails). My assumption is maps and
lists that I use locally will be recollected by gc. Am I missing

Dmitriy Ryaboy 2011-02-24, 03:56
Jai Krishna 2011-02-24, 08:58
Aniket Mokashi 2011-02-24, 23:49
Dmitriy Ryaboy 2011-02-25, 00:13
Daniel Dai 2011-02-25, 00:25
Aniket Mokashi 2011-02-25, 00:47
Aniket Mokashi 2011-02-25, 01:26