Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> UDF problem: Java Heap space


Copy link to this message
-
UDF problem: Java Heap space
I ve written a simple UDF that parses a chararray (which looks like
...[a].....[b]...[a]...) to capture stuff inside brackets and return them
as String a=2;b=1; and so on. The input chararray are rarely more than
1000 characters and are not more than 100000 (I ve added log.warn in my
udf to ensure this). But, I still see java heap error while running this
udf (even in local mode, the job simply fails). My assumption is maps and
lists that I use locally will be recollected by gc. Am I missing
something?

Thanks,
Aniket
+
Dmitriy Ryaboy 2011-02-24, 03:56
+
Jai Krishna 2011-02-24, 08:58
+
Aniket Mokashi 2011-02-24, 23:49
+
Dmitriy Ryaboy 2011-02-25, 00:13
+
Daniel Dai 2011-02-25, 00:25
+
Aniket Mokashi 2011-02-25, 00:47
+
Aniket Mokashi 2011-02-25, 01:26