Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Interaction between Java and Transform Scripts on Hive


Copy link to this message
-
Interaction between Java and Transform Scripts on Hive
John Omernik 2013-01-17, 03:32
I am perplexed  if I run a transform script on a file by itself, it runs
fine, outputs to standard out life is good. If I run the transform script
on that same file (with the path and filename being passed into the script
via transform so that the python script is doing the exact same thing) I
get a java heap space error. This process works on 99% of files, and I just
can't figure out why this file is different.  How does say a python
transform script run "in" the java process (if that is even what it is
doing) so that it causes a heap error in a transform script but not run
without java around?

I am curious on what steps I can take to trouble shoot or eliminate this
problem.