Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Interaction between Java and Transform Scripts on Hive


Copy link to this message
-
Interaction between Java and Transform Scripts on Hive
I am perplexed  if I run a transform script on a file by itself, it runs
fine, outputs to standard out life is good. If I run the transform script
on that same file (with the path and filename being passed into the script
via transform so that the python script is doing the exact same thing) I
get a java heap space error. This process works on 99% of files, and I just
can't figure out why this file is different.  How does say a python
transform script run "in" the java process (if that is even what it is
doing) so that it causes a heap error in a transform script but not run
without java around?

I am curious on what steps I can take to trouble shoot or eliminate this
problem.
+
Dean Wampler 2013-01-17, 04:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB