Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Good practices using a jar with hadoop jobs


Copy link to this message
-
Good practices using a jar with hadoop jobs
Let's say I have a /home/me/foo.jar which contains a main that runs a hadoop
job and once it get completed launches another job (a pipeline of a couple
of jobs). The jar contains all the hadoop libs and other stuff needed too. I
launch it with hadoop jar /home/me/foo.jar.
If while the first job is running I do some changes to the code (that just
affect the second job, or neither of them) and upload the new compiled jar
to /home/me/foo.jar, once the job that was running finishes and the second
tries to start, everything gets broken. If I launch everything form the
beginning again, it works with no problems.
The thing is that I have this execution croned, so , every time I change
something, I have to stop the cron, wait for the execution to end, upload
the new jar and activate the cron again. This way I can avid the crash.
Are there any good practices about doing this kind of uploads?
Thanks in advance.

--
View this message in context: http://lucene.472066.n3.nabble.com/Good-practices-using-a-jar-with-hadoop-jobs-tp3085755p3085755.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB