Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - copyToLocal Failed inside the cleanup(.........) of Map task


Copy link to this message
-
copyToLocal Failed inside the cleanup(.........) of Map task
samir das mohapatra 2013-06-03, 21:11
Dear All,

     Is there any way to copy the intermediate output file of the mapper
into     local  folder  after each map task complete.

   Right now I am using

   FileSystem.copyToLocalFile(hdfsLocation,localLocation);
 indiste the cleanup of mapper task , but it is failing .

Exception file not found.

But if I am giving same statement after the job complete in driver class
,it is working fine. that i dont want.

protected void cleanup(Context context){
   FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
}

Note: I need to copy the inter mediate output of the mapper to local file
system just after  each map task complete. I dont want any reducer.

   If this is not the right solution then how to solve this type of
scenario.

Any help.............

regards,
samir.