Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> copyToLocal Failed inside the cleanup(.........) of Map task


Copy link to this message
-
copyToLocal Failed inside the cleanup(.........) of Map task
Dear All,

     Is there any way to copy the intermediate output file of the mapper
into     local  folder  after each map task complete.

   Right now I am using

   FileSystem.copyToLocalFile(hdfsLocation,localLocation);
 indiste the cleanup of mapper task , but it is failing .

Exception file not found.

But if I am giving same statement after the job complete in driver class
,it is working fine. that i dont want.

protected void cleanup(Context context){
   FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
}

Note: I need to copy the inter mediate output of the mapper to local file
system just after  each map task complete. I dont want any reducer.

   If this is not the right solution then how to solve this type of
scenario.

Any help.............

regards,
samir.