Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - copyToLocal Failed inside the cleanup(.........) of Map task


Copy link to this message
-
Re: copyToLocal Failed inside the cleanup(.........) of Map task
Jay Vyas 2013-06-03, 22:05
Just for context and so that we can understand the question --- why are you needing to copy intermediate mapper output?

On Jun 3, 2013, at 4:11 PM, samir das mohapatra <[EMAIL PROTECTED]> wrote:

> Dear All,
>
>      Is there any way to copy the intermediate output file of the mapper into     local  folder  after each map task complete.
>
>    Right now I am using
>    
>    FileSystem.copyToLocalFile(hdfsLocation,localLocation);
>  indiste the cleanup of mapper task , but it is failing .
>
> Exception file not found.
>
> But if I am giving same statement after the job complete in driver class ,it is working fine. that i dont want.
>
> protected void cleanup(Context context){
>    FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
> }
>
> Note: I need to copy the inter mediate output of the mapper to local file system just after  each map task complete. I dont want any reducer.
>
>    If this is not the right solution then how to solve this type of scenario.
>  
> Any help.............
>
> regards,
> samir.
> --
>  
>  
>