Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: copyToLocal Failed inside the cleanup(.........) of Map task


Copy link to this message
-
Re: copyToLocal Failed inside the cleanup(.........) of Map task
Do you have any link or example ? could you please send me ?
On Tue, Jun 4, 2013 at 2:53 AM, Shahab Yunus <[EMAIL PROTECTED]> wrote:

> Have you taken a look into extending the FileOutputFormat class and
> overriding the OutputCommitter API functionality?
>
> Regards,
> Shahab
>
>
> On Mon, Jun 3, 2013 at 5:11 PM, samir das mohapatra <
> [EMAIL PROTECTED]> wrote:
>
>> Dear All,
>>
>>      Is there any way to copy the intermediate output file of the mapper
>> into     local  folder  after each map task complete.
>>
>>    Right now I am using
>>
>>    FileSystem.copyToLocalFile(hdfsLocation,localLocation);
>>  indiste the cleanup of mapper task , but it is failing .
>>
>> Exception file not found.
>>
>> But if I am giving same statement after the job complete in driver class
>> ,it is working fine. that i dont want.
>>
>> protected void cleanup(Context context){
>>    FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
>> }
>>
>> Note: I need to copy the inter mediate output of the mapper to local file
>> system just after  each map task complete. I dont want any reducer.
>>
>>    If this is not the right solution then how to solve this type of
>> scenario.
>>
>> Any help.............
>>
>> regards,
>> samir.
>>
>
>
+
Shahab Yunus 2013-06-03, 22:38
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB