Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - MR - Input from Hbase output to HDFS


Copy link to this message
-
Re: MR - Input from Hbase output to HDFS
Joey Echeverria 2011-11-11, 17:08
There are two APIs (old and new), and you appear to be mixing them.
TableMapReduceUtil only works with the new API. The solution is to
import the new version of FileOutputFormat which takes a Job:
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

-Joey

On Fri, Nov 11, 2011 at 12:55 AM, Stuti Awasthi <[EMAIL PROTECTED]> wrote:
> The method " setOutputPath (JobConf,Path)" take JobConf as a parameter not the Job object.
> At least this is the error Im getting while compiling with Hadoop 0.20.2 jar with eclipse.
>
> FileOutputFormat.setOutputPath(conf, new Path("/output"));
>
> -----Original Message-----
> From: Prashant Sharma [mailto:[EMAIL PROTECTED]]
> Sent: Friday, November 11, 2011 11:20 AM
> To: [EMAIL PROTECTED]
> Subject: Re: MR - Input from Hbase output to HDFS
>
> Hi stuti,
> I was wondering why  you are not using job object to set output path like this.
>
> FileOutputFormat.setOutputPath(job, new Path("outputReadWrite") );
>
>
> thanks
>
> On Fri, Nov 11, 2011 at 10:43 AM, Stuti Awasthi <[EMAIL PROTECTED]>wrote:
>
>> Hi Andrie,
>> Well I am bit confused. When I use Jobconf , and associate with
>> JobClient to run the job then I get the error that "Input directory is not set".
>> Since I want my input to be taken by Hbase table which I already
>> configured with "TableMapReduceUtil.initTableMapperJob". I don't want
>> to set input directory via jobconf.
>> How to mix these 2 so that I can get input from Hbase and write ouput
>> to HDFS.
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Andrei Cojocaru [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, November 10, 2011 7:09 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: MR - Input from Hbase output to HDFS
>>
>> Stuti,
>>
>> I don't see you associating JobConf with Job anywhere.
>> -Andrei
>>
>> ::DISCLAIMER::
>>
>> ----------------------------------------------------------------------
>> -------------------------------------------------
>>
>> The contents of this e-mail and any attachment(s) are confidential and
>> intended for the named recipient(s) only.
>> It shall not attach any liability on the originator or HCL or its
>> affiliates. Any views or opinions presented in this email are solely
>> those of the author and may not necessarily reflect the opinions of
>> HCL or its affiliates.
>> Any form of reproduction, dissemination, copying, disclosure,
>> modification, distribution and / or publication of this message
>> without the prior written consent of the author of this e-mail is
>> strictly prohibited. If you have received this email in error please
>> delete it and notify the sender immediately. Before opening any mail
>> and attachments please check them for viruses and defect.
>>
>>
>> ----------------------------------------------------------------------
>> -------------------------------------------------
>>
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434