Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> how to write custom object using M/R


+
Joan 2011-01-14, 12:57
+
MONTMORY Alain 2011-01-14, 18:27
+
Joan 2011-01-17, 08:19
+
Harsh J 2011-01-17, 09:13
Copy link to this message
-
Re: how to write custom object using M/R
Does you custom object have Writable implemented? Also, does it have
toString() implemented? I think this means the Writable code does not
work:

K  CustomObject@2b237512
K  CustomObject@24db06de

This is Java's default toString() method.

On Mon, Jan 17, 2011 at 12:19 AM, Joan <[EMAIL PROTECTED]> wrote:
> Hi Alain,
>
> I put it, but It didn't work.
>
> Joan
>
> 2011/1/14 MONTMORY Alain <[EMAIL PROTECTED]>
>>
>> Hi,
>>
>>
>>
>> I think you have to put :
>>
>>             job.setOutputFormatClass(SequenceFileOutputFormat.class);
>>
>> to make it works..
>>
>> hopes this help
>>
>>
>>
>> Alain
>>
>>
>>
>> [@@THALES GROUP RESTRICTED@@]
>>
>>
>>
>> De : Joan [mailto:[EMAIL PROTECTED]]
>> Envoyé : vendredi 14 janvier 2011 13:58
>> À : mapreduce-user
>> Objet : how to write custom object using M/R
>>
>>
>>
>> Hi,
>>
>> I'm trying to write (K,V) where K is a Text object and V's CustomObject.
>> But It doesn't run.
>>
>> I'm configuring output job like: SequenceFileInputFormat so I have job
>> with:
>>
>>         job.setMapOutputKeyClass(Text.class);
>>         job.setMapOutputValueClass(CustomObject.class);
>>         job.setOutputKeyClass(Text.class);
>>         job.setOutputValueClass(CustomObject.class);
>>
>>         SequenceFileOutputFormat.setOutputPath(job, new Path("myPath");
>>
>> And I obtain the next output (this is a file: part-r-00000):
>>
>> K  CustomObject@2b237512
>> K  CustomObject@24db06de
>> ...
>>
>> When this job finished I run other job which input is
>> SequenceFileInputFormat but It doesn't run:
>>
>> The configuration's second job is:
>>
>>         job.setInputFormatClass(SequenceFileInputFormat.class);
>>         SequenceFileInputFormat.addInputPath(job, new Path("myPath"));
>>
>> But I get an error:
>>
>> java.io.IOException: hdfs://localhost:30000/user/hadoop/out/part-r-00000
>> not a SequenceFile
>>         at
>> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1523)
>>         at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1483)
>>         at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1451)
>>         at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1432)
>>         at
>> org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:60)
>>
>>
>> Can someone help me? Because I don't understand it. I don't know to save
>> my object in first M/R and how to use it in second M/R
>>
>> Thanks
>>
>> Joan
>>
>>
>
>

--
Lance Norskog
[EMAIL PROTECTED]
+
Joan 2011-01-19, 14:36
+
David Rosenstrauch 2011-01-19, 20:04
+
David Rosenstrauch 2011-01-18, 18:56
+
Joan 2011-01-19, 14:42
+
Joan 2011-01-19, 14:50
+
David Rosenstrauch 2011-01-18, 18:49