Guys,

Is there another way to set the output from within the Mapper?

My Mapper reads from various serialised files and generates different type of objects for values depending on the returned value of instanceof. I wanted to change the class name within the Mapper as opposed to the driver class. Anyway of doing it?

Thanks,
Andy

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel

Hi Andy,

It should be called within the setup of your job and it should NOT be
called within the Mapper.
You could compile the call within the mapper of course. At best it should
do nothing. I would say in practice the job will fail. But that's something
that you have to verify if you are curious.

Back to your original problem, the mapper need to output the same kind of
object in order to be able to unserialize them for the mapper. (More
generally the serialization strategy must be able to figure out what was
serialized.) It does not mean your problem is not solvable but it means
that you will need to come up with a wrapper or a common structure.

How do you read different kind of objects with the Mapper? Maybe you could
use the same approach.

Regards

Bertrand

On Thu, Oct 11, 2012 at 7:49 PM, Kartashov, Andy <[EMAIL PROTECTED]>wrote:

>    Guys,
>
>
>
> Is there another way to set the output from within the Mapper?
>
>
>
> My Mapper reads from various serialised files and generates different type
> of objects for values depending on the returned value of instanceof. I
> wanted to change the class name within the Mapper as opposed to the driver
> class. Anyway of doing it?
>
>
>
> Thanks,
>
> Andy
>
>
>  NOTICE: This e-mail message and any attachments are confidential,
> subject to copyright and may be privileged. Any unauthorized use, copying
> or disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>

--
Bertrand Dechoux

Bertand

I am reading my input by declaring it in the driver as follows:

conf.setInputFormat(SequenceFileInputFormat.class);
In the mapper:
public void map(LongWritable key, SomeInterfaceClass value,
                        OutputCollector<Text, SomeInterfaceClass > output, Reporter reporter)
                                throws IOException {
                SomeInterfaceClass o = null;
                        if ( value instanceof classA) {
                          o = (classA)value;
                  }
                        if ( value instanceof classB) {
                          o = (classB)value;
                  }
                output.collect(new Text(o.get_property_id().toString()), o);
        } // end of map()
     } // end of static class

As u can see "o" can be an instance of classA or classB, where both implement SomeInerfaceClass.

However, when I try to write in the driver:
Conf.setOutputValueClass(SomeIterfaceClass.class);
And run the job I get the following error

Error: java.io.IOException: Type mismatch in value from map: expected SomeInterfaceClass, received ClassA

I wished I could change the outputClass name depending on the instance of "o" to either ClassA or ClassB.

From: Bertrand Dechoux [mailto:[EMAIL PROTECTED]]
Sent: Thursday, October 11, 2012 2:12 PM
To: [EMAIL PROTECTED]
Subject: Re: setOutputValueClass(*.class) method

Hi Andy,

It should be called within the setup of your job and it should NOT be called within the Mapper.
You could compile the call within the mapper of course. At best it should do nothing. I would say in practice the job will fail. But that's something that you have to verify if you are curious.

Back to your original problem, the mapper need to output the same kind of object in order to be able to unserialize them for the mapper. (More generally the serialization strategy must be able to figure out what was serialized.) It does not mean your problem is not solvable but it means that you will need to come up with a wrapper or a common structure.

How do you read different kind of objects with the Mapper? Maybe you could use the same approach.

Regards

Bertrand
On Thu, Oct 11, 2012 at 7:49 PM, Kartashov, Andy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Guys,

Is there another way to set the output from within the Mapper?

My Mapper reads from various serialised files and generates different type of objects for values depending on the returned value of instanceof. I wanted to change the class name within the Mapper as opposed to the driver class. Anyway of doing it?

Thanks,
Andy

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

--
Bertrand Dechoux
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

Bertrand,

I did it!!!!  :)))))))))))))))))))))))))))  using your suggested wrapper. Cheers man.
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel