Guys,

The notorious error. Used all possible clues to resolve this.

Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord

This exact same job ran before.

I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside  sqoop-1.4.1-cdh4.1.2.jar...

I copied this jar file to every possible directory:

/usr/lib/hadoop

/usr/lib/hadoop/lib

As well as mapreduce-0.20 libs

Even tried to include this jar at command line -libjars which is added to the job...

But MR still doesn't pick it up, throwing ClassNotFoundError error.

Rgds,

AK

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel

Oh my Gosh!!!

This exact error happened to me before, the very first time I installed hadoop. Spent days solving that exact problem. It got resolved on its own so I had no idea what exactly I did.

Today, I was installing hadoop on a brand new node from afresh, and... exact the same ClassNotfoundError no matter where to  I copied this jar file to.

I did it again, accidentally resolved this issue .. but THIS TIME I took a note what I did.

Ready?

So, no matter where the jar file was located, no matter if it was indicated in CLASSPATH or HADOOP_HOME or -libjars for that matter, it would not pick up the .jar file until I restarted the TaskTracker.  Heheheh

You know the saying that apple a day keeps doctors away? Well, in the future, as the VERY FIRST possible solution I will always try to restart the nodes first.

For those who find this post ever useful, "you're welcome in advance".

Happy hadooping..

AK-47
From: Kartashov, Andy [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 16, 2012 4:13 PM
To: [EMAIL PROTECTED]
Subject: notorious ClassNotFoundError
Guys,

The notorious error. Used all possible clues to resolve this.

Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord

This exact same job ran before.

I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside  sqoop-1.4.1-cdh4.1.2.jar...

I copied this jar file to every possible directory:

/usr/lib/hadoop

/usr/lib/hadoop/lib

As well as mapreduce-0.20 libs

Even tried to include this jar at command line -libjars which is added to the job...

But MR still doesn't pick it up, throwing ClassNotFoundError error.

Rgds,

AK

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

External libs must exist on all task nodes, in case you haven't already
done that.

/* Joey */
On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <[EMAIL PROTECTED]>wrote:

>  Guys,
>
>
>
> The notorious error. Used all possible clues to resolve this.
>
>
>
> Ran sqoop --import.... at command line without problems. But whenever am
> trying to execute MR job get Error: java.lang.ClassNotFoundException:
> com.cloudera.sqoop.lib.SqoopRecord
>
> This exact same job ran before.
>
>
>
> I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside
> sqoop-1.4.1-cdh4.1.2.jar...
>
>
>
> I copied this jar file to every possible directory:
>
> /usr/lib/hadoop
>
> /usr/lib/hadoop/lib
>
> As well as mapreduce-0.20 libs
>
>
>
> Even tried to include this jar at command line -libjars which is added to
> the job...
>
>
>
> But MR still doesn't pick it up, throwing ClassNotFoundError error.
>
>
>
> Rgds,
>
> AK
>
>
>  NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel
>

I am running MR in pseudo distributed mode - so it is only one Node involced.

From: Joey Krabacher [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 16, 2012 4:16 PM
To: [EMAIL PROTECTED]
Subject: Re: notorious ClassNotFoundError

External libs must exist on all task nodes, in case you haven't already done that.

/* Joey */

On Fri, Nov 16, 2012 at 3:13 PM, Kartashov, Andy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

Guys,

The notorious error. Used all possible clues to resolve this.

Ran sqoop --import.... at command line without problems. But whenever am trying to execute MR job get Error: java.lang.ClassNotFoundException: com.cloudera.sqoop.lib.SqoopRecord

This exact same job ran before.

I can clearly see the com/cloudera/sqoop/lib/SqoopRecord.class inside  sqoop-1.4.1-cdh4.1.2.jar...

I copied this jar file to every possible directory:

/usr/lib/hadoop

/usr/lib/hadoop/lib

As well as mapreduce-0.20 libs

Even tried to include this jar at command line -libjars which is added to the job...

But MR still doesn't pick it up, throwing ClassNotFoundError error.

Rgds,

AK

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel