Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar


Copy link to this message
-
Re: Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar
Jarek Jarcec Cecho 2013-11-02, 01:29
Hi Amit,
what Hadoop version are you using? Would you mind sharing with us listing of the directory that you specified in the --hadoop-home argument?

Please note that Sqoop 1 do not have officially supported client API and thus directly using Sqoop.runTool() is not supported and you might run into a lot of issues such as this one.

Jarcec

On Thu, Oct 31, 2013 at 11:09:05AM +0200, Amit Sela wrote:
> That's the stack trace for the exception I got:
> Encountered IOException running import job: java.io.IOException: Could not
> find hadoop core jar!
> at
> org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:146)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at
> com.infolinks.redmap.services.impl.utils.Sqooper.importMysqlToHBase(Sqooper.java:35)
> at
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:79)
> at
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:76)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> My infrastructure allows me to execute MR jobs and to access HBase so I
> don't understand why executing Sqoop results with "Could not find hadoop
> core jar!" unless Sqoop is actually looking for the jar to deploy it for
> some reason...
>
>
>
> On Wed, Oct 30, 2013 at 11:14 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Amit,
> > would you mind sharing the entire stack trace that you are getting?
> >
> > Please note that Sqoop 1 do not have officially supported client API and
> > thus directly using Sqoop.runTool() is not supported and you might run into
> > a lot of issues such as this one.
> >
> > Jarcec
> >
> > On Sun, Oct 27, 2013 at 05:52:30PM +0200, Amit Sela wrote:
> > > Which jar ?
> > >
> > > Hadoop jars are in "HADOOP_HOME/lib" and also sqoop jar.
> > > I'm trying to execute the import from an OSGi running client, with a
> > maven
> > > dependency and embedding on sqoop.
> > >
> > >
> > >
> > >
> > > On Sun, Oct 27, 2013 at 5:15 PM, Krishnan K <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Can you try adding the jar to HADOOP_CLASSPATH / CLASSPATH ?
> > > >
> > > >
> > > > On Sun, Oct 27, 2013 at 7:34 PM, Amit Sela <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I'm new to Sqoop and I'm trying to execute (from Java code) a sqoop
> > > >> import from MySQL to HBase.
> > > >>
> > > >> I'm using Sqoop.runTool() with a String[] containing all the necessary
> > > >> parameters, I even added "--hadoop-home" with the value of my
> > $HADOOP_HOME.
> > > >>
> > > >> Anyone had this problem before ? any ideas ?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Amit.
> > > >>
> > > >
> > > >
> >