Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar


Copy link to this message
-
Re: Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar
Hi Amit,
what Hadoop version are you using? Would you mind sharing with us listing of the directory that you specified in the --hadoop-home argument?

Please note that Sqoop 1 do not have officially supported client API and thus directly using Sqoop.runTool() is not supported and you might run into a lot of issues such as this one.

Jarcec

On Thu, Oct 31, 2013 at 11:09:05AM +0200, Amit Sela wrote:
> That's the stack trace for the exception I got:
> Encountered IOException running import job: java.io.IOException: Could not
> find hadoop core jar!
> at
> org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:146)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at
> com.infolinks.redmap.services.impl.utils.Sqooper.importMysqlToHBase(Sqooper.java:35)
> at
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:79)
> at
> com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:76)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
>
> My infrastructure allows me to execute MR jobs and to access HBase so I
> don't understand why executing Sqoop results with "Could not find hadoop
> core jar!" unless Sqoop is actually looking for the jar to deploy it for
> some reason...
>
>
>
> On Wed, Oct 30, 2013 at 11:14 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Amit,
> > would you mind sharing the entire stack trace that you are getting?
> >
> > Please note that Sqoop 1 do not have officially supported client API and
> > thus directly using Sqoop.runTool() is not supported and you might run into
> > a lot of issues such as this one.
> >
> > Jarcec
> >
> > On Sun, Oct 27, 2013 at 05:52:30PM +0200, Amit Sela wrote:
> > > Which jar ?
> > >
> > > Hadoop jars are in "HADOOP_HOME/lib" and also sqoop jar.
> > > I'm trying to execute the import from an OSGi running client, with a
> > maven
> > > dependency and embedding on sqoop.
> > >
> > >
> > >
> > >
> > > On Sun, Oct 27, 2013 at 5:15 PM, Krishnan K <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Can you try adding the jar to HADOOP_CLASSPATH / CLASSPATH ?
> > > >
> > > >
> > > > On Sun, Oct 27, 2013 at 7:34 PM, Amit Sela <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I'm new to Sqoop and I'm trying to execute (from Java code) a sqoop
> > > >> import from MySQL to HBase.
> > > >>
> > > >> I'm using Sqoop.runTool() with a String[] containing all the necessary
> > > >> parameters, I even added "--hadoop-home" with the value of my
> > $HADOOP_HOME.
> > > >>
> > > >> Anyone had this problem before ? any ideas ?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Amit.
> > > >>
> > > >
> > > >
> >