Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar


Copy link to this message
-
Re: Programmatic (java) Sqoop import from MySQL to HBase Could not find hadoop core jar
Hi Amit,
the parameter --hadoop-home should be pointing to $HADOOP_HOME, not to a $HADOOP_HOME/bin.

Jarcec

On Sun, Nov 03, 2013 at 03:06:38PM +0200, Amit Sela wrote:
> I'm running Hadoop 1.0.4 (and HBAse 0.94.2) and the directory I specify for
> --hadoop-home is: "$HADOOP_HOME/bin/..".
> That's the exact definition in our system.
>
>
> On Sat, Nov 2, 2013 at 3:29 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Amit,
> > what Hadoop version are you using? Would you mind sharing with us listing
> > of the directory that you specified in the --hadoop-home argument?
> >
> > Please note that Sqoop 1 do not have officially supported client API and
> > thus directly using Sqoop.runTool() is not supported and you might run into
> > a lot of issues such as this one.
> >
> > Jarcec
> >
> > On Thu, Oct 31, 2013 at 11:09:05AM +0200, Amit Sela wrote:
> > > That's the stack trace for the exception I got:
> > > Encountered IOException running import job: java.io.IOException: Could
> > not
> > > find hadoop core jar!
> > > at
> > >
> > org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:146)
> > > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)
> > > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)
> > > at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
> > > at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> > > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> > > at
> > >
> > com.infolinks.redmap.services.impl.utils.Sqooper.importMysqlToHBase(Sqooper.java:35)
> > > at
> > >
> > com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:79)
> > > at
> > >
> > com.infolinks.redmap.services.impl.submitters.SqoopSubmitter$1.call(SqoopSubmitter.java:76)
> > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > at
> > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> > > at
> > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> > > at
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> > > at
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> > > at java.lang.Thread.run(Thread.java:722)
> > >
> > > My infrastructure allows me to execute MR jobs and to access HBase so I
> > > don't understand why executing Sqoop results with "Could not find hadoop
> > > core jar!" unless Sqoop is actually looking for the jar to deploy it for
> > > some reason...
> > >
> > >
> > >
> > > On Wed, Oct 30, 2013 at 11:14 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]
> > >wrote:
> > >
> > > > Hi Amit,
> > > > would you mind sharing the entire stack trace that you are getting?
> > > >
> > > > Please note that Sqoop 1 do not have officially supported client API
> > and
> > > > thus directly using Sqoop.runTool() is not supported and you might run
> > into
> > > > a lot of issues such as this one.
> > > >
> > > > Jarcec
> > > >
> > > > On Sun, Oct 27, 2013 at 05:52:30PM +0200, Amit Sela wrote:
> > > > > Which jar ?
> > > > >
> > > > > Hadoop jars are in "HADOOP_HOME/lib" and also sqoop jar.
> > > > > I'm trying to execute the import from an OSGi running client, with a
> > > > maven
> > > > > dependency and embedding on sqoop.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Sun, Oct 27, 2013 at 5:15 PM, Krishnan K <[EMAIL PROTECTED]>
> > > > wrote:
> > > > >
> > > > > > Can you try adding the jar to HADOOP_CLASSPATH / CLASSPATH ?
> > > > > >
> > > > > >
> > > > > > On Sun, Oct 27, 2013 at 7:34 PM, Amit Sela <[EMAIL PROTECTED]>
> > > > wrote:
> > > > > >
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I'm new to Sqoop and I'm trying to execute (from Java code) a
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB