Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> TableMapReduceUtil.findOrCreateJar throwing NPE


Copy link to this message
-
Re: TableMapReduceUtil.findOrCreateJar throwing NPE
In your scenario, jar is null.
Mind filing a JIRA for the null check ?

For your second question, a config parameter can be introduced to control whether the error should be logged or exception should be thrown

Cheers

On Dec 1, 2013, at 6:49 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> I'm testing HBase 0.94.12 (Hadoop 1.0.4) in our systems and I get NPE when
> setting up bulk load.
> I'll start by noting that we run a Hadoop/HBase OSGi client that runs with
> a bundled version of Hadoop and HBase.
>
> We currently run in production with a bundled 0.94.2 client we created and
> everything works although TableMapReduceUtil.addDependencyJars is logging
> WARNs like "Could not find jar for class class... in order to ship it to
> the cluster" and that happens because of CL issues. It doesn't matter much
> because those classes are available in all nodes/region servers classpath.
>
> When testing HBase 0.94.12 (again bundled here) I get the following NPE:
>
> Caused by: java.lang.NullPointerException: null
> at java.io.File.<init>(File.java:251)
> at java.util.zip.ZipFile.<init>(ZipFile.java:115)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.updateMap(TableMapReduceUtil.java:617)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:597)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:557)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:518)
> at
> com.infolinks.hadoop.framework.InfolinksHFileOutputFormat.configureIncrementalLoad(InfolinksHFileOutputFormat.java:114)
> at
> com.infolinks.redmap.services.impl.dailybatchprocess.HadoopJobBulkLoader.initBulkLoad(HadoopJobBulkLoader.java:95)
> at
> com.infolinks.redmap.services.impl.dailybatchprocess.HadoopJobBulkLoader.preSplitAndInitBulkLoad(HadoopJobBulkLoader.java:60)
> at
> com.infolinks.redmap.services.impl.dailybatchprocess.UrlsBulkLoadProcess.init(UrlsBulkLoadProcess.java:40)
> ... 9 common frames omitted
>
> Since getJar(my_class) (in TableMapReduceUtil) may return null, calling
> updateMap(null, ) will throw NPE.
> Should check null==jar before calling updateMap ?
> Why not allow failure to add dependency jars and just log a WARN ?
>
> Thanks,
> Amit.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB