Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> TableMapReduceUtil.findOrCreateJar throwing NPE


Copy link to this message
-
Re: TableMapReduceUtil.findOrCreateJar throwing NPE
Oh, got it, I'll add to Jira :)
On Sun, Dec 1, 2013 at 5:43 PM, Amit Sela <[EMAIL PROTECTED]> wrote:

> Created https://issues.apache.org/jira/browse/HBASE-10061
>
> What is the config parameter that controls whether the error should be
> logged or exception should be thrown ?
>
>
> On Sun, Dec 1, 2013 at 5:23 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> In your scenario, jar is null.
>> Mind filing a JIRA for the null check ?
>>
>> For your second question, a config parameter can be introduced to control
>> whether the error should be logged or exception should be thrown
>>
>> Cheers
>>
>> On Dec 1, 2013, at 6:49 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>>
>> > I'm testing HBase 0.94.12 (Hadoop 1.0.4) in our systems and I get NPE
>> when
>> > setting up bulk load.
>> > I'll start by noting that we run a Hadoop/HBase OSGi client that runs
>> with
>> > a bundled version of Hadoop and HBase.
>> >
>> > We currently run in production with a bundled 0.94.2 client we created
>> and
>> > everything works although TableMapReduceUtil.addDependencyJars is
>> logging
>> > WARNs like "Could not find jar for class class... in order to ship it to
>> > the cluster" and that happens because of CL issues. It doesn't matter
>> much
>> > because those classes are available in all nodes/region servers
>> classpath.
>> >
>> > When testing HBase 0.94.12 (again bundled here) I get the following NPE:
>> >
>> > Caused by: java.lang.NullPointerException: null
>> > at java.io.File.<init>(File.java:251)
>> > at java.util.zip.ZipFile.<init>(ZipFile.java:115)
>> > at
>> >
>> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.updateMap(TableMapReduceUtil.java:617)
>> > at
>> >
>> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:597)
>> > at
>> >
>> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:557)
>> > at
>> >
>> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:518)
>> > at
>> >
>> com.infolinks.hadoop.framework.InfolinksHFileOutputFormat.configureIncrementalLoad(InfolinksHFileOutputFormat.java:114)
>> > at
>> >
>> com.infolinks.redmap.services.impl.dailybatchprocess.HadoopJobBulkLoader.initBulkLoad(HadoopJobBulkLoader.java:95)
>> > at
>> >
>> com.infolinks.redmap.services.impl.dailybatchprocess.HadoopJobBulkLoader.preSplitAndInitBulkLoad(HadoopJobBulkLoader.java:60)
>> > at
>> >
>> com.infolinks.redmap.services.impl.dailybatchprocess.UrlsBulkLoadProcess.init(UrlsBulkLoadProcess.java:40)
>> > ... 9 common frames omitted
>> >
>> > Since getJar(my_class) (in TableMapReduceUtil) may return null, calling
>> > updateMap(null, ) will throw NPE.
>> > Should check null==jar before calling updateMap ?
>> > Why not allow failure to add dependency jars and just log a WARN ?
>> >
>> > Thanks,
>> > Amit.
>>
>
>