Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Filter - Dynamic Jar Load - FilterList not using DynamicClassLoader


Copy link to this message
-
Filter - Dynamic Jar Load - FilterList not using DynamicClassLoader
Federico Gaule 2013-12-02, 18:02
Hi everyone,

I've tried to use dynamic jar load (
https://issues.apache.org/jira/browse/HBASE-1936) but seems to have an
issue with FilterList.
Here is some log from my app where i send a Get with a FilterList
containing AFilter and other with BFilter.

2013-12-02 13:55:42,564 DEBUG
org.apache.hadoop.hbase.util.DynamicClassLoader: Class d.p.AFilter not
found - using dynamical class loader
2013-12-02 13:55:42,564 DEBUG
org.apache.hadoop.hbase.util.DynamicClassLoader: Finding class:
d.p.AFilter
2013-12-02 13:55:42,564 DEBUG
org.apache.hadoop.hbase.util.DynamicClassLoader: Loading new jar
files, if any
2013-12-02 13:55:42,677 DEBUG
org.apache.hadoop.hbase.util.DynamicClassLoader: Finding class again:
d.p.AFilter
2013-12-02 13:55:43,004 ERROR
org.apache.hadoop.hbase.io.HbaseObjectWritable: Can't find class
d.p.BFilter
java.lang.ClassNotFoundException: d.p.BFilter
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.getClassByName(HbaseObjectWritable.java:792)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:679)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
at org.apache.hadoop.hbase.filter.FilterList.readFields(FilterList.java:324)
at org.apache.hadoop.hbase.client.Get.readFields(Get.java:405)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
at org.apache.hadoop.hbase.client.Action.readFields(Action.java:101)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:594)
at org.apache.hadoop.hbase.client.MultiAction.readFields(MultiAction.java:116)
at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:690)
at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:126)
at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1311)
at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1226)
at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

AFilter is not found so it tries with DynamicClassLoader, but when it
tries to load AFilter, it uses URLClassLoader and fails without
checking out for dynamic jars.
I think the issue is releated to FilterList#readFields

public void readFields(final DataInput in) throws IOException { byte opByte
= in.readByte(); operator = Operator.values()[opByte]; int size in.readInt(); if (size > 0) { filters = new ArrayList<Filter>(size); for
(int i = 0; i < size; i++) { Filter filter (Filter)*HbaseObjectWritable.readObject(in,
conf); * filters.add(filter); } } }

*HbaseObjectWritable#readObject *uses a conf (created by calling
HBaseConfiguration.create())
which i suppose doesn't include a DynamicClassLoader instance.

Cheers