|
|
-
Filter Implementation - Accumulo 1.3
Victoria Bare 2012-10-23, 13:54
Hello,
I am currently using Accumulo 1.3 to implement a Filter. Since I'm using 1.3, I realize that the Filter class is not an iterator so I have created a MyFilter class that implements Filter to use when I initialize my Scanner. When I run my code, I am getting an AccumuloServerException.
I was referencing the posts from December 2011 on "Filter Use" to initialize my scanner with MyFilter. My scanner initialization currently appears as so:
Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers*); Connector connector = zooInstance.getConnector(*userName*, *password*); Authorizations authorizations = new Authorizations(); Scanner scanner = connector.createScanner(*tableName*, authorizations);
scanner.setRange(*range*);
scanner.setScanIterators(1, "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter"); scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter"); scanner.setScanIteratorOption("myFilter", "0.start", *start*);
Iterator<Entry<Key,Value>> iterator = scanner.iterator();
while(iterator.hasNext()) { <--- Exception here
...
}
-------------------------------------------------------------------------------------------------------------------------
public class MyFilter implements Filter{ long startOfRange = 0; @Override public boolean accept(Key key, Value value) { String colqual = key.getColumnQualifier().toString(); long end = Long.parseLong(colqual.substring(20, 39)); if(end < startOfRange){ return false; } return true; }
@Override public void init(Map<String, String> options) {
if(options == null){ throw new IllegalArgumentException("'start' must be set for filter"); } String start = options.get("start"); if(start == null){ throw new IllegalArgumentException("'start' must be set for filter"); } startOfRange = Long.parseLong(start); }
}
-------------------------------------------------------------------------------------------------------------------------
The Exception that I'm receiving is:
Exception in thread "main" java.lang.RuntimeException: org.apache.accumulo.core.client.impl.AccumuloServerException: at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java) at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java) at test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java) at test.offsets.TestFilter.main(TestFilter.java)
-------------------------------------------------------------------------------------------------------------------------
I was thinking that maybe the server couldn't find the MyFilter class, or maybe it was a permissions error, but I wasn't sure. When I initialize my Scanner to use MyFilter, is it looking on the server for the file or in my project?
Any assistance you can provide would be greatly appreciated, thanks! Tori
-
Re: Filter Implementation - Accumulo 1.3
Eric Newton 2012-10-23, 14:15
Check the tablet server logs... you'll see the real problem using the filter in there.
-Eric
On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <[EMAIL PROTECTED]>wrote:
> Hello, > > I am currently using Accumulo 1.3 to implement a Filter. Since I'm using > 1.3, I realize that the Filter class is not an iterator so I have created a > MyFilter class that implements Filter to use when I initialize my Scanner. > When I run my code, I am getting an AccumuloServerException. > > I was referencing the posts from December 2011 on "Filter Use" to > initialize my scanner with MyFilter. > My scanner initialization currently appears as so: > > Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers* > ); > Connector connector = zooInstance.getConnector(*userName*, *password*); > Authorizations authorizations = new Authorizations(); > Scanner scanner = connector.createScanner(*tableName*, authorizations); > > scanner.setRange(*range*); > > scanner.setScanIterators(1, > "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter"); > scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter"); > scanner.setScanIteratorOption("myFilter", "0.start", *start*); > > Iterator<Entry<Key,Value>> iterator = scanner.iterator(); > > while(iterator.hasNext()) { <--- Exception here > > ... > > } > > > ------------------------------------------------------------------------------------------------------------------------- > > public class MyFilter implements Filter{ > long startOfRange = 0; > @Override > public boolean accept(Key key, Value value) { > String colqual = key.getColumnQualifier().toString(); > long end = Long.parseLong(colqual.substring(20, 39)); > if(end < startOfRange){ > return false; > } > return true; > } > > @Override > public void init(Map<String, String> options) { > > if(options == null){ > throw new IllegalArgumentException("'start' must be set for filter"); > } > String start = options.get("start"); > if(start == null){ > throw new IllegalArgumentException("'start' must be set for filter"); > } > startOfRange = Long.parseLong(start); > } > > } > > > ------------------------------------------------------------------------------------------------------------------------- > > The Exception that I'm receiving is: > > Exception in thread "main" java.lang.RuntimeException: > org.apache.accumulo.core.client.impl.AccumuloServerException: > at > org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java) > at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java) > at > test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java) > at test.offsets.TestFilter.main(TestFilter.java) > > > ------------------------------------------------------------------------------------------------------------------------- > > I was thinking that maybe the server couldn't find the MyFilter class, or > maybe it was a permissions error, but I wasn't sure. When I initialize my > Scanner to use MyFilter, is it looking on the server for the file or in my > project? > > Any assistance you can provide would be greatly appreciated, thanks! > Tori >
-
Re: Filter Implementation - Accumulo 1.3
William Slacum 2012-10-23, 18:41
Make sure that the class is available to the the tserver process. This is done by putting the jar containing your class on all nodes under the $ACCUMULO_HOME/lib/ext directory. If you put it under lib/ext, then you won't need to stop and restart the process for the tserver to pick it up.
On Tue, Oct 23, 2012 at 10:15 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
> Check the tablet server logs... you'll see the real problem using the > filter in there. > > -Eric > > > On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <[EMAIL PROTECTED]>wrote: > >> Hello, >> >> I am currently using Accumulo 1.3 to implement a Filter. Since I'm using >> 1.3, I realize that the Filter class is not an iterator so I have created a >> MyFilter class that implements Filter to use when I initialize my Scanner. >> When I run my code, I am getting an AccumuloServerException. >> >> I was referencing the posts from December 2011 on "Filter Use" to >> initialize my scanner with MyFilter. >> My scanner initialization currently appears as so: >> >> Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers* >> ); >> Connector connector = zooInstance.getConnector(*userName*, *password*); >> Authorizations authorizations = new Authorizations(); >> Scanner scanner = connector.createScanner(*tableName*, authorizations); >> >> scanner.setRange(*range*); >> >> scanner.setScanIterators(1, >> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter"); >> scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter"); >> scanner.setScanIteratorOption("myFilter", "0.start", *start*); >> >> Iterator<Entry<Key,Value>> iterator = scanner.iterator(); >> >> while(iterator.hasNext()) { <--- Exception here >> >> ... >> >> } >> >> >> ------------------------------------------------------------------------------------------------------------------------- >> >> public class MyFilter implements Filter{ >> long startOfRange = 0; >> @Override >> public boolean accept(Key key, Value value) { >> String colqual = key.getColumnQualifier().toString(); >> long end = Long.parseLong(colqual.substring(20, 39)); >> if(end < startOfRange){ >> return false; >> } >> return true; >> } >> >> @Override >> public void init(Map<String, String> options) { >> >> if(options == null){ >> throw new IllegalArgumentException("'start' must be set for filter"); >> } >> String start = options.get("start"); >> if(start == null){ >> throw new IllegalArgumentException("'start' must be set for filter"); >> } >> startOfRange = Long.parseLong(start); >> } >> >> } >> >> >> ------------------------------------------------------------------------------------------------------------------------- >> >> The Exception that I'm receiving is: >> >> Exception in thread "main" java.lang.RuntimeException: >> org.apache.accumulo.core.client.impl.AccumuloServerException: >> at >> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java) >> at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java) >> at >> test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java) >> at test.offsets.TestFilter.main(TestFilter.java) >> >> >> ------------------------------------------------------------------------------------------------------------------------- >> >> I was thinking that maybe the server couldn't find the MyFilter class, or >> maybe it was a permissions error, but I wasn't sure. When I initialize my >> Scanner to use MyFilter, is it looking on the server for the file or in my >> project? >> >> Any assistance you can provide would be greatly appreciated, thanks! >> Tori >> > >
-
Re: Filter Implementation - Accumulo 1.3
Victoria Bare 2012-10-25, 19:47
Thank you guys for you input, I really appreciate it! I just had to put the jar into the $ACCUMULO_HOME/lib/ext directory and everything seems to be working.
On Tue, Oct 23, 2012 at 2:41 PM, William Slacum < [EMAIL PROTECTED]> wrote:
> Make sure that the class is available to the the tserver process. This is > done by putting the jar containing your class on all nodes under the > $ACCUMULO_HOME/lib/ext directory. If you put it under lib/ext, then you > won't need to stop and restart the process for the tserver to pick it up. > > > On Tue, Oct 23, 2012 at 10:15 AM, Eric Newton <[EMAIL PROTECTED]>wrote: > >> Check the tablet server logs... you'll see the real problem using the >> filter in there. >> >> -Eric >> >> >> On Tue, Oct 23, 2012 at 9:54 AM, Victoria Bare <[EMAIL PROTECTED]>wrote: >> >>> Hello, >>> >>> I am currently using Accumulo 1.3 to implement a Filter. Since I'm >>> using 1.3, I realize that the Filter class is not an iterator so I have >>> created a MyFilter class that implements Filter to use when I initialize my >>> Scanner. When I run my code, I am getting an AccumuloServerException. >>> >>> I was referencing the posts from December 2011 on "Filter Use" to >>> initialize my scanner with MyFilter. >>> My scanner initialization currently appears as so: >>> >>> Instance zooInstance = new ZooKeeperInstance(*instanceName*, *zooServers >>> *); >>> Connector connector = zooInstance.getConnector(*userName*, *password*); >>> Authorizations authorizations = new Authorizations(); >>> Scanner scanner = connector.createScanner(*tableName*, authorizations); >>> >>> scanner.setRange(*range*); >>> >>> scanner.setScanIterators(1, >>> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter"); >>> scanner.setScanIteratorOption("myFilter", "0", "test.offsets.MyFilter"); >>> scanner.setScanIteratorOption("myFilter", "0.start", *start*); >>> >>> Iterator<Entry<Key,Value>> iterator = scanner.iterator(); >>> >>> while(iterator.hasNext()) { <--- Exception here >>> >>> ... >>> >>> } >>> >>> >>> ------------------------------------------------------------------------------------------------------------------------- >>> >>> public class MyFilter implements Filter{ >>> long startOfRange = 0; >>> @Override >>> public boolean accept(Key key, Value value) { >>> String colqual = key.getColumnQualifier().toString(); >>> long end = Long.parseLong(colqual.substring(20, 39)); >>> if(end < startOfRange){ >>> return false; >>> } >>> return true; >>> } >>> >>> @Override >>> public void init(Map<String, String> options) { >>> >>> if(options == null){ >>> throw new IllegalArgumentException("'start' must be set for filter"); >>> } >>> String start = options.get("start"); >>> if(start == null){ >>> throw new IllegalArgumentException("'start' must be set for filter"); >>> } >>> startOfRange = Long.parseLong(start); >>> } >>> >>> } >>> >>> >>> ------------------------------------------------------------------------------------------------------------------------- >>> >>> The Exception that I'm receiving is: >>> >>> Exception in thread "main" java.lang.RuntimeException: >>> org.apache.accumulo.core.client.impl.AccumuloServerException: >>> at >>> org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java) >>> at test.offsets.TestFilter.getFilterEntrySetRange(TestFilter.java) >>> at >>> test.offsets.TestFilter.getAnalysisProductsByClassFilteredOffset(TestFilter.java) >>> at test.offsets.TestFilter.main(TestFilter.java) >>> >>> >>> ------------------------------------------------------------------------------------------------------------------------- >>> >>> I was thinking that maybe the server couldn't find the MyFilter class, >>> or maybe it was a permissions error, but I wasn't sure. When I initialize >>> my Scanner to use MyFilter, is it looking on the server for the file or in >>> my project? >>> >>> Any assistance you can provide would be greatly appreciated, thanks!
|
|