Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # dev >> (Re)Introducing Culvert - A secondary indexing framework for BigTable like systems


Copy link to this message
-
Re: (Re)Introducing Culvert - A secondary indexing framework for BigTable like systems
Wow, that's embarrassing - project not building...

It's because accumulo's release is no longer deployed into the standard
apache maven repository. Maybe one of the accumulo committers can shed some
light on where to find it?

I'll make some changes and have it at least compiling from the raw tonight
:)

The alternative is to download accumulo source (
https://github.com/apache/accumulo) and "mvn clean install" to get it
working on your local machine.

Thanks Ted!

-Jesse

On Thu, Dec 22, 2011 at 1:54 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Thanks for the update, Jesse.
> Let us know of any feature Culvert needs from HBase.
>
> After cloning Culvert, I got:
>
> [INFO] Culvert - Accumulo Integration .................... FAILURE [0.431s]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 1:06.638s
> [INFO] Finished at: Thu Dec 22 13:51:34 PST 2011
> [INFO] Final Memory: 20M/81M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project culvert-accumulo: Could not
> resolve dependencies for project
> com.bah.culvert:culvert-accumulo:jar:0.4.0-SNAPSHOT: Could not find
> artifact org.apache.accumulo:accumulo-core:jar:1.4.0-incubating-SNAPSHOT in
> apache-snapshots (http://repository.apache.org/snapshots/) -> [Help 1]
>
> Can someone provide hint ?
>
> On Thu, Dec 22, 2011 at 11:44 AM, Jesse Yates <[EMAIL PROTECTED]
> >wrote:
>
> > Culvert was originally introduced at Hadoop Summit 2011, but recent
> updates
> > have made it very applicable to current systems. Recently, we added
> support
> > for Accumulo as well as upgraded HBase support to 0.92. Since Hadoop
> > Summit, there have also been significant code cleanup and added some
> small
> > features. However, we found that most people hadn't heard of Culvert, so
> we
> > wanted to re-release the framework.
> >
> > For an introduction to using Culvert, check out the blog post here:
> > http://jyates.github.com/2011/11/17/intro-to-culvert.html
> >
> > Also, the original presentation (where we discuss the internals) is
> > available on slideshare<
> >
> http://www.slideshare.net/jesse_yates/culvert-a-robust-framework-for-secondary-indexing-of-structured-and-unstructured-data
> > >
> > .
> >
> > There is a Culvert hackathon in the middle of January:
> > http://culverthackathon2012.eventbrite.com/
> >
> > Oh, and you can find the code on
> > github<https://github.com/booz-allen-hamilton/culvert>
> > .
> >
> > Below is an overview of why we wrote Culvert and what it does.
> >
> > Secondary indexing is a common design pattern in BigTable-like databases
> > that allows users to index one or more columns in a table. This technique
> > enables fast search of records in a database based on a particular column
> > instead of the row id, thus enabling relational-style semantics in a
> NoSQL
> > environment. Frequently, the index is stored either in a reserved
> namespace
> > in the table or another index table.
> >
> > Despite the fact that this is a common design pattern in BigTable-based
> > applications, most implementations of this practice to date have been
> > tightly coupled with a particular application. As a result, few
> > general-purpose frameworks for secondary indexing on BigTable-like
> > databases exist, and those that do are tied to a particular
> implementation
> > of the BigTable model.
> >
> > There are several existing tools (Solr, Lily), but these are focused on
> > doing text based search and are highly restrictive to indexes created
> > through their framework. What if you want to use your existing indexes?
> Or
> > leverage the indexes to do complex queries?
> >
> > We developed a solution to this problem called Culvert that supports
> online
> > index updates as well as a variation of the HIVE query language. In
> > designing Culvert, we sought to make the solution pluggable so that it

Jesse Yates
240-888-2200
@jesse_yates