-
Re: Titan (graph database) Support For Accumulo?Jason Trost 2012-07-22, 16:59
I experimented briefly with Titan, and wasn't impressed with its
performance when backed by BigTable DBs. I started making mods
necessary to use Accumulo for Titan, but got busy and had to put it on
hold. This code is definitely not finished (or pretty).
Here is the branch:
https://github.com/jt6211/titan/tree/feature.Accumulo-Backend
Here is what I added:
https://github.com/jt6211/titan/compare/master...feature.Accumulo-Backend
For the same reasons the HBase code is not ready for production, the
accumulo code is not either. The Titan API seems to lack some
features that may make working with Big Data more efficient (like
batching reads). I feel like the API could/should be changed to take
advantage of BatchScanners when performing many get()'s, which it does
not currently do, and I feel like this system would benefit from some
MapReduce jobs that would enable building graphs in the format
required. There are probably many other optimizations that could be
done here too.
I haven't had time to finish it and I am definitely open to someone
else picking this up. I "Stopped Progress" on this in case someone
else is interested in taking this JIRA:
https://issues.apache.org/jira/browse/ACCUMULO-638
--Jason
On Fri, Jul 20, 2012 at 1:24 PM, William Slacum <[EMAIL PROTECTED]> wrote:
> Incredibly poor choice of words (and quoting) on my part. The idea was
> essentially "oh there's HBase code-- let's port it to Accumulo." The
> HBase code is advertised as "not ready for production"
> (https://github.com/thinkaurelius/titan/blob/master/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseStorageManager.java).
> I'll defer to Jason to make a more technical comment.
>
> On Fri, Jul 20, 2012 at 9:56 AM, Jim Klucar <[EMAIL PROTECTED]> wrote:
>> We are looking at using Titan also. The developers have a lot of
>> experience developing graph databases, so I expect good things.
>>
>> Sent from my iPhone
>>
>> On Jul 20, 2012, at 12:53 PM, Jeremy Jenkins <[EMAIL PROTECTED]> wrote:
>>
>>> I'm currently using accumulo, and have been looking at Titan. So, I
>>> definitely have interest in both the integration as well as the
>>> rationale behind "...he wasn't very impressed with Titan".
>>>
>>> Does anyone have any further information as to why?
>>>
>>>
>>> On Fri, Jul 20, 2012 at 9:16 AM, John Vines <[EMAIL PROTECTED]> wrote:
>>>> I had spoken with Jason last week, as he made the ticket for it (
>>>> https://issues.apache.org/jira/browse/ACCUMULO-638 ). I don't want to
>>>> misquote him, but the short of it was that he doesn't think Titan is needing
>>>> of Accumulo integration currently so he was planning on dropping it.
>>>>
>>>> John
>>>>
>>>>
>>>> On Fri, Jul 20, 2012 at 10:09 AM, David Medinets <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>
>>>>> I recall hearing someone at the July OSSI meeting that Titan could
>>>>> read information from Accumulo tables. Does anyone here know if that
>>>>> support is standard in Titan?
>>>>
>>>>
performance when backed by BigTable DBs. I started making mods
necessary to use Accumulo for Titan, but got busy and had to put it on
hold. This code is definitely not finished (or pretty).
Here is the branch:
https://github.com/jt6211/titan/tree/feature.Accumulo-Backend
Here is what I added:
https://github.com/jt6211/titan/compare/master...feature.Accumulo-Backend
For the same reasons the HBase code is not ready for production, the
accumulo code is not either. The Titan API seems to lack some
features that may make working with Big Data more efficient (like
batching reads). I feel like the API could/should be changed to take
advantage of BatchScanners when performing many get()'s, which it does
not currently do, and I feel like this system would benefit from some
MapReduce jobs that would enable building graphs in the format
required. There are probably many other optimizations that could be
done here too.
I haven't had time to finish it and I am definitely open to someone
else picking this up. I "Stopped Progress" on this in case someone
else is interested in taking this JIRA:
https://issues.apache.org/jira/browse/ACCUMULO-638
--Jason
On Fri, Jul 20, 2012 at 1:24 PM, William Slacum <[EMAIL PROTECTED]> wrote:
> Incredibly poor choice of words (and quoting) on my part. The idea was
> essentially "oh there's HBase code-- let's port it to Accumulo." The
> HBase code is advertised as "not ready for production"
> (https://github.com/thinkaurelius/titan/blob/master/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseStorageManager.java).
> I'll defer to Jason to make a more technical comment.
>
> On Fri, Jul 20, 2012 at 9:56 AM, Jim Klucar <[EMAIL PROTECTED]> wrote:
>> We are looking at using Titan also. The developers have a lot of
>> experience developing graph databases, so I expect good things.
>>
>> Sent from my iPhone
>>
>> On Jul 20, 2012, at 12:53 PM, Jeremy Jenkins <[EMAIL PROTECTED]> wrote:
>>
>>> I'm currently using accumulo, and have been looking at Titan. So, I
>>> definitely have interest in both the integration as well as the
>>> rationale behind "...he wasn't very impressed with Titan".
>>>
>>> Does anyone have any further information as to why?
>>>
>>>
>>> On Fri, Jul 20, 2012 at 9:16 AM, John Vines <[EMAIL PROTECTED]> wrote:
>>>> I had spoken with Jason last week, as he made the ticket for it (
>>>> https://issues.apache.org/jira/browse/ACCUMULO-638 ). I don't want to
>>>> misquote him, but the short of it was that he doesn't think Titan is needing
>>>> of Accumulo integration currently so he was planning on dropping it.
>>>>
>>>> John
>>>>
>>>>
>>>> On Fri, Jul 20, 2012 at 10:09 AM, David Medinets <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>
>>>>> I recall hearing someone at the July OSSI meeting that Titan could
>>>>> read information from Accumulo tables. Does anyone here know if that
>>>>> support is standard in Titan?
>>>>
>>>>
