Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - RE: EXTERNAL: Re: Large files in Accumulo


+
Cardon, Tejay E 2012-08-23, 20:37
+
Eric Newton 2012-08-23, 21:05
Copy link to this message
-
Re: EXTERNAL: Re: Large files in Accumulo
Billie Rinaldi 2012-08-23, 21:33
The filedata example shows one way to split a file into multiple Values.

Billie
On Thu, Aug 23, 2012 at 2:05 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

> An entire mutation needs to fit in memory several times, so you should not
> attempt to push in a single mutation larger than a 100MB unless you have a
> lot of memory in your tserver/logger.
>
> And while I'm at it, large keys will create large indexes, so try to keep
> your (row,cf,cq,cv) under 100K.
>
> -Eric
>
>
> On Thu, Aug 23, 2012 at 4:37 PM, Cardon, Tejay E <[EMAIL PROTECTED]>wrote:
>
>>  In my case I’ll be doing a document based index store (like the
>> wikisearch example), but my documents may be as large as several GB.  I
>> just wanted to pick the collective brain of the group to see if I’m walking
>> into a major headache.  If it’s never been tried before, then I’ll give it
>> a shot and report back.****
>>
>>
>> Tejay****
>>
>> ** **
>>
>> *From:* William Slacum [mailto:[EMAIL PROTECTED]]
>> *Sent:* Thursday, August 23, 2012 2:07 PM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* EXTERNAL: Re: Large files in Accumulo****
>>
>> ** **
>>
>> Are these RFiles as a whole? I know at some point HBase needed to have
>> entire rows fit into memory; Accumulo does not have this restriction.****
>>
>> On Thu, Aug 23, 2012 at 12:55 PM, Cardon, Tejay E <
>> [EMAIL PROTECTED]> wrote:****
>>
>> Alright, this one’s a quick question.  I’ve been told that HBase does not
>> perform well if large (> 100MB) files are stored in it).  Does Accumulo
>> have similar trouble?  If so, can it be overcome by storing the large files
>> in their own locality group?****
>>
>>  ****
>>
>> Thanks,****
>>
>> Tejay****
>>
>> ** **
>>
>
>
+
Cardon, Tejay E 2012-08-23, 21:34
+
Christopher Tubbs 2012-08-23, 22:11