Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Table splitting


+
Mathias Herberts 2012-12-11, 14:32
+
William Slacum 2012-12-11, 14:37
+
ameet kini 2012-12-12, 18:08
+
John Vines 2012-12-12, 18:17
+
Eric Newton 2012-12-12, 18:18
Copy link to this message
-
Re: Table splitting
ameet kini 2012-12-12, 18:27
Ok, so in short, assuming that there's sufficient local disk space, a given
tablet having all its blocks local relies on HDFS's guarantee that the
first replica of a block will be local as long as the tablet server is also
a data node. Yes?

On Wed, Dec 12, 2012 at 1:18 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

> Check out o.a.a.server.util.LocalityCheck
>
> -Eric
>
>
> On Wed, Dec 12, 2012 at 1:17 PM, John Vines <[EMAIL PROTECTED]> wrote:
>
>> When a file gets written to hdfs, there is a guarantee the file is local
>> as long as that systems disks are not full. Accumulo does not have a
>> locality guarantee as tablets will migrate on occasion. However, as data is
>> added, major compactions will occur which will restore locality.
>>  On Dec 12, 2012 1:09 PM, "ameet kini" <[EMAIL PROTECTED]> wrote:
>>
>>>
>>> Along these lines....
>>>
>>> Can someone help me understand how tablets map to files on disk in HDFS?
>>> From what I understand, after a compaction, there may be one (or more?)
>>> files on HDFS for a given tablet. Each file can consist of multiple HDFS
>>> blocks. Does Accumulo guarantee that the tablet serving a given data range
>>> finds all its blocks locally? If so, how does it keep this guarantee?
>>> Wouldn't HDFS distribute these blocks around based on HDFS balancing
>>> strategy?
>>>
>>> Thanks,
>>> Ameet
>>>
>>> On Tue, Dec 11, 2012 at 9:37 AM, William Slacum <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Tablets will split automatically, down to the granularity of a row.
>>>>
>>>>
>>>> On Tue, Dec 11, 2012 at 9:32 AM, Mathias Herberts <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've read the user manual for v1.4.2 and I have not seen any mention
>>>>> of automatic tablet splitting. Is there such a thing in Accumulo or is
>>>>> pre-splitting the only way to split a table?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Mathias.
>>>>>
>>>>
>>>>
>>>
>
+
Eric Newton 2012-12-12, 18:29
+
ameet kini 2012-12-12, 18:32
+
Eric Newton 2012-12-12, 18:36
+
ameet kini 2012-12-12, 18:45
+
Mathias Herberts 2012-12-11, 14:38
+
Keith Turner 2012-12-11, 14:58
+
Mathias Herberts 2012-12-11, 15:01