This question was asked at a presentation gave yesterday and I
realized that I don't know the answer. A tablet serves a specific key
of keys which is disjoint from all other tablets. Say a table has ten
tablets with perfectly distributed keys so each tablet serves 10% of
the data. If tablet A is lost, does the system lose access to the
first 10% of the data?

isn't this inherently handled by replication?

On Thu, Jul 26, 2012 at 7:54 AM, David Medinets <[EMAIL PROTECTED]>wrote:

> This question was asked at a presentation gave yesterday and I
> realized that I don't know the answer. A tablet serves a specific key
> of keys which is disjoint from all other tablets. Say a table has ten
> tablets with perfectly distributed keys so each tablet serves 10% of
> the data. If tablet A is lost, does the system lose access to the
> first 10% of the data?
>

well, the master would identify the failure, and transition access to that
key extent to the replicant data

On Thu, Jul 26, 2012 at 9:32 AM, Marc Parisi <[EMAIL PROTECTED]> wrote:

> isn't this inherently handled by replication?
>
>
> On Thu, Jul 26, 2012 at 7:54 AM, David Medinets <[EMAIL PROTECTED]>wrote:
>
>> This question was asked at a presentation gave yesterday and I
>> realized that I don't know the answer. A tablet serves a specific key
>> of keys which is disjoint from all other tablets. Say a table has ten
>> tablets with perfectly distributed keys so each tablet serves 10% of
>> the data. If tablet A is lost, does the system lose access to the
>> first 10% of the data?
>>
>
>

A tablets persistent data is composed of a set of files in HDFS and a
set of write ahead logs.  The !METADATA table stores the names of the
HDFS files and write ahead logs.  When a tablet server goes down, the
master ask another tablet server to load the tablet.  As long as the
other tablet server can read the write ahead logs and HDFS files, no
data is lost.   If enough servers go down such that HDFS files or
write ahead logs lose access to data, then the tablet will not be
accessible until the data node and/or loggers come back online.

On Thu, Jul 26, 2012 at 7:54 AM, David Medinets
<[EMAIL PROTECTED]> wrote:
> This question was asked at a presentation gave yesterday and I
> realized that I don't know the answer. A tablet serves a specific key
> of keys which is disjoint from all other tablets. Say a table has ten
> tablets with perfectly distributed keys so each tablet serves 10% of
> the data. If tablet A is lost, does the system lose access to the
> first 10% of the data?

Thanks, Keith. That explanation is just what I wanted.

Keith,

>From your response...
"A tablets persistent data is composed of a set of files in HDFS and a set
of write ahead logs."
"As long as the other tablet server can read the write ahead logs and HDFS
files, no data is lost."

Question.
I understand when a write arrives at a TabletServer it is written to a
Write-Ahead Log and then stored in the MemTable until flusing to HDFS via
minor compaction.
1) Are write ahead logs even used when data is bulk inserted into hdfs then
handed to Accumulo?
2) If not, how is data replication achieved when using bulk insert methods?

--
Thank you,

Hugh Xedni
On Thu, Jul 26, 2012 at 12:31 PM, David Medinets
<[EMAIL PROTECTED]>wrote:

> Thanks, Keith. That explanation is just what I wanted.
>

Responding inline

John

Sent from my phone, so pardon the typos and brevity.
On Aug 9, 2012 9:02 AM, "Hugh Xedni" <[EMAIL PROTECTED]> wrote:
>
> Keith,
>
> From your response...
>
> "A tablets persistent data is composed of a set of files in HDFS and a
set of write ahead logs."
> "As long as the other tablet server can read the write ahead logs and
HDFS files, no data is lost."
>
> Question.
> I understand when a write arrives at a TabletServer it is written to a
Write-Ahead Log and then stored in the MemTable until flusing to HDFS via
minor compaction.
> 1) Are write ahead logs even used when data is bulk inserted into hdfs
then handed to Accumulo?

Only for the metadata entries regarding that table and those files.

> 2) If not, how is data replication achieved when using bulk insert
methods?

Hdfs replicates the data, so when the files are created in hdfs they are
replicated.
>
> --
> Thank you,
>
> Hugh Xedni
>
>
>
> On Thu, Jul 26, 2012 at 12:31 PM, David Medinets <[EMAIL PROTECTED]>
wrote:
>>
>> Thanks, Keith. That explanation is just what I wanted.
>
>
>
>
>

On Thu, Aug 9, 2012 at 9:01 AM, Hugh Xedni <[EMAIL PROTECTED]> wrote:
> Keith,
>
> From your response...
>
> "A tablets persistent data is composed of a set of files in HDFS and a set
> of write ahead logs."
> "As long as the other tablet server can read the write ahead logs and HDFS
> files, no data is lost."
>
> Question.
> I understand when a write arrives at a TabletServer it is written to a
> Write-Ahead Log and then stored in the MemTable until flusing to HDFS via
> minor compaction.
> 1) Are write ahead logs even used when data is bulk inserted into hdfs then
> handed to Accumulo?

No write ahead logs are used directly.  Bulk import moves a file into
the tables dir in HDFS. Then an entry is written to the metadata table
for the tablet.  This write to the metadata table ends up in the write
ahead log of the tablet server serving the metadata tablet.

> 2) If not, how is data replication achieved when using bulk insert methods?

Bulk import just moves file already in hdfs.  So the file retains the
hdfs replication it had before bulk import.

>
> --
> Thank you,
>
> Hugh Xedni
>
>
>
> On Thu, Jul 26, 2012 at 12:31 PM, David Medinets <[EMAIL PROTECTED]>
> wrote:
>>
>> Thanks, Keith. That explanation is just what I wanted.
>
>
>
>
>