Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Re: persistence in Hbase


Copy link to this message
-
Re: persistence in Hbase
This is a broad topic by itself. In short often people use battery backed
cache or leave the write cache disabled for such concern. There are various
factors involved when deciding if to leave caches enabled or not. Caches
are often good for OLTP type application or even light OLAP workload. But
for huge workload you might saturate caches and is better off leaving it
disabled, assuming these are not super computers with TBs of caches :).
There are other factors in play like number of replica copies you have and
what gurantees or SLA you have put in place around data availability and
loss in the event of site disaster. Most people are ok with some data loss
on site failure and leave caches enabled with multiple replica copies
enabled.

On Mon, Jan 14, 2013 at 7:19 PM, Otis Gospodnetic <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Just for my own edification - isn't data loss always going to be possible
> due to caches present in HDDs and inability(?) for force them to flush.  I
> believe I've read even fsync lies...
>
> Thanks,
> Otis
> --
> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>
>
>
>
>
> On Thu, Jan 10, 2013 at 10:54 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > Not entirely true, though.
> > Data is not sync'ed to disk, but only distributed to all HDFS replicas.
> > During a power outage event across all HDFS failure zones (such as a data
> > center) you can lose data.
> >
> >
> > -- Lars
> >
> >
> >
> > ----- Original Message -----
> > From: anil gupta <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Cc:
> > Sent: Thursday, January 10, 2013 2:38 PM
> > Subject: Re: persistence in Hbase
> >
> > Hi Mohammad,
> >
> > If the Write Ahead Log(WAL) is "turned on" then in **NO** case data
> should
> > be lost. HBase is strongly-consistent. If you know of any case when WAL
> is
> > turned on and data is lost then IMO that's a Critical bug in HBase.
> >
> > Thanks,
> > Anil Gupta
> >
> > On Thu, Jan 10, 2013 at 7:37 AM, Mohit Anchlia <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Data also gets written in WAL. See:
> > >
> > > http://hbase.apache.org/book/perf.writing.html
> > >
> > > On Thu, Jan 10, 2013 at 7:36 AM, ramkrishna vasudevan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Yes definitely you will get back the data.
> > > >
> > > > Please read the HBase Book that explains things in detail.
> > > > http://hbase.apache.org/book.html.
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > >
> > > > On Thu, Jan 10, 2013 at 8:48 PM, Panshul Gupta <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > >  > Hello,
> > > > >
> > > > > I was wondering if it is possible that I have data stored in Hbase
> > > tables
> > > > > on my 10 node cluster. I switch off (power down) my cluster. When I
> > > power
> > > > > up my cluster again, and run the HDFS and hadoop daemons, will the
> > > Hbase
> > > > > have my old data persisted in the form I left it?? or will I have
> to
> > re
> > > > > import all the data??
> > > > >
> > > > > Thankyou for the help.
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Panshul.
> > > > > http://about.me/panshulgupta
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
> >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB