Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # dev - Storing znode on disks


Copy link to this message
-
Re: Storing znode on disks
Edward Ribeiro 2013-03-15, 22:11
On Thu, Mar 14, 2013 at 3:13 PM, Abhishek .E.S <[EMAIL PROTECTED]>wrote:

> Could I build a large scale data-store using Zookeeper though ?
>

Yep, but ZooKeeper should be used as the coordinator of this data-store.
For example, suppose you are building a sharded database. Then, ZK could be
used to store the *mapping* of data partitions to machines. Or ZK could be
used to provide high availability of servers so that in case of failure of
some machines the data-store would still be able to serve requests (this
scenario is used by HBase, by the way). Or you could use ZK to implement
some sort of distributed transaction protocol for the data-store (as used
by the Calvin, a research database system being developed by Yale
university). As you can see by the cited examples, ZK is used as a
component of the data-store, but it's in charge of meta-data and
coordination, not the large scale user data.

Edward
>
> On Thu, Mar 14, 2013 at 12:52 PM, Edward Ribeiro
> <[EMAIL PROTECTED]>wrote:
>
> > >> For me, latency is acceptable but I require the znodes to be on  disk.
> >
> > Why would you need to do that?
> >
> > ZooKeeper stores the dataTree in memory, but it performs periodic
> snapshots
> > to disk, besides sync-ing a commit log also to disk, so that a node can
> > recover in case of failures. If you are asking to store znodes *only* in
> > disk then the answer is no (afaik!).
> >
> > Last but not least, you should be aware that znodes are not intended to
> > store large quantities of data, it's not mean to be a database, but a
> > coordination system.
> >
> > Edward
> >
> > On Thu, Mar 14, 2013 at 1:49 PM, Abhishek .E.S <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > I am new to Zookeeper .
> > > I had a question. Zookeeper places znodes in memory to optimize data
> > > access.
> > > I am working on an experiment for which I intend to use zookeeper.
> > > For me, latency is acceptable but I require the znodes to be on  disk.
> > >
> > > Can this be achieved.
> > > If so, could someone please provide me the pointers for the same.
> > >
> > > Thanks and Regards,
> > > Abhishek
> > >
> >
> >
> >
> > --
> > *"Matar um Leão por dia é fácil. O difícil é desviar das antas.",
> anônimo*
> >
>

--
*"Matar um Leão por dia é fácil. O difícil é desviar das antas.", anônimo*