Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> which approach is better


Copy link to this message
-
Re: which approach is better
The best way to restore is from a backup. We use distcp to keep this
scalable : http://hadoop.apache.org/docs/r1.2.0/distcp2.html
The data we feed to hdfs also gets pushed to this backup and the
metadatabase from hive also gets pushed here. So this combination works
well for us (had to use it once).
Even if a namenode could never crash and all software worked fine 100%
of the time there is always the one crazy user/admin who will find a way
to wipe all data.
To me backups are not optional.

Op 17-7-2013 20:17, Hamza Asad schreef:
> I use data to generates reports on daily basis, Do couple of analysis
> and its insert once and read many on daily basis.  But My main purpose
> is to secure my data and easily recover it even if my hadoop(datanode)
> OR HDFS crashes. As uptill now, i'm using approach in which data has
> been retrieved directly from HDFS and few days back my hadoop crashes
> and when i repair it, i was unable to recover my Old data which
> resides on HDFS. So please let me know do i have to make architectural
> change OR is there any way to recover data which resides in crashed HDFS
>
>
> On Wed, Jul 17, 2013 at 11:00 PM, Nitin Pawar <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     what's the purpose of data storage?
>     whats the read and write throughput you expect?
>     whats the way you will access data while read?
>     whats are your SLAs on both read and write?
>
>     there will be more questions others will ask so be ready for that :)
>
>
>
>     On Wed, Jul 17, 2013 at 11:10 PM, Hamza Asad
>     <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
>
>         Please let me knw which approach is better. Either i save my
>         data directly to HDFS and run hive (shark) queries over it OR
>         store my data in HBASE, and then query it.. as i want to
>         ensure efficient data retrieval and data remains safe and can
>         easily recover if hadoop crashes.
>
>         --
>         */Muhammad Hamza Asad/*
>
>
>
>
>     --
>     Nitin Pawar
>
>
>
>
> --
> */Muhammad Hamza Asad/*