Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Amazon s3 and Hadoop/Hbase


Copy link to this message
-
Re: Amazon s3 and Hadoop/Hbase
Sonal Goyal 2011-11-25, 04:45
If your use case is pure file modification, I am not sure how HBase is
helping you. Do you want to save some metadata as well? Are you doing some
file analysis and want to access the results? If not, you are probably
better off not adding another layer.

This is not to say HBase can not save file blobs. Just that in your
scenario, you dont seem to need the value HBase adds.

Thats my point of view! Lets see if someone on the list has some other
thoughts.

Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>

On Fri, Nov 25, 2011 at 9:25 AM, Aditya Kumar <[EMAIL PROTECTED]>wrote:

> Thanks Sonal for the excellent reply.
>
> I am looking for a use case:
> where in I want to store 10000 files and they will be accessed and
> modified and kept back in the S3.
>
> so there will be  modifications of Files on S3 and trying to access them.
>
> For a use case like this do I need Hbase on S3? (or) if I just use S3,
> create bucket and keep on adding files and modify them on the need bases
>
>
> ________________________________
>  From: Sonal Goyal <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; Aditya Kumar <[EMAIL PROTECTED]>
> Sent: Thursday, November 24, 2011 7:11 PM
> Subject: Re: Amazon s3 and Hadoop/Hbase
>
> Aditya,
>
> S3 is a file storage service. It does provide versioning and key based file
> access, but it does not provide any processing paradigm, like Map Reduce
> for Hadoop. HBase provides random access to data - its more like a database
> in that you can have a key based access for records, and it is a columnar
> store. S3 can be compared with HDFS, the hadoop distributed file system,
> and in fact one can spin Hadoop clusters off S3 file system.
>
> Maybe if you explain your problem in detail, we can help you further on the
> mailing list. You can also read up the fundamentals on the Hadoop and HBase
> wiki, there is a high level introduction which can clear the basic doubts.
>
> Hope that helps.
>
> Best Regards,
> Sonal
> Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
> On Fri, Nov 25, 2011 at 7:10 AM, Aditya Kumar <[EMAIL PROTECTED]
> >wrote:
>
> > Thanks Li Pi.
> >
> > so,
> > does it mean that s3 is hadoop?
> > can you please let me know what I will be missing if I use s3 only with
> > out Hbase?
> >
> > 2) In s3 there is delete operation, does it really delete the key and
> > there by value?
> >
> >
> >
> > ________________________________
> >  From: Li Pi <[EMAIL PROTECTED]>
> > To: Aditya Kumar <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
> > Sent: Thursday, November 24, 2011 5:29 PM
> > Subject: Re: Amazon s3 and Hadoop/Hbase
> >
> > You can run hbase against s3.
> > On Nov 24, 2011 5:11 PM, "Aditya Kumar" <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi,
> > > I am looking at Amazon S3.
> > > is it like HDFS/Hadoop? .
> > > If it is like Hadoop can we add only data to the tables.. than can we
> use
> > > Hbase?
> > > I tried to read online,
> > > all I see that S3 will not delete the data  and will do versioing
> > > (something similar to Hadoop)
> > >
> > > TIA,
> > > Adi
> >
>