Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: Re: How can I insert large image or video into HBase?


+
Jack Chan 2013-10-24, 02:30
+
Jack Chan 2013-10-23, 06:38
+
Michael Segel 2013-10-23, 12:07
Copy link to this message
-
Re: How can I insert large image or video into HBase?
To reply to Jack: There is no stupid question. What is stupid is to go in
the wrong direction, close its eyes, and run. Asking is always good.

You got a very nicely detailed respons from Michael, so I don't think I
have anything to add ;)

JM
2013/10/23 Michael Segel <[EMAIL PROTECTED]>

> Actually you didn't.
>
> Its actually a good question and JM gave you the short answer. ;-)
>
> First to clarify... depending on the size of the image or video you may
> want to actually store the image/vid in a sequence file.
> Then use HBase to index the file. (Its pretty much the same idea as what
> JM was saying but that you have to remember that even Vid and image files
> may be too large to fit into HBase yet too small to be separate files. )
>
> This is one of the patterns that we've talked about at conferences. (Big
> Data TechCon) and its covered in Boris Lublinsky, Kevin Smith and Alexey
> Y's new book.
> (Its a Wiley publication.)
>
> The longer answer...
>
> In HBase your row cannot exceed the width of the region.  So you end up
> with very large regions, with a single row and you have to cap the size of
> the video or image.
> HBase doesn't really perform well and of course performance would suck
> when you tried to insert or read the row.
> (You can go through examples in either Lars George's book or Nick Dimiduk
> ?sp? book on what happens during reads and writes.) Unless the row is in
> cache, reads and writes to HBase are slower than reading or writing to HDFS.
>
> The reason I like the question is that it forces people to think about how
> HBase works and why it may not be the best option as a stand alone
> solution. ;-)
>
> On Oct 23, 2013, at 1:38 AM, Jack Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi JM,
> >
> > Thanks for your reply,It seems I really asked a stupid question :)
> >
> >
> >
> >
> > Jack Chan.
> >
> > From: Jean-Marc Spaggiari
> > Date: 2013-10-23 12:12
> > To: user; cdj0579
> > Subject: Re: How can I insert large image or video into HBase?
> > Put your file into HDFS and store only the name into HBase. HBase is not
> done do store large files.
> >
> >
> > JM
> >
> >
> >
> >
> > 2013/10/23 Jack Chan <[EMAIL PROTECTED]>
> >
> > Hi All:
> >
> > This could be a stupid question.But here it goes....
> > We knew that we can use "put" to insert some small files by converting
> it to bytes first.
> > But for a large file,I think we would better stream it first.
> > So,how can we insert the large file into HBase through Java code using
> the stream way?
> >
> > Thanks and regards
> >
> >
> >
> > Jack Chan.
>
> The opinions expressed here are mine, while they may reflect a cognitive
> thought, that is purely accidental.
> Use at your own risk.
> Michael Segel
> michael_segel (AT) hotmail.com
>
>
>
>
>
>
+
Jack Chan 2013-10-23, 04:07
+
Jean-Marc Spaggiari 2013-10-23, 04:12
+
Roman Nikitchenko 2013-10-23, 06:50
+
Wei Tan 2013-10-24, 04:19
+
Julian Zhou 2013-10-24, 12:23
+
谢良 2013-10-23, 06:56
+
Roman Nikitchenko 2013-10-23, 07:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB