Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - How can I insert large image or video into HBase?


Copy link to this message
-
Re: How can I insert large image or video into HBase?
Jean-Marc Spaggiari 2013-10-23, 14:46
To reply to Jack: There is no stupid question. What is stupid is to go in
the wrong direction, close its eyes, and run. Asking is always good.

You got a very nicely detailed respons from Michael, so I don't think I
have anything to add ;)

JM
2013/10/23 Michael Segel <[EMAIL PROTECTED]>

> Actually you didn't.
>
> Its actually a good question and JM gave you the short answer. ;-)
>
> First to clarify... depending on the size of the image or video you may
> want to actually store the image/vid in a sequence file.
> Then use HBase to index the file. (Its pretty much the same idea as what
> JM was saying but that you have to remember that even Vid and image files
> may be too large to fit into HBase yet too small to be separate files. )
>
> This is one of the patterns that we've talked about at conferences. (Big
> Data TechCon) and its covered in Boris Lublinsky, Kevin Smith and Alexey
> Y's new book.
> (Its a Wiley publication.)
>
> The longer answer...
>
> In HBase your row cannot exceed the width of the region.  So you end up
> with very large regions, with a single row and you have to cap the size of
> the video or image.
> HBase doesn't really perform well and of course performance would suck
> when you tried to insert or read the row.
> (You can go through examples in either Lars George's book or Nick Dimiduk
> ?sp? book on what happens during reads and writes.) Unless the row is in
> cache, reads and writes to HBase are slower than reading or writing to HDFS.
>
> The reason I like the question is that it forces people to think about how
> HBase works and why it may not be the best option as a stand alone
> solution. ;-)
>
> On Oct 23, 2013, at 1:38 AM, Jack Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi JM,
> >
> > Thanks for your reply,It seems I really asked a stupid question :)
> >
> >
> >
> >
> > Jack Chan.
> >
> > From: Jean-Marc Spaggiari
> > Date: 2013-10-23 12:12
> > To: user; cdj0579
> > Subject: Re: How can I insert large image or video into HBase?
> > Put your file into HDFS and store only the name into HBase. HBase is not
> done do store large files.
> >
> >
> > JM
> >
> >
> >
> >
> > 2013/10/23 Jack Chan <[EMAIL PROTECTED]>
> >
> > Hi All:
> >
> > This could be a stupid question.But here it goes....
> > We knew that we can use "put" to insert some small files by converting
> it to bytes first.
> > But for a large file,I think we would better stream it first.
> > So,how can we insert the large file into HBase through Java code using
> the stream way?
> >
> > Thanks and regards
> >
> >
> >
> > Jack Chan.
>
> The opinions expressed here are mine, while they may reflect a cognitive
> thought, that is purely accidental.
> Use at your own risk.
> Michael Segel
> michael_segel (AT) hotmail.com
>
>
>
>
>
>