Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - how to store 100billion short text messages with hbase


Copy link to this message
-
how to store 100billion short text messages with hbase
tgh 2012-12-06, 02:51
Hi
I try to use hbase to store 100billion short texts messages, each
message has less than 1000 character and some other items, that is, each
messages has less than 10 items,
The whole data is a stream for about one year, and I want to create
multi tables to store these data, I have two ideas, the one is to store the
data in one hour in one table, and for one year data, there are 365*24
tables, the other is to store the date in one day in one table, and for one
year , there are 365 tables,

And I have about 15 computer nodes to handle these data, and I want
to know how to deal with these data, the one for 365*24 tables , or the one
for 365 tables, or other better ideas,

I am really confused about hbase, it is powerful yet a bit complex
for me , is it?
Could you give me some advice for hbase data schema and others,
Could you help me,
Thank you
---------------------------------
Tian Guanhua