Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: MapReduce to load data in HBase


+
Mohammad Tariq 2013-02-07, 11:59
+
Panshul Whisper 2013-02-07, 12:55
Copy link to this message
-
Re: MapReduce to load data in HBase
Size is not a prob, frequently changing schema might be.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 7, 2013 at 6:25 PM, Panshul Whisper <[EMAIL PROTECTED]>wrote:

> Hello,
>
> Thank you for the replies.
>
> I have not used pig yet. I am looking into it. I wanted to implement both
> the approaches.
> Are pig scripts maintainable? Because the Json structure that I will be
> receiving will be changing quite often. Almost 3 times a month.
> I will be processing 24 million Json files per month.
> I am getting one big file with almost 3 million Json files aggregated. One
> Json per line. I need to process this file and store all values into HBase.
>
> Thanking You,
>
>
>
>
> On Thu, Feb 7, 2013 at 12:59 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
>
> > Good point sir. If Pig fits into Panshul's requirements then it's a much
> > better option.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Thu, Feb 7, 2013 at 5:25 PM, Damien Hardy <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hello,
> > > Why not using a PIG script for that ?
> > > make the json file available on HDFS
> > > Load with
> > >
> > >
> >
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/builtin/JsonLoader.html
> > > Store with
> > >
> > >
> >
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/backend/hadoop/hbase/HBaseStorage.html
> > >
> > > http://pig.apache.org/docs/r0.10.0/
> > >
> > > Cheers,
> > >
> > > --
> > > Damien
> > >
> >
>
>
>
> --
> Regards,
> Ouch Whisper
> 010101010101
>
+
Michael Segel 2013-02-07, 13:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB