Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Re: MapReduce to load data in HBase


Copy link to this message
-
Re: MapReduce to load data in HBase
Mohammad Tariq 2013-02-07, 13:23
Size is not a prob, frequently changing schema might be.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Feb 7, 2013 at 6:25 PM, Panshul Whisper <[EMAIL PROTECTED]>wrote:

> Hello,
>
> Thank you for the replies.
>
> I have not used pig yet. I am looking into it. I wanted to implement both
> the approaches.
> Are pig scripts maintainable? Because the Json structure that I will be
> receiving will be changing quite often. Almost 3 times a month.
> I will be processing 24 million Json files per month.
> I am getting one big file with almost 3 million Json files aggregated. One
> Json per line. I need to process this file and store all values into HBase.
>
> Thanking You,
>
>
>
>
> On Thu, Feb 7, 2013 at 12:59 PM, Mohammad Tariq <[EMAIL PROTECTED]>
> wrote:
>
> > Good point sir. If Pig fits into Panshul's requirements then it's a much
> > better option.
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > On Thu, Feb 7, 2013 at 5:25 PM, Damien Hardy <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hello,
> > > Why not using a PIG script for that ?
> > > make the json file available on HDFS
> > > Load with
> > >
> > >
> >
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/builtin/JsonLoader.html
> > > Store with
> > >
> > >
> >
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/backend/hadoop/hbase/HBaseStorage.html
> > >
> > > http://pig.apache.org/docs/r0.10.0/
> > >
> > > Cheers,
> > >
> > > --
> > > Damien
> > >
> >
>
>
>
> --
> Regards,
> Ouch Whisper
> 010101010101
>