Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: MapReduce to load data in HBase


+
Mohammad Tariq 2013-02-07, 11:59
+
Panshul Whisper 2013-02-07, 12:55
+
Mohammad Tariq 2013-02-07, 13:23
Copy link to this message
-
Re: MapReduce to load data in HBase
Pig is like any other scripting language. Therefore maintainable.

Your script defines the schema of the data set and then does the processing you tell it to do.

On Feb 7, 2013, at 6:55 AM, Panshul Whisper <[EMAIL PROTECTED]> wrote:

> Hello,
>
> Thank you for the replies.
>
> I have not used pig yet. I am looking into it. I wanted to implement both
> the approaches.
> Are pig scripts maintainable? Because the Json structure that I will be
> receiving will be changing quite often. Almost 3 times a month.
> I will be processing 24 million Json files per month.
> I am getting one big file with almost 3 million Json files aggregated. One
> Json per line. I need to process this file and store all values into HBase.
>
> Thanking You,
>
>
>
>
> On Thu, Feb 7, 2013 at 12:59 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
>
>> Good point sir. If Pig fits into Panshul's requirements then it's a much
>> better option.
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Thu, Feb 7, 2013 at 5:25 PM, Damien Hardy <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Hello,
>>> Why not using a PIG script for that ?
>>> make the json file available on HDFS
>>> Load with
>>>
>>>
>> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/builtin/JsonLoader.html
>>> Store with
>>>
>>>
>> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/backend/hadoop/hbase/HBaseStorage.html
>>>
>>> http://pig.apache.org/docs/r0.10.0/
>>>
>>> Cheers,
>>>
>>> --
>>> Damien
>>>
>>
>
>
>
> --
> Regards,
> Ouch Whisper
> 010101010101

The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com