Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Partitions in pig


+
abhishek 2012-12-18, 22:39
+
Russell Jurney 2012-12-18, 23:13
+
abhishek 2012-12-19, 00:11
Copy link to this message
-
Re: Partitions in pig
Are you doing a directory-based partition with Hive, or are you
letting Hive's RCFile partition data for you?

Russell Jurney http://datasyndrome.com

On Dec 18, 2012, at 4:12 PM, abhishek <[EMAIL PROTECTED]> wrote:

> Hi Russell,
>
> Thanks for the reply.How RCFile loader is related to partitions?
>
> I did not get your point in this.
>
> Regards
> Abhi
>
> Sent from my iPhone
>
> On Dec 18, 2012, at 6:13 PM, Russell Jurney <[EMAIL PROTECTED]> wrote:
>
>> This is what HCatalog and Pig's HCatStorage is for, to access data
>> from Hive from Pig. Unfortunately you are running CDH, which doesn't
>> support the Apache HCatalog project. HDP includes Apache HCatalog:
>> http://hortonworks.com/hdp/hdp-hcatalog-metadata-services/ More info
>> on Apache HCatalog is available here:
>> http://www.infoq.com/articles/HadoopMetadata
>>
>> However, there is an RCFile loader in Piggybank:
>> http://svn.apache.org/viewvc/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/HiveColumnarLoader.java?view=markup
>>
>> Russell Jurney http://datasyndrome.com
>>
>> On Dec 18, 2012, at 2:39 PM, abhishek <[EMAIL PROTECTED]> wrote:
>>
>>> Hi all,
>>>
>>> I have a use case which is implemented in hive with partitions.
>>>
>>> Say
>>> Customer_data/2012-12-18/....
>>>                      /2012-12-17/....
>>>                      /2012-12-16/....
>>>                      /
>>>                      /
>>>
>>> I want implement this in pig.
>>>
>>> How will partitions work in pig?
>>>
>>> Regards
>>> Abhishek
+
abhishek 2012-12-19, 00:27
+
Russell Jurney 2012-12-19, 00:43
+
abhishek 2012-12-19, 04:33
+
abhishek 2012-12-19, 01:03
+
Cheolsoo Park 2012-12-18, 23:43