Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Partitions in pig


+
abhishek 2012-12-18, 22:39
Copy link to this message
-
Re: Partitions in pig
Russell Jurney 2012-12-18, 23:13
This is what HCatalog and Pig's HCatStorage is for, to access data
from Hive from Pig. Unfortunately you are running CDH, which doesn't
support the Apache HCatalog project. HDP includes Apache HCatalog:
http://hortonworks.com/hdp/hdp-hcatalog-metadata-services/ More info
on Apache HCatalog is available here:
http://www.infoq.com/articles/HadoopMetadata

However, there is an RCFile loader in Piggybank:
http://svn.apache.org/viewvc/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/HiveColumnarLoader.java?view=markup

Russell Jurney http://datasyndrome.com

On Dec 18, 2012, at 2:39 PM, abhishek <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I have a use case which is implemented in hive with partitions.
>
> Say
> Customer_data/2012-12-18/....
>                        /2012-12-17/....
>                        /2012-12-16/....
>                        /
>                        /
>
> I want implement this in pig.
>
> How will partitions work in pig?
>
> Regards
> Abhishek
+
abhishek 2012-12-19, 00:11
+
Russell Jurney 2012-12-19, 00:20
+
abhishek 2012-12-19, 00:27
+
Russell Jurney 2012-12-19, 00:43
+
abhishek 2012-12-19, 04:33
+
abhishek 2012-12-19, 01:03
+
Cheolsoo Park 2012-12-18, 23:43