Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Partitions in pig


Copy link to this message
-
Re: Partitions in pig
It works for me thanks.

Regards
Abhi

Sent from my iPhone

On Dec 18, 2012, at 7:43 PM, Russell Jurney <[EMAIL PROTECTED]> wrote:

> It will work like so:
> http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur
>
> Russell Jurney http://datasyndrome.com
>
> On Dec 18, 2012, at 4:27 PM, abhishek <[EMAIL PROTECTED]> wrote:
>
>> Directory based partition in hive.
>>
>> Partition by date
>>
>> Thanks
>> Abhi
>>
>> Sent from my iPhone
>>
>> On Dec 18, 2012, at 7:20 PM, Russell Jurney <[EMAIL PROTECTED]> wrote:
>>
>>> Are you doing a directory-based partition with Hive, or are you
>>> letting Hive's RCFile partition data for you?
>>>
>>> Russell Jurney http://datasyndrome.com
>>>
>>> On Dec 18, 2012, at 4:12 PM, abhishek <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Russell,
>>>>
>>>> Thanks for the reply.How RCFile loader is related to partitions?
>>>>
>>>> I did not get your point in this.
>>>>
>>>> Regards
>>>> Abhi
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Dec 18, 2012, at 6:13 PM, Russell Jurney <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> This is what HCatalog and Pig's HCatStorage is for, to access data
>>>>> from Hive from Pig. Unfortunately you are running CDH, which doesn't
>>>>> support the Apache HCatalog project. HDP includes Apache HCatalog:
>>>>> http://hortonworks.com/hdp/hdp-hcatalog-metadata-services/ More info
>>>>> on Apache HCatalog is available here:
>>>>> http://www.infoq.com/articles/HadoopMetadata
>>>>>
>>>>> However, there is an RCFile loader in Piggybank:
>>>>> http://svn.apache.org/viewvc/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/HiveColumnarLoader.java?view=markup
>>>>>
>>>>> Russell Jurney http://datasyndrome.com
>>>>>
>>>>> On Dec 18, 2012, at 2:39 PM, abhishek <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I have a use case which is implemented in hive with partitions.
>>>>>>
>>>>>> Say
>>>>>> Customer_data/2012-12-18/....
>>>>>>                   /2012-12-17/....
>>>>>>                   /2012-12-16/....
>>>>>>                   /
>>>>>>                   /
>>>>>>
>>>>>> I want implement this in pig.
>>>>>>
>>>>>> How will partitions work in pig?
>>>>>>
>>>>>> Regards
>>>>>> Abhishek
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB