Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> easiest way to get loops in PIG?


Copy link to this message
-
Re: easiest way to get loops in PIG?
As mentioned mbedding is the way to go now. We have done it before with bash scripts and our custom workflow system.

On Jun 20, 2012, at 10:20 PM, "Norbert Burger" <[EMAIL PROTECTED]> wrote:

> Yang -- have you seen Hortonworks' blogpost on this?
>
> http://hortonworks.com/blog/transitive-closure-in-apache-pig/
>
> Norbert
>
> On Wed, Jun 20, 2012 at 10:15 PM, Prashant Kommireddi
> <[EMAIL PROTECTED]>wrote:
>
>> Would embedding Pig in java or other languages work?
>>
>> http://pig.apache.org/docs/r0.10.0/cont.html#embed-java
>>
>>
>> On Jun 20, 2012, at 7:12 PM, Yang <[EMAIL PROTECTED]> wrote:
>>
>>> I agree that pig does not have loop probably for a good reason.
>>>
>>> but currently I need to write a code to find the transitive closures of
>>> many edges in a graph.
>>> so I need to iterate a code snippet several times, so finally I can find
>> a
>>> connected component of size 2^N
>>>
>>> right now I just copy-paste the snippet several times.
>>>
>>> I guess I could take out the snippet and make it into a separate pig
>>> script, and load and store intermediate data
>>> at the beginning and end. but loading data is kind of a waste.
>>>
>>> any suggestions?
>>>
>>> Thanks
>>> Yang
>>

Will Duckworth  Senior Vice President, Software Engineering  | comScore, Inc.(NASDAQ:SCOR)
o +1 (703) 438-2108 | m +1 (301) 606-2977 | mailto:[EMAIL PROTECTED]
.....................................................................................................

Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
www.comscore.com/MobileMetrix