Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - easiest way to get loops in PIG?


Copy link to this message
-
Re: easiest way to get loops in PIG?
Duckworth, Will 2012-06-21, 02:23
As mentioned mbedding is the way to go now. We have done it before with bash scripts and our custom workflow system.

On Jun 20, 2012, at 10:20 PM, "Norbert Burger" <[EMAIL PROTECTED]> wrote:

> Yang -- have you seen Hortonworks' blogpost on this?
>
> http://hortonworks.com/blog/transitive-closure-in-apache-pig/
>
> Norbert
>
> On Wed, Jun 20, 2012 at 10:15 PM, Prashant Kommireddi
> <[EMAIL PROTECTED]>wrote:
>
>> Would embedding Pig in java or other languages work?
>>
>> http://pig.apache.org/docs/r0.10.0/cont.html#embed-java
>>
>>
>> On Jun 20, 2012, at 7:12 PM, Yang <[EMAIL PROTECTED]> wrote:
>>
>>> I agree that pig does not have loop probably for a good reason.
>>>
>>> but currently I need to write a code to find the transitive closures of
>>> many edges in a graph.
>>> so I need to iterate a code snippet several times, so finally I can find
>> a
>>> connected component of size 2^N
>>>
>>> right now I just copy-paste the snippet several times.
>>>
>>> I guess I could take out the snippet and make it into a separate pig
>>> script, and load and store intermediate data
>>> at the beginning and end. but loading data is kind of a waste.
>>>
>>> any suggestions?
>>>
>>> Thanks
>>> Yang
>>

Will Duckworth  Senior Vice President, Software Engineering  | comScore, Inc.(NASDAQ:SCOR)
o +1 (703) 438-2108 | m +1 (301) 606-2977 | mailto:[EMAIL PROTECTED]
.....................................................................................................

Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
www.comscore.com/MobileMetrix