Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - run pig explain command over the entire script in java


+
Petar Jovanovic 2013-02-18, 16:18
+
Jonathan Coveney 2013-02-18, 18:04
+
Petar Jovanovic 2013-02-18, 21:18
+
Jonathan Coveney 2013-02-18, 21:21
Copy link to this message
-
Re: run pig explain command over the entire script in java
Petar Jovanovic 2013-02-18, 22:24
Well no. I want to do explain over the entire script as I show you in
the command I wrote.
That would produce the compact execution plan for the whole script and
not the several separate ones (one for each alias).
On 2/18/2013 10:21 PM, Jonathan Coveney wrote:
> I guess I'm confused at what you want then.
>
> So we have a script:
>
> a = ..
> b = ...
> c = ..
>
> my impression was that you essentially wanted to print "explain a; explain
> b; explain c;" is this not what you mean/
>
>
> 2013/2/18 Petar Jovanovic <[EMAIL PROTECTED]>
>
>> But pig already has this, it's jut that the java API seems not to have it.
>> The point would be to have the execution flow from the entire script.
>> Indeed, the output is not exactly the same if the explain is called over
>> the entire script than if it is called for each query (alias).
>>
>> If from the shell we execute command
>>
>> $ pig -x local -e 'explain -script Temp1/TPC_test.pig -out
>> explain-out9.txt'
>> it will not give the same output as if we did it for each operation
>> separately.
>>
>>
>> On 2/18/2013 7:04 PM, Jonathan Coveney wrote:
>>
>>> Hacky way: grep for "^\S =", pull out the names, and then do the explains.
>>>
>>> Why is doing the progressive explains useful? it wouldn't be too hard to
>>> build this into pig but the results would be pretty unwieldy, it'd be
>>> really big, and pretty redundant.
>>>
>>>
>>> 2013/2/18 Petar Jovanovic <[EMAIL PROTECTED]>
>>>
>>>   Hi,
>>>> I am trying to find the way to run the explain command over the entire
>>>> pig
>>>> script in java. I was using PigServer but it offers only to do explain
>>>> over
>>>> the single query (alias) not the entire script. Is there a way to do
>>>> something like:
>>>> $ pig -x local -e 'explain -script Temp1/TPC_test.pig -out
>>>> explain-out9.txt'
>>>> but from my Java code?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>
+
Jonathan Coveney 2013-02-18, 22:51
+
Petar Jovanovic 2013-02-18, 23:27