Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> run pig explain command over the entire script in java


Copy link to this message
-
Re: run pig explain command over the entire script in java
I would like to do it from Java automatically and also for the scripts
with multiple stores.

On 2/18/2013 11:51 PM, Jonathan Coveney wrote:
> so what is the issue with just doing "explain final_alias;" ? Or do you
> want it to do explain on the final alias of any given script?
>
>
> 2013/2/18 Petar Jovanovic <[EMAIL PROTECTED]>
>
>> Well no. I want to do explain over the entire script as I show you in the
>> command I wrote.
>> That would produce the compact execution plan for the whole script and not
>> the several separate ones (one for each alias).
>>
>>
>>
>> On 2/18/2013 10:21 PM, Jonathan Coveney wrote:
>>
>>> I guess I'm confused at what you want then.
>>>
>>> So we have a script:
>>>
>>> a = ..
>>> b = ...
>>> c = ..
>>>
>>> my impression was that you essentially wanted to print "explain a; explain
>>> b; explain c;" is this not what you mean/
>>>
>>>
>>> 2013/2/18 Petar Jovanovic <[EMAIL PROTECTED]>
>>>
>>>   But pig already has this, it's jut that the java API seems not to have
>>>> it.
>>>> The point would be to have the execution flow from the entire script.
>>>> Indeed, the output is not exactly the same if the explain is called over
>>>> the entire script than if it is called for each query (alias).
>>>>
>>>> If from the shell we execute command
>>>>
>>>> $ pig -x local -e 'explain -script Temp1/TPC_test.pig -out
>>>> explain-out9.txt'
>>>> it will not give the same output as if we did it for each operation
>>>> separately.
>>>>
>>>>
>>>> On 2/18/2013 7:04 PM, Jonathan Coveney wrote:
>>>>
>>>>   Hacky way: grep for "^\S =", pull out the names, and then do the
>>>>> explains.
>>>>>
>>>>> Why is doing the progressive explains useful? it wouldn't be too hard to
>>>>> build this into pig but the results would be pretty unwieldy, it'd be
>>>>> really big, and pretty redundant.
>>>>>
>>>>>
>>>>> 2013/2/18 Petar Jovanovic <[EMAIL PROTECTED]>
>>>>>
>>>>>    Hi,
>>>>>
>>>>>> I am trying to find the way to run the explain command over the entire
>>>>>> pig
>>>>>> script in java. I was using PigServer but it offers only to do explain
>>>>>> over
>>>>>> the single query (alias) not the entire script. Is there a way to do
>>>>>> something like:
>>>>>> $ pig -x local -e 'explain -script Temp1/TPC_test.pig -out
>>>>>> explain-out9.txt'
>>>>>> but from my Java code?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB