Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: how to get the mapreduce code which was pig/hive script translated to?


Copy link to this message
-
Re: how to get the mapreduce code which was pig/hive script translated to?
Lance Norskog 2013-06-16, 23:06
Both Pig and Hive have an 'explain plan' command that prints a schematic
version. This might make it easier to see what M/R algorithms are used.
Mostly the data goes through single-threaded transforms inside a mapper
or reducer.

https://cwiki.apache.org/Hive/languagemanual-explain.html

On 06/16/2013 09:17 AM, Edward Capriolo wrote:
> Hive serializes the entire plan into an XML file if you set the log 4j
> settings to debug you should get the locations to the files
> itgenerates before launching the job.
>
>
> On Sun, Jun 16, 2013 at 11:08 AM, Harsh J <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     This is a question for the Hive/Pig lists to answer best.
>
>     Note though that they only compile a plan, not the code. The code is
>     available already, the compiled plan just structures the execution
>     flow. If you take a look at the sources, you'll find the bits and
>     pieces that get linked together depending on the operation.
>
>     On Sun, Jun 16, 2013 at 10:15 AM, 王洪军 <[EMAIL PROTECTED]
>     <mailto:[EMAIL PROTECTED]>> wrote:
>     > hi,all
>     >
>     >      how to get the mapreduce code which was pig/hive script
>     translated to?
>     >
>     > Best regards,
>     > wanghj
>     >
>
>
>
>     --
>     Harsh J
>
>