Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: how to get the mapreduce code which was pig/hive script translated to?


Copy link to this message
-
Re: how to get the mapreduce code which was pig/hive script translated to?
Both Pig and Hive have an 'explain plan' command that prints a schematic
version. This might make it easier to see what M/R algorithms are used.
Mostly the data goes through single-threaded transforms inside a mapper
or reducer.

https://cwiki.apache.org/Hive/languagemanual-explain.html

On 06/16/2013 09:17 AM, Edward Capriolo wrote:
> Hive serializes the entire plan into an XML file if you set the log 4j
> settings to debug you should get the locations to the files
> itgenerates before launching the job.
>
>
> On Sun, Jun 16, 2013 at 11:08 AM, Harsh J <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     This is a question for the Hive/Pig lists to answer best.
>
>     Note though that they only compile a plan, not the code. The code is
>     available already, the compiled plan just structures the execution
>     flow. If you take a look at the sources, you'll find the bits and
>     pieces that get linked together depending on the operation.
>
>     On Sun, Jun 16, 2013 at 10:15 AM, 王洪军 <[EMAIL PROTECTED]
>     <mailto:[EMAIL PROTECTED]>> wrote:
>     > hi,all
>     >
>     >      how to get the mapreduce code which was pig/hive script
>     translated to?
>     >
>     > Best regards,
>     > wanghj
>     >
>
>
>
>     --
>     Harsh J
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB