Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Review Request 15565: PIG-3567 LogicalPlanPrinter throws OOM for large scripts


Copy link to this message
-
Re: Review Request 15565: PIG-3567 LogicalPlanPrinter throws OOM for large scripts
Daniel Dai 2013-11-15, 21:53

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15565/#review28990
-----------------------------------------------------------

Ship it!
Didn't go through line by line, but general approach looks good. You can add a unit test to check the memory footprint.

- Daniel Dai
On Nov. 15, 2013, 5:25 a.m., Aniket Mokashi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15565/
> -----------------------------------------------------------
>
> (Updated Nov. 15, 2013, 5:25 a.m.)
>
>
> Review request for pig, Cheolsoo Park, Daniel Dai, Julien Le Dem, and Rohini Palaniswamy.
>
>
> Bugs: PIG-3567
>     https://issues.apache.org/jira/browse/PIG-3567
>
>
> Repository: pig
>
>
> Description
> -------
>
> Changed LPPrinter recursion to directly write to stream so that memory footprint is reduced.
>
>
> Diffs
> -----
>
>   trunk/src/org/apache/pig/newplan/logical/optimizer/LogicalPlanPrinter.java 1538708
>
> Diff: https://reviews.apache.org/r/15565/diff/
>
>
> Testing
> -------
>
> Diff'ed explain plan of few existing scripts. No diffs found. I will try to submit a testcase to show that LP print of before after is same (test need not be committed).
>
>
> Thanks,
>
> Aniket Mokashi
>
>