Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Review Request 13535: [PIG-3204] Reduce the number of getSchema calls during script parsing


Copy link to this message
-
Re: Review Request 13535: [PIG-3204] Reduce the number of getSchema calls during script parsing
Cheolsoo Park 2013-08-14, 02:13

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13535/#review25098
-----------------------------------------------------------
Overall looks good. I have few minor comments.
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestGrunt.java
<https://reviews.apache.org/r/13535/#comment49348>

    I believe we shouldn't remove xargs. It was added by PIG-3099 to avoid some race condition.

http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigServer.java
<https://reviews.apache.org/r/13535/#comment49349>

    This is a bit confusing to me. 10 - 4 + 6 = 12, but numTimesInitiated is set to 10.
    
    _testSkipParseInRegisterForBatch(false, 10, 4);

http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigServer.java
<https://reviews.apache.org/r/13535/#comment49351>

    If I understand correctly, this is equivalent to calling the followings:
    
    GruntParser grunt = new GruntParser(in);
    grunt.setInteractive(false);
    grunt.setParams(pigServer);
    grunt.parseStopOnError(false); //batch
    
    Can you explicitly call them, so it will be easier to identify the difference when skipParseInRegisterForBatch is on and off?
- Cheolsoo Park
On Aug. 13, 2013, 2:34 p.m., Rohini Palaniswamy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13535/
> -----------------------------------------------------------
>
> (Updated Aug. 13, 2013, 2:34 p.m.)
>
>
> Review request for pig.
>
>
> Bugs: PIG-3204
>     https://issues.apache.org/jira/browse/PIG-3204
>
>
> Repository: pig
>
>
> Description
> -------
>
> Change parsing from line by line to whole script at once.
>
>
> Diffs
> -----
>
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigServer.java 1510960
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/tools/grunt/GruntParser.java 1510960
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestGrunt.java 1510960
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPigServer.java 1510960
>
> Diff: https://reviews.apache.org/r/13535/diff/
>
>
> Testing
> -------
>
> New unit tests added to track the number of times a line is parsed. TestGrunt and TestShortcuts test failures fixed.
>
>
> Thanks,
>
> Rohini Palaniswamy
>
>