Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - Review Request 13781: Changes to add support for streaming_python udfs.


+
Jeremy Karn 2013-08-23, 20:19
Copy link to this message
-
Re: Review Request 13781: Changes to add support for streaming_python udfs.
Julien Le Dem 2013-09-04, 23:36

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13781/#review25913
-----------------------------------------------------------

src/org/apache/pig/builtin/StreamUDFToPig.java
<https://reviews.apache.org/r/13781/#comment50560>

    decoding string is actually quite costly as java strings are utf-16 and creating a string always copies the content of the byte array.
    Can't you just use the binary format of the type?

src/org/apache/pig/builtin/StreamUDFToPig.java
<https://reviews.apache.org/r/13781/#comment50566>

    would StreamTokenizer help ?

src/org/apache/pig/builtin/StreamUDFToPig.java
<https://reviews.apache.org/r/13781/#comment50562>

    if you know the size in advance you can pass it here and avoid resizing

src/org/apache/pig/builtin/StreamUDFToPig.java
<https://reviews.apache.org/r/13781/#comment50561>

    you probably want to do getInstance() once

src/org/apache/pig/builtin/StreamingDelimiters.java
<https://reviews.apache.org/r/13781/#comment50565>

    why do we need a hashmap to access those statically defined values?

src/org/apache/pig/builtin/StreamingUDF.java
<https://reviews.apache.org/r/13781/#comment50568>

    enum?

src/org/apache/pig/builtin/StreamingUDF.java
<https://reviews.apache.org/r/13781/#comment50569>

    chain exceptions

src/org/apache/pig/builtin/StreamingUDF.java
<https://reviews.apache.org/r/13781/#comment50571>

    if (info) log.info

src/org/apache/pig/builtin/StreamingUDFException.java
<https://reviews.apache.org/r/13781/#comment50574>

    if you don't make the parent see the cause printStackTrace won't display the cause's stackTrace

src/org/apache/pig/impl/util/StorageUtil.java
<https://reviews.apache.org/r/13781/#comment50575>

    why is that?
- Julien Le Dem
On Aug. 23, 2013, 8:19 p.m., Jeremy Karn wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13781/
> -----------------------------------------------------------
>
> (Updated Aug. 23, 2013, 8:19 p.m.)
>
>
> Review request for pig.
>
>
> Repository: pig-git
>
>
> Description
> -------
>
> Changes for PIG-2417 (https://issues.apache.org/jira/browse/PIG-2417)
>
>
> Diffs
> -----
>
>   build.xml b20eb3d
>   src/org/apache/pig/PigToStream.java 7cc2950
>   src/org/apache/pig/StreamToPig.java ff24b27
>   src/org/apache/pig/builtin/PigStreaming.java 5467693
>   src/org/apache/pig/builtin/PigToStreamUDF.java PRE-CREATION
>   src/org/apache/pig/builtin/StreamUDFToPig.java PRE-CREATION
>   src/org/apache/pig/builtin/StreamingDelimiters.java PRE-CREATION
>   src/org/apache/pig/builtin/StreamingUDF.java PRE-CREATION
>   src/org/apache/pig/builtin/StreamingUDFException.java PRE-CREATION
>   src/org/apache/pig/builtin/StreamingUDFOutputSchemaException.java PRE-CREATION
>   src/org/apache/pig/impl/streaming/DefaultInputHandler.java 301bea3
>   src/org/apache/pig/impl/streaming/DefaultOutputHandler.java 1b46e7d
>   src/org/apache/pig/impl/streaming/ExecutableManager.java cf79c83
>   src/org/apache/pig/impl/streaming/InputHandler.java 690d94e
>   src/org/apache/pig/impl/streaming/OutputHandler.java 6e9262a
>   src/org/apache/pig/impl/streaming/StreamingUDFOutputHandler.java PRE-CREATION
>   src/org/apache/pig/impl/streaming/StreamingUtil.java PRE-CREATION
>   src/org/apache/pig/impl/util/JarManager.java 5c4acb0
>   src/org/apache/pig/impl/util/StorageUtil.java dcb62ec
>   src/org/apache/pig/scripting/ScriptEngine.java 29a9e1f
>   src/org/apache/pig/scripting/ScriptingIllustrateOutputCapturer.java PRE-CREATION
>   src/org/apache/pig/scripting/streaming/python/PythonScriptEngine.java PRE-CREATION
>   src/python/streaming/controller.py PRE-CREATION
>   src/python/streaming/pig_util.py PRE-CREATION
>   test/org/apache/pig/builtin/TestPigToStreamUDF.java PRE-CREATION
>   test/org/apache/pig/builtin/TestStreamUDFToPig.java PRE-CREATION
+
Jeremy Karn 2013-09-05, 20:27
+
Jeremy Karn 2013-09-05, 20:23
+
Daniel Dai 2013-09-07, 00:04
+
Jeremy Karn 2013-09-07, 15:38
+
Daniel Dai 2013-09-16, 04:18
+
Jeremy Karn 2013-09-07, 15:39