Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Review Request 14552: PIG-3480 TFile-based tmpfile compression crashes in some cases


Copy link to this message
-
Re: Review Request 14552: PIG-3480 TFile-based tmpfile compression crashes in some cases
Aniket Mokashi 2013-10-09, 01:10

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14552/#review26808
-----------------------------------------------------------

trunk/src/org/apache/pig/impl/util/Utils.java
<https://reviews.apache.org/r/14552/#comment52164>

    Spoke to Dmitriy offline. This approach does not allow us support more compression types (other than these 4) for sequencefile. We should avoid throwing exception when the codec for sequencefile is not specified.
- Aniket Mokashi
On Oct. 9, 2013, 1:02 a.m., Aniket Mokashi wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14552/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2013, 1:02 a.m.)
>
>
> Review request for pig, Cheolsoo Park, Dmitriy Ryaboy, and Rohini Palaniswamy.
>
>
> Bugs: PIG-3480
>     https://issues.apache.org/jira/browse/PIG-3480
>
>
> Repository: pig
>
>
> Description
> -------
>
> - Added a new parameter to make SequenceFileInterStorage optional.
> - Added tests
> - Refactored apis
>
>
> Diffs
> -----
>
>   trunk/conf/pig.properties 1530468
>   trunk/src/org/apache/pig/PigConfiguration.java 1530468
>   trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java 1530468
>   trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java 1530468
>   trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartitionRearrange.java 1530468
>   trunk/src/org/apache/pig/backend/hadoop/executionengine/util/MapRedUtil.java 1530468
>   trunk/src/org/apache/pig/impl/io/InterStorage.java 1530468
>   trunk/src/org/apache/pig/impl/io/SequenceFileInterStorage.java PRE-CREATION
>   trunk/src/org/apache/pig/impl/io/TFileStorage.java 1530468
>   trunk/src/org/apache/pig/impl/util/Utils.java 1530468
>   trunk/test/org/apache/pig/test/TestTmpFileCompression.java 1530468
>
> Diff: https://reviews.apache.org/r/14552/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Aniket Mokashi
>
>