Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request: Add BigInteger and BigDecimal to Pig

Copy link to this message
Re: Review Request: Add BigInteger and BigDecimal to Pig

This is an automatically generated e-mail. To reply, visit:
Looks good.  The only comment that I think needs to be addressed before checking it in is the one on TypeAwareTuple.java.  After that I'm +1 on checking this in.  The rest are questions or JIRAs can be filed for later.

I also think we should file a JIRA to add e2e tests for these two types and make it a blocker for 0.12.  I know with the datetime stuff we found a bunch of new issues when we added those tests.

As a general question, I'm trying to figure out the utility of BigInteger.  I totally get BigDecimal, since people want fixed precision calculations.  BigInteger would be useful if you want integer semantics for numbers larger than 2^63, but I can't think of a use case for that off the top of my head.  I'm not saying we shouldn't do it, I just wonder if it will be used.

    Is this because HBase doesn't have a standard representation for this?


    Why not implement these?  You have read and write functions in the BytesWritable implementations.


    Again, why not implement these?  There are a string to bigint and bigdecimal functions.


    I wonder if there's a way to avoid conversions to and from and strings here.  That can hardly be efficient.  I don't think it's something we need to change now but down the road maybe something to think about.


    Seems like the second argument here should be a BigInteger, not a boolean.  Same comment for the next line.
- Alan Gates
On Jan. 18, 2013, 10:11 p.m., Jonathan Coveney wrote:
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9012/
> -----------------------------------------------------------
> (Updated Jan. 18, 2013, 10:11 p.m.)
> Review request for pig, Alan Gates and Mathias Herberts.
> Description
> -------
> This patch adds big integer and big decimal support to Pig. It could use more tests, something I'd appreciate feedback on (but I wanted to make sure the core implementation is good)
> This addresses bug PIG-2764.
>     https://issues.apache.org/jira/browse/PIG-2764
> Diffs
> -----
>   .gitignore cc62d7d
>   src/org/apache/pig/LoadCaster.java 574769b
>   src/org/apache/pig/PigWarning.java 5de075f
>   src/org/apache/pig/StoreCaster.java 5fe48de
>   src/org/apache/pig/backend/hadoop/BigDecimalWritable.java PRE-CREATION
>   src/org/apache/pig/backend/hadoop/BigIntegerWritable.java PRE-CREATION
>   src/org/apache/pig/backend/hadoop/HDataType.java 84a56b8
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java 96fba6b
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigBigDecimalRawComparator.java PRE-CREATION
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigBigIntegerRawComparator.java PRE-CREATION
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java 9749339
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java f40eb43
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/Add.java c84b767
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/ConstantExpression.java db3840f