Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 15105: HIVE-3844: Unix timestamps don't seem to be read correctly from HDFS as Timestamp column


Copy link to this message
-
Re: Review Request 15105: HIVE-3844: Unix timestamps don't seem to be read correctly from HDFS as Timestamp column

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15105/#review28864
-----------------------------------------------------------

serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java
<https://reviews.apache.org/r/15105/#comment55925>

    Maybe consider using TimestampWritable.decimalToTimestamp() here. Looks like this is what the cast operator ends up using.

serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java
<https://reviews.apache.org/r/15105/#comment55926>

    If you are allowing numeric value to timestamp conversion, not sure if it needs to be so strict that it returns null because of the nanos. Looks like TimestampWritable.decimalToTimestamp() behavior isn't so strict.
- Jason Dere
On Oct. 30, 2013, 10:05 p.m., Venki Korukanti wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15105/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2013, 10:05 p.m.)
>
>
> Review request for hive and Mark Grover.
>
>
> Bugs: HIVE-3844
>     https://issues.apache.org/jira/browse/HIVE-3844
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Currently LazyTimestamp can only interpret timestamps in JDBC format. If there are UNIX epoch styles timestamps it treats them as invalid and output NULL. This patch is to support all three types of formats mentioned here https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-Timestamps.
>
> BigDecimal is used to avoid rounding off errors with atoi/atof type of conversions.
>
>
> Diffs
> -----
>
>   data/files/timestamp_data.txt PRE-CREATION
>   ql/src/test/queries/clientpositive/timestamp_4.q PRE-CREATION
>   ql/src/test/queries/clientpositive/timestamp_null.q efd5bc4
>   ql/src/test/results/clientpositive/timestamp_4.q.out PRE-CREATION
>   ql/src/test/results/clientpositive/timestamp_null.q.out d21b880
>   serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyTimestamp.java 27895c5
>
> Diff: https://reviews.apache.org/r/15105/diff/
>
>
> Testing
> -------
>
> Fix includes a unittest which reads from a file that has most types of timestamp formats.
>
>
> Thanks,
>
> Venki Korukanti
>
>