Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Type Casting


Copy link to this message
-
RE: Type Casting
Great Jinfeng,
The parsing part is something I was stuck yesterday and was trying to figure out. Nice to know you are working on it.
I would then continue with the isCastable check. Would implement it similar to Optiq's Map rather than Hive style.
You can keep working on the parsing part. I would let you know when my part is done.

- Yash
-----Original Message-----
From: Jinfeng Ni [mailto:[EMAIL PROTECTED]]
Sent: Monday, November 25, 2013 11:56 PM
To: [EMAIL PROTECTED]
Subject: Re: Type Casting

Hi Yash,

For the checking of isCastable, I agree that we can not directly use Optiq's SqlAssignmentRules, since the date types are different. But looks like the implementation would be quite similar, as we could use Map<type, Set<type>> to check if one type is allowed to cast to another type.

I'm working on the parser part to support explicit cast function,   so that
we can put cast function in logical/physical plan.
How about you continue work on implicit cast and the check of isCastable?
Once both the explicit cast and implicit cast is ready, we can discuss how to merge them together ( I assume implicit will eventually call the explicit cast function to do the real cast).

On Sun, Nov 24, 2013 at 9:27 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:

> For now, you need to do all cast work using logical or physical plans.
> The Optiq currently removes any explicit casts from the logical
> output.  Once we depend on an updated Optiq with Mehant's ANY changes,
> we can add explicit cast support back in.
>
> Jinfeng was working on some cast infrastructure as well.  Jinfeng, can
> you comment on the second part given the design brainstorming you've done?
>
> thanks,
> Jacques
>
>
> On Sun, Nov 24, 2013 at 1:45 AM, Yash Sharma
> <[EMAIL PROTECTED]
> >wrote:
>
> > Observations while providing explicit cast functions.
> >
> > 1.
> > On hitting any explicit cast query on Sqlline I get the below results.
> > While debugging the flow I see that there is no lookup happening in
> > FunctionImplementationRegistry for CAST() function, and therefore it
> > is
> not
> > casting the argument and returning the passed argument directly.
> > It seems like some part of query validation is bypassing the lookup
> > for 'cast' function from FunctionRegistry.
> >
> > 0: jdbc:drill:schema=parquet-local> select cast(2.3 as int) from
> > "sample-data/region.parquet";
> > +---------+
> > | EXPR$0  |
> > +---------+
> > | 2.3     |
> > | 2.3     |
> > | 2.3     |
> > | 2.3     |
> > | 2.3     |
> > +---------+
> > 5 rows selected (0.489 seconds)
> >
> >
> >
> > 2.
> > For the 'Is Castable' check, we might have to create our own class
> similar
> > to Optiq's SqlAssignmentRules. Since Drill's datatypes are different
> > from those used in Optiq we might not be able to reuse the class directly.
> > (Ref:
> >
> https://github.com/julianhyde/optiq/blob/master/core/src/main/java/org
> /eigenbase/sql/type/SqlTypeAssignmentRules.java
> > )
> >
> > Alternatively, an approach can be adopted from Hive's
> > org.apache.hadoop.hive.ql.exec.FunctionRegistry's
> > implicitConvertable() method. It uses datatype grouping via enum
> > (PrimitiveGrouping). Then
> > implicitlyConvertable() method checks the common groups to check if
> > types are convertible implicitly.
> >
> > Let me know your thoughts on the two approaches.
> >
> > Regards,
> > Yash
> >
> >
> >
> >
> > ________________________________________
> > From: Julian Hyde [[EMAIL PROTECTED]]
> > Sent: Friday, November 22, 2013 4:16 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: hangout
> >
> > On Nov 20, 2013, at 9:12 PM, Jinfeng Ni <[EMAIL PROTECTED]> wrote:
> >
> > > 5. Nullable vs Non-nullable. I think implicit cast probably need
> > > support cast from Non-nullable exp
> > >
> > > Nullable exp. Otherwise, for each operator and each type, we at
> > > least have to implement 4 versions:
> > >
> > >     1) Nullable  int +  Nullable int
> > >
> > >     2) Nullable  int + Non-Nullalbe int

________________________________
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.