Hi,

I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?

Thank you,
Chalcy

You need to write a UDF, like this person did:

http://search-hadoop.com/m/HFWE32CYs6x/v=plain

On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:

> Hi,
>  
> I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?
>  
> Thank you,
> Chalcy

Thank you! Loren.  So far I have only using hive as is.  Looks like I have to start extending :)

--Chalcy

From: Loren Siebert [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 15, 2011 12:28 PM
To: [EMAIL PROTECTED]
Subject: Re: urldecode hive column

You need to write a UDF, like this person did:

http://search-hadoop.com/m/HFWE32CYs6x/v=plain

On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:
Hi,

I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?

Thank you,
Chalcy

It's possible to call Java static methods directly using the built-in
'reflect' UDF. The performance will not be as good as writing your
own UDF, but you'll get results a lot faster.

On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja
<[EMAIL PROTECTED]>wrote:

>  Thank you! Loren.  So far I have only using hive as is.  Looks like I
> have to start extending J****
>
> ** **
>
> --Chalcy****
>
> ** **
>
> *From:* Loren Siebert [mailto:[EMAIL PROTECTED]]
> *Sent:* Thursday, September 15, 2011 12:28 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: urldecode hive column****
>
> ** **
>
> You need to write a UDF, like this person did:****
>
> ** **
>
> http://search-hadoop.com/m/HFWE32CYs6x/v=plain****
>
> ** **
>
> On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:****
>
>
>
> ****
>
> Hi,****
>
>  ****
>
> I have a situation where I need to do urldecode on one particular column.
> Is there any hive built in function available?****
>
>  ****
>
> Thank you,****
>
> Chalcy****
>
> ** **
>

That is cool! I didn’t know this feature existed at all.

I’ve been using https://cwiki.apache.org/confluence/display/Hive/LanguageManual as the online manual for Hive, and I don’t see ‘reflect’ mentioned there. I see it here, though: http://archive.cloudera.com/cdh/3/hive/udf/reflect.html. This archive site also contains some good info on variable substitution in Hive scripts, which I didn’t know was available either.

Thanks for the tip on ‘reflect’ !
On Sep 15, 2011, at 11:36 AM, Carl Steinbach wrote:

> It's possible to call Java static methods directly using the built-in
> 'reflect' UDF. The performance will not be as good as writing your
> own UDF, but you'll get results a lot faster.
>
> On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja <[EMAIL PROTECTED]> wrote:
> Thank you! Loren.  So far I have only using hive as is.  Looks like I have to start extending J
>
>  
>
> --Chalcy
>
>  
>
> From: Loren Siebert [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, September 15, 2011 12:28 PM
> To: [EMAIL PROTECTED]
> Subject: Re: urldecode hive column
>
>  
>
> You need to write a UDF, like this person did:
>
>  
>
> http://search-hadoop.com/m/HFWE32CYs6x/v=plain
>
>  
>
> On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:
>
>
>
>
> Hi,
>
>  
>
> I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?
>
>  
>
> Thank you,
>
> Chalcy
>
>  
>
>

It's easy to write a udf by yourself.
such as:
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
import java.net.URLDecoder;
import java.io.UnsupportedEncodingException;
public class urlDecoder extends UDF{
public urlDecoder(){
}
private Text text = new Text();
public Text evaluate(final Text column) throws UnsupportedEncodingException{
text.clear();
String columnString = column.toString();
if(column != null){
text = new Text(URLDecoder.decode(columnString, "UTF-8"));
}
   return text;
}
}

At 2011-09-16 00:03:59,"Chalcy Raja" <[EMAIL PROTECTED]> wrote:
Hi,

 

I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?

 

Thank you,

Chalcy

I used reflect to urldecode like below and it works great.

SELECT reflect("java.net.URLDecoder", "decode","to be decoded value) from footable limit 1;

I have to start building UDF at somepoint.  I am deferring that for now.

Thank you Loren and Carl for the ideas and your time.

Thank you,
Chalcy

From: Loren Siebert [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 15, 2011 2:52 PM
To: [EMAIL PROTECTED]
Subject: Re: urldecode hive column

That is cool! I didn't know this feature existed at all.

I've been using https://cwiki.apache.org/confluence/display/Hive/LanguageManual as the online manual for Hive, and I don't see 'reflect' mentioned there. I see it here, though: http://archive.cloudera.com/cdh/3/hive/udf/reflect.html. This archive site also contains some good info on variable substitution in Hive scripts, which I didn't know was available either.

Thanks for the tip on 'reflect' !
On Sep 15, 2011, at 11:36 AM, Carl Steinbach wrote:
It's possible to call Java static methods directly using the built-in
'reflect' UDF. The performance will not be as good as writing your
own UDF, but you'll get results a lot faster.

On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Thank you! Loren.  So far I have only using hive as is.  Looks like I have to start extending :)

--Chalcy

From: Loren Siebert [mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>]
Sent: Thursday, September 15, 2011 12:28 PM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: Re: urldecode hive column

You need to write a UDF, like this person did:

http://search-hadoop.com/m/HFWE32CYs6x/v=plain

On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:

Hi,

I have a situation where I need to do urldecode on one particular column.  Is there any hive built in function available?

Thank you,
Chalcy