Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Calling same UDF multiple times in a SELECT query


Copy link to this message
-
Re: Calling same UDF multiple times in a SELECT query
Sanjay Subramanian 2013-07-23, 19:49
Hi Nitin

Thanks
Yes I did actually do a nested query but it spawns reducers that I did not want…I wanted to keep it to one select so that only mappers are called and then I can invoke several mappers to call the we b service

Thanks

sanjay

From: Nitin Pawar <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, July 23, 2013 12:25 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Calling same UDF multiple times in a SELECT query

fucntion return values are not stored for repeat use of same (as per my understanding)

I know you may have already thought about other approach as

select a , if (call <-1, -1 call) as b from (select a, fooudf(a) as call from table
On Wed, Jul 24, 2013 at 12:42 AM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi

V r using version hive-exec-0.9.0-cdh4.1.2 in production

I need to check and use the output from a UDF in a query to assign values to 2 columns in a SELECT query

Example

SELECT
     a,
     IF(fooUdf(a) < -1  , -1, fooUdf(a)) as b,
     IF(fooUdf(a) < -1  , fooUdf(a), 0) as c
FROM
     my_hive_table
So will fooUdf be called 4 times ? Or once ?

Why this is important is because in our case this UDF calls a web service and I don't want so many calls to the service.

Thanks

sanjay

CONFIDENTIALITY NOTICE
=====================This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.

--
Nitin Pawar

CONFIDENTIALITY NOTICE
=====================This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.