I am trying to use a parameter as the expression in a filter.

Assuming:

colors_in = load ‘$in_path’ as (color:chararray);
flt = filter colors_in by color == ‘blue’ or color == ‘green’;

I would like to use this instead:

flt = filter colors_in by ‘$color_filter’;

But I get an unexpected symbol near ‘color’.  I have tried a couple different formats for the string that I am passing in an none seem to work.

I can get this to work:

flt = filter colors_in by (color == ‘$color_filter1’ or color == ‘$color_filter2’);

But it means I have to know the complete filter for anything I want to run.

I know I could do this with embedded or dynamically building the pig but was trying to do it with just passing parameters to an existing pig script.

Any ideas?

Thanks.

Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)

o +1 (703) 438-2108 | m +1 (301) 606-2977 | [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

...........................................................................................................

Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>

I think you just want this:

filt = filter colors_in by $color_filter;

(no quotes)

D

On Mon, Aug 27, 2012 at 1:50 PM, Duckworth, Will
<[EMAIL PROTECTED]> wrote:
> I am trying to use a parameter as the expression in a filter.
>
> Assuming:
>
> colors_in = load ‘$in_path’ as (color:chararray);
> flt = filter colors_in by color == ‘blue’ or color == ‘green’;
>
> I would like to use this instead:
>
> flt = filter colors_in by ‘$color_filter’;
>
> But I get an unexpected symbol near ‘color’.  I have tried a couple different formats for the string that I am passing in an none seem to work.
>
> I can get this to work:
>
> flt = filter colors_in by (color == ‘$color_filter1’ or color == ‘$color_filter2’);
>
> But it means I have to know the complete filter for anything I want to run.
>
> I know I could do this with embedded or dynamically building the pig but was trying to do it with just passing parameters to an existing pig script.
>
> Any ideas?
>
> Thanks.
>
>
>
> Will Duckworth Senior Vice President, Software Engineering | comScore, Inc. (NASDAQ:SCOR)
>
> o +1 (703) 438-2108 | m +1 (301) 606-2977 | [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
> ...........................................................................................................
>
> Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral measurement
> www.comscore.com/MobileMetrix<http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
>
>

There are a couple ways to do this. One is to turn your filter into a
replicated join. IE put the colors you want in a separate file, and then
just join on color. Since it is an equijoin, it's essentially a big or
statement.

The second way would be the use the parameter approach, and then make a UDF
returning boolean that takes that parameter as it's argument

ie

DEFINE FilterUdf com.comscore.pig.MyFilterUdf('$color_param')

filt = filter colors_in by FilterUdf(color);

Make sense?

2012/8/27 Duckworth, Will <[EMAIL PROTECTED]>

> I am trying to use a parameter as the expression in a filter.
>
> Assuming:
>
> colors_in = load ‘$in_path’ as (color:chararray);
> flt = filter colors_in by color == ‘blue’ or color == ‘green’;
>
> I would like to use this instead:
>
> flt = filter colors_in by ‘$color_filter’;
>
> But I get an unexpected symbol near ‘color’.  I have tried a couple
> different formats for the string that I am passing in an none seem to work.
>
> I can get this to work:
>
> flt = filter colors_in by (color == ‘$color_filter1�� or color => ‘$color_filter2’);
>
> But it means I have to know the complete filter for anything I want to run.
>
> I know I could do this with embedded or dynamically building the pig but
> was trying to do it with just passing parameters to an existing pig script.
>
> Any ideas?
>
> Thanks.
>
>
>
> Will Duckworth Senior Vice President, Software Engineering | comScore,
> Inc. (NASDAQ:SCOR)
>
> o +1 (703) 438-2108 | m +1 (301) 606-2977 | [EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>
>
>
> ...........................................................................................................
>
> Introducing Mobile Metrix 2.0 - The next generation of mobile behavioral
> measurement
> www.comscore.com/MobileMetrix<
> http://www.comscore.com/Products_Services/Product_Index/Mobile_Metrix_2.0>
>
>
>