Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Regex variable using param file


Copy link to this message
-
Re: Regex variable using param file
Try removing those 3 comments and running again. The token substitution
looks for $ followed by [key]=[value] and my guess is that there's a bug
where $ in comments isn't being ignored. Please report back if that works
and if so, I'll file a JIRA.

On Thu, Apr 5, 2012 at 8:34 AM, Shin Chan <[EMAIL PROTECTED]> wrote:

>  Hi Bill
>
> My Pig file looks like something
>
> inputRaw = load 'rawDetails' using PigStorage(',') as (
> score,
> average,
> date
> );
>
> filterRaw = filter inputRaw by(
>
> --average matches '^[0-9]{1,5}(.)[0-9]{1,2}$' AND
> --date matches '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z' AND
> --score matches '^[0-9]{1,4}$'
>
> average matches matches $decimalRegex AND
> date matches matches $dateRegex AND
> score matches $intRegex
>
> );
>
> dump filterRaw;
>
> Now i want to remove this hard coded matching to external property file.
>
>
> Property file looks like
>
> intRegex = '^[0-9]{1,9}$'
> decimalRegex = '^[0-9]{1,12}(.)[0-9]{1,5}$'
> dateRegex =  '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z'
>
> Can you suggest how to add regex parameters to property file.
>
> Thanks in advance
>
>
>
>
>
>
>
>
>
>  ----- Original Message -----
>
> From: Bill Graham
>
> Sent: 04/04/12 11:54 PM
>
> To: [EMAIL PROTECTED]
>
> Subject: Re: Regex variable using param file
>
>
> This error is happening when trying to substitute this value into your pig
> script. What does mypig.pig look like?
>
> On Wed, Apr 4, 2012 at 3:23 AM, Shin Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi All
> >
> >  I am trying to use param file to import certain regex variables into Pig
> >
> >  I have file written something like
> >
> >  isoDate = '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z'
> >
> >  When i try to use it with
> >
> >  pig -x local -param_file myparam mypig.pig
> >
> >  I get error as
> >
> >  2012-04-03 05:12:35,319 [main] ERROR org.apache.pig.Main - ERROR 2999:
> > Unexpected internal error. String index out of range: 26
> >
> >  The error log is
> >
> >  ERROR 2999: Unexpected internal error. String index out of range: 26
> >
> >  java.lang.StringIndexOutOfBoundsException: String index out of range: 26
> >  at java.lang.String.charAt(String.java:686)
> >  at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
> >  at java.util.regex.Matcher.replaceFirst(Matcher.java:861)
> >  at java.lang.String.replaceFirst(String.java:2146)
> >  at
> > org.apache.pig.tools.parameters.PreprocessorContext.substitute(PreprocessorContext.java:236)
> >  at
> > org.apache.pig.tools.parameters.PigFileParser.input(PigFileParser.java:65)
> >  at
> > org.apache.pig.tools.parameters.PigFileParser.Parse(PigFileParser.java:43)
> >  at
> > org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.parsePigFile(ParameterSubstitutionPreprocessor.java:105)
> >  at
> > org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:98)
> >  at org.apache.pig.Main.runParamPreprocessor(Main.java:718)
> >  at org.apache.pig.Main.run(Main.java:525)
> >  at org.apache.pig.Main.main(Main.java:111)
> >  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >  at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >  at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >  at java.lang.reflect.Method.invoke(Method.java:597)
> >  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
> >  ===========================================================> ===================> >
> >  The error i feel points at incorrect parsing of regex , but i was not
> > able to figure it out.
> >
> >  Can you guys help.
> >
> >  Thanks in advance
> >
> >  :)
> >
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at [EMAIL PROTECTED] going forward.*
>
>
>
>
> Thanks and Regards ,
>

--
*Note that I'm no longer using my Yahoo! email address. Please email me at
[EMAIL PROTECTED] going forward.*