Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Regex variable using param file


Copy link to this message
-
Re: Regex variable using param file
Try removing those 3 comments and running again. The token substitution
looks for $ followed by [key]=[value] and my guess is that there's a bug
where $ in comments isn't being ignored. Please report back if that works
and if so, I'll file a JIRA.

On Thu, Apr 5, 2012 at 8:34 AM, Shin Chan <[EMAIL PROTECTED]> wrote:

>  Hi Bill
>
> My Pig file looks like something
>
> inputRaw = load 'rawDetails' using PigStorage(',') as (
> score,
> average,
> date
> );
>
> filterRaw = filter inputRaw by(
>
> --average matches '^[0-9]{1,5}(.)[0-9]{1,2}$' AND
> --date matches '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z' AND
> --score matches '^[0-9]{1,4}$'
>
> average matches matches $decimalRegex AND
> date matches matches $dateRegex AND
> score matches $intRegex
>
> );
>
> dump filterRaw;
>
> Now i want to remove this hard coded matching to external property file.
>
>
> Property file looks like
>
> intRegex = '^[0-9]{1,9}$'
> decimalRegex = '^[0-9]{1,12}(.)[0-9]{1,5}$'
> dateRegex =  '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z'
>
> Can you suggest how to add regex parameters to property file.
>
> Thanks in advance
>
>
>
>
>
>
>
>
>
>  ----- Original Message -----
>
> From: Bill Graham
>
> Sent: 04/04/12 11:54 PM
>
> To: [EMAIL PROTECTED]
>
> Subject: Re: Regex variable using param file
>
>
> This error is happening when trying to substitute this value into your pig
> script. What does mypig.pig look like?
>
> On Wed, Apr 4, 2012 at 3:23 AM, Shin Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi All
> >
> >  I am trying to use param file to import certain regex variables into Pig
> >
> >  I have file written something like
> >
> >  isoDate = '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z'
> >
> >  When i try to use it with
> >
> >  pig -x local -param_file myparam mypig.pig
> >
> >  I get error as
> >
> >  2012-04-03 05:12:35,319 [main] ERROR org.apache.pig.Main - ERROR 2999:
> > Unexpected internal error. String index out of range: 26
> >
> >  The error log is
> >
> >  ERROR 2999: Unexpected internal error. String index out of range: 26
> >
> >  java.lang.StringIndexOutOfBoundsException: String index out of range: 26
> >  at java.lang.String.charAt(String.java:686)
> >  at java.util.regex.Matcher.appendReplacement(Matcher.java:711)
> >  at java.util.regex.Matcher.replaceFirst(Matcher.java:861)
> >  at java.lang.String.replaceFirst(String.java:2146)
> >  at
> > org.apache.pig.tools.parameters.PreprocessorContext.substitute(PreprocessorContext.java:236)
> >  at
> > org.apache.pig.tools.parameters.PigFileParser.input(PigFileParser.java:65)
> >  at
> > org.apache.pig.tools.parameters.PigFileParser.Parse(PigFileParser.java:43)
> >  at
> > org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.parsePigFile(ParameterSubstitutionPreprocessor.java:105)
> >  at
> > org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor.genSubstitutedFile(ParameterSubstitutionPreprocessor.java:98)
> >  at org.apache.pig.Main.runParamPreprocessor(Main.java:718)
> >  at org.apache.pig.Main.run(Main.java:525)
> >  at org.apache.pig.Main.main(Main.java:111)
> >  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >  at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >  at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >  at java.lang.reflect.Method.invoke(Method.java:597)
> >  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
> >  ===========================================================> ===================> >
> >  The error i feel points at incorrect parsing of regex , but i was not
> > able to figure it out.
> >
> >  Can you guys help.
> >
> >  Thanks in advance
> >
> >  :)
> >
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at [EMAIL PROTECTED] going forward.*
>
>
>
>
> Thanks and Regards ,
>

--
*Note that I'm no longer using my Yahoo! email address. Please email me at
[EMAIL PROTECTED] going forward.*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB