Pig, mail # user - Re: Registering *.jar

Registering *.jar

Mohit Anchlia 2012-05-16, 23:51

I have something like this in pig script. But pig doesn't recognize *.jar
regular expression. It seems to be looking for a file name "*.jar" Is there
a way to register using regular expression?

SET mapred.map.tasks.speculative.execution false;

%default ES_JAR_DIR '/root/elasticsearch/lib'

%default INDEX 'testIdx'

%default OBJ 'testType'

register /home/hadoop/pigtest/*.jar;

Re: Registering *.jar

Prashant Kommireddi 2012-05-16, 23:58

What version are you using?

Sent from my iPhone

On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:

> I have something like this in pig script. But pig doesn't recognize *.jar
> regular expression. It seems to be looking for a file name "*.jar" Is there
> a way to register using regular expression?
>
> SET mapred.map.tasks.speculative.execution false;
>
> %default ES_JAR_DIR '/root/elasticsearch/lib'
>
> %default INDEX 'testIdx'
>
> %default OBJ 'testType'
>
> register /home/hadoop/pigtest/*.jar;

Re: Registering *.jar

Mohit Anchlia 2012-05-17, 00:04

.8.1

On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> What version are you using?
>
> Sent from my iPhone
>
> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>
> > I have something like this in pig script. But pig doesn't recognize *.jar
> > regular expression. It seems to be looking for a file name "*.jar" Is
> there
> > a way to register using regular expression?
> >
> > SET mapred.map.tasks.speculative.execution false;
> >
> > %default ES_JAR_DIR '/root/elasticsearch/lib'
> >
> > %default INDEX 'testIdx'
> >
> > %default OBJ 'testType'
> >
> > register /home/hadoop/pigtest/*.jar;
>

Re: Registering *.jar

Ashutosh Chauhan 2012-05-17, 00:04

This I think is a useful feature to have. Another possibility is to let
user specify dir name and then add all the jars in that directory.

register /home/hadoop/pigtest/

Ashutosh

On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> What version are you using?
>
> Sent from my iPhone
>
> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>
> > I have something like this in pig script. But pig doesn't recognize *.jar
> > regular expression. It seems to be looking for a file name "*.jar" Is
> there
> > a way to register using regular expression?
> >
> > SET mapred.map.tasks.speculative.execution false;
> >
> > %default ES_JAR_DIR '/root/elasticsearch/lib'
> >
> > %default INDEX 'testIdx'
> >
> > %default OBJ 'testType'
> >
> > register /home/hadoop/pigtest/*.jar;
>

Re: Registering *.jar

Prashant Kommireddi 2012-05-17, 00:07

I think 0.10 already supports globbing.

Sent from my iPhone

On May 16, 2012, at 5:05 PM, Ashutosh Chauhan <[EMAIL PROTECTED]> wrote:

> This I think is a useful feature to have. Another possibility is to let
> user specify dir name and then add all the jars in that directory.
>
> register /home/hadoop/pigtest/
>
> Ashutosh
>
> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>
>> What version are you using?
>>
>> Sent from my iPhone
>>
>> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>
>>> I have something like this in pig script. But pig doesn't recognize *.jar
>>> regular expression. It seems to be looking for a file name "*.jar" Is
>> there
>>> a way to register using regular expression?
>>>
>>> SET mapred.map.tasks.speculative.execution false;
>>>
>>> %default ES_JAR_DIR '/root/elasticsearch/lib'
>>>
>>> %default INDEX 'testIdx'
>>>
>>> %default OBJ 'testType'
>>>
>>> register /home/hadoop/pigtest/*.jar;
>>

Re: Registering *.jar

Prashant Kommireddi 2012-05-17, 00:16

That's a quite old version. You might want to upgrade to 0.10 which
has a bunch of new features, improvements and bug fixes.

Sent from my iPhone

On May 16, 2012, at 5:05 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:

> .8.1
>
> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>
>> What version are you using?
>>
>> Sent from my iPhone
>>
>> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>
>>> I have something like this in pig script. But pig doesn't recognize *.jar
>>> regular expression. It seems to be looking for a file name "*.jar" Is
>> there
>>> a way to register using regular expression?
>>>
>>> SET mapred.map.tasks.speculative.execution false;
>>>
>>> %default ES_JAR_DIR '/root/elasticsearch/lib'
>>>
>>> %default INDEX 'testIdx'
>>>
>>> %default OBJ 'testType'
>>>
>>> register /home/hadoop/pigtest/*.jar;
>>

Re: Registering *.jar

Russell Jurney 2012-05-17, 00:30

Does 0.10+ do jar globbing? I need this too!

Russell Jurney
twitter.com/rjurney
[EMAIL PROTECTED]
datasyndrome.com

On May 16, 2012, at 5:16 PM, Prashant Kommireddi <[EMAIL PROTECTED]> wrote:

> That's a quite old version. You might want to upgrade to 0.10 which
> has a bunch of new features, improvements and bug fixes.
>
> Sent from my iPhone
>
> On May 16, 2012, at 5:05 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>
>> .8.1
>>
>> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>>
>>> What version are you using?
>>>
>>> Sent from my iPhone
>>>
>>> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>>
>>>> I have something like this in pig script. But pig doesn't recognize *.jar
>>>> regular expression. It seems to be looking for a file name "*.jar" Is
>>> there
>>>> a way to register using regular expression?
>>>>
>>>> SET mapred.map.tasks.speculative.execution false;
>>>>
>>>> %default ES_JAR_DIR '/root/elasticsearch/lib'
>>>>
>>>> %default INDEX 'testIdx'
>>>>
>>>> %default OBJ 'testType'
>>>>
>>>> register /home/hadoop/pigtest/*.jar;
>>>

Re: Registering *.jar

Alan Gates 2012-05-17, 00:34

See http://hortonworks.com/blog/new-features-in-apache-pig-0-10/

Alan.

On May 16, 2012, at 5:30 PM, Russell Jurney wrote:

> Does 0.10+ do jar globbing? I need this too!
>
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com
>
> On May 16, 2012, at 5:16 PM, Prashant Kommireddi <[EMAIL PROTECTED]> wrote:
>
>> That's a quite old version. You might want to upgrade to 0.10 which
>> has a bunch of new features, improvements and bug fixes.
>>
>> Sent from my iPhone
>>
>> On May 16, 2012, at 5:05 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>
>>> .8.1
>>>
>>> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>>>
>>>> What version are you using?
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> I have something like this in pig script. But pig doesn't recognize *.jar
>>>>> regular expression. It seems to be looking for a file name "*.jar" Is
>>>> there
>>>>> a way to register using regular expression?
>>>>>
>>>>> SET mapred.map.tasks.speculative.execution false;
>>>>>
>>>>> %default ES_JAR_DIR '/root/elasticsearch/lib'
>>>>>
>>>>> %default INDEX 'testIdx'
>>>>>
>>>>> %default OBJ 'testType'
>>>>>
>>>>> register /home/hadoop/pigtest/*.jar;
>>>>

Re: Registering *.jar

Prashant Kommireddi 2012-05-17, 01:00

Daniel's blog post talks about it. Try it out.

Sent from my iPhone

On May 16, 2012, at 5:34 PM, Alan Gates <[EMAIL PROTECTED]> wrote:

> See http://hortonworks.com/blog/new-features-in-apache-pig-0-10/
>
> Alan.
>
> On May 16, 2012, at 5:30 PM, Russell Jurney wrote:
>
>> Does 0.10+ do jar globbing? I need this too!
>>
>> Russell Jurney
>> twitter.com/rjurney
>> [EMAIL PROTECTED]
>> datasyndrome.com
>>
>> On May 16, 2012, at 5:16 PM, Prashant Kommireddi <[EMAIL PROTECTED]> wrote:
>>
>>> That's a quite old version. You might want to upgrade to 0.10 which
>>> has a bunch of new features, improvements and bug fixes.
>>>
>>> Sent from my iPhone
>>>
>>> On May 16, 2012, at 5:05 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>>
>>>> .8.1
>>>>
>>>> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> What version are you using?
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> I have something like this in pig script. But pig doesn't recognize *.jar
>>>>>> regular expression. It seems to be looking for a file name "*.jar" Is
>>>>> there
>>>>>> a way to register using regular expression?
>>>>>>
>>>>>> SET mapred.map.tasks.speculative.execution false;
>>>>>>
>>>>>> %default ES_JAR_DIR '/root/elasticsearch/lib'
>>>>>>
>>>>>> %default INDEX 'testIdx'
>>>>>>
>>>>>> %default OBJ 'testType'
>>>>>>
>>>>>> register /home/hadoop/pigtest/*.jar;
>>>>>
>

Re: Registering *.jar

Raghu Angadi 2012-05-18, 07:42

if you still want to use 8.1, you can apply 0.8 patch from
https://issues.apache.org/jira/browse/PIG-2142

On Wed, May 16, 2012 at 5:04 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> .8.1
>
> On Wed, May 16, 2012 at 4:58 PM, Prashant Kommireddi <[EMAIL PROTECTED]
> >wrote:
>
> > What version are you using?
> >
> > Sent from my iPhone
> >
> > On May 16, 2012, at 4:52 PM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> >
> > > I have something like this in pig script. But pig doesn't recognize
> *.jar
> > > regular expression. It seems to be looking for a file name "*.jar" Is
> > there
> > > a way to register using regular expression?
> > >
> > > SET mapred.map.tasks.speculative.execution false;
> > >
> > > %default ES_JAR_DIR '/root/elasticsearch/lib'
> > >
> > > %default INDEX 'testIdx'
> > >
> > > %default OBJ 'testType'
> > >
> > > register /home/hadoop/pigtest/*.jar;
> >
>