|
|
-
Re: Is there any way to add jar when invoking hadoop command
James Seigel 2011-03-23, 01:41
Hello, some quick advice for you....
which portion of your job needs the jar? if answer = mapper or reducer, then add it to the -libjars flag.
If it is in the job initiation..bundle it in your job jar for fun.
Cheers
James.
On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
> Another work around I can think of is that have my own copy of hadoop, and
> copy extra jars to my hadoop. But it result into more maintenance effort
>
> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>
>> Hi all,
>>
>> When I use command "hadoop fs -text" I need to add extra jar to CLASSPATH,
>> because there's custom type in my sequence file. One way is that copying jar
>> to $HADOOP_HOME/lib
>> But in my case, I am not administrator, so I do not have the permission to
>> copy files under $HADOOP_HOME/lib
>>
>> Is there any other ways to add extra jar to CLASSPATH ?
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
-
Re: Is there any way to add jar when invoking hadoop command
Jeff Zhang 2011-03-23, 02:01
Actually I don't use the jar for mapreduce job. I only need it to display
sequence file.
On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <[EMAIL PROTECTED]> wrote:
> Hello, some quick advice for you....
>
>
> which portion of your job needs the jar? if answer = mapper or reducer,
> then add it to the -libjars flag.
>
> If it is in the job initiation..bundle it in your job jar for fun.
>
> Cheers
> James.
> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
>
> > Another work around I can think of is that have my own copy of hadoop,
> and
> > copy extra jars to my hadoop. But it result into more maintenance effort
> >
> > On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> >
> >> Hi all,
> >>
> >> When I use command "hadoop fs -text" I need to add extra jar to
> CLASSPATH,
> >> because there's custom type in my sequence file. One way is that copying
> jar
> >> to $HADOOP_HOME/lib
> >> But in my case, I am not administrator, so I do not have the permission
> to
> >> copy files under $HADOOP_HOME/lib
> >>
> >> Is there any other ways to add extra jar to CLASSPATH ?
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
-
Re: Is there any way to add jar when invoking hadoop command
James Seigel 2011-03-23, 02:04
Ahhhhh lib jars should do...let me see if I can dig up an example.
hadoop fs -libjars ./adhoc/xxxxx.jar -text /xxxx/xxx/xxx/xxxx/20110101/part-r-00001.evt
cheers
James.
On 2011-03-22, at 8:01 PM, Jeff Zhang wrote:
> Actually I don't use the jar for mapreduce job. I only need it to display
> sequence file.
>
>
>
> On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <[EMAIL PROTECTED]> wrote:
>
>> Hello, some quick advice for you....
>>
>>
>> which portion of your job needs the jar? if answer = mapper or reducer,
>> then add it to the -libjars flag.
>>
>> If it is in the job initiation..bundle it in your job jar for fun.
>>
>> Cheers
>> James.
>> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
>>
>>> Another work around I can think of is that have my own copy of hadoop,
>> and
>>> copy extra jars to my hadoop. But it result into more maintenance effort
>>>
>>> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi all,
>>>>
>>>> When I use command "hadoop fs -text" I need to add extra jar to
>> CLASSPATH,
>>>> because there's custom type in my sequence file. One way is that copying
>> jar
>>>> to $HADOOP_HOME/lib
>>>> But in my case, I am not administrator, so I do not have the permission
>> to
>>>> copy files under $HADOOP_HOME/lib
>>>>
>>>> Is there any other ways to add extra jar to CLASSPATH ?
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
-
Re: Is there any way to add jar when invoking hadoop command
Jeff Zhang 2011-03-23, 02:37
Thanks, it works
On Wed, Mar 23, 2011 at 10:04 AM, James Seigel <[EMAIL PROTECTED]> wrote:
> Ahhhhh lib jars should do...let me see if I can dig up an example.
>
>
> hadoop fs -libjars ./adhoc/xxxxx.jar -text
> /xxxx/xxx/xxx/xxxx/20110101/part-r-00001.evt
>
>
> cheers
> James.
>
>
>
> On 2011-03-22, at 8:01 PM, Jeff Zhang wrote:
>
> > Actually I don't use the jar for mapreduce job. I only need it to display
> > sequence file.
> >
> >
> >
> > On Wed, Mar 23, 2011 at 9:41 AM, James Seigel <[EMAIL PROTECTED]> wrote:
> >
> >> Hello, some quick advice for you....
> >>
> >>
> >> which portion of your job needs the jar? if answer = mapper or reducer,
> >> then add it to the -libjars flag.
> >>
> >> If it is in the job initiation..bundle it in your job jar for fun.
> >>
> >> Cheers
> >> James.
> >> On 2011-03-22, at 7:35 PM, Jeff Zhang wrote:
> >>
> >>> Another work around I can think of is that have my own copy of hadoop,
> >> and
> >>> copy extra jars to my hadoop. But it result into more maintenance
> effort
> >>>
> >>> On Wed, Mar 23, 2011 at 9:19 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> When I use command "hadoop fs -text" I need to add extra jar to
> >> CLASSPATH,
> >>>> because there's custom type in my sequence file. One way is that
> copying
> >> jar
> >>>> to $HADOOP_HOME/lib
> >>>> But in my case, I am not administrator, so I do not have the
> permission
> >> to
> >>>> copy files under $HADOOP_HOME/lib
> >>>>
> >>>> Is there any other ways to add extra jar to CLASSPATH ?
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>
> >>
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
|
|
