What is the rationale for having the hadoop JARs outside of the lib/
directory?

It would definitely simplify packaging configuration if they are under lib/
as well.

Any objection to it?

Thanks.

Alejandro

On Aug 4, 2011, at 11:03 AM, Alejandro Abdelnur wrote:

> What is the rationale for having the hadoop JARs outside of the lib/
> directory?
>
> It would definitely simplify packaging configuration if they are under lib/
> as well.
>
> Any objection to it?
It needs a big release note as this will break users in bad bad ways.

The jar files placement  outside of lib directory is to ensure the project generated jar files are not mixed with it's dependencies.
Hence, if another project tries to import current project's jar files without dependencies, it is possible to do so.

regards,
Eric

On Aug 4, 2011, at 11:03 AM, Alejandro Abdelnur wrote:

> What is the rationale for having the hadoop JARs outside of the lib/
> directory?
>
> It would definitely simplify packaging configuration if they are under lib/
> as well.
>
> Any objection to it?
>
> Thanks.
>
> Alejandro

[moving to core-dev@, general@ BCCed]

Eric,

Even if the JAR is in lib/ you could import/use that JAR only.

How would you use Hadoop JARs without its dependencies? Many things will
break unless you add the dependency JARs.

Granted, there are JARs that are used by Hadoop server side only
(JT/NN/TT/DN/SNN), but that is a different thing. Having a client side set
of JARs would help handle this (MAPREDUCE-1638).

Thoughts?

Thanks.

Alejandro

On Thu, Aug 4, 2011 at 1:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> The jar files placement  outside of lib directory is to ensure the project
> generated jar files are not mixed with it's dependencies.
> Hence, if another project tries to import current project's jar files
> without dependencies, it is possible to do so.
>
> regards,
> Eric
>
> On Aug 4, 2011, at 11:03 AM, Alejandro Abdelnur wrote:
>
> > What is the rationale for having the hadoop JARs outside of the lib/
> > directory?
> >
> > It would definitely simplify packaging configuration if they are under
> lib/
> > as well.
> >
> > Any objection to it?
> >
> > Thanks.
> >
> > Alejandro
>
>

It is easier to write shell script to import jar files by directory instead of explicitly reference to a few jars with specific versions.

The common use case is:

HBase needs to use hadoop jar files, but HBase depends on more recent version of log4j.  The construction of the class path would be:

$HBASE_PREFIX/share/hbase/hbase*.jar:$HBASE_PREFIX/share/hbase/lib/*.jar:$HADOOP_PREFIX/share/hadoop/*.jar

This provides a way to segment the library loading with least amount of scripting and loosely coupled.

regards,
Eric

On Aug 4, 2011, at 2:40 PM, Alejandro Abdelnur wrote:

> [moving to core-dev@, general@ BCCed]
>
> Eric,
>
> Even if the JAR is in lib/ you could import/use that JAR only.
>
> How would you use Hadoop JARs without its dependencies? Many things will
> break unless you add the dependency JARs.
>
> Granted, there are JARs that are used by Hadoop server side only
> (JT/NN/TT/DN/SNN), but that is a different thing. Having a client side set
> of JARs would help handle this (MAPREDUCE-1638).
>
> Thoughts?
>
> Thanks.
>
> Alejandro
>
> On Thu, Aug 4, 2011 at 1:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>
>> The jar files placement  outside of lib directory is to ensure the project
>> generated jar files are not mixed with it's dependencies.
>> Hence, if another project tries to import current project's jar files
>> without dependencies, it is possible to do so.
>>
>> regards,
>> Eric
>>
>> On Aug 4, 2011, at 11:03 AM, Alejandro Abdelnur wrote:
>>
>>> What is the rationale for having the hadoop JARs outside of the lib/
>>> directory?
>>>
>>> It would definitely simplify packaging configuration if they are under
>> lib/
>>> as well.
>>>
>>> Any objection to it?
>>>
>>> Thanks.
>>>
>>> Alejandro
>>
>>