Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 3)


Copy link to this message
-
Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
+1 on two separate distributions

On 11/4/11 3:35 PM, Neha Narkhede wrote:
>> Therefore a binary distribution must include the dependent libraries
>> to make it run out of the box.
> I agree with Taylor here. That means our binary distribution will not
> have the jars in boot/test.
>
>>>> If someone wants to run tests they are a developer and should get a
>> source distribution.
> This is also a good suggestion. That means we can upload a source
> distribution along with the binary distribution.
>
> I would encourage everyone to speak up here, to avoid delaying this
> release any further.
>
> Thanks,
> Neha
>
> On Fri, Nov 4, 2011 at 9:16 AM, Taylor Gautier<[EMAIL PROTECTED]>  wrote:
>> My $.02
>>
>> There are different audiences for different target distributions
>>   - binary distribution : end user (developer) or sysadmin
>>   - source distribution : a developer
>>
>> Therefore a binary distribution must include the dependent libraries
>> to make it run out of the box.
>>
>> That doesn't include tests because the audience for a binary
>> distribution doesn't run tests.
>>
>> If someone wants to run tests they are a developer and should get a
>> source distribution.
>>
>> The source distribution should NOT contain binary dependencies. In
>> this case Maven or another suitable build tool should resolve any
>> dependencies during the build stage.
>>
>>
>>
>> On Nov 4, 2011, at 8:51 AM, Neha Narkhede<[EMAIL PROTECTED]>  wrote:
>>
>>> Let me state why we included *all* the dependencies in the package
>>> distribution. Initially I thought this distribution should just work
>>> out-of-the-box after the download. That includes all unit tests, all
>>> scripts in core as well as contrib. Note that the assumption was to not
>>> have the user run ./sbt udpate to download dependencies or ./sbt package to
>>> build the sub projects.
>>>
>>> Now, assuming we have the user do both, here is the set of jars we can
>>> include -
>>>
>>> ./core/lib/zkclient-20110412.jar
>>> ./lib/apache-rat-0.8-SNAPSHOT.jar
>>> ./lib/sbt-launch.jar
>>> ./contrib/hadoop-consumer/lib/avro-1.4.0.jar
>>> ./contrib/hadoop-consumer/lib/commons-logging-1.0.4.jar
>>> ./contrib/hadoop-consumer/lib/hadoop-0.20.2-core.jar
>>> ./contrib/hadoop-consumer/lib/jackson-core-asl-1.5.5.jar
>>> ./contrib/hadoop-consumer/lib/jackson-mapper-asl-1.5.5.jar
>>> ./contrib/hadoop-consumer/lib/pig-0.8.0-core.jar
>>> ./contrib/hadoop-consumer/lib/piggybank.jar
>>> ./contrib/hadoop-producer/lib/avro-1.4.0.jar
>>> ./contrib/hadoop-producer/lib/commons-logging-1.0.4.jar
>>> ./contrib/hadoop-producer/lib/hadoop-0.20.2-core.jar
>>> ./contrib/hadoop-producer/lib/jackson-core-asl-1.5.5.jar
>>> ./contrib/hadoop-producer/lib/jackson-mapper-asl-1.5.5.jar
>>> ./contrib/hadoop-producer/lib/pig-0.8.0-core.jar
>>> ./contrib/hadoop-producer/lib/piggybank.jar
>>> .*
>>> /contrib/hadoop-consumer/target/scala_2.8.0/hadoop-consumer_2.8.0-0.7.0.jar
>>> ./contrib/hadoop-producer/target/scala_2.8.0/hadoop-producer_2.8.0-0.7.0.jar
>>> ./examples/target/scala_2.8.0/kafka-java-examples-0.7.0.jar
>>> ./core/target/scala_2.8.0/kafka-0.7.0.jar*
>>>
>>> The jars in bold are Kafka jars. The question is how will the user be able
>>> to run our jars, with just the stripped set of dependent jars we package ?
>>>
>>>>> Many of the issues about distribution would automatically be solved if
>>> Maven were used.
>>>
>>> We use maven. All the jars in "lib_managed" are downloaded from Maven. The
>>> question is not whether or not to use Maven. The question is whether you
>>> have the user download dependencies build the jars themselves or not.
>>>
>>> Once that is clear, we can reduce the set of dependent jars we include.
>>>
>>> I would encourage everyone to give your inputs now, since this is important
>>> to iron out for further releases.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Fri, Nov 4, 2011 at 8:26 AM, Alan D. Cabrera<[EMAIL PROTECTED]>
>>> wrote:
>>>> It would...
>>>>
>>>> Many of the issues about distribution would automatically be solved if