Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 3)


Copy link to this message
-
Re: [VOTE] Release Kafka 0.7.0-incubating (candidate 5)
+1 on two separate distributions

On 11/4/11 3:35 PM, Neha Narkhede wrote:
>> Therefore a binary distribution must include the dependent libraries
>> to make it run out of the box.
> I agree with Taylor here. That means our binary distribution will not
> have the jars in boot/test.
>
>>>> If someone wants to run tests they are a developer and should get a
>> source distribution.
> This is also a good suggestion. That means we can upload a source
> distribution along with the binary distribution.
>
> I would encourage everyone to speak up here, to avoid delaying this
> release any further.
>
> Thanks,
> Neha
>
> On Fri, Nov 4, 2011 at 9:16 AM, Taylor Gautier<[EMAIL PROTECTED]>  wrote:
>> My $.02
>>
>> There are different audiences for different target distributions
>>   - binary distribution : end user (developer) or sysadmin
>>   - source distribution : a developer
>>
>> Therefore a binary distribution must include the dependent libraries
>> to make it run out of the box.
>>
>> That doesn't include tests because the audience for a binary
>> distribution doesn't run tests.
>>
>> If someone wants to run tests they are a developer and should get a
>> source distribution.
>>
>> The source distribution should NOT contain binary dependencies. In
>> this case Maven or another suitable build tool should resolve any
>> dependencies during the build stage.
>>
>>
>>
>> On Nov 4, 2011, at 8:51 AM, Neha Narkhede<[EMAIL PROTECTED]>  wrote:
>>
>>> Let me state why we included *all* the dependencies in the package
>>> distribution. Initially I thought this distribution should just work
>>> out-of-the-box after the download. That includes all unit tests, all
>>> scripts in core as well as contrib. Note that the assumption was to not
>>> have the user run ./sbt udpate to download dependencies or ./sbt package to
>>> build the sub projects.
>>>
>>> Now, assuming we have the user do both, here is the set of jars we can
>>> include -
>>>
>>> ./core/lib/zkclient-20110412.jar
>>> ./lib/apache-rat-0.8-SNAPSHOT.jar
>>> ./lib/sbt-launch.jar
>>> ./contrib/hadoop-consumer/lib/avro-1.4.0.jar
>>> ./contrib/hadoop-consumer/lib/commons-logging-1.0.4.jar
>>> ./contrib/hadoop-consumer/lib/hadoop-0.20.2-core.jar
>>> ./contrib/hadoop-consumer/lib/jackson-core-asl-1.5.5.jar
>>> ./contrib/hadoop-consumer/lib/jackson-mapper-asl-1.5.5.jar
>>> ./contrib/hadoop-consumer/lib/pig-0.8.0-core.jar
>>> ./contrib/hadoop-consumer/lib/piggybank.jar
>>> ./contrib/hadoop-producer/lib/avro-1.4.0.jar
>>> ./contrib/hadoop-producer/lib/commons-logging-1.0.4.jar
>>> ./contrib/hadoop-producer/lib/hadoop-0.20.2-core.jar
>>> ./contrib/hadoop-producer/lib/jackson-core-asl-1.5.5.jar
>>> ./contrib/hadoop-producer/lib/jackson-mapper-asl-1.5.5.jar
>>> ./contrib/hadoop-producer/lib/pig-0.8.0-core.jar
>>> ./contrib/hadoop-producer/lib/piggybank.jar
>>> .*
>>> /contrib/hadoop-consumer/target/scala_2.8.0/hadoop-consumer_2.8.0-0.7.0.jar
>>> ./contrib/hadoop-producer/target/scala_2.8.0/hadoop-producer_2.8.0-0.7.0.jar
>>> ./examples/target/scala_2.8.0/kafka-java-examples-0.7.0.jar
>>> ./core/target/scala_2.8.0/kafka-0.7.0.jar*
>>>
>>> The jars in bold are Kafka jars. The question is how will the user be able
>>> to run our jars, with just the stripped set of dependent jars we package ?
>>>
>>>>> Many of the issues about distribution would automatically be solved if
>>> Maven were used.
>>>
>>> We use maven. All the jars in "lib_managed" are downloaded from Maven. The
>>> question is not whether or not to use Maven. The question is whether you
>>> have the user download dependencies build the jars themselves or not.
>>>
>>> Once that is clear, we can reduce the set of dependent jars we include.
>>>
>>> I would encourage everyone to give your inputs now, since this is important
>>> to iron out for further releases.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Fri, Nov 4, 2011 at 8:26 AM, Alan D. Cabrera<[EMAIL PROTECTED]>
>>> wrote:
>>>> It would...
>>>>
>>>> Many of the issues about distribution would automatically be solved if
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB