Hello,

New to Kafka

And have an existing java project where I want to use kafka and I use maven
as my build tool.

So is there anyway , I can build kafka as maven project , or build a jar
out of kafka source code ..
Thanks
Ajit

The latest release is 0.7.1 http://incubator.apache.org/kafka/downloads.html

once un packaged

./sbt update
./sbt package

to build the kafka jar from source

On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]> wrote:

> Hello,
>
> New to Kafka
>
> And have an existing java project where I want to use kafka and I use maven
> as my build tool.
>
> So is there anyway , I can build kafka as maven project , or build a jar
> out of kafka source code ..
>
>
> Thanks
> Ajit
>

--

/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/

I included these in my pom.xml (the first two I put in our own maven repository for now out of the sbt build - I wasn't sure what version of zkclient is included with kafka - note the former is just the built kafka jar) - I have scala.version set to 2.9.2 which was what I got 0.7 to build with.

        <dependency>
            <groupId>com.vast.temporary</groupId>
            <artifactId>kafka</artifactId>
            <version>0.7.0-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>com.vast.temporary</groupId>
            <artifactId>zkclient</artifactId>
            <version>20110412-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>${scala.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.3.3</version>
            <exclusions>
                <exclusion>
                    <groupId>com.sun.jmx</groupId>
                    <artifactId>jmxri</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.sun.jdmk</groupId>
                    <artifactId>jmxtools</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>javax.jms</groupId>
                    <artifactId>jms</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>log4j-over-slf4j</artifactId>
            <scope>runtime</scope>
        </dependency>

On Jul 17, 2012, at 1:38 PM, Joe Stein wrote:

> The latest release is 0.7.1 http://incubator.apache.org/kafka/downloads.html
>
> once un packaged
>
> ./sbt update
> ./sbt package
>
> to build the kafka jar from source
>
> On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> New to Kafka
>>
>> And have an existing java project where I want to use kafka and I use maven
>> as my build tool.
>>
>> So is there anyway , I can build kafka as maven project , or build a jar
>> out of kafka source code ..
>>
>>
>> Thanks
>> Ajit
>>
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */

yup, you need to add "com.github.sgroschupf" % "zkclient" % "0.1"

i would also recommend using Kafka 0.7.1 and Zookeeper 3.3.4

On Tue, Jul 17, 2012 at 2:45 PM, graham sanderson <[EMAIL PROTECTED]> wrote:

> I included these in my pom.xml (the first two I put in our own maven
> repository for now out of the sbt build - I wasn't sure what version of
> zkclient is included with kafka - note the former is just the built kafka
> jar) - I have scala.version set to 2.9.2 which was what I got 0.7 to build
> with.
>
>         <dependency>
>             <groupId>com.vast.temporary</groupId>
>             <artifactId>kafka</artifactId>
>             <version>0.7.0-SNAPSHOT</version>
>         </dependency>
>
>         <dependency>
>             <groupId>com.vast.temporary</groupId>
>             <artifactId>zkclient</artifactId>
>             <version>20110412-SNAPSHOT</version>
>         </dependency>
>
>         <dependency>
>             <groupId>org.scala-lang</groupId>
>             <artifactId>scala-library</artifactId>
>             <version>${scala.version}</version>
>         </dependency>
>
>         <dependency>
>             <groupId>org.apache.zookeeper</groupId>
>             <artifactId>zookeeper</artifactId>
>             <version>3.3.3</version>
>             <exclusions>
>                 <exclusion>
>                     <groupId>com.sun.jmx</groupId>
>                     <artifactId>jmxri</artifactId>
>                 </exclusion>
>                 <exclusion>
>                     <groupId>com.sun.jdmk</groupId>
>                     <artifactId>jmxtools</artifactId>
>                 </exclusion>
>                 <exclusion>
>                     <groupId>javax.jms</groupId>
>                     <artifactId>jms</artifactId>
>                 </exclusion>
>             </exclusions>
>         </dependency>
>
>         <dependency>
>             <groupId>org.slf4j</groupId>
>             <artifactId>log4j-over-slf4j</artifactId>
>             <scope>runtime</scope>
>         </dependency>
>
> On Jul 17, 2012, at 1:38 PM, Joe Stein wrote:
>
> > The latest release is 0.7.1
> http://incubator.apache.org/kafka/downloads.html
> >
> > once un packaged
> >
> > ./sbt update
> > ./sbt package
> >
> > to build the kafka jar from source
> >
> > On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]> wrote:
> >
> >> Hello,
> >>
> >> New to Kafka
> >>
> >> And have an existing java project where I want to use kafka and I use
> maven
> >> as my build tool.
> >>
> >> So is there anyway , I can build kafka as maven project , or build a jar
> >> out of kafka source code ..
> >>
> >>
> >> Thanks
> >> Ajit
> >>
> >
> >
> >
> > --
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
>
>
--

/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/

Thanks Joe, I wasn't sure what maven version the zkclient JAR in kafka referred to (which is why I installed the JAR from kafka)… if it is 0.1 I will update my pom, and yes, I now need to move to 0.7.1 and 3.3.4

On Jul 17, 2012, at 2:28 PM, Joe Stein wrote:

> yup, you need to add "com.github.sgroschupf" % "zkclient" % "0.1"
>
> i would also recommend using Kafka 0.7.1 and Zookeeper 3.3.4
>
> On Tue, Jul 17, 2012 at 2:45 PM, graham sanderson <[EMAIL PROTECTED]> wrote:
>
>> I included these in my pom.xml (the first two I put in our own maven
>> repository for now out of the sbt build - I wasn't sure what version of
>> zkclient is included with kafka - note the former is just the built kafka
>> jar) - I have scala.version set to 2.9.2 which was what I got 0.7 to build
>> with.
>>
>>        <dependency>
>>            <groupId>com.vast.temporary</groupId>
>>            <artifactId>kafka</artifactId>
>>            <version>0.7.0-SNAPSHOT</version>
>>        </dependency>
>>
>>        <dependency>
>>            <groupId>com.vast.temporary</groupId>
>>            <artifactId>zkclient</artifactId>
>>            <version>20110412-SNAPSHOT</version>
>>        </dependency>
>>
>>        <dependency>
>>            <groupId>org.scala-lang</groupId>
>>            <artifactId>scala-library</artifactId>
>>            <version>${scala.version}</version>
>>        </dependency>
>>
>>        <dependency>
>>            <groupId>org.apache.zookeeper</groupId>
>>            <artifactId>zookeeper</artifactId>
>>            <version>3.3.3</version>
>>            <exclusions>
>>                <exclusion>
>>                    <groupId>com.sun.jmx</groupId>
>>                    <artifactId>jmxri</artifactId>
>>                </exclusion>
>>                <exclusion>
>>                    <groupId>com.sun.jdmk</groupId>
>>                    <artifactId>jmxtools</artifactId>
>>                </exclusion>
>>                <exclusion>
>>                    <groupId>javax.jms</groupId>
>>                    <artifactId>jms</artifactId>
>>                </exclusion>
>>            </exclusions>
>>        </dependency>
>>
>>        <dependency>
>>            <groupId>org.slf4j</groupId>
>>            <artifactId>log4j-over-slf4j</artifactId>
>>            <scope>runtime</scope>
>>        </dependency>
>>
>> On Jul 17, 2012, at 1:38 PM, Joe Stein wrote:
>>
>>> The latest release is 0.7.1
>> http://incubator.apache.org/kafka/downloads.html
>>>
>>> once un packaged
>>>
>>> ./sbt update
>>> ./sbt package
>>>
>>> to build the kafka jar from source
>>>
>>> On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hello,
>>>>
>>>> New to Kafka
>>>>
>>>> And have an existing java project where I want to use kafka and I use
>> maven
>>>> as my build tool.
>>>>
>>>> So is there anyway , I can build kafka as maven project , or build a jar
>>>> out of kafka source code ..
>>>>
>>>>
>>>> Thanks
>>>> Ajit
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> /*
>>> Joe Stein
>>> http://www.linkedin.com/in/charmalloc
>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> */
>>
>>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */

We are using an unreleased version of zkclient built from their trunk as of
April 2010. They have not uploaded a later version of the jar on Maven
since then (I think), al though Jun might know more about this.

Thanks,
Neha

On Tue, Jul 17, 2012 at 1:24 PM, graham sanderson <[EMAIL PROTECTED]> wrote:

> Thanks Joe, I wasn't sure what maven version the zkclient JAR in kafka
> referred to (which is why I installed the JAR from kafka)… if it is 0.1 I
> will update my pom, and yes, I now need to move to 0.7.1 and 3.3.4
>
> On Jul 17, 2012, at 2:28 PM, Joe Stein wrote:
>
> > yup, you need to add "com.github.sgroschupf" % "zkclient" % "0.1"
> >
> > i would also recommend using Kafka 0.7.1 and Zookeeper 3.3.4
> >
> > On Tue, Jul 17, 2012 at 2:45 PM, graham sanderson <[EMAIL PROTECTED]>
> wrote:
> >
> >> I included these in my pom.xml (the first two I put in our own maven
> >> repository for now out of the sbt build - I wasn't sure what version of
> >> zkclient is included with kafka - note the former is just the built
> kafka
> >> jar) - I have scala.version set to 2.9.2 which was what I got 0.7 to
> build
> >> with.
> >>
> >>        <dependency>
> >>            <groupId>com.vast.temporary</groupId>
> >>            <artifactId>kafka</artifactId>
> >>            <version>0.7.0-SNAPSHOT</version>
> >>        </dependency>
> >>
> >>        <dependency>
> >>            <groupId>com.vast.temporary</groupId>
> >>            <artifactId>zkclient</artifactId>
> >>            <version>20110412-SNAPSHOT</version>
> >>        </dependency>
> >>
> >>        <dependency>
> >>            <groupId>org.scala-lang</groupId>
> >>            <artifactId>scala-library</artifactId>
> >>            <version>${scala.version}</version>
> >>        </dependency>
> >>
> >>        <dependency>
> >>            <groupId>org.apache.zookeeper</groupId>
> >>            <artifactId>zookeeper</artifactId>
> >>            <version>3.3.3</version>
> >>            <exclusions>
> >>                <exclusion>
> >>                    <groupId>com.sun.jmx</groupId>
> >>                    <artifactId>jmxri</artifactId>
> >>                </exclusion>
> >>                <exclusion>
> >>                    <groupId>com.sun.jdmk</groupId>
> >>                    <artifactId>jmxtools</artifactId>
> >>                </exclusion>
> >>                <exclusion>
> >>                    <groupId>javax.jms</groupId>
> >>                    <artifactId>jms</artifactId>
> >>                </exclusion>
> >>            </exclusions>
> >>        </dependency>
> >>
> >>        <dependency>
> >>            <groupId>org.slf4j</groupId>
> >>            <artifactId>log4j-over-slf4j</artifactId>
> >>            <scope>runtime</scope>
> >>        </dependency>
> >>
> >> On Jul 17, 2012, at 1:38 PM, Joe Stein wrote:
> >>
> >>> The latest release is 0.7.1
> >> http://incubator.apache.org/kafka/downloads.html
> >>>
> >>> once un packaged
> >>>
> >>> ./sbt update
> >>> ./sbt package
> >>>
> >>> to build the kafka jar from source
> >>>
> >>> On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> New to Kafka
> >>>>
> >>>> And have an existing java project where I want to use kafka and I use
> >> maven
> >>>> as my build tool.
> >>>>
> >>>> So is there anyway , I can build kafka as maven project , or build a
> jar
> >>>> out of kafka source code ..
> >>>>
> >>>>
> >>>> Thanks
> >>>> Ajit
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>> /*
> >>> Joe Stein
> >>> http://www.linkedin.com/in/charmalloc
> >>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >>> */
> >>
> >>
> >
> >
> > --
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
>
>

Thanks guys for the help , was able to get the kafka jar

Happy kafkaing !!!

Ajit

On Wed, Jul 18, 2012 at 3:35 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> We are using an unreleased version of zkclient built from their trunk as of
> April 2010. They have not uploaded a later version of the jar on Maven
> since then (I think), al though Jun might know more about this.
>
> Thanks,
> Neha
>
> On Tue, Jul 17, 2012 at 1:24 PM, graham sanderson <[EMAIL PROTECTED]> wrote:
>
> > Thanks Joe, I wasn't sure what maven version the zkclient JAR in kafka
> > referred to (which is why I installed the JAR from kafka)… if it is 0.1 I
> > will update my pom, and yes, I now need to move to 0.7.1 and 3.3.4
> >
> > On Jul 17, 2012, at 2:28 PM, Joe Stein wrote:
> >
> > > yup, you need to add "com.github.sgroschupf" % "zkclient" % "0.1"
> > >
> > > i would also recommend using Kafka 0.7.1 and Zookeeper 3.3.4
> > >
> > > On Tue, Jul 17, 2012 at 2:45 PM, graham sanderson <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> I included these in my pom.xml (the first two I put in our own maven
> > >> repository for now out of the sbt build - I wasn't sure what version
> of
> > >> zkclient is included with kafka - note the former is just the built
> > kafka
> > >> jar) - I have scala.version set to 2.9.2 which was what I got 0.7 to
> > build
> > >> with.
> > >>
> > >>        <dependency>
> > >>            <groupId>com.vast.temporary</groupId>
> > >>            <artifactId>kafka</artifactId>
> > >>            <version>0.7.0-SNAPSHOT</version>
> > >>        </dependency>
> > >>
> > >>        <dependency>
> > >>            <groupId>com.vast.temporary</groupId>
> > >>            <artifactId>zkclient</artifactId>
> > >>            <version>20110412-SNAPSHOT</version>
> > >>        </dependency>
> > >>
> > >>        <dependency>
> > >>            <groupId>org.scala-lang</groupId>
> > >>            <artifactId>scala-library</artifactId>
> > >>            <version>${scala.version}</version>
> > >>        </dependency>
> > >>
> > >>        <dependency>
> > >>            <groupId>org.apache.zookeeper</groupId>
> > >>            <artifactId>zookeeper</artifactId>
> > >>            <version>3.3.3</version>
> > >>            <exclusions>
> > >>                <exclusion>
> > >>                    <groupId>com.sun.jmx</groupId>
> > >>                    <artifactId>jmxri</artifactId>
> > >>                </exclusion>
> > >>                <exclusion>
> > >>                    <groupId>com.sun.jdmk</groupId>
> > >>                    <artifactId>jmxtools</artifactId>
> > >>                </exclusion>
> > >>                <exclusion>
> > >>                    <groupId>javax.jms</groupId>
> > >>                    <artifactId>jms</artifactId>
> > >>                </exclusion>
> > >>            </exclusions>
> > >>        </dependency>
> > >>
> > >>        <dependency>
> > >>            <groupId>org.slf4j</groupId>
> > >>            <artifactId>log4j-over-slf4j</artifactId>
> > >>            <scope>runtime</scope>
> > >>        </dependency>
> > >>
> > >> On Jul 17, 2012, at 1:38 PM, Joe Stein wrote:
> > >>
> > >>> The latest release is 0.7.1
> > >> http://incubator.apache.org/kafka/downloads.html
> > >>>
> > >>> once un packaged
> > >>>
> > >>> ./sbt update
> > >>> ./sbt package
> > >>>
> > >>> to build the kafka jar from source
> > >>>
> > >>> On Tue, Jul 17, 2012 at 2:23 PM, Ajit Koti <[EMAIL PROTECTED]>
> wrote:
> > >>>
> > >>>> Hello,
> > >>>>
> > >>>> New to Kafka
> > >>>>
> > >>>> And have an existing java project where I want to use kafka and I
> use
> > >> maven
> > >>>> as my build tool.
> > >>>>
> > >>>> So is there anyway , I can build kafka as maven project , or build a
> > jar
> > >>>> out of kafka source code ..
> > >>>>
> > >>>>
> > >>>> Thanks
> > >>>> Ajit
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> /*
> > >>> Joe Stein
> > >>> http://www.linkedin.com/in/charmalloc
> > >>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>

Hello,

I would like to introduce Kafka into an existing application that uses
Maven.  I couldn't find a Maven repository.  I tried installing it to my
local repository as follows:

mvn install:install-file –Dfile=kafka-0.7.1.jar –DgroupId=org.apache.kafka
–DartifactId=kafka-core –Dversion=0.7.1-incubating -Dpackaging=jar
-DgeneratePom-true

But I keep getting this error message:

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-install-plugin:2.3.1:install-file
(default-cli) on project gurkha: The parameters 'file' for goal
org.apache.maven.plugins:maven-install-plugin:2.3.1:install-file are
missing or invalid -> [Help 1]
Needless to say, I am NOT  a Maven expert.  Can someone help me integrate
Kafka into an existing Maven project?

Thanks in advance.