Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> how to use hadoop apis with cloudera distribution ?


Copy link to this message
-
Re: how to use hadoop apis with cloudera distribution ?
If write your code within a Maven project (which you can open from Eclipse)
then you should the following in your pom.xml:

* Define Cloudera repository:

...
        <repository>
            <id>cdh.repo</id>
            <url>
https://repository.cloudera.com/content/groups/cloudera-repos</url>
            <name>Cloudera Repositories</name>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
...

* Add Hadoop dependency:

...
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-core</artifactId>
                <version>0.20.2-cdh3B4</version>
            </dependency>
...
You can browse the Cloudera Maven repositories (for avail JARs and versions)
at:

    https://repository.cloudera.com/

You'll find Hadoop/Pig/Hive/Sqoop/Zookeeper/Hbase/Flume/Oozie JARs (with all
the *cdh3B4 of them versions working with each other)

Then your app should run with the Cloudera releases of Hadoop/...etc.

Hope this helps.

Thanks.

Alejandro

On Wed, Mar 9, 2011 at 6:03 AM, Marcos Ortiz Valmaseda <[EMAIL PROTECTED]>wrote:

> You can check the Cloudera Training Videos, where is a screencast
> explaining how to develop Hadoop using Eclipse.
> http://www.cloudera.com/presentations
> http://vimeo.com/cloudera
>
> Now, For working with Hadoop APIs using Eclipse, for developing
> applications based on Hadoop, you can use the Kamasphere Plugin for Hadoop
> Development, or if you are a NetBeans user, they have a module for that.
>
> Regards.
>
>
> ----- Mensaje original -----
> De: "Mapred Learn" <[EMAIL PROTECTED]>
> Para: "Marcos Ortiz" <[EMAIL PROTECTED]>
> CC: [EMAIL PROTECTED]
> Enviados: Martes, 8 de Marzo 2011 12:26:00 (GMT-0500) Auto-Detected
> Asunto: Re: how to use hadoop apis with cloudera distribution ?
>
>
> Thanks Marco !
> I was trying to use CDH3 with eclipse and not able to know why eclipse
> complains for the import statement for hadoop apis when cloudera already
> includes them.
>
> I did not understand how CDH3 works with eclipse, does it download hadoop
> apis when we add svn urls ?
>
>
>
> On Tue, Mar 8, 2011 at 7:22 AM, Marcos Ortiz < [EMAIL PROTECTED] > wrote:
>
>
>
> On Tue, 2011-03-08 at 07:16 -080s,
>
>
>
> 0, Mapred Learn wrote:
> >
> > > Hi,
> > > I downloaded CDH3 VM for hadoop but if I want to use something like:
> > >
> > > import org.apache.hadoop.conf.Configuration;
> > >
> > > in my java code, what else do I need to do ?
>
> Can you see all tutorial that Cloudera has on its site
> http://www.cloudera.com/presentations
> http://www.cloudera.com/info/training
> http://www.cloudera.com/developers/learn-hadoop/
>
> Can you check the CDH3 Official Documentation and the last news about
> the new release:
>
> http://docs.cloudera.com
> http://www.cloudera.com/blog/category/cdh/
>
>
>
> > > Do i need to download hadoop from apache ?
> No, CDH beta 3 has with all required tools to work with Hadoop, even
> more applications like HUE, Oozie, Zookepper, Pig, Hive, Chukwa, HBase,
> Flume, etc
>
> > >
> > > if yes, then what does cdh3 do ?
> The Cloudera' colleagues has a excelent work packaging the most used
> applications with Hadoop on a single virtual machine for testing and
> they did a better approach to use Hadoop.
>
> They has Red Hat and Ubuntu/Debian compatible packages to do more easy
> the installation, configuration and use of Hadoop on these operating
> systems.
>
> Please, read http://docs.cloudera.com
>
>
> > >
> > > if not, then where can i find hadoop code on cdh VM ?
> > >
> > >
> > > I am using above line in my java code in eclipse and eclipse is not
> able to find it.
> Do you set JAVA_HOME, and HADOOP_HOME on your system?
>
> If you have any doubt with this, you can check the excellent DZone'
> refcards about "Getting Started with Hadood" and "Deploying Hadoop"
> written by Eugene Ciurana( http://eugeneciurana.eu ), VP of Technology at
> Badoo.com
>
> Regards, and I hope that this information could be useful for you.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB