Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> how to use hadoop apis with cloudera distribution ?


Copy link to this message
-
Re: how to use hadoop apis with cloudera distribution ?
If write your code within a Maven project (which you can open from Eclipse)
then you should the following in your pom.xml:

* Define Cloudera repository:

...
        <repository>
            <id>cdh.repo</id>
            <url>
https://repository.cloudera.com/content/groups/cloudera-repos</url>
            <name>Cloudera Repositories</name>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
...

* Add Hadoop dependency:

...
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-core</artifactId>
                <version>0.20.2-cdh3B4</version>
            </dependency>
...
You can browse the Cloudera Maven repositories (for avail JARs and versions)
at:

    https://repository.cloudera.com/

You'll find Hadoop/Pig/Hive/Sqoop/Zookeeper/Hbase/Flume/Oozie JARs (with all
the *cdh3B4 of them versions working with each other)

Then your app should run with the Cloudera releases of Hadoop/...etc.

Hope this helps.

Thanks.

Alejandro

On Wed, Mar 9, 2011 at 6:03 AM, Marcos Ortiz Valmaseda <[EMAIL PROTECTED]>wrote:

> You can check the Cloudera Training Videos, where is a screencast
> explaining how to develop Hadoop using Eclipse.
> http://www.cloudera.com/presentations
> http://vimeo.com/cloudera
>
> Now, For working with Hadoop APIs using Eclipse, for developing
> applications based on Hadoop, you can use the Kamasphere Plugin for Hadoop
> Development, or if you are a NetBeans user, they have a module for that.
>
> Regards.
>
>
> ----- Mensaje original -----
> De: "Mapred Learn" <[EMAIL PROTECTED]>
> Para: "Marcos Ortiz" <[EMAIL PROTECTED]>
> CC: [EMAIL PROTECTED]
> Enviados: Martes, 8 de Marzo 2011 12:26:00 (GMT-0500) Auto-Detected
> Asunto: Re: how to use hadoop apis with cloudera distribution ?
>
>
> Thanks Marco !
> I was trying to use CDH3 with eclipse and not able to know why eclipse
> complains for the import statement for hadoop apis when cloudera already
> includes them.
>
> I did not understand how CDH3 works with eclipse, does it download hadoop
> apis when we add svn urls ?
>
>
>
> On Tue, Mar 8, 2011 at 7:22 AM, Marcos Ortiz < [EMAIL PROTECTED] > wrote:
>
>
>
> On Tue, 2011-03-08 at 07:16 -080s,
>
>
>
> 0, Mapred Learn wrote:
> >
> > > Hi,
> > > I downloaded CDH3 VM for hadoop but if I want to use something like:
> > >
> > > import org.apache.hadoop.conf.Configuration;
> > >
> > > in my java code, what else do I need to do ?
>
> Can you see all tutorial that Cloudera has on its site
> http://www.cloudera.com/presentations
> http://www.cloudera.com/info/training
> http://www.cloudera.com/developers/learn-hadoop/
>
> Can you check the CDH3 Official Documentation and the last news about
> the new release:
>
> http://docs.cloudera.com
> http://www.cloudera.com/blog/category/cdh/
>
>
>
> > > Do i need to download hadoop from apache ?
> No, CDH beta 3 has with all required tools to work with Hadoop, even
> more applications like HUE, Oozie, Zookepper, Pig, Hive, Chukwa, HBase,
> Flume, etc
>
> > >
> > > if yes, then what does cdh3 do ?
> The Cloudera' colleagues has a excelent work packaging the most used
> applications with Hadoop on a single virtual machine for testing and
> they did a better approach to use Hadoop.
>
> They has Red Hat and Ubuntu/Debian compatible packages to do more easy
> the installation, configuration and use of Hadoop on these operating
> systems.
>
> Please, read http://docs.cloudera.com
>
>
> > >
> > > if not, then where can i find hadoop code on cdh VM ?
> > >
> > >
> > > I am using above line in my java code in eclipse and eclipse is not
> able to find it.
> Do you set JAVA_HOME, and HADOOP_HOME on your system?
>
> If you have any doubt with this, you can check the excellent DZone'
> refcards about "Getting Started with Hadood" and "Deploying Hadoop"
> written by Eugene Ciurana( http://eugeneciurana.eu ), VP of Technology at
> Badoo.com
>
> Regards, and I hope that this information could be useful for you.