Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Compiling programs with Hadoop 0.21.0


Copy link to this message
-
RE: Compiling programs with Hadoop 0.21.0
Dear Arko,

 

The class org.apache.hadoop.fs.Path is available in the
hadoop-common-0.21.0.jar.

>From your code I can see you are trying to create a file in HDFS and write
into it.

 

The code snippet needs the hdfs and common jar for compiling.

 

Regards,
Subroto Sanyal

  _____  

From: Arko Provo Mukherjee [mailto:[EMAIL PROTECTED]]
Sent: Friday, September 02, 2011 10:18 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling programs with Hadoop 0.21.0

 

Hello,

 

Thanks for your reply.

Following your suggestion, I tried to compile using the following command:
$ javac -classpath
/Users/arko/Documents/hadoop-0.21.0/hadoop-hdfs-0.21.0.jar  -d class/
FileTest.java

I got the following errors:
FileTest.java:9: package org.apache.hadoop does not exist
import org.apache.hadoop.*;
^
FileTest.java:16: cannot find symbol
symbol  : class Path
location: class FileTest
               Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
               ^
FileTest.java:16: cannot find symbol
symbol  : class Path
location: class FileTest
               Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
                           ^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
               FileSystem fs = FileSystem.get(new Configuration());
               ^
FileTest.java:17: cannot find symbol
symbol  : class Configuration
location: class FileTest
               FileSystem fs = FileSystem.get(new Configuration());
                                                  ^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
               FileSystem fs = FileSystem.get(new Configuration());
                               ^
6 errors

Many thanks in advance!
Warm regards

Arko

On Thu, Sep 1, 2011 at 10:13 PM, Rottinghuis, Joep <[EMAIL PROTECTED]>
wrote:
> Hadoop-hdfs will contain the HDFS (Hadoop FileSystem) classes.
>
> In your code you point to hdfs:// in your path, therefore you use HDFS,
> hence they are needed.
>
>  
>
> Add the hdfs jar to your classpath and see if that improves things.
>
>  
>
> Iif you organize imports on you class in an IDE such as Eclipse you'll see
> that when you restrict the org.apache.hadoop.* import only to packages you
> need, that indeed you are using hdfs classes.
>
>  
>
> Thanks,
>
>  
>
> Joep
>
>  
>
> From: Arko Provo Mukherjee [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, August 31, 2011 5:40 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Compiling programs with Hadoop 0.21.0
>
>  
>
> Hi,
>
>  
>
> Thanks for the kind reply.
>
>  
>
> If only the common and mapred are used, then what is
hadoop-hdfs-0.21.0.jar
> used for?
>
>  
>
> My code looks like this:
>
>  
>
> import java.util.*;
>
> import java.lang.*;
>
> import java.io.*;
>
> import org.apache.hadoop.*;
>
>  
>
> class MapReduce  {
>
>    
>
> public static void main(String[] args) throws Exception  {
>
>         try  {
>
>                 Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>
>                 FileSystem fs = FileSystem.get(new Configuration());
>
>                 BufferedWriter br=new BufferedWriter(new
> OutputStreamWriter(fs.create(pt,true)));
>
>                 String line;
>
>                 line="Testing";
>
>                 System.out.println(line);
>
>                 br.write(line);
>
>                 br.close();
>
>         }catch(Exception e){
>
>                 System.out.println("File not found");
>
>         }
>
>     }
>
> }
>
>  
>
> When I compile the code, I get the following erros:
>
> $javac -classpath
> /Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
> FileSystemCat.java clear
>
>  
>
> FileTest.java:16: cannot find symbol
>
> symbol  : class Path
>
> location: class FileTest
>
>                 Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
cannot
0.21.0
the
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB