Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Compiling programs with Hadoop 0.21.0


+
Arko Provo Mukherjee 2011-08-31, 21:34
+
Robert Evans 2011-08-31, 22:00
+
Arko Provo Mukherjee 2011-09-01, 00:40
+
Rottinghuis, Joep 2011-09-02, 03:13
+
Arko Provo Mukherjee 2011-09-02, 04:48
Copy link to this message
-
RE: Compiling programs with Hadoop 0.21.0
Subroto Sanyal 2011-09-02, 05:35
Dear Arko,

 

The class org.apache.hadoop.fs.Path is available in the
hadoop-common-0.21.0.jar.

>From your code I can see you are trying to create a file in HDFS and write
into it.

 

The code snippet needs the hdfs and common jar for compiling.

 

Regards,
Subroto Sanyal

  _____  

From: Arko Provo Mukherjee [mailto:[EMAIL PROTECTED]]
Sent: Friday, September 02, 2011 10:18 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling programs with Hadoop 0.21.0

 

Hello,

 

Thanks for your reply.

Following your suggestion, I tried to compile using the following command:
$ javac -classpath
/Users/arko/Documents/hadoop-0.21.0/hadoop-hdfs-0.21.0.jar  -d class/
FileTest.java

I got the following errors:
FileTest.java:9: package org.apache.hadoop does not exist
import org.apache.hadoop.*;
^
FileTest.java:16: cannot find symbol
symbol  : class Path
location: class FileTest
               Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
               ^
FileTest.java:16: cannot find symbol
symbol  : class Path
location: class FileTest
               Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
                           ^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
               FileSystem fs = FileSystem.get(new Configuration());
               ^
FileTest.java:17: cannot find symbol
symbol  : class Configuration
location: class FileTest
               FileSystem fs = FileSystem.get(new Configuration());
                                                  ^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
               FileSystem fs = FileSystem.get(new Configuration());
                               ^
6 errors

Many thanks in advance!
Warm regards

Arko

On Thu, Sep 1, 2011 at 10:13 PM, Rottinghuis, Joep <[EMAIL PROTECTED]>
wrote:
> Hadoop-hdfs will contain the HDFS (Hadoop FileSystem) classes.
>
> In your code you point to hdfs:// in your path, therefore you use HDFS,
> hence they are needed.
>
>  
>
> Add the hdfs jar to your classpath and see if that improves things.
>
>  
>
> Iif you organize imports on you class in an IDE such as Eclipse you'll see
> that when you restrict the org.apache.hadoop.* import only to packages you
> need, that indeed you are using hdfs classes.
>
>  
>
> Thanks,
>
>  
>
> Joep
>
>  
>
> From: Arko Provo Mukherjee [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, August 31, 2011 5:40 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Compiling programs with Hadoop 0.21.0
>
>  
>
> Hi,
>
>  
>
> Thanks for the kind reply.
>
>  
>
> If only the common and mapred are used, then what is
hadoop-hdfs-0.21.0.jar
> used for?
>
>  
>
> My code looks like this:
>
>  
>
> import java.util.*;
>
> import java.lang.*;
>
> import java.io.*;
>
> import org.apache.hadoop.*;
>
>  
>
> class MapReduce  {
>
>    
>
> public static void main(String[] args) throws Exception  {
>
>         try  {
>
>                 Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>
>                 FileSystem fs = FileSystem.get(new Configuration());
>
>                 BufferedWriter br=new BufferedWriter(new
> OutputStreamWriter(fs.create(pt,true)));
>
>                 String line;
>
>                 line="Testing";
>
>                 System.out.println(line);
>
>                 br.write(line);
>
>                 br.close();
>
>         }catch(Exception e){
>
>                 System.out.println("File not found");
>
>         }
>
>     }
>
> }
>
>  
>
> When I compile the code, I get the following erros:
>
> $javac -classpath
> /Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
> FileSystemCat.java clear
>
>  
>
> FileTest.java:16: cannot find symbol
>
> symbol  : class Path
>
> location: class FileTest
>
>                 Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
cannot
0.21.0
the