|
|
Arko Provo Mukherjee 2011-08-31, 08:13
Hi,
I am using HDFS on 0.20.203 for my project.
I can see the API has evolved much over the different versions.
Can you please guide me to some link where I can find some example code for
any version >= 0.20.203 so that I can quick start the process.
Thanks much in advance!
Warm regards
Arko
Arun C Murthy 2011-09-01, 07:24
What kind of example code are you looking for? It will help if you describe your use case...
On Aug 31, 2011, at 1:13 AM, Arko Provo Mukherjee wrote:
> Hi,
>
> I am using HDFS on 0.20.203 for my project.
>
> I can see the API has evolved much over the different versions.
>
> Can you please guide me to some link where I can find some example code for any version >= 0.20.203 so that I can quick start the process.
>
> Thanks much in advance!
>
> Warm regards
> Arko
Arko Provo Mukherjee 2011-09-01, 14:52
Hi,
My code looks like this:
*import java.util.*;*
*import java.lang.*;*
*import java.io.*;*
*import org.apache.hadoop.*;*
*
*
*class MapReduce {*
* *
*public static void main(String[] args) throws Exception {*
**
* try {*
* Path pt=new Path(
"hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");*
* FileSystem fs = FileSystem.get(new Configuration());*
* BufferedWriter br=new BufferedWriter(new
OutputStreamWriter(fs.create(pt,true)));*
* String line;*
* line="Testing";*
* System.out.println(line);*
* br.write(line);*
* br.close();*
* }catch(Exception e){*
* System.out.println("File not found");*
* }*
* }*
*}*
When I compile the code, I get the following erros:
*$javac -classpath
/Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
FileSystemCat.java clear*
FileTest.java:16: cannot find symbol
symbol : class Path
location: class FileTest
Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
^
FileTest.java:16: cannot find symbol
symbol : class Path
location: class FileTest
Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
FileSystem fs = FileSystem.get(new Configuration());
^
FileTest.java:17: cannot find symbol
symbol : class Configuration
location: class FileTest
FileSystem fs = FileSystem.get(new Configuration());
^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
FileSystem fs = FileSystem.get(new Configuration());
^
5 errors
Also I would like to learn the FileContext API as I want to use 0.21 for
making appends to my file in each MR round.
Would be great if you have a place where I can have a look at some code
snippets that have demonstrated the use of FileContext for easy learning.
Many thanks in advance!
Warm regards
Arko
On Thu, Sep 1, 2011 at 2:24 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> What kind of example code are you looking for? It will help if you describe
> your use case...
>
> On Aug 31, 2011, at 1:13 AM, Arko Provo Mukherjee wrote:
>
> > Hi,
> >
> > I am using HDFS on 0.20.203 for my project.
> >
> > I can see the API has evolved much over the different versions.
> >
> > Can you please guide me to some link where I can find some example code
> for any version >= 0.20.203 so that I can quick start the process.
> >
> > Thanks much in advance!
> >
> > Warm regards
> > Arko
>
>
Arun Murthy 2011-09-01, 15:14
You are using to hadoop-0.21 jars on your classpath - try with the 0.20.203
jars?
Sent from my iPhone
On Sep 1, 2011, at 7:52 AM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:
Hi,
My code looks like this:
*import java.util.*;*
*import java.lang.*;*
*import java.io.*;*
*import org.apache.hadoop.*;*
*
*
*class MapReduce {*
* *
*public static void main(String[] args) throws Exception {*
**
* try {*
* Path pt=new Path(
"hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");*
* FileSystem fs = FileSystem.get(new Configuration());*
* BufferedWriter br=new BufferedWriter(new
OutputStreamWriter(fs.create(pt,true)));*
* String line;*
* line="Testing";*
* System.out.println(line);*
* br.write(line);*
* br.close();*
* }catch(Exception e){*
* System.out.println("File not found");*
* }*
* }*
*}*
When I compile the code, I get the following erros:
*$javac -classpath
/Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
FileSystemCat.java clear*
FileTest.java:16: cannot find symbol
symbol : class Path
location: class FileTest
Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
^
FileTest.java:16: cannot find symbol
symbol : class Path
location: class FileTest
Path pt=new
Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
FileSystem fs = FileSystem.get(new Configuration());
^
FileTest.java:17: cannot find symbol
symbol : class Configuration
location: class FileTest
FileSystem fs = FileSystem.get(new Configuration());
^
FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
accessed from outside package
FileSystem fs = FileSystem.get(new Configuration());
^
5 errors
Also I would like to learn the FileContext API as I want to use 0.21 for
making appends to my file in each MR round.
Would be great if you have a place where I can have a look at some code
snippets that have demonstrated the use of FileContext for easy learning.
Many thanks in advance!
Warm regards
Arko
On Thu, Sep 1, 2011 at 2:24 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> What kind of example code are you looking for? It will help if you describe
> your use case...
>
> On Aug 31, 2011, at 1:13 AM, Arko Provo Mukherjee wrote:
>
> > Hi,
> >
> > I am using HDFS on 0.20.203 for my project.
> >
> > I can see the API has evolved much over the different versions.
> >
> > Can you please guide me to some link where I can find some example code
> for any version >= 0.20.203 so that I can quick start the process.
> >
> > Thanks much in advance!
> >
> > Warm regards
> > Arko
>
>
Sudharsan Sampath 2011-09-02, 04:42
Hi,
Some of your imports are incorrect. Both FileSystem and Path classes are
present in o.a.h.fs
Thanks
Sudhan S
On Thu, Sep 1, 2011 at 8:22 PM, Arko Provo Mukherjee <
[EMAIL PROTECTED]> wrote:
> Hi,
>
> My code looks like this:
>
> *import java.util.*;*
>
> *import java.lang.*;*
>
> *import java.io.*;*
>
> *import org.apache.hadoop.*;*
>
> *
> *
>
> *class MapReduce {*
>
> * *
>
> *public static void main(String[] args) throws Exception {*
>
> **
>
> * try {*
>
> * Path pt=new Path(
> "hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");*
>
> * FileSystem fs = FileSystem.get(new Configuration());*
>
> * BufferedWriter br=new BufferedWriter(new
> OutputStreamWriter(fs.create(pt,true)));*
>
> * String line;*
>
> * line="Testing";*
>
> * System.out.println(line);*
>
> * br.write(line);*
>
> * br.close();*
>
> * }catch(Exception e){*
>
> * System.out.println("File not found");*
>
> * }*
>
> * }*
>
> *}*
>
> When I compile the code, I get the following erros:
> *$javac -classpath
> /Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
> FileSystemCat.java clear*
>
> FileTest.java:16: cannot find symbol
> symbol : class Path
> location: class FileTest
> Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
> ^
> FileTest.java:16: cannot find symbol
> symbol : class Path
> location: class FileTest
> Path pt=new
> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
> ^
> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
> accessed from outside package
> FileSystem fs = FileSystem.get(new Configuration());
> ^
> FileTest.java:17: cannot find symbol
> symbol : class Configuration
> location: class FileTest
> FileSystem fs = FileSystem.get(new Configuration());
> ^
> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
> accessed from outside package
> FileSystem fs = FileSystem.get(new Configuration());
> ^
> 5 errors
>
>
> Also I would like to learn the FileContext API as I want to use 0.21 for
> making appends to my file in each MR round.
> Would be great if you have a place where I can have a look at some code
> snippets that have demonstrated the use of FileContext for easy learning.
>
> Many thanks in advance!
> Warm regards
> Arko
>
> On Thu, Sep 1, 2011 at 2:24 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
>> What kind of example code are you looking for? It will help if you
>> describe your use case...
>>
>> On Aug 31, 2011, at 1:13 AM, Arko Provo Mukherjee wrote:
>>
>> > Hi,
>> >
>> > I am using HDFS on 0.20.203 for my project.
>> >
>> > I can see the API has evolved much over the different versions.
>> >
>> > Can you please guide me to some link where I can find some example code
>> for any version >= 0.20.203 so that I can quick start the process.
>> >
>> > Thanks much in advance!
>> >
>> > Warm regards
>> > Arko
>>
>>
>
Arko Provo Mukherjee 2011-09-02, 05:05
Hi,
Many thanks for your kind reply.
I switched to 0.203.0 as per the advice of Arun Murthy.
Also I replaced import org.apache.hadoop.*; by
importorg.apache.hadoop.fs.*; as per the advice of Sudharsan.
Now I get just one error:
FileTest.java:19: cannot find symbol
symbol : class Configuration
location: class FileTest
FileSystem fs = FileSystem.get(new Configuration());
Am I missing some other import statements?
Many thanks in advance!
Warm Regards
Arko
On Thu, Sep 1, 2011 at 11:42 PM, Sudharsan Sampath <[EMAIL PROTECTED]>wrote:
> Hi,
>
> Some of your imports are incorrect. Both FileSystem and Path classes are
> present in o.a.h.fs
>
> Thanks
> Sudhan S
>
> On Thu, Sep 1, 2011 at 8:22 PM, Arko Provo Mukherjee <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> My code looks like this:
>>
>> *import java.util.*;*
>>
>> *import java.lang.*;*
>>
>> *import java.io.*;*
>>
>> *import org.apache.hadoop.*;*
>>
>> *
>> *
>>
>> *class MapReduce {*
>>
>> * *
>>
>> *public static void main(String[] args) throws Exception {*
>>
>> **
>>
>> * try {*
>>
>> * Path pt=new Path(
>> "hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");*
>>
>> * FileSystem fs = FileSystem.get(new Configuration());*
>>
>> * BufferedWriter br=new BufferedWriter(new
>> OutputStreamWriter(fs.create(pt,true)));*
>>
>> * String line;*
>>
>> * line="Testing";*
>>
>> * System.out.println(line);*
>>
>> * br.write(line);*
>>
>> * br.close();*
>>
>> * }catch(Exception e){*
>>
>> * System.out.println("File not found");*
>>
>> * }*
>>
>> * }*
>>
>> *}*
>>
>> When I compile the code, I get the following erros:
>> *$javac -classpath
>> /Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
>> FileSystemCat.java clear*
>>
>> FileTest.java:16: cannot find symbol
>> symbol : class Path
>> location: class FileTest
>> Path pt=new
>> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>> ^
>> FileTest.java:16: cannot find symbol
>> symbol : class Path
>> location: class FileTest
>> Path pt=new
>> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>> ^
>> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
>> accessed from outside package
>> FileSystem fs = FileSystem.get(new Configuration());
>> ^
>> FileTest.java:17: cannot find symbol
>> symbol : class Configuration
>> location: class FileTest
>> FileSystem fs = FileSystem.get(new Configuration());
>> ^
>> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot be
>> accessed from outside package
>> FileSystem fs = FileSystem.get(new Configuration());
>> ^
>> 5 errors
>>
>>
>> Also I would like to learn the FileContext API as I want to use 0.21 for
>> making appends to my file in each MR round.
>> Would be great if you have a place where I can have a look at some code
>> snippets that have demonstrated the use of FileContext for easy learning.
>>
>> Many thanks in advance!
>> Warm regards
>> Arko
>>
>> On Thu, Sep 1, 2011 at 2:24 AM, Arun C Murthy <[EMAIL PROTECTED]>wrote:
>>
>>> What kind of example code are you looking for? It will help if you
>>> describe your use case...
>>>
>>> On Aug 31, 2011, at 1:13 AM, Arko Provo Mukherjee wrote:
>>>
>>> > Hi,
>>> >
>>> > I am using HDFS on 0.20.203 for my project.
>>> >
>>> > I can see the API has evolved much over the different versions.
>>> >
>>> > Can you please guide me to some link where I can find some example code
>>> for any version >= 0.20.203 so that I can quick start the process.
>>> >
>>> > Thanks much in advance!
>>> >
>>> > Warm regards
>>> > Arko
>>>
>>>
Arko Provo Mukherjee 2011-09-02, 06:27
Hi,
Thanks much!
It worked for both 0.20.203.0 and 0.21.0!
I am using Eclipse IDE. I tried once to use the plugin but it didn't work :(
Thanks & Regards
Arko
On Fri, Sep 2, 2011 at 12:21 AM, Sudharsan Sampath <[EMAIL PROTECTED]>wrote:
> Hi,
>
> Import o.a.h.conf . Also, if possible try and use an IDE so that u can
> avoid these kind of issues.
>
> - Sudhan S
>
> On Fri, Sep 2, 2011 at 10:35 AM, Arko Provo Mukherjee <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Many thanks for your kind reply.
>>
>> I switched to 0.203.0 as per the advice of Arun Murthy.
>>
>> Also I replaced import org.apache.hadoop.*; by importorg.apache.hadoop.fs.*; as per the advice of Sudharsan.
>>
>> Now I get just one error:
>> FileTest.java:19: cannot find symbol
>> symbol : class Configuration
>> location: class FileTest
>> FileSystem fs = FileSystem.get(new Configuration());
>>
>> Am I missing some other import statements?
>>
>> Many thanks in advance!
>> Warm Regards
>> Arko
>>
>> On Thu, Sep 1, 2011 at 11:42 PM, Sudharsan Sampath <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> Some of your imports are incorrect. Both FileSystem and Path classes are
>>> present in o.a.h.fs
>>>
>>> Thanks
>>> Sudhan S
>>>
>>> On Thu, Sep 1, 2011 at 8:22 PM, Arko Provo Mukherjee <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>> My code looks like this:
>>>>
>>>> *import java.util.*;*
>>>>
>>>> *import java.lang.*;*
>>>>
>>>> *import java.io.*;*
>>>>
>>>> *import org.apache.hadoop.*;*
>>>>
>>>> *
>>>> *
>>>>
>>>> *class MapReduce {*
>>>>
>>>> * *
>>>>
>>>> *public static void main(String[] args) throws Exception {*
>>>>
>>>> **
>>>>
>>>> * try {*
>>>>
>>>> * Path pt=new Path(
>>>> "hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");*
>>>>
>>>> * FileSystem fs = FileSystem.get(new Configuration());*
>>>>
>>>> * BufferedWriter br=new BufferedWriter(new
>>>> OutputStreamWriter(fs.create(pt,true)));*
>>>>
>>>> * String line;*
>>>>
>>>> * line="Testing";*
>>>>
>>>> * System.out.println(line);*
>>>>
>>>> * br.write(line);*
>>>>
>>>> * br.close();*
>>>>
>>>> * }catch(Exception e){*
>>>>
>>>> * System.out.println("File not found");*
>>>>
>>>> * }*
>>>>
>>>> * }*
>>>>
>>>> *}*
>>>>
>>>> When I compile the code, I get the following erros:
>>>> *$javac -classpath
>>>> /Users/arko/Documents/hadoop-0.21.0/hadoop-common-0.21.0.jar -d class/
>>>> FileSystemCat.java clear*
>>>>
>>>> FileTest.java:16: cannot find symbol
>>>> symbol : class Path
>>>> location: class FileTest
>>>> Path pt=new
>>>> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>>>> ^
>>>> FileTest.java:16: cannot find symbol
>>>> symbol : class Path
>>>> location: class FileTest
>>>> Path pt=new
>>>> Path("hdfs://localhost:54310//Users/arko/Documents/Research/HDFS/abc");
>>>> ^
>>>> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot
>>>> be accessed from outside package
>>>> FileSystem fs = FileSystem.get(new Configuration());
>>>> ^
>>>> FileTest.java:17: cannot find symbol
>>>> symbol : class Configuration
>>>> location: class FileTest
>>>> FileSystem fs = FileSystem.get(new Configuration());
>>>> ^
>>>> FileTest.java:17: java.io.FileSystem is not public in java.io; cannot
>>>> be accessed from outside package
>>>> FileSystem fs = FileSystem.get(new Configuration());
>>>> ^
>>>> 5 errors
>>>>
>>>>
>>>> Also I would like to learn the FileContext API as I want to use 0.21 for
>>>> making appends to my file in each MR round.
>>>> Would be great if you have a place where I can have a look at some code
>>>> snippets that have demonstrated the use of FileContext for easy learning.
|
|
