Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> jni files


Amit,

On Fri, Jul 9, 2010 at 3:00 PM, amit kumar verma <[EMAIL PROTECTED]> wrote:
>  Hi  Hemanth,
>
> Yeah I have gone through the api documentation and there is no issue in
> accessing files from HDFS, but my concern is what about the API which
> already got developed without hadoop. OK, what I mean, I developed an
> application when I didn't know about the hadoop, but as now I need to
> implement grid environment so I am looking for Hadoop.
>
> So no the question is, how can use the same code to work for HDFS, do I need
> to change my code and use hadoop API to used the HDFS. If that is the case
> then the change will be major, or there is any way where the default
> java.file can be integrated with hdfs.
>
> Did you get the issue ??
>

Yes, I think I do. Unfortunately, AFAIK, there's no easy way out. If
your application had previously used Java I/O File APIs, they need to
be migrated to the Hadoop FS API. If you are moving from a
non-Distributed application to Hadoop for a reason (such as handling
scale for e.g.) the investment will be well worth the effort, IMHO.

> Thanks,
> Amit Kumar Verma
> Verchaska Infotech Pvt. Ltd.
>
>
>
> On 07/09/2010 02:47 PM, Hemanth Yamijala wrote:
>>
>> Amit,
>>
>> On Fri, Jul 9, 2010 at 2:39 PM, amit kumar verma<[EMAIL PROTECTED]>
>>  wrote:
>>>
>>>  Hi Hemant,
>>>
>>> The version are same as copied it to all client machine.
>>>
>>> I think I got a solution. As I read more about hadoop and JNI, I learned
>>> that I need to copy jni files to
>>> HADOOP_INSTALLATION_DIR//lib/native/Linux-xxx-xxx. I though my linux
>>> machine
>>> is Linux-i386-32. then I found in "org.apache.hadoop.util.PlatformName"
>>> class gives you your machine type and its Linux-amd64-64 and asa I copied
>>> jni files to this directory error are not coming.
>>>
>>> Though full code is still not running as I developed the application
>>> using
>>> java.file class and i am still thinking how to make changes so that it
>>> can
>>> access hdfs !!!  Do i need to change my all API with respect to HDFS and
>>> rewrite using hadoop fs or ??!!!
>>>
>> To access files from HDFS, you should use the Hadoop FileSystem API.
>> Please take a look at the Javadoc and also a tutorial such as this:
>> http://developer.yahoo.com/hadoop/tutorial/module2.html#programmatically
>> for more information.
>>
>>> It will be great if someone advice on this.
>>>
>>>
>>>
>>> Thanks,
>>> Amit Kumar Verma
>>> Verchaska Infotech Pvt. Ltd.
>>>
>>>
>>>
>>> On 07/09/2010 02:04 PM, Hemanth Yamijala wrote:
>>>>
>>>> Hi,
>>>>
>>>> Possibly another silly question, but can you cross check if the
>>>> versions of Hadoop on the client and the server are the same ?
>>>>
>>>> Thanks
>>>> hemanth
>>>>
>>>> On Thu, Jul 8, 2010 at 10:57 PM, Allen Wittenauer
>>>> <[EMAIL PROTECTED]>    wrote:
>>>>>
>>>>> On Jul 8, 2010, at 1:08 AM, amit kumar verma wrote:
>>>>>
>>>>>>     DistributedCache.addCacheFile("hdfs://*
>>>>>>     /192.168.0.153:50075*/libraries/mylib.so.1#mylib.so", conf);
>>>>>
>>>>> Do you actually have asterisks in this?  If so, that's the problem.
>>>>>
>>>>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB