Sorry, resending on hdfs-dev; apparently I'm not on -user.

Begin forwarded message:

> From: Brian Bockelman <[EMAIL PROTECTED]>
> Date: March 1, 2011 6:24:28 PM CST
> Cc: hdfs-user <[EMAIL PROTECTED]>
> Subject: Re: problems with fuse-dfs
>
>
> Side note: Do not cross post to multiple lists.  It annoys folks.
>
> On Mar 1, 2011, at 11:50 AM, Aastha Mehta wrote:
>
>> Hello,
>>
>> I am facing problems in running fuse-dfs over hdfs. I came across this
>> thread while searching for my problem:
>> http://www.mail-archive.com/[EMAIL PROTECTED]/msg00341.html
>> OR
>> http://search-hadoop.com/m/T1Bjv17q0eF1&subj=Re+Fuse+DFS
>>
>> and it exactly mentions some of the symptoms I am looking at.
>>
>> To quote Eli,
>> "fuse_impls_getattr.c connects via hdfsConnectAsUser so you should see a log
>> (unless its returning from a case that doesn't print an error). Next
>> step is to determine that you're actually reaching the code you modified by
>> adding a syslog to the top of the function (need to make sure you're
>> actually loading the libhdfs you've built vs an older one or another one
>> installed on your system), and then determine which error case in that
>> function you're seeing. It's strange that -d would cause that path to
>> change."
>>
>> I get the error from fuse_impls_getattr.c that could not connect to my host
>> and port. I checked the syslogs and only this error is printed there. No
>> error is printed from hdfsConnectAsUser(). I even wrote down syslog
>> functions in hdfsConnectAsUser, but it is not printed. I used the following
>> commands to compile libhdfs and then fuse-dfs from $HADOOP_HOME:
>>
>> $ export
>> LD_LIBRARY_PATH=$HADOOP_HOME/build/libhdfs:/usr/lib/jvm/java-1.5.0-sun:
>> /usr/lib/jvm/java-1.5.0-sun-1.5.0.19/include:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/i386/client
>> $ ant compile-c++-libhdfs -Dlibhdfs=1
>> $ ant compile-contrib -Dlibhdfs=1 -Dfusedfs=1
>> -Djava5.home=/usr/lib/jvm/java-1.5.0-sun-1.5.0.19
>> $ mkdir /media/myDrive/newhdfs
>> $ cd $HADOOP_HOME/build/contrib/fuse-dfs/
>> $ sudo sh fuse_dfs_wrapper.sh dfs://aastha-desktop:9000
>> /media/myDrive/newhdfs
>>
>> And when i do:
>> $ sudo cat /var/log/syslog
>>
>> at the end, I get
>> aastha-desktop fuse_dfs: ERROR: could not connect to aastha-desktop:9000,
>> fuse_impls_getattr.c:37
>>
>> EIO value is 5, which I understand implies an Input/output error.
>>
>> When i try to ls /media/myDrive/newhdfs, i get the input/output error.
>> Also if i try to unmount newhdfs, it says that it is in use and cannot be
>> unmounted.
>>
>> I even tried -d option, but that just runs into an infinite loop of
>> connection retries and shows some socketconnection exception.
>>
>> I have been working with fuse-dfs and hdfs for more than a month now. Till
>> two days back I could atleast create fuse-dfs instances and mount them using
>> the wrapper script. I could also create files inside it, modify them, etc.
>> and view the changes being updated in the nodes of the hadoop cluster. Since
>> yesterday, I am trying to create a LUN image inside fuse-dfs (using dd
>> if=/dev/zero of=... command), and check whether it is created properly on
>> the cluster or not. Yesterday, a normal file system was working and when I
>> tried to create a LUN, my syslog showed that an error was coming from
>> fuse_impls_open.c. But now a normal file system instance is also not getting
>> created and mounted properly.
>>
>> Following are my questions based on this observation:
>> 1. Please tell me, why is the fuse_impls_getattr.c error is coming now and
>> how can it be resolved.
>
> Did you try adding "-d" to the options to run in debug mode?
>
> What is the resulting options for "fuse_dfs" if you do "ps faux"?
>
>> 2. Does ant compile-c++-libhdfs compile libhdfs? Will the changes that I
>> make in source files in libhdfs be updated by the command? Since I can't see
>> any syslog statements getting printed from hdfsConnectAsUser(), I am not
>> sure if libhdfs has been recompiled. Also I cannot see any object files