|
|
-
potential bug when read directly from local block
hadoop 2012-08-09, 09:36
<div><br>
<p class="p1">Hi all</p> <p class="p2"><br></p> <p class="p1">I found some warning in the log, and it may the bug of hdfs. Is there anyone can confirm that?</p> <p class="p2"><br></p> <p class="p1">I make a simple test case and the output is attached.</p> <p class="p2"><br></p> <p class="p1">I test it on hadoop-dist-2.0.0-alpha</p> <p class="p2"><br></p> <p class="p2"><br></p> <p class="p1">Best Regards</p> <p class="p2"><br></p> <p class="p1">-- </p> <p class="p3"><b><i>Zhanwei Wang</i></b> </p></div>
+
hadoop 2012-08-09, 09:36
-
Re: potential bug when read directly from local block
Harsh J 2012-08-09, 13:13
Hi,
If you're running this as a non-superuser (i.e. not the user thats running the DataNodes), have you set your dfs.datanode.data.dir.perm to be 755 instead of the default 700?
On Thu, Aug 9, 2012 at 3:06 PM, hadoop <[EMAIL PROTECTED]> wrote: > > Hi all > > > I found some warning in the log, and it may the bug of hdfs. Is there anyone can confirm that? > > > I make a simple test case and the output is attached. > > > I test it on hadoop-dist-2.0.0-alpha > > > > > Best Regards > > > -- > Zhanwei Wang >
-- Harsh J
+
Harsh J 2012-08-09, 13:13
-
Re:Re: potential bug when read directly from local block
hadoop 2012-08-09, 13:30
<span ></span>Hi, <div><br></div><div>I run this test as superuser. </div><div><br></div><div>The reason of this warning is that, hdfs client buffer the block information and then the block is appended. For example, the generation stamp buffered on the client is 1000, after append, the real generation stamp of block is 1001. When the client ask datanode to return the local block path, client send 1000 to datanode(in the block information in the rpc call), and datanode return metadata file path which is "blockid_1000.meta", and this metadata file does not exist, it should be "blockid_1001.meta"</div><div><br></div><div><br></div><div><br></div><div><br><pre>----- Original Message -----From: Harsh J <[EMAIL PROTECTED]> To: [EMAIL PROTECTED]
Hi,
If you're running this as a non-superuser (i.e. not the user thats running the DataNodes), have you set your dfs.datanode.data.dir.perm to be 755 instead of the default 700?
On Thu, Aug 9, 2012 at 3:06 PM, hadoop <[EMAIL PROTECTED]> wrote: > > Hi all > > > I found some warning in the log, and it may the bug of hdfs. Is there anyone can confirm that? > > > I make a simple test case and the output is attached. > > > I test it on hadoop-dist-2.0.0-alpha > > > > > Best Regards > > > -- > Zhanwei Wang >
-- Harsh J </pre></div>
+
hadoop 2012-08-09, 13:30
|
|