We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
shortcut enabled.  Since updating to this version I've seen a bunch of WARN
messages corresponding to exceptions trying to connect to what looks like
the local DN.

As far as I can tell it doesn't appear to be having much affect on the RS
but it'd be nice to clear it up and know for sure that it doesn't indicate
some problem. Has anyone ever seen this and know what it is or how to fix
it?

http://pastebin.com/PA6Y9pJN

Thanks!

Bryan,

  What permissions did you set for the SCRs?  The DNs need

<property>
<name>dfs.datanode.data.dir.perm</name>
<value>755</name>
</property>
On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> shortcut enabled.  Since updating to this version I've seen a bunch of WARN
> messages corresponding to exceptions trying to connect to what looks like
> the local DN.
>
> As far as I can tell it doesn't appear to be having much affect on the RS
> but it'd be nice to clear it up and know for sure that it doesn't indicate
> some problem. Has anyone ever seen this and know what it is or how to fix
> it?
>
> http://pastebin.com/PA6Y9pJN
>
> Thanks!
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

Yep we do have that property set to that value.  The file does not seem to
exist when I try ls'ing it myself.  I'm not sure where it comes from or how
it should be created.
On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Bryan,
>
>   What permissions did you set for the SCRs?  The DNs need
>
> <property>
> <name>dfs.datanode.data.dir.perm</name>
> <value>755</name>
> </property>
>
>
> On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
> > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > shortcut enabled.  Since updating to this version I've seen a bunch of
> WARN
> > messages corresponding to exceptions trying to connect to what looks like
> > the local DN.
> >
> > As far as I can tell it doesn't appear to be having much affect on the RS
> > but it'd be nice to clear it up and know for sure that it doesn't
> indicate
> > some problem. Has anyone ever seen this and know what it is or how to fix
> > it?
> >
> > http://pastebin.com/PA6Y9pJN
> >
> > Thanks!
> >
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>

Hi Byran,

One of engineers wanted me to pass this recommendation along:

The issue here is mostly likely a missing configuration.

We need to have this configuration on both the DataNode and RegionServer:
<property>
  <name>dfs.domain.socket.path</name>
  <value>/var/run/hadoop-hdfs/dn._PORT</value>
</property>
<property>
  <name>dfs.client.read.shortcircuit</name>
  <value>true</value>
</property>

In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
so most likely he does not have dfs.client.read.shortcircuit set in
hdfs-site.xml.  Adding that should fix it

On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> Yep we do have that property set to that value.  The file does not seem to
> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
> it should be created.
>
>
> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Bryan,
> >
> >   What permissions did you set for the SCRs?  The DNs need
> >
> > <property>
> > <name>dfs.datanode.data.dir.perm</name>
> > <value>755</name>
> > </property>
> >
> >
> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> > <[EMAIL PROTECTED]>wrote:
> >
> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > > shortcut enabled.  Since updating to this version I've seen a bunch of
> > WARN
> > > messages corresponding to exceptions trying to connect to what looks
> like
> > > the local DN.
> > >
> > > As far as I can tell it doesn't appear to be having much affect on the
> RS
> > > but it'd be nice to clear it up and know for sure that it doesn't
> > indicate
> > > some problem. Has anyone ever seen this and know what it is or how to
> fix
> > > it?
> > >
> > > http://pastebin.com/PA6Y9pJN
> > >
> > > Thanks!
> > >
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
> >
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

Great, thanks for the help Kevin.  He's correct that we only have
dfs.client.read.shortcircuit set in our hbase-site.xml, and we don't
have dfs.domain.socket.path
set anywhere.  I'll get that added.

Thanks again!
On Wed, Mar 6, 2013 at 3:43 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Hi Byran,
>
> One of engineers wanted me to pass this recommendation along:
>
> The issue here is mostly likely a missing configuration.
>
> We need to have this configuration on both the DataNode and RegionServer:
> <property>
>   <name>dfs.domain.socket.path</name>
>   <value>/var/run/hadoop-hdfs/dn._PORT</value>
> </property>
> <property>
>   <name>dfs.client.read.shortcircuit</name>
>   <value>true</value>
> </property>
>
> In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
> so most likely he does not have dfs.client.read.shortcircuit set in
> hdfs-site.xml.  Adding that should fix it
>
> On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
> > Yep we do have that property set to that value.  The file does not seem
> to
> > exist when I try ls'ing it myself.  I'm not sure where it comes from or
> how
> > it should be created.
> >
> >
> > On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Bryan,
> > >
> > >   What permissions did you set for the SCRs?  The DNs need
> > >
> > > <property>
> > > <name>dfs.datanode.data.dir.perm</name>
> > > <value>755</name>
> > > </property>
> > >
> > >
> > > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > > > shortcut enabled.  Since updating to this version I've seen a bunch
> of
> > > WARN
> > > > messages corresponding to exceptions trying to connect to what looks
> > like
> > > > the local DN.
> > > >
> > > > As far as I can tell it doesn't appear to be having much affect on
> the
> > RS
> > > > but it'd be nice to clear it up and know for sure that it doesn't
> > > indicate
> > > > some problem. Has anyone ever seen this and know what it is or how to
> > fix
> > > > it?
> > > >
> > > > http://pastebin.com/PA6Y9pJN
> > > >
> > > > Thanks!
> > > >
> > >
> > >
> > >
> > > --
> > > Kevin O'Dell
> > > Customer Operations Engineer, Cloudera
> > >
> >
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>

Hi Kevin
              Thanks for the information.  In HBase book, we have added a note on how to use short circuit reads.. Can we update it accordingly? Which version of HDFS need this attribute to be present in DN side also?  It would be great if you can file a JIRA and give a change in the description on the usage...

-Anoop-
________________________________________
From: Kevin O'dell [[EMAIL PROTECTED]]
Sent: Thursday, March 07, 2013 2:13 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Odd WARN in hbase 0.94.2

Hi Byran,

One of engineers wanted me to pass this recommendation along:

The issue here is mostly likely a missing configuration.

We need to have this configuration on both the DataNode and RegionServer:
<property>
  <name>dfs.domain.socket.path</name>
  <value>/var/run/hadoop-hdfs/dn._PORT</value>
</property>
<property>
  <name>dfs.client.read.shortcircuit</name>
  <value>true</value>
</property>

In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
so most likely he does not have dfs.client.read.shortcircuit set in
hdfs-site.xml.  Adding that should fix it

On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> Yep we do have that property set to that value.  The file does not seem to
> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
> it should be created.
>
>
> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Bryan,
> >
> >   What permissions did you set for the SCRs?  The DNs need
> >
> > <property>
> > <name>dfs.datanode.data.dir.perm</name>
> > <value>755</name>
> > </property>
> >
> >
> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> > <[EMAIL PROTECTED]>wrote:
> >
> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > > shortcut enabled.  Since updating to this version I've seen a bunch of
> > WARN
> > > messages corresponding to exceptions trying to connect to what looks
> like
> > > the local DN.
> > >
> > > As far as I can tell it doesn't appear to be having much affect on the
> RS
> > > but it'd be nice to clear it up and know for sure that it doesn't
> > indicate
> > > some problem. Has anyone ever seen this and know what it is or how to
> fix
> > > it?
> > >
> > > http://pastebin.com/PA6Y9pJN
> > >
> > > Thanks!
> > >
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
> >
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

I'll add that this did fix my issue.  The dn.50010 proc file is now in
place and I have not see any other exceptions.

Thanks guys!
On Wed, Mar 6, 2013 at 10:45 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

> Hi Kevin
>               Thanks for the information.  In HBase book, we have added a
> note on how to use short circuit reads.. Can we update it accordingly?
> Which version of HDFS need this attribute to be present in DN side also?
>  It would be great if you can file a JIRA and give a change in the
> description on the usage...
>
> -Anoop-
> ________________________________________
> From: Kevin O'dell [[EMAIL PROTECTED]]
> Sent: Thursday, March 07, 2013 2:13 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Odd WARN in hbase 0.94.2
>
> Hi Byran,
>
> One of engineers wanted me to pass this recommendation along:
>
> The issue here is mostly likely a missing configuration.
>
> We need to have this configuration on both the DataNode and RegionServer:
> <property>
>   <name>dfs.domain.socket.path</name>
>   <value>/var/run/hadoop-hdfs/dn._PORT</value>
> </property>
> <property>
>   <name>dfs.client.read.shortcircuit</name>
>   <value>true</value>
> </property>
>
> In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
> so most likely he does not have dfs.client.read.shortcircuit set in
> hdfs-site.xml.  Adding that should fix it
>
> On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
> > Yep we do have that property set to that value.  The file does not seem
> to
> > exist when I try ls'ing it myself.  I'm not sure where it comes from or
> how
> > it should be created.
> >
> >
> > On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Bryan,
> > >
> > >   What permissions did you set for the SCRs?  The DNs need
> > >
> > > <property>
> > > <name>dfs.datanode.data.dir.perm</name>
> > > <value>755</name>
> > > </property>
> > >
> > >
> > > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > > > shortcut enabled.  Since updating to this version I've seen a bunch
> of
> > > WARN
> > > > messages corresponding to exceptions trying to connect to what looks
> > like
> > > > the local DN.
> > > >
> > > > As far as I can tell it doesn't appear to be having much affect on
> the
> > RS
> > > > but it'd be nice to clear it up and know for sure that it doesn't
> > > indicate
> > > > some problem. Has anyone ever seen this and know what it is or how to
> > fix
> > > > it?
> > > >
> > > > http://pastebin.com/PA6Y9pJN
> > > >
> > > > Thanks!
> > > >
> > >
> > >
> > >
> > > --
> > > Kevin O'Dell
> > > Customer Operations Engineer, Cloudera
> > >
> >
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>

Hi Byran,
                          This change is needed with usage of any of the open src HDFS release or is it only in CDH? Is this related with HDFS-347?
In such a case forget abt my previous mail abt adding in book  :)

-Anoop-
________________________________________
From: Kevin O'dell [[EMAIL PROTECTED]]
Sent: Thursday, March 07, 2013 2:13 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Odd WARN in hbase 0.94.2

Hi Byran,

One of engineers wanted me to pass this recommendation along:

The issue here is mostly likely a missing configuration.

We need to have this configuration on both the DataNode and RegionServer:
<property>
  <name>dfs.domain.socket.path</name>
  <value>/var/run/hadoop-hdfs/dn._PORT</value>
</property>
<property>
  <name>dfs.client.read.shortcircuit</name>
  <value>true</value>
</property>

In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
so most likely he does not have dfs.client.read.shortcircuit set in
hdfs-site.xml.  Adding that should fix it

On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> Yep we do have that property set to that value.  The file does not seem to
> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
> it should be created.
>
>
> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Bryan,
> >
> >   What permissions did you set for the SCRs?  The DNs need
> >
> > <property>
> > <name>dfs.datanode.data.dir.perm</name>
> > <value>755</name>
> > </property>
> >
> >
> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
> > <[EMAIL PROTECTED]>wrote:
> >
> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
> > > shortcut enabled.  Since updating to this version I've seen a bunch of
> > WARN
> > > messages corresponding to exceptions trying to connect to what looks
> like
> > > the local DN.
> > >
> > > As far as I can tell it doesn't appear to be having much affect on the
> RS
> > > but it'd be nice to clear it up and know for sure that it doesn't
> > indicate
> > > some problem. Has anyone ever seen this and know what it is or how to
> fix
> > > it?
> > >
> > > http://pastebin.com/PA6Y9pJN
> > >
> > > Thanks!
> > >
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
> >
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

The datanode data directory permission settings applies to Apache HDFS
0.22, 0.23 and 2.x, is unrelated and unnecessary for HDFS-347 but is
necessary for the older SCR. If not mentioned in the book yet, they
ought to go in I think as even some other distributions (other than
HDFS 1.x) included the specific security settings in their
past-version releases.

The other settings (the domain socket options), which remove the need
to expose the data directory permissions are part of HDFS-347, which
are in a branch upstream right now, awaiting integration into an
upcoming HDFS release. We could document these better options once
such a release is out, as its a further gain and is also secure.

On Thu, Mar 7, 2013 at 2:38 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:
> Hi Byran,
>                           This change is needed with usage of any of the open src HDFS release or is it only in CDH? Is this related with HDFS-347?
> In such a case forget abt my previous mail abt adding in book  :)
>
> -Anoop-
> ________________________________________
> From: Kevin O'dell [[EMAIL PROTECTED]]
> Sent: Thursday, March 07, 2013 2:13 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Odd WARN in hbase 0.94.2
>
> Hi Byran,
>
> One of engineers wanted me to pass this recommendation along:
>
> The issue here is mostly likely a missing configuration.
>
> We need to have this configuration on both the DataNode and RegionServer:
> <property>
>   <name>dfs.domain.socket.path</name>
>   <value>/var/run/hadoop-hdfs/dn._PORT</value>
> </property>
> <property>
>   <name>dfs.client.read.shortcircuit</name>
>   <value>true</value>
> </property>
>
> In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
> so most likely he does not have dfs.client.read.shortcircuit set in
> hdfs-site.xml.  Adding that should fix it
>
> On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
>> Yep we do have that property set to that value.  The file does not seem to
>> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
>> it should be created.
>>
>>
>> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Bryan,
>> >
>> >   What permissions did you set for the SCRs?  The DNs need
>> >
>> > <property>
>> > <name>dfs.datanode.data.dir.perm</name>
>> > <value>755</name>
>> > </property>
>> >
>> >
>> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
>> > <[EMAIL PROTECTED]>wrote:
>> >
>> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
>> > > shortcut enabled.  Since updating to this version I've seen a bunch of
>> > WARN
>> > > messages corresponding to exceptions trying to connect to what looks
>> like
>> > > the local DN.
>> > >
>> > > As far as I can tell it doesn't appear to be having much affect on the
>> RS
>> > > but it'd be nice to clear it up and know for sure that it doesn't
>> > indicate
>> > > some problem. Has anyone ever seen this and know what it is or how to
>> fix
>> > > it?
>> > >
>> > > http://pastebin.com/PA6Y9pJN
>> > >
>> > > Thanks!
>> > >
>> >
>> >
>> >
>> > --
>> > Kevin O'Dell
>> > Customer Operations Engineer, Cloudera
>> >
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera

--
Harsh J