Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - question about clienttrace logs in hdfs and shortcircuit read


Copy link to this message
-
Re: question about clienttrace logs in hdfs and shortcircuit read
Anoop John 2013-07-05, 08:07
Thanks Viral.. Confusion came to me as the heading of the mails say abt SCR
metric and finally speaks abt hbase handled checksum metric which are 2
features..  Agree that hbase checksum works only with SCR as of now, this
might change later..  That is why I was just making it clear  .. NP :)

As I said pls check the DFS client side log(RS log) for BlockReaderLocal.
AFAIK there should not be HDFS_READ op log in DN side when the SCR is
happening..

-Anoop-

On Fri, Jul 5, 2013 at 1:31 PM, Viral Bajaria <[EMAIL PROTECTED]>wrote:

> No worries, Anoop. Here is some clarification for this chain.
>
> It started initially to figure out how to check whether SCR is effective at
> the RS or not. I could not find the metric anywhere in ganglia/JMX and
> didn't find any RegionServer level metric either and so started looking at
> my DN logs. I saw a lot of "clienttrace" logs which are basically logs for
> HDFS_READ and HDFS_WRITE ops. I wanted to see if it's a valid assumption
> that SCR is working if I don't see any clienttrace logs for the RS that is
> hosted on the same box as the DN.
>
> Hopefully I clarified it.
>
> On Fri, Jul 5, 2013 at 12:55 AM, Anoop John <[EMAIL PROTECTED]> wrote:
>
> > Agree that right now the HBase handled checksum will work only with SCR.
> > But it might work without SCR also later (as support come from HDFS)..
>  So
> > I am not getting the direct relation btw the SCR metric and this one. Am
> I
> > missed some thing in the mail loop? Sorry just trying to be clear on this
> > :)
> >
>