Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> question about clienttrace logs in hdfs and shortcircuit read


+
Viral Bajaria 2013-07-04, 06:25
+
Azuryy Yu 2013-07-04, 08:57
+
Viral Bajaria 2013-07-04, 09:00
+
Asaf Mesika 2013-07-04, 10:50
+
Viral Bajaria 2013-07-05, 06:09
+
ramkrishna vasudevan 2013-07-05, 06:21
+
Viral Bajaria 2013-07-05, 07:03
+
ramkrishna vasudevan 2013-07-05, 07:30
+
Viral Bajaria 2013-07-05, 07:38
Copy link to this message
-
Re: question about clienttrace logs in hdfs and shortcircuit read
Viral
             DFS client uses org.apache.hadoop.hdfs.BlockReaderLocal  for
SCR..  I can see some debug level logs in this

*

LOG*.debug("New BlockReaderLocal for file " + blkfile + " of size "

+ blkfile.length() + " startOffset " + startOffset + " length "

+ length + " short circuit checksum " + !skipChecksumCheck);

If debug is enabled u can check for this in ur DFS client side (ie RS
logs).. I hope u have enabled debug level for hadoop.. Some metric from the
DFS client side would be useful..
-Anoop-
On Fri, Jul 5, 2013 at 1:08 PM, Viral Bajaria <[EMAIL PROTECTED]>wrote:

> Yes I was checking 0.94 code.
>
> And sorry for the brain fart, I just spotted the metric in ganglia. There
> are just too many metrics in ganglia and skipped this one! It was under the
> group "hbase.regionserver", while I was expected it to be
> "hbase.regionserver.RegionServerStatistics".
>
> The chart shows no failures (i.e. it has no data to show). So now I need to
> figure out if shortcircuit is on or not. Is there an easier way to do via
> JMX if not by logs ? I tried to grep the logs for "shortcircuit", "circuit"
> and didn't find anything in the last 2 days. I restarted a region server
> yesterday and would have expected it to show up on that one atleast.
>
> On Fri, Jul 5, 2013 at 12:30 AM, ramkrishna vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> > Are you checking with 0.94 code? I believe so.
>
+
Anoop John 2013-07-05, 07:55
+
Viral Bajaria 2013-07-05, 08:01
+
Anoop John 2013-07-05, 08:07
+
Viral Bajaria 2013-07-05, 08:21
+
ramkrishna vasudevan 2013-07-04, 09:02
+
lars hofhansl 2013-07-04, 09:07
+
Viral Bajaria 2013-07-04, 09:12
+
Asaf Mesika 2013-07-04, 06:33
+
Viral Bajaria 2013-07-04, 08:50