Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Doubt Regading HLogs


Copy link to this message
-
Re: Doubt Regading HLogs
Yes, it's by design.
The last log file the one beeing written by HBase The safe option is to
wait for this file to be closed by HBase.  As Yong said, you can change the
roll parameter if you want it to be terminated sooner, but changing this
parameter impacts the hdfs namenode load.10 minutes is likely safer than 3
seconds.
On Fri, May 17, 2013 at 1:40 PM, yonghu <[EMAIL PROTECTED]> wrote:

> In this situation, you can set the
>
> > <property>
> >
> >     <name>hbase.regionserver.
> logroll.period</name>
> >
> >     <value>3600000</value>
> >
> > </property>
>
> to a short value, let's say 3000 and then you can see your log file with
> current size after 3 seconds.
>
> To Nicolas,
>
> I guess he wants somehow to analyze the HLog.
>
> regards!
>
> Yong
>
>
>
> On Fri, May 17, 2013 at 1:27 PM, Rishabh Agrawal <
> [EMAIL PROTECTED]> wrote:
>
> > Thanks Nicolas,
> >
> > When will  this file be finalized. Is it time bound? Or it will be always
> > be zero for last one (even if it contains the data)
> >
> > -----Original Message-----
> > From: Nicolas Liochon [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, May 17, 2013 4:39 PM
> > To: user
> > Subject: Re: Doubt Regading HLogs
> >
> > That's HDFS.
> >
> > When a file is currently written, the size is not known, as the write is
> > in progress. So the namenode reports a size of zero (more exactly, it
> does
> > not take into account the hdfs block beeing written when it calculates
> the
> > size). When you read, you go to the datanode owning the data, so you see
> > the real content as it is at the time of reading.
> >
> > btw, why do you want to read the HLog?
> >
> >
> > On Fri, May 17, 2013 at 12:53 PM, Rishabh Agrawal <
> > [EMAIL PROTECTED]> wrote:
> >
> > >  Hello,
> > >
> > > I am working with Hlogs of Hbase and I have this doubt that HDFS shows
> > > size of last log file as zero. But when I open it I see data in it.
> > > When I add extra data a new file with zero size is created and
> > > previous HLog file gets its size.  This thing applies to each region
> > > server.  Following is the purged screen shot of same:
> > >
> > >
> > >
> > > I have set following parameters in hbase-site.xml for logs:
> > >
> > > <property>
> > >
> > >     <name>hbase.regionserver.logroll.period</name>
> > >
> > >     <value>3600000</value>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > >    <name>hbase.master.logcleaner.ttl</name>
> > >
> > >    <value>604800000</value>
> > >
> > >     </property>
> > >
> > > <property>
> > >
> > >    <name>hbase.regionserver.optionallogflushinterval</name>
> > >
> > >    <value>3000</value>
> > >
> > > </property>
> > >
> > >
> > >
> > >
> > >
> > > I plan to read log files for some validation work, Please guide me
> > > through this behavior of Hbase.
> > >
> > >
> > >
> > >
> > >
> > > Thanks and Regards
> > >
> > > Rishabh Agrawal
> > >
> > > Software Engineer
> > >
> > > Impetus Infotech (India) Pvt. Ltd.
> > >
> > > (O) +91.731.426.9300 x4526
> > >
> > > (M) +91.812.026.2722
> > >
> > > www.impetus.com
> > >
> > >
> > >
> > > ------------------------------
> > >
> > >
> > >
> > >
> > >
> > >
> > > NOTE: This message may contain information that is confidential,
> > > proprietary, privileged or otherwise protected by law. The message is
> > > intended solely for the named addressee. If received in error, please
> > > destroy and notify the sender. Any use of this email is prohibited
> > > when received in error. Impetus does not represent, warrant and/or
> > > guarantee, that the integrity of this communication has been
> > > maintained nor that the communication is free of errors, virus,
> > interception or interference.
> > >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> > proprietary, privileged or otherwise protected by law. The message is
> > intended solely for the named addressee. If received in error, please
> > destroy and notify the sender. Any use of this email is prohibited when
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB