Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> .SpoolingFileLineReader warning....


+
Dan Young 2012-11-17, 15:02
+
Brock Noland 2012-11-17, 15:57
+
Dan Young 2012-11-17, 16:01
+
Brock Noland 2012-11-17, 16:15
+
Dan Young 2012-11-17, 16:33
+
Dan Young 2012-11-19, 19:33
+
Patrick Wendell 2012-11-19, 23:03
+
Patrick Wendell 2012-11-19, 23:04
+
Brock Noland 2012-11-19, 23:29
+
Brock Noland 2012-11-20, 12:25
+
Dan Young 2012-11-20, 15:02
+
Brock Noland 2012-11-20, 16:21
+
Dan Young 2012-11-20, 16:59
+
Brock Noland 2012-11-20, 17:01
+
Dan Young 2012-11-20, 17:10
+
Brock Noland 2012-11-20, 17:14
Copy link to this message
-
Re: .SpoolingFileLineReader warning....
Ahh...thank you.  Let me check some things and make some changes....
On Tue, Nov 20, 2012 at 10:14 AM, Brock Noland <[EMAIL PROTECTED]> wrote:

> OK yeah, so basically it's doing a copy and then a delete. See "man 2
> rename" then ERRORS -> EXDEV but basically atomically renaming a file
> only works on the same mount point.
>
> On Tue, Nov 20, 2012 at 11:10 AM, Dan Young <[EMAIL PROTECTED]> wrote:
> > This is running on an EC2 instance, and these are the ephemeral drive(s).
> >
> > Filesystem     1K-blocks    Used Available Use% Mounted on
> > /dev/xvda1       8256952 1618248   6219276  21% /
> > udev             1912272       8   1912264   1% /dev
> > tmpfs             768100     180    767920   1% /run
> > none                5120       0      5120   0% /run/lock
> > none             1920244       0   1920244   0% /run/shm
> > /dev/xvdb      412814512 1127172 390717564   1% /mnt
> >
> > ubuntu@ip-foo:~$ cat /etc/fstab
> > LABEL=cloudimg-rootfs / ext4 defaults 0 0
> > /dev/xvdb /mnt auto defaults,nobootwait,comment=cloudconfig 0 2
> > ubuntu@ip-foo:~$
> >
> > ubuntu@ip-foo:~$ mount
> > /dev/xvda1 on / type ext4 (rw)
> > proc on /proc type proc (rw,noexec,nosuid,nodev)
> > sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
> > none on /sys/fs/fuse/connections type fusectl (rw)
> > none on /sys/kernel/debug type debugfs (rw)
> > none on /sys/kernel/security type securityfs (rw)
> > udev on /dev type devtmpfs (rw,mode=0755)
> > devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
> > tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
> > none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
> > none on /run/shm type tmpfs (rw,nosuid,nodev)
> > /dev/xvdb on /mnt type ext3 (rw)
> >
> > I do see that / is ext4 and /mnt is ext3.....hmm, not sure it matters.
> >
> > Dano
> >
> >
> >
> > On Tue, Nov 20, 2012 at 10:01 AM, Brock Noland <[EMAIL PROTECTED]>
> wrote:
> >>
> >> Are /var/log and /mnt/flume on the same file system? mv across file
> >> systems is a cp and then delete.
> >>
> >> On Tue, Nov 20, 2012 at 10:59 AM, Dan Young <[EMAIL PROTECTED]>
> wrote:
> >> > Coolio, thank you Brock.
> >> >
> >> > Did a quick test with a mv vs cp.....this is one test, repeated a few
> >> > times....the smaller file (~60M) seemed to work fine w/ a mv, but a
> >> > larger
> >> > file we're seeing the same behavior....
> >> >
> >> >
> >> > Start with no logs in the SpoolingDirectory:
> >> >
> >> > ls -lrt /mnt/flume/clickstream
> >> >
> >> > /mnt/flume/clickstream:
> >> > total 0
> >> >
> >> >
> >> >
> >> > Review the logs that will be rotated via logrotate.d into the
> respective
> >> > directory; /mnt/flume/clickstream
> >> >
> >> > ls -lrt /var/log/clickstream
> >> > /var/log/clickstream:
> >> > total 64112
> >> > -rw-rw-r-- 1 ubuntu ubuntu 65648336 Nov 20 16:05 clickstream.log
> >> >
> >> >
> >> >
> >> > Review logrotate config in /etc/logrotate.d. Note here, I changed from
> >> > cp -p
> >> > to a mv.....
> >> >
> >> > /var/log/clickstream/clickstream.log
> >> > {
> >> >   missingok
> >> >   rotate 3
> >> >   compress
> >> >   delaycompress
> >> >   copytruncate
> >> >   notifempty
> >> >   size 50M
> >> >   dateext
> >> >   dateformat -%Y-%m-%d-%s
> >> >   create 666 ubuntu ubuntu
> >> >   postrotate
> >> >   mv $1 /mnt/flume/clickstream/ 2>&1
> >> >   endscript
> >> > }
> >> >
> >> >
> >> > I run logrotate.d/clickstream.POST with the ~60MB file, and everything
> >> > looked fine....now I try a ~190MB file....
> >> >
> >> >
> >> >
> >> > ls -lrt /var/log/clickstream
> >> > /var/log/clickstream:
> >> > total 192336
> >> > -rw-rw-r-- 1 ubuntu ubuntu 196945008 Nov 20 16:42 clickstream.log
> >> >
> >> >
> >> > Run logrotate.d/clickstream.POST, and we see the WARNING in the FLume
> >> > log.
> >> >
> >> > ....
> >> > ....
> >> > 20 Nov 2012 16:45:07,117 WARN  [pool-13-thread-1]
> >> > (org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile:328)
>  -
> >> > Could not find file:
> >> > /mnt/flume/clickstream/clickstream.log-2012-11-20-1353429906
+
Dan Young 2012-11-20, 20:03
+
Brock Noland 2012-11-20, 20:06
+
Patrick Wendell 2012-11-23, 12:46
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB