/var full on server
2007-12-25 8:46:00
ORIGINAL POST
==================================================================
>I'm having problems with a Sparc 2 acting as a server in an engineering dept.
>I'm getting the standard message on the console:
>
> /var: write failed, file system full
>
>Here's the appropriate line of the 'df' command:
>
>/dev/sd2g 282463 254231 0 100% /var
>
>Here's the du command:
>
>sona8:/var# du -s *
>1429 adm
>1 crash
>424 etc
>2097 log
>288 lost+found
>6 net
>7 preserve
>10380 spool
>3073 tmp
>715 yp
>sona8:/var# ls -a
>./ crash/ lost+found/ spool/
>../ etc/ net/ tmp/
>adm/ log/ preserve/ yp/
>sona8:/var#
>
>How does that add up to 254 MB? What's du not reporting?
>How do I find out what's hogging it?
==================================================================
ANSWER
==================================================================
The solution was (quoting from Bryan Curnutt):
"If a process has a file open (for read or write), and someone
uses "rm" to remove the file, the file will be gone from the
directory structure but the disk space won't be freed up until
the process dies or closes the file."
Sure enough, it turned out that a logged-in user was running
xvnews (beware!) and it went crazy, making a 200+ MB file in
/var/tmp. I had deleted the file (which is why du didn't report
it) but xvnews still had an open file descriptor, so the space
wasn't freed up until I killed the process.
You can get a report of all open files using "lsof":
>LSOF(8) MAINTENANCE COMMANDS LSOF(8)
>
>NAME
> lsof - list open files
>
>SYNOPSIS
> lsof [ -dhlLnNostTU ] [ -c c ] [ -i i ] [ -k k ] [ -p l ] [
> -u l ] names
>
>DESCRIPTION
> Lsof version 2.16 lists information about files opened by
> processes for the following operating systems:
>
> AIX 3.2 for the IBM RISC/System 6000
> Dynix 3.0.12 and 3.1.2 for the Sequent Symmetry S81
> EP/IX 1.4.3 for the CDC 4680
> ETAV (SYSV3.1 with Lachman TCP/IP 3.0) for the ETA-10P*
> HP-UX 7.x and 8.x for Hewlett Packard work stations
> IRIX 4.0.5 for the Silicon Graphics Indigo
> NeXTStep 2.1 and 3.0 for the NeXT Work Station
> SunOS 4.1.1, 4.1.2 and 4.1.3
>
>......
>
>AUTHORS
> Lsof was written by Victor A. Abell of the Purdue University
> Computing Center (PUCC), based on the fstat and ofiles pro-
> grams.....
>
>.......
>
> You should be able to ftp it from: cc.purdue.edu
Thanks to Dave Fetrow for the above tip on lsof.
Other comments made were that du doesn't include . files, but I
didn't have a chance to test that. Many people suggested that I
reboot (wasn't an option for me), or that perhaps an NFS mount
was taking place "over" existing files (wasn't the case).
==================================================================
CREDITS
==================================================================
Thanks to all who responded:
sckhoo@emtds1.nsc.com (Swee-Chuan Khoo)
Ric Anderson <ric@Artisoft.COM>
Dave Fetrow <fetrow@biostat.washington.edu>
"A. Bryan Curnutt" <bryan@Stoner.COM>
Claus Assmann <ca@informatik.uni-kiel.de>
vc@glass.geol.lsu.edu (Venkat Chalasani)
brw@hertz.njit.edu (Brian White)
hoogs@SynOptics.COM (Tim Hoogasian)
vina@hub.eng.wayne.edu (vina parikh)
allyn@bigbird.ca.boeing.com (Mark Allyn)
david.warm@fi.gs.com (David Warm)
Jerry de Raad <jerry@mafalda.univalle.edu.co>
pallas@oclc.org (George Pallas)
Pat Cain <pjc@denver.ssds.com>
weitzel@burke.com (David Weitzel)
dotty@elvis.tgivan.wimsey.bc.ca (Dotty Pon)
glenn@uniq.com.au (Glenn Satchell - Uniq Professional Services)
jna@concorde.com (J. Adams)
hkatz@nucmed.med.nyu.edu (Henry Katz)
sommer@vsun02.ag01.Kodak.com (Tilman Sommer)
epl@Kodak.COM (Gene Loriot (epl@Caps.Kodak.COM.))
don@mars.dgrc.doc.ca (Donald McLachlan)
pburyk@leis.leis.bellcore.com (Patrick Buryk)
kumeda@tds.com (Andy Kumeda)
cammaa@mcsunx.gs.com (Aaron Cammarata)
louis@meg.meg.saic.com (Dances on keyboards (Louis Brune))
poffen@San-Jose.ate.slb.com (Russ Poffenberger)
etnibsd!vsh@uunet.UU.NET (Steve Harris)
edk@mach10.utica1.kaman.com (Edward F Killian)
paulo@nova.dcc.unicamp.br (Paulo Licio de Geus)
alek@spatial.com (Alek O. Komarnitsky)
________________________________________________
Mark Thomas internet:Thomas@Sonatech.COM
Network Analyst AppleLink:Sonatech
Sonatech, Inc. AOL: Hackmaster
Comments
Got something to say?
You must be logged in to post a comment.
