Archive for the ‘Tips & tricks’ Category

Missing diskspace in Linux

Today I had a problem with a server that had no more disk space. And I learned something new in the process.

df -h told me that 100% was in use. Which was about 29GB on that server.

But if I checked the root partition with du -shx / i got about 9GB of used space.

So off to checking where the space could have gone:

Inode usage

I know “du” does not take account for inodes, etc. But according to dumpe2fs /dev/sdx1 my Inode size * Inode count = about 700MB.
So that was not it.

“Hidden” directories under mountpoints

“du” will not see used space of files located in a path that is later mounted to another file-system. For example, if you have files in /home/ on your root partition, but has later mounted /home to its own partition. The files will be hidden behind the new mountpoint.

To be able to check these files without unmounting anything in use, I did the following:

mount --bind / /mnt
du -shx /mnt

If “du” would give me a different result now, I would have known that the files where hidden under one of my mountpoints. But sadly, they where not. I was starting to run out of options.

Deleted files

If a process opens a file, and then you delete it by rm thefile you will have the file deleted from the filesystem, but the inodes will not be freed before the process closes/releases the file. This is something I love about Linux/Posix systems, since that means that processes does not need to lock my files and I have full control over my files as opposed to other operating systems(windows). But I thought that when you delete a opened file, there is no easy way of knowing which deleted files are “held back” by processes. But there is!

lsof | grep deleted quickly gave me a list of files that has been deleted, but is held open by processes, and their size. In my case a deleted log file of 17GB in size was held by asterisk. So i reloaded the logger module of asterisk, and hey presto! The diskspace was available again.

Now only 9GB was “in use” according to df -h.

Following status of a file or command

A command most people know about in POSIX systems, is “tail”. A command showing you the ‘tail'(in most cases, the last lines) of a file or pipe. Or with the option -f, it actually monitors your files, and will show the newly appended lines live.

But a command that is not that much known is “watch”. Say someone is uploading a file to your FTP server, and you want to see the file grow in size. If you want to “monitor” how big the file has become without making a “bash loop” that clears the screen and runs the command again, often making a flickering output, you can easily just type:

watch -n 1 du -h

Watch will now every second run the command “du -sh” and display it on your screen, along with the current time.

Or if you want to look at a whole folder being uploaded to:

watch -n 1 ls -lah ~/upload

Would at all times show the latest directory information about your upload folder. Any files growing in size, or any new files appearing would be shown each second. This if of course just a few simple examples of how to use the tool “watch”.

Variable initialization in C

I’m feel that I am starting to get the hold of C programming. But every now and then, I get these “ahaaa” moments, that I am a bit embarrassed about. One of these “aha”‘s I experienced today is how/when C handles initialization of variables where you don’t explicitly initialize them yourself.

Consider this code:

int var1;
int main(char **argv, int argc) {
  static int var2;
  int var3;
  return 0;

Here i deliberately did not initialize any of the variables to make an example. The first two variables will automatically be initialized with 0 before your main() function is executed. The last variable lives in the stack and will not be initialized unless you explicitly do it yourself. So it will probably have a ‘random’ value. Thats kind of handy to know about ;) I feel like this probably is one of the first things you usually learn about C, but I had totally missed this. Nice to know. Also if you use Valgrind to check your code, it can be very good at following uninitialized stack variables. It will actually follow uninitialized memory bit-wise.

First I thought the memory was initialized to 0 at compile-time, but after some reading, I learnt that the compiler records the amount of uninitialized memory for global and static variables, and stores the amount of data required in the BSS segment of the program. This way the executable will not grow in size in line with the amount of uninitialized data. For this reason some people call the BSS segment for the “Better Save Space” segment.

X applications and Socks proxy via SSH

I have often been very thankful of people who blog/share about things they find out on their own, and want to help others find it out quicker, or just to make little known, but awesome, features more visible. So I thought, maybe I should start to write these small posts about stuff i find, or stuff I want others to know about. So here goes the first.

Remote X applications

Have you ever wanted to run an x application from a distant server, but don’t want to bother with installing VNC or installing anything at all? There is this little known feature in the standard OpenSSH client, which will proxy your X display connection. It’s as easy as running this from a terminal window:

ssh -X

Now, after you have authenticated, you can start x applications. For example xclock, or kdevelop or other nifty stuff, and it comes right to your local X windows.


Remote connections using SOCKS

When you develop against remote web services, people often restrict your access by your servers IP address. This way only your web server can connect to the web services. But sometimes the only documentation you get from them are the wsdl pages on their server, or maybe they even expect you to have a web-browser on your server to browse the documentation on their web services server. Then you have a problem right? And links/lynx isn’t always web 2.0 enough. Thankfully I recently found this nice Socks feature in OpenSSH. Just connect as follows:

ssh -D 8080

Now when you are logged in, you automatically have a Socks server running at Go to your favourite browser, and go to proxy settings, and enter port 8080 under the “Socks” configuration, and save. Now all your web-browsing is tunneled through your web server, and you can easily open the documentation you wanted.

If you have an application that does not know of Socks proxying, and still want to ‘tunnel’ your data through your remote server, you can of course always use the more known port forwarding functions in OpenSSH (-L, and -R).