sizeup.sh for cleaning house… I mean cloud.

In a previous post, I described an experimental method of mounting S3 as a virtual file system within a cloud instance. I’m still in the process of doing spring cleaning… although fall is basically here… but anyway cleanliness is generally overrated until it comes to the idea of getting web files organized properly in the cloud. So before I take a shower this morning I think I’ll finish moving some static content into content distribution networked storage bit buckets.

#!/bin/bash
# Asher Bond 2010
# http://www.asherbond.com/blog/2010/09/23/sizeup-sh-for-cleaning-house-i-mean-cloud/
# sizeup.sh [dir]
# sizes up the present working directory or some other directory
# by summarizing the directories inside. I use this script to
# make sure my cloud compute instances are storing files properly
# in walrus and s3 filesystems or google storage instead of cluttering
# the compute instance's internal file systems

if [ "$#" -gt 0 ] ; then
        cd $1;
fi

find . -maxdepth 1 -type d -exec du -hs {} ;

A Content Distribution Network is often more efficient than one point of delivery such as a single or centralized web server / VPS.

[m-img src=”http://cdn.asherbond.com/public-domain/traditional-content-delivery-compared-to-content-distribution-network-diagram-by-kanoha-wikimedia-commons.png” alt=”try to store static content in a CDN for better content distribution at the edge” /]

Please note this is risky business down here
Here’s how a thrill seeker could try to move a whole bunch of stuff into a cdn real fast:

sizeup.sh
# oh wow what’s up here? that one is like fulla mp3s and videos
s3-mount.sh cdn.somedomain.com /mnt/cdn.somedomain.com
mv /var/some-directory/some-big-podcast-2010* /mnt/cdn.somedomain.com
echo “hey web master dude I just moved all ur files into the cdn, so update ur links.” | wall
# oh no I’m still the web master…
tar -cvf backup-in-case-my-links-dont-get-updated.tar /var/some-web-site
gzip *.tar
umount fuse
s3-mount.sh backups.somedomain.com /mnt/backups.somedomain.com
mv backup-in-case-my-links-dont-get-updated.tar.gz /mnt/backups.somedomain.com
cd /var/some-web-site
find . -type f -exec perl -p -i -e ‘s/some-old-links/some-new-cdn-links/g’ {} ;
# whoa i hope that worked… LOL!

Tagged with: , , , ,
Posted in cloud computing, designing scalable systems