The Apache Bandwidth module allows you to limit the allocated bandwidth on a per-VirtualHost basis, which is very useful on hosted scenarios (many sites on a single IP) or if you're simply hosting your own site at home.
First check if you have it installed already:
apachectl -M |grep mod_bw
Then, on CentOS/Ubuntu, install it by doing:
yum install mod_bw; apachectl -k graceful
sudo apt-get install libapache2-mod-bw; sudo a2enmod bw
Here's an example I successfully used to limit the available bandwidth on a named virtual host site:
Bandwidth all 20480
MinBandwidth all 500
LargeFileLimit * 1024 500
mod_bw vhost config.
Bandwidth all 20480 allocated 20Kb/s (20480/1024) to every user (it was a very fascist limit, but hey, I was in Italy!). Instead of "all", you may specify IP Addresses or subnets with an /XX notation and you can also declare multiple Bandwidth-entries (ie. BW localhost XX; BW 192.168.0.0/24 YY).
ForceBandWidthModule On forces the module to apply to every request (the man says mod_bw by default doesn't apply to every request), but I suppose this may increase the server load - just speculating here, so bear with me.
MinBandwidth all 500 allocated a minimum bandwidth of 500 bytes (0.48k!) to every user.
Both the previous commands, when applied, meant the 1st user would have a total of 20Kb/s, from the 2nd client on, the total Bandwidth would've been splitted (10k each), with a MinBandwidth for every user of 0.48k.
LargeFileLimit * 1024 500 is a limit on a per-file basis. Here I did limit ALL the files with an asterisk, but I coulda specified different specific extensions (i.e.: by specifying multiple LargeFileLimit-entries - ".avi", ".tbz2", ".mp3", etc.). 1024 specifies the single file size limit in Kbytes, the 500(!) specifies the maximum bandwidth available to filetypes in bytes/s.
A good alternative way to use mod_bw is to declare the "MinBandwidth -1" parameter, which means each user will have a top Bandwidth specified by the "Bandwidth" directive (ie.: everybody will have the same top bandwidth) like so:
Bandwidth all 20480
MinBandwidth all -1
MaxConnection all 8
In this example all woulda had a top limit of 20Kb/s, which sounds decisively less fascist!
I've also added a statement to limit the maximum number of connections. Please note this can be used only if you also declared a "Bandwidth" statement AND it must also be applied only to the same Bandwidth "XXX"-users (i.e.: don't use MaxConnection if you don't use Bandwidth). BTW, this should limit the maximum number of connections per IP to 8.
That said, mod_bw is a beautiful toy that allows you to set your bandwidth limits on a per customer-basis, and that's the best way (absolutely IMHO) to use it.
You need to first identify your "resource-hog" sites, study how much bandwidth per-user they need and, depending on how much bandwidth you have available, verify how it works under different scenarios, but this is also risky since how a sites develops is (generally) not easy to predict (and good values for today usage are not always good values for tomorrow scenarios).
That could represent a good intro, for more info, look at the man page: http://bwmod.sourceforge.net/files/mod_bw-0.7.txt