Traffic Shaping with CrashPlan
Posted on: Oct. 6, 2011, 5:09 p.m.
I wrote this guide for CrashPlan specifically, because it was the inspiration for configuring traffic shaping on my router but traffic shaping isn’t limited to CrashPlan, it has many other applications as well.
All of my important data is backed up using CrashPlan. To date, it is the only backup software I’ve been truly happy to use. I used to backup with their free account but earlier this year they updated their subscription plans making their cloud service much more affordable. Now I backup to CrashPlan Central (their cloud). Previously I used Time Machine, but I find CrashPlan to be superior because it supports OS X, Windows, and Linux. All three are Operating Systems I use on a daily basis, having the same backup program for all of them is more efficient and saves a lot of time on my part (not to mention the ability to send your data off-site).
My problem was that I had over 600GB of data that I needed to upload to CrashPlan Central (Hey, their CEO said he wanted my data! Ask the Code 42 CEO). If you’re not familiar with CrashPlan, all of the data you send whether to CrashPlan Central or another CrashPlan user is encrypted. My “high speed” internet connection is capped at a measly 1Mbps of upstream (any more than that costs an arm and a leg, but I digress). How could I upload all of that data without making my internet connection slow for months on end?
One thought (and Code 42 recommendation: Bandwidth Caps and You) is to cap the rate at which CrashPlan transmits data. I could configure CrashPlan to only send data at 30KiB/s, which under normal circumstances probably works well but I have hundreds of gigabytes spread across multiple computers. Either I have to let them seed one at a time (taking a month or more each) or I need to set the cap significantly lower so they can all seed together without putting a drag on my internet connection and making the seed process take much longer.
The solutions presented so far will work for the average user with less data to upload but for the rest of us they leave much to be desired. What about the other 70% of my upstream that remains un-utilized most of the day? If I could use all of my allocated bandwidth it would significantly reduce the time it takes to seed (and later send differentials). The problem with that is it makes latency sensitive applications (streaming, games, voip, etc.) almost unusable while making even normal browsing sluggish.
How could I get the best of both worlds? I want to use all of my upstream without introducing latency. That’s where Traffic Control (tc) comes in. I’m using CentOS on my router but if you have any recent Linux distribution it should already come with QoS support entirely enabled in the kernel (if not then enable it), IMQ device, HTB queue, iptables, and two ethernet cards (for NAT).
Once you have Linux installed, save the following iptables script as iptables.bsh in your root home directory (for now).
#!/bin/bash -x
IPTABLES=/sbin/iptables
EXTERNAL=eth0
INTERNAL=eth1
RATEUP=868
# reset/clear rules
$IPTABLES -F
tc qdisc del dev $EXTERNAL root
$IPTABLES -t mangle -D POSTROUTING -o $EXTERNAL -j BWSHAPER
$IPTABLES -t mangle -F BWSHAPER
$IPTABLES -t mangle -X BWSHAPER
# NAT
$IPTABLES -t nat -A POSTROUTING -o $EXTERNAL -j MASQUERADE
# this is where you would put your port forwarding rules
# bandwidth shaper
$IPTABLES -t mangle -N BWSHAPER
$IPTABLES -t mangle -A POSTROUTING -o $EXTERNAL -j BWSHAPER
$IPTABLES -t mangle -A BWSHAPER -p tcp -m length --length :64 -j MARK --set-mark 10
$IPTABLES -t mangle -A BWSHAPER -p tcp --dport 443 -m tos --tos 0x04 -j MARK --set-mark 12
$IPTABLES -t mangle -A BWSHAPER -p tcp --dport 4242 -m tos --tos 0x04 -j MARK --set-mark 12
$IPTABLES -t filter -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -t filter -A INPUT -p icmp -j ACCEPT
$IPTABLES -t filter -A INPUT -i lo -j ACCEPT
$IPTABLES -t filter -A INPUT -j REJECT
$IPTABLES -t filter -A FORWARD -i eth1 -o $EXTERNAL -j ACCEPT
$IPTABLES -t filter -A OUTPUT -j ACCEPT
$IPTABLES -t filter -A OUTPUT -o lo -j ACCEPT
$IPTABLES -t filter -A OUTPUT -o eth0 -j ACCEPT
tc qdisc add dev $EXTERNAL root handle 1: htb default 11
tc class add dev $EXTERNAL parent 1: classid 1:1 htb rate ${RATEUP}kbit ceil ${RATEUP}kbit
tc class add dev $EXTERNAL parent 1:1 classid 1:10 htb rate $[$RATEUP/4]kbit ceil ${RATEUP}kbit prio 0
tc class add dev $EXTERNAL parent 1:1 classid 1:11 htb rate $[$RATEUP/2]kbit ceil ${RATEUP}kbit prio 1
tc class add dev $EXTERNAL parent 1:1 classid 1:12 htb rate $[$RATEUP/4]kbit ceil ${RATEUP}kbit prio 2
tc qdisc add dev $EXTERNAL parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $EXTERNAL parent 1:11 handle 11: sfq perturb 11
tc qdisc add dev $EXTERNAL parent 1:12 handle 12: sfq perturb 12
tc filter add dev $EXTERNAL parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10
tc filter add dev $EXTERNAL parent 1:0 prio 0 protocol ip handle 11 fw flowid 1:11
tc filter add dev $EXTERNAL parent 1:0 prio 0 protocol ip handle 12 fw flowid 1:12First make sure the variables at the top of the script point to the correct network device names. EXTERNAL should be the public interface while INTERNAL should be the NIC connecting directly to your computer(s).
Next you will need to adjust the RATEUP variable to the amount of upstream bandwidth you have. You can use the speakeasy bandwidth test to get an idea of your actual upstream (don’t go by what your ISP says just in case your real upstream is slower).
The key here is to set the RATEUP variable low enough so that packets never get queued at your modem. We want to retain control over all of the packets because if they are getting bottlenecked somewhere beyond our router then we will be subject to that devices QoS or lack thereof. You may need to experiment with this value.
Now we skip down to the last 13 lines of iptables.bsh. I didn’t want to spend a lot of time fine-tuning my queues so I took the simplest approach by making only three queues.
queue 10: high priority queue with 1/4 of RATEUP reserved for it. All of my latency sensitive applications are configured to use this queue.
queue 11: medium/unclassified priority queue with 1/2 of RATEUP reserved for it. Any unclassified packets or packets marked for queue 11 go here.
queue 12: low priority queue with 1/4 of RATEUP reserved for it. This is the queue I have CrashPlan configured to use.
All of the queues can share unused portions of their reserved bandwidth with other queues (goes to higher priority queues first), meaning that if queue 10 was only 10% utilized but queue 12 has a transfer going then queue 10 would let queue 12 “borrow” the free portion of its reserved bandwidth. Borrowing would probably be the wrong term to use here since queue 12 doesn’t return the bandwidth (unless the situation was reversed) but that is the term the developers chose.
Now we have queues to control data flow but how does it know what data should go into which queue? That’s where iptables comes in. iptables has a “--set-mark” option that allows you to mark packets that match specific criteria.
Here’s the one we use to catch CrashPlan packets.
$IPTABLES -t mangle -A BWSHAPER -p tcp --dport 443 -m tos --tos 0x04 -j MARK --set-mark 12
There are two conditions this rule is searching for:
all packets with a destination port of 443
all packets with a TOS flag of 0x04
When the rule finds a packet that matches both of those conditions it is marked for queue 12. Port 443 is the destination port CrashPlan uses when connecting to CrashPlan Central (their subscription service). If you’re connecting to another CrashPlan user this port will need to be 4242 (I included a second rule for this). The TOS flag is set within CrashPlan itself. “Settings -> Network”. The option at the bottom that says “TCP packet TOS” is the option we want. I chose reliability (0x04) because I don’t have any other applications on my network that use 0x04. Don’t forget to set the TOS in CrashPlan or it will not match the rules we’ve specified.
Make the iptables script executable.
# chmod u+x /root/iptables.bsh
Execute iptables script.
# /root/iptables.bsh
At this point you should see all of the rules scroll by as they are added to iptables. If it is the first time you’ve run the script you will see it complain that the queue it tried to remove (in the reset/clear section) didn’t exist, which is to be expected.
You’re all set! You can monitor the queues by executing
# watch -n1 tc -s -d class show dev eth0
I use an old PC as my router, however you may be able to get the same results by using any of the Linux embedded device operating systems such as OpenWRT or Tomato but I’m not sure if they support rules based on TOS which is required since CrashPlan uses the standard SSL port when connecting to their subscription service; otherwise you would be limiting all of your SSL traffic in addition to CrashPlan.
Disclaimer: Some internet service providers impose transfer quotas. Keep that in mind when backing up data.
Happy backup shaping!
Comments
Rowsreatows said...
Get up to 100000 forum backlinks with our backlinks service & massive targeted traffic Get great web traffic using amazing backlink service today. We are able post your custom message up to 100’000 forums around the web, get insane amount of backlinks and incredible targeted web traffic in very short time. Most affordable and most powerful service for web traffic and backlinks in the world!!!! Your post will be published up to 100000 forums worldwide your website or blog will get instant traffic and massive increase in seo rankings just after few days or weeks so your site will get targeted long term traffic from search engines. Order now: <a href=http://xrumerservice.org>backlink service</a>