Although we’ve moved the crawlers and a large part of our workflow to a co-location facility, we still do some of our processing here at the office. So on a daily basis I hop on my bike and ride down the road (a little over a mile) to pick up the daily data dump. We do have a VPN to the data center, but the 100 gigabytes in a daily dump is more than we could download.
We have two USB hard drives that we use for this shuttle service. It didn’t take us long to learn that it’s best to have two of the same type of drive. That way all you need to carry back and forth is the drive itself. You can leave a USB cable and power supply at each location.
In the short time we’ve been doing this, we’ve tested a number of different drives. My favorite is the 500 GB Maxtor Personal Storage 3200. It has a very conveinent form factor (just a brick), the power supply is not a wall wart, and it has a standard USB 2.0 connector. We have used several of these around the office, and I have one at home. It’s been a reliable drive and a good performer. Unfortunately, you can’t get them anymore.
We picked up a couple of Iomega eGo 1TB Desktop Hard Drives on sale at Fry’s for about $160 each. Hard to argue with the price, and the form factor is nice. However, the drive made a lot of clicking noises, got very hot (uncomfortable to touch), and one of them failed outright. The other started giving errors and we ended up taking them both back. Considering my experience with the eGo, and also the experience I’ve had with the 160 GB unit I bought several years ago (slow, noisy, and hot), I would not recommend Iomega removable hard drives.
About two weeks ago we picked up two Seagate FreeAgent 1 TB (it’s a PDF) drives on sale at Fry’s for around $150 each. I’m not real wild about the form factor and the wall wart power supply, nor do I much care for the mini-USB connector. But the drives are quiet, fast, and reliable. They do get significantly hotter than the 500 GB Maxtors that I prefer, but not near as hot as the Iomega drives. We also have two of the 750 GB FreeAgent drives in the office for backups and haven’t had any problems with them.
So far, I haven’t encountered the mysterious drive removal problem with the FreeAgent drives. I’m beginning to think that permissions might have something to do with it. Let me explain why.
I edited the security properties on the FreeAgent drives (right-click on the drive, select Properties, and then click on the Security tab) to give Everyone full control. This allows me to attach the drive to any of our machines at the office or at the data center, and be able to add, copy, rename, delete, or otherwise manipulate files without trouble. Since I did that, I haven’t had any trouble removing the drive. I’ve yet to try this on the other drives I was having trouble with.