Comments on: Copying large files on Windows

By: Jason

Jason — Thu, 25 Dec 2008 20:34:12 +0000

“Every program starts running as though you’re emulating a Pentium on a 2 MHz 6502, using a cassette tape as virtual memory.”

Made me LOL.

By: Jim

Jim — Wed, 03 Dec 2008 23:10:39 +0000

As Roy Harvey pointed out, I’m using Windows Server 2008. This is supposed to be a server operating system, and one would expect it to handle large files without trouble.

Windows doesn’t refuse to copy the file. In fact, it copies the file just fine. The problem is that the operating system somehow decides that the file operation needs all of the physical memory, which pages every other process’s working set.

These types of problems aren’t limited to Windows or other closed-source solutions. I’ve run into plenty of similar limitations in open source tools. And it’s not like I can’t write or obtain a different copy utility that tells CopyFileEx not to buffer the file. It’s just that I shouldn’t have to.

By: Matt Simmons

Matt Simmons — Sun, 30 Nov 2008 20:44:26 +0000

Darrin’s point really highlights the strength of open source solutions. Limiting the size of files able to be copied to the amount of physical and virtual memory shouldn’t be considered a “low priority” bug.

Recommended memory for 2008 server is 2GB. You’d better not be copying any oracle datafiles on that machine.

Dumb arbitrary limits implemented by dumb programming.

By: Roy Harvey

Roy Harvey — Wed, 15 Oct 2008 22:28:53 +0000

Windows is not just a desktop OS. Windows Server is supposed to handle a diverse concurrent workload gracefully. I suspect that Jim is having the problem with one of the Server versions.

By: Darrin Chandler

Darrin Chandler — Wed, 15 Oct 2008 16:31:21 +0000

The problem hasn’t been addressed because it’s such a very low priority. These kinds of problems aren’t cropping up that much for ordinary users, and it’s not hurting sales figures at all. MS has other problems that are hurting sales, and they will be focused on those.

CopyFileEx probably works fine for typical daily use. The silly caching stuff may actually perform well in those common situations (files up to a certain size, copied to local/removable media). It’s a shame they didn’t put some sensible limits on it to keep it sane.

I’m afraid you’re going to keep hitting these sorts of problems using Windows in such unorthodox ways. Your app and its data are bound to be outside the limits (in several areas) for which Windows is optimized.