As your TFS environment grows, as well as your software stack that’s built within it, it can be easy to overlook some pretty simple things to keep performance where it needs to be.
On occasion we had noticed that builds slowed down dramatically, resulting in cranky developers and dev managers. While space on the build drop location was tight (builds, unless aggressively managed will, like work, fill all the space / time available!), but that was a normal constraint, not something new. It took a while to figure out, but when I did it was obvious. In retrospect!.
Make sure to disable both real time and periodic virus scanning on your build drive (or at least folders). We had real time scanning disabled, but overlooked the periodic scans. At first glance, the periodic scans would seem to not be an issue since they occurred on a Sunday when there was minimal development (and thus builds) happening; however, the massive amount of space, and the huge number of files involved in our build drops meant that scanning continued from Sunday well into the work week! Note that I’m talking about the build server itself, not the drop location, but the same rule really applies there too.
The only way to identify this issue was to watch the Performance Monitor and catch that the MSMPENG process had a ton of open file handles to build files. The CPU and disk overheard was initially lost in the “noise” of many simultaneous builds.
Hope this helps someone!