Is it possible to change the location of packages for NuGet?

I have the following convention for most of my projects:

/src    /Solution.sln    /SolutionFolder        /Project1        /Project2        /etc../lib    /Moq        moq.dll        license.txt    /Yui-Compressor        yui.compressor.dll/tools    /ILMerge        ilmerge.exe

You'll notice that I do not keep external libraries inside the source folder. I'm also very interested in using NuGet but don't want these external libraries inside the source folder. Does NuGet have a setting to change the directory that all packages are loaded into?