can I simply plug in a USB printer, load the software, and print?
Yes, if you use Win 2000 (or Win XP) but only a "definite maybe" if you use Win 9x. In that case you may need to download and install some updated software. Also, when you're shopping for a printer, be sure to check its manual to see what versions of Windows it supports.
Here's a good resource on how to get USB to work with Win9x:
Win 98 http://www.usbman.com/win98seusbguide.htm
Win 95 http://www.usbman.com/Win95%20USB%20Guide.htm