MB's answer is correct, you can format with either FAT32 or NTFS, its your choice. If you want the explanation on what all this means, read on;
Older versions of Windows, like WindowsME, Win98 and Win95, all use FAT32 and cannot read the newer NTFS file system. When Windows NT came out, it was mostly geared towards business users and servers, and needed a better file system than what FAT12/16/32 could provide. Microsoft then came out with NTFS (NT File System) to fix the shortcomings of the FAT file systems.
Windows NT, Windows 2000, Windows XP, Vista and now, Windows 7, all can read both FAT32 and NTFS. Linux can read FAT32, and I believe, with some distro's (but not all), can also write to NTFS disks. Mac's can read NTFS, but if you try to write to the drive, it must make changes to it which makes the file system unable to be written to on a Windows system. Mac's can read and write to a FAT32 file system with no problem.
So FAT32 is technically compatible with more systems. However, my personal choice would be to use NTFS, since I don't know anyone that has a Mac, nor do any of my computers, nor my friends / families computers, have an OS installed that cannot read NTFS. NTFS is a better file system overall, less prone to file damage, errors, and fragmentation. Its also slight faster, which is good if you are running applications directly from the drive.
I've got a couple of flash drives, one set up to be used on "working" systems, the other for 'crashed' systems. The 'crashed' flash drive is set up to be able to boot a computer, and run a full diagnostics suite, as well as install several versions of Windows, and even an Ubuntu linux distro, and has an array of common drivers.
The flash drive I have set up for 'working' systems is what you will be setting up. Its dead simple to do. Most of the work is already done by coders and made available for download. You can't install just any app on a flash drive, though. Applications these days are geared towards being fully installed on a system. This means, not only does it copy all its files to the computer's hard drive, it also makes changes to the registry, possibly creates files and saves temporary info in the %temp% path, and might even create a folder in your "my documents" area as a default path to save files, and a whole host of other changes, like adding "send to" entries, changing the "open with" entries, etc.. Some applications can be modified to keep all its data on one drive, and make no registry changes or system wide alterations. These are generally referred to as "portable apps". There are several good sites out there that specialize in free open source portable apps. A few links:
http://portableapps.com/
http://lifehacker.com/tag/portable-applications/
http://www.pendriveapps.com/software/portable-games/