First of all, Android is Linux. It's a custom Linux kernel with a special software layer on top. And the customizations to the kernel and the software layer sitting on top of it are pretty important for it to run on the rather specialized smartphone architecture.
Second, a kernel has to be compiled to run on a certain architecture. So you'd have to find a version of Linux compiled for an ARM processor. That's not much of a problem. Several distributions, including Ubuntu, have ARM kernels. But then you'd have to have kernel modules for the drivers of the smartphone hardware, which... isn't nearly as likely. And you'd probably need a specially-designed interface, since Gnome or KDE really wouldn't work on a 4-inch touchscreen. And all of that would also have to be specially compiled for the kernel. And that's before we start with the application software. And by the time you got all of that working... you'd essentially have Android.
And even if you could do all of that, you'd have to have some way of booting from the USB, which you can't since smartphone firmware boots the kernel directly, without any sort of bootloader like you have on a personal computer.
If you want Android to be more Linux-y -- whatever that means, since the Linux experience differs greatly based on the graphical interface (assuming you use one) -- you can install a terminal emulator and even the bash shell. If you play around with it a bit, you might be able to install some command-line Linux software. But even then, it wouldn't be terribly useful, given the limited ability to enter text on a smartphone.