Different people want to reduce the size of the Linux kernel for different reasons. This page holds a few use cases for a small Linux kernel, as a means of illustrating some of the requirements and possibilities for this "feature" of the kernel.
Linux can be used on some devices at the primary bootloader. The size requirements on the first-stage bootloader are more stringent than for the ultimate running operating system.
There are projects along these lines, such as the coreboot project, or petitboot
There is currently a migration from 8-bit micro-controllers to 32-bit micro-controllers in the deeply embedded space. These types of processors are used in industrial automation, sensors, infrastructure, and more… Some of these devices may need to be powered for a long time on a coin cell battery, or be powered by ambient energy.
emcraft makes boards with these types of processors
Also, see this article about using uClinux on Cortex-M3 devices: Practical Advice on Running uClinux on Cortex-M3/M4
A program may want to run at high performance entirely within the L2 cache of the processor it is on. If the combination of kernel and program is small enough, it can avoid accessing main memory entirely.
Here are some chips with on-chip memory that could run Linux with no external RAM: