Copying in a response I recently wrote for someone else.
On a CE device, most of the file system and the whole registry is in memory. If the memory gets reset, you lose your files and any new registry entries. The Application folder is in flash, so that will remain after a cold boot. Our remcapture tool is a good way to persist your app/files/registry entries. It lets you take a snapshot of your file system before you install your apps and files, then takes another snapshot again when you are done installing them, then it lets you build a package to put on the device to reinstall everything on a cold boot.
That's very useful, thanks!
For the benefit of anyone reading this in the future, I also discovered the /application directory is, as Dan says persistent. Loading files into /application/StartUp will ensure these programs are run after a boot.