Changing a reg key on the device resolved the issue. Turns out WinCE 6 and 7 (both use.NET CF 3.5) limit the stack to grow only up to 32MB in debug mode unless the reg key is changed explicitly. See MS article below:
Managed application crashes with debugger attached on ARMv7
If your managed application consumes more than 32MB on an ARMv7 platform you may run in to problems when debugging with your application crashing. Typically the application runs fine when launched on the device, but when launched from the Visual Studio debugger the application will crash as soon as the stack consumption goes above 32MB. This hotfix describes the problem better than I can:
You do not need to apply the hotfix to WEC7 as PB will have already placed the fixed .netCF assemblies on to your image, however you will need to add the registry value to activate the fix:
Below is another VS 2008 patch that might come in handy also. But the above registry fix alone solved the issue for us.