Figured I should be a responsible IT person and update the program. After doing so I attempted to relaunch and got the two attached error messages.
This is running on Windows 7 Enterprise N, I do not care if I have to roll back to a different update of this program, or if you can make this new update work.
I restarted the server, and am installing a few more windows updates to see if it is a problem with the System.Data.SQLServerCE namespace.
Still me, I believe it is an issue that this update if using .Net 4.0 which houses that System.Data.SqlServerCE, installing .Net 4.0 should fix it, however it will also break my legacy scripts running on this server.
I was wrong, neither of those things worked.
This is the final and correct solution:
32bit, then 64bit version.
After this it should work.
Final solution was to install SQLServerCE V3.5 Service pack 1. Install the 32bit version then the 64bit version and it worked.