Note that this is not a bug with the server, but with Empyrion 4.0 itself but it’s a big enough pain that might still help others.
Prior to 4.0 I never really had any problems playing. Since 4.0, I and many other players were unable to play anymore as hitting tab in game to access backpack, or selecting player options from the menu, or various other things would cause Empyrion to crash to desktop with an error about ‘graphics driver has stopped responding and started again’ or something along those lines.
None of the other suggestions I saw online worked until I tried this and it worked perfectly.
Basically go to where your Empyrion.exe file is stored. C:\Program Files (x86)\Steam\SteamApps\common\Empyrion - Galactic Survival
Create shortcut of EACLauncher.exe, right click it, select properties, in the “Target” field add a space after the final " and then type -force-d3d9. Click apply, and okay. Then open Empyrion from your EACLauncher.exe shortcut you just made and edited.
That solved the problem for me and now I have been able to play 4.0 again 