Developing in-house game tools presents a myriad of debugging issues. You can’t always nail down bugs to reproducible steps (if you even have QA resources to concentrate on that). Frequently content creators will complain about rare issues that force them to reboot the tools or use bizarre workarounds then things go wrong. Remote debugging works in some of these scenarios, but is mainly useful for debugging crash bugs. Errant “drag and drop” or “click and drag” problems require sitting at the machine to properly deal with.
In these cases its handy to be able to deploy a debugger onto the user’s machine so you can dive in and see where your code is going wrong. To be successful at this you need a couple of components: the debug symbols from the compile, the source code, and a debugger.
On Windows the debugging symbols are separate files from the executables. PDB files contains the information debuggers need to map addresses of code and data in a running tool to the source code counterparts. In Visual Studio, PDBs are only generated in the Debug configuration by default, so assuming you distribute something like a Release build to your users you will need to turn on PDB generation in that configuration. Its under Linker… Debugging… Generate Debugging Info. Set it to Yes (/DEBUG). When you prepare and publish your tool set, make sure to include these PDBs with the executables (EXE and DLL files).
PDBs can get quite large, so it may be a good idea to not always pull down PDB files when users get the latest tools. Insomniac’s tools retrieval script has some command line flags to pull down PDBs and code only when we know we want to debug something on a user’s machine. Using -pdb will get the executables and PDBs, and -code will get the executables, PDBs, and source (all from a label populated when the tools executables were checked in).
Once you have the PDB and code on the machine you just need a Debugger to dig in with. On Windows you have a choice: Visual C++ Express Editions or WinDBG (from Debugging Tools for Windows). Both are free to install so you aren’t bending any license agreements here. Visual C++ should work pretty much like you expect on your development box, but can take a while to install and patch to the latest service pack. WinDBG on the other hand is very quick to install, but takes a little getting used to. Typically you must show the UI you want to use (Callstack, Memory, etc…), as well as potentially manually setting the PDB search path (from File… Symbol File Path). It’s a very different experience but its so quick to deploy it may be worth checking out.
RE: Windows debugging
Setting up a symbol server was one thing that saved me most time in these situations, when combined with a minidump to help determine crash problems the amount of time saved is incredible.
http://entland.homelinux.com/blog/2006/07/06/setting-up-a-symbol-server/
Mike
Right now I deal a lot with some code I’ve rigged to compile an assembly from a C# project in memory. I’m not an expert at most of this; if I’m using a dynamically compiled assembly, can I debug it?
For context, I have a C# tool that I am using from 3ds Max 2010, and I dynamically compile because 3ds Max grabs ahold of your .dll files and won’t release them – no unloading at the moment, unfortunately. So, I can either restart Max after every…single…change, or compile the assembly on the fly and use that. There are a few other options but they seem much ‘heavier’ workflow-wise than my setup.
The only negative so far is that I can’t debug it at all (not that I could, before…but if I could it would really be nice…)
Any advice or suggestions? I would really appreciate it!
@Alex:
Have you considered instead having your Max .dll just load your secondary .dll in separate App Domain, which can be closed and refreshed by the main .dll? That way, you can still have Visual Studio do your building for you (it’s a bit more efficient at it anyway) which will give you the .pdb for debugging. Any “attach to process” from visual studio should show you all App Domains loaded by that process, so you should be okay.
If you must dynamically compile code, have you tried using CompilerParameters.IncludeDebugInformation? I would think that would create a .pdb for you.
@Jeff:
Thanks for the suggestion. I will have to look that up – I’m not exactly sure how to have one .dll load another in a separate App Domain. So far I’ve always just used “Add Reference” in Visual Studio to use any .dll’s I’ve created. I will Google about for that later tonight.
I have turned on IncludeDebugInformation, then added a break to a few lines, followed by a throw(), then attached Visual Studio to the 3dsMax.exe process and ran my code – I’m afraid Visual Studio didn’t catch anything. I’ll try again if I can get the separate App Domain working.
Again, thanks for the reply – I appreciate it!