A Day with VS11 Beta – part 1

A large chunk of our customers still use XP so we won’t be upgrading VS any time soon. Still, out of curiosity I spent some free time with the VS11 Beta and below are some bits and pieces I noticed that I (mostly) haven’t read elsewhere.

Bug Fixes

I’m sure there are plenty, but MS doesn’t make the full list public (please join me in trying to fix that). I can say that two minor bugs that bothered me are indeed solved and one isn’t. Also, MS finally solved a major-ish issue I reported that prevented using STL containers on aligned elements. This last fix opens up some serious potential optimizations – I’d finally be able to make our matrices 16-byte aligned and apply SSE to them.

UI enhancements

I agree that the colorless GUI and ALL-CAPS-TITLES do not increase productivity, and I hope the options dialog does become resizable one day, but by and large I don’t understand the heat of this discussion .  Maybe it’s the most heavily discussed aspect of every new version just because it’s the most widely accessible aspect. Maybe it’s just fertile ground for MS-bashers – covering the entire range from ‘why did they change the GUI, I can’t find anything now’ up to ‘they just slapped some new GUI around the same product’.

Here’s a thoughtful little addition to the UI: the Project Properties dialog now includes an ‘All Options’ sub-category, for most property categories (C/C++, Linker, etc.):

I didn’t think of it until after seeing this addition, but it entails some deep usability analysis. It probably seemed like a good UI practice to divide the jungle of compiler/linker switches into digestible-size categories, but in practice I often find myself wasting time looking for a single familiar option within inevitably vague categories.

They might have gotten a bit carried away, though.. :

Here’s another nice addition to the editor context menu:

Well, it will be nice when it actually works, anyway. In the mean time /showincludes does the work for me.

Editor enhancements

There are a ton of those – in syntax coloring, code completion, reference highlighting, code snippets and more, but I use the almighty Visual Assist extensively and the native IDE still has a way to go to catch up with it.

Debugger Enhancements

New debugging scenarios are now supported, and the visibility of different debugger configurations is vastly improved:

I didn’t experiment with any web/GPU debugging myself yet, but these do seem promising.

I do get a fair share of optimized builds debugging, and so had high hopes from the new advertised PDB improvements, that allow watching more variables and stepping into inlined functions in release builds.  So far, I have very little to write home about:

The vast majority of local variables (probably all those which are stored only in registers) are still un-watchable.  The ‘Variable is optimized away’ message is a nice direction, but still appears very sporadically. It doesn’t show in the screenshot, but I had no luck stepping into inlined functions too.

Next time…

The part I care most about are compiler enhancements, and specifically the auto vectorizer. More on that –in the coming post.

Windows Internals Course by Sela College

A word just for Israeli readers:

Sela College has agreed to open a Windows Internals Class in Ramat-Gan. It’s a 5 day course, every Sunday starting June 24.

It’s obviously based on the identically named book, and the syllabus is as in-depth as you can possibly get in a course. Planned instructor is internals veteran Noam Sheffer.

I consider this course a rare opportunity (no, really, next one is in 1.5Y if at all) for an assisted dive into some really deep water. I’m going, so are 6 of my colleagues, and if you’re interested – I recommend you join. Either online or otherwise.

Disclaimer: It was my request that started it and so I feel somewhat responsible and am trying to help spread the word. I’ve no affiliation whatsoever with Sela or any of their employees, and am genuinely thrilled about this course.

QuickTip: Toggle disassembly with Ctrl+F11

Short post today. For a long while I’ve been switching to disassembly with Alt+8, as appears under the Debug/Windows menu:

- and back with Ctrl+Alt+0, as appears under View menu:

A while ago I discovered there’s a more pleasant shortcut, usable for both directions: Ctrl+F11, which is the default assignment to the IDE command Debug.ToggleDisassembly.

Happy disassembling.

Viewing types, part 3: Exceptions

Last time a way was shown to use internal RTTI mechanics to view C++ type names without direct debugger aid. There is one place in particular where such type names carry substantial information, and that is c++ exceptions. These can also require some additional treatment.

Native SEH exceptions are characterized – and can be caught – by a code. In contrast, C++ exceptions are characterized and caught by a type. So if you are debugging already within a catch-block (which is after some hefty compiler and runtime machinery already kicked in), you already have a direct view of the caught type, and need no extra trickery.

Where things get delicate is when a C++ exception goes uncaught in any C++ catch block, and is either caught in an SEH __except clause or goes completely uncaught and (hopefully) included in a core dump for you to analyze.  C++ exceptions, which are implemented on top of SEH ones, all utilize the code E06D7363 (standing for Exception, ‘M’ ‘V’ ‘C’ in ascii) and so cannot be distinguished by code. Some low-level trickery is in order.

The Old New Recipe

Sometime in 2010 Raymond Chen posted a recipe on how to view types of such exceptions (i.e.: originating in VC++, viewed in a context of SEH). Briefly, once you obtained an EXCEPTION_RECORD, do as follows:

Take Parameter 2 [of the exception record, O.S.] and go to the fourth DWORD and treat it as a pointer. (On 64-bit systems, you have to add this value to the HINSTANCE passed as Parameter 3 to convert it to a pointer.)

Next, go to the second DWORD and treat it as a pointer. (Again, on 64-bit systems, it’s really an offset from the HINSTANCE.)

Next, go to the second DWORD and treat it as a pointer. (64-bit systems: you know the drill.)

Finally, skip over the first two void*s and the rest is the class name.

And here’s a shameless cut & paste of his diagram:

EXCEPTION_RECORD
+----------+
| E06D7363 |
+----------+
|  ~~~     |
+----------+
|* ~~~     |
+----------+
|* ~~~     |
+----------+
| 3 or 4   |
+----------+
|* ~~~     |
+----------+
|*Object   |
+----------+     +---+
|*       ------> |~~~|
+----------+     +---+
|*HINSTANCE|     |~~~|
+----------+     +---+
                 |~~~|
                 +---+    +---+
                 | -----> |~~~|
                 +---+    +---+    +---+
                          | -----> |~~~|
                          +---+    +---+    +----------+
                                   | -----> |*   ~~~   |
                                   +---+    +----------+
                                            |*   ~~~   |
                                            +----------+
                                            |Class name|
                                            +----------+

Decorate with type names

If you’re thinking there’s gotta be a better way, you’d be right. Dumping VC++ intrinsic compiler types by compiling an empty file with /d1reportAllClassLayout – as done for type analysis – reveals a few types that directly relate to exception handling:

class _s__CatchableType    size(28):
+---
0    | properties
4    | pType
8    | _PMD thisDisplacement
20    | sizeOrOffset
24    | copyFunction
+---
class _s__CatchableTypeArray    size(4):
+---
0    | nCatchableTypes
4    | arrayOfCatchableTypes
+---
class _s__ThrowInfo    size(16):
+---
0    | attributes
4    | pmfnUnwind
8    | pForwardCompat
12    | pCatchableTypeArray
+---

Fitting these into Raymond’s diagram gives -

EXCEPTION_RECORD
+----------+
| E06D7363 |
+----------+
|  ~~~     |
+----------+
|* ~~~     |
+----------+
|* ~~~     |
+----------+
| 3 or 4   |
+----------+
|* ~~~     |
+----------+
|*Object   |  _s__ThrowInfo 
+----------+     +---+
|*       ------> |~~~|
+----------+     +---+
                 |~~~|
                 +---+
                 |~~~|  _s__CatchableTypeArray
                 +---+    +---+
                 | -----> |~~~|    _s__CatchableType
                 +---+    +---+    +---+
                          | -----> |~~~|   _TypeDescriptor
                          +---+    +---+    +----------+
                                   | -----> |*   ~~~   |
                                   +---+    +----------+
                                     .      |*   ~~~   |
                                     .      +----------+
                                     .      |Class name|
                                            +----------+

Into the debugger

Now for a toy – but real – example in VC. Take this code:

__try
{
throw std::runtime_error("adsf"); // throw a C++ exception
}
__except (EXCEPTION_EXECUTE_HANDLER ) // catch as a native exception
{
__debugbreak(); // can we identify the C++ type of the thrown exception?
}

Here’s an initial inspection of the exception record in the watch window:

NumberParameters being 3 indicates that this is an x86 system. The interesting parts are the contents of ExceptionInformation. Forget about parameter 1 for a minute, and let’s inspect parameter 2 – which by the analysis above should be of type _s__ThrowInfo:

There seem to already exist multiple indications of the thrown C++ type name, as circled in red, with no need to go further down Raymond’s recipe. However this is only due to the fact that our toy code throws a standard type for which symbols are available. If such is the kind exception you’re facing – you can happily stop here. If you’re facing a custom type – follow this game a bit further.   For some reason VS properly recognizes the type of arrayOfCatchableTypes but does not expand it, so let’s do it ourselves:

And the name is available as part of the TypeDescriptor, as for a regular C++ type (bottom red rect). Undname.exe it – if the decorated name isn’t obvious enough – and view the type name as appeared in the source code.

Bonus: An Alternate (Arguably Better) Recipe

As Raymond notes, parameter 1 of the ExceptionInformation (red rect ‘1’ above) is usually a pointer to the C++ object being thrown. It might contain much more information that it’s type name (well, at least if it’s not a custom type it should be accessible to you), and as seen in the previous post – an object contains its own route to the TypeDescriptor and hence the type name.  Here’s how it looks in the watch window:

Raymond notes that parameter 1 usually points to the thrown object, but -

…sometimes there is some junk in front that you have to skip over. Once you figure out what it is, you can dump it. (I haven’t bothered trying to figure out exactly how much; I just dump bytes and figure out the correct start of the object by inspection.)

You can easily do that from within VS too: if the expression in the example

  (_s__RTTICompleteObjectLocator**)(*(int*)0x003bf75c-4)

(containing the direct parameter 1, 0x003bf75c) expands to rubbish, start increasing the pointer until you arrive at meaningful struct contents. This seems a faster way to get to the information of interest since you get to the thrown object and its type name in a single search.

Viewing types, part 2: the manual way

Compiling with /d1reportAllClassLayout, even an empty file, dumps many compiler-intrinsic types. Six of these are interesting in the present context:  _PMD, _TypeDescriptor, _s__RTTIBaseClassDescriptor2, _s__RTTIBaseClassArray, _s__RTTIClassHierarchyDescriptor, and _s__RTTICompleteObjectLocator.

Most of this machinery exists solely for multiple inheritance, and virtual inheritance in particular – which modern languages eschew and is considered (I hope) a painful legacy by all current C++ professionals, and so I won’t go further in. Go here for an excellent survey on these deep internals of multiple inheritance and RTTI.

Turns out that for viewing types in sane (== single inheritance) code, two of these intrinsic types suffice:

1> class _TypeDescriptor    size(8):
1>      +—
1>   0    | pVFTable
1>   4    | spare
1>   8    | name
1>      +—
1>
1> class _s__RTTICompleteObjectLocator    size(20):
1>      +—
1>   0    | signature
1>   4    | offset
1>   8    | cdOffset
1>  12    | pTypeDescriptor
1>  16    | pClassDescriptor
1>      +—

The gem mentioned above tells us that an RTTICompleteObjectLocator pointer is placed right before the vftable. (This struct is responsible for locating the adjoint vftable within a larger vftable in the case of multiple inheritence, where method dispatching requires adjustor thunks, dynamic-casts require traversal of the full inheritance tree and things get hairy in general.  Not another word on that, I swear).

RTTICompleteObjectLocator contains a pointer to a TypeDescriptor, which happily points to a name!   To view this in the debugger, take a typed pointer (say, pb -sticking with the setup from last time), and type at a watch window

(_s__RTTICompleteObjectLocator**)((int*)pb->__vfptr – 1)

Here’s what you’d see:

That is (naturally) a decorated name, so pop open a VS command prompt and call undname.exe with the 0×2000 flag (decipher types) and that name string, omitting the leading ‘.’ :

And now, for something completely useful.

All the above may seem like a very roundabout – albeit cool – way of seeing what the debugger already shows, but this does present a very useful trick:

What if you don’t have symbols?

It is very easy to recognize dispatching of virtual functions (call on a register argument), and typically a few instructions before that $ecx would be populated with the ‘this’ pointer.  Now, you can know the name of the type whose methods you call, even without symbols!   (assuming the code was built with RTTI on).

E.g., around this virtual method call:

Type this at a watch window:

(_s__RTTICompleteObjectLocator**)(*(uintptr_t*)$ecx-sizeof(int*))

(some complexity added to conform also to x64 systems, thanks @Martin Ridgers!), and you’ll be greeted be a magnificent view:

Viewing types, part 1: The Normal way

Suppose C derives from B, which derives from A:

class A
{
public:
virtual void A_virt() {}
};

class B : public A
{
public:
virtual void B_virt() {}
};

class C : public B
{
public:
virtual void C_virt() {}
};

Now create A-pointers which point to children types, B and C, and watch them in VS:

The types in square brackets are the most derived types of the referenced objects. How does the debugger know them?

It can’t be pdb-voodoo, as evidently dynamic types are runtime only info. This info has to lurk somewhere in memory accessible at run time to the debugger.

Now where would be a good place to store such info?  We (wearing compiler designer hats for a second) could embed type info into every object instance, but that would be a gross duplication. We need a place that holds info common to all instances of the same class.

Such as, say, a virtual table.

- which is exactly where compiler writers do put run-time-type-info (RTTI).

Evidence (1): Watch the type name at the __vfptr value (circled in red above).

Evidence (2): mess with the __vfptr directly. Change pb->__vfptr to contain C::vftable:

And watch pb’s type miraculously change to C in the watch window:

Evidence (3): delete all virtual methods from A, B, and C, and watch the most-derived-type info disappear.

Evidence (4): compile without RTTI,

and watch the most-derived-type data disappear again.

So the debugger uses RTTI information, that is somehow attached to the vtable, to display type information. If the debugger can, perhaps we can do it ourselves when we must?…