_DllMain@12 already defined

We recently faced this linkage error:

error LNK2005: _DllMain@12 already defined in MSVCRT.lib(dllmain.obj)

Searching gives ~36K results as of July 2012, many of which seem high quality (StackOverflow, MS, CodeProject etc.), and I was certain it would be a simple matter of finding a fix online and blindly applying it. However it seems the root cause in our particular case wasn’t covered yet (AFAIK), and it seems worthwhile to document.

The MS KB article teaches that this is a linkage order problem – MFC libs must be linked before the CRT ones – but none of the fixes the article proposes worked. We did have one build configuration which was successful and one which failed with the above LNK2005 (Release – but it really doesn’t matter) so I dumped two /VERBOSE linker outputs for the two configurations and diffed them. After some admittedly tedious inspection, an interesting difference came up – these lines were dumped only in the successful build:

Found __afxForceUSRDLL

Referenced in Stdafx.obj
Loaded mfcs100d.lib(dllmodul.obj)

The symbol name implies that it is intended to force some linkage, and including it seems to have the beneficial effect of loading the mfc lib mfcs100d.lib.  Indeed, searching reveals the following lines in dllmodul.cpp:

#ifdef _X86_
extern "C" { int _afxForceUSRDLL; }
#else
extern "C" { int __afxForceUSRDLL; }
#endif

and the following in afx.h:

// force inclusion of DLLMODUL.OBJ for _USRDLL
#ifdef _USRDLL
#pragma comment(linker, "/include:__afxForceUSRDLL")
#endif

So it turns out there’s a single condition that governs the linkage to the MFC library mfcs100/d (the one containing DllModul.obj, which exports _afxForceUSRDLL), and that condition is – _USRDLL being defined.   Our linking project was indeed a dll and somehow the default _USRDLL preprocessor macro was missing from it – restoring the definition fixed the linkage.

So bottom line, if you get a ‘DllMain@12 already defined’ linkage error for a dll, here’s another thing to try: make sure _USRDLL is defined in your project C++ property sheets.

On AFX_MODULE_STATE, Appendix

My recent AFX_MODULE_STATE plunge was driven by some real world problems. In parallel to the write-up I mailed the current MFC chief maintainer, Pat Brenner – who directly invited such user interaction.

The correspondence that ensued makes semi-official claims that aren’t documented elsewhere. I got Pat’s approval to blog it – and here it is, almost verbatim (slight abridges to keep to the technical stuff).  

Thanks a lot Pat!

 

From: Me
To: Pat
Subject: MFC Module States Question

suppose a dll calls back into an executable, as described here. If the callback uses AFX_MANAGE_STATE(AfxGetStaticModuleState()) the program asserts, and doesn’t find the (exe’s) resource handle. The solution that worked for me – which I suggested at that forum – is to use instead the undocumented AFX_MANAGE_STATE(AfxGetAppModuleState()).

Next, suppose some functionality that consumes resources is written in a static lib, and linked into both an executable and a dll (of the same application). Now neither Static nor App module states can work for both cases…

As a user I would expect that AfxGetStaticModuleState would have identical semantics when called from a dll and from an executable (that’s how I view the ‘Static’ description). Digging around the MFC sources I have a conjectured reason why it isn’t so:

RawDllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID)

includes the lines –
// set module state before initialization
_AFX_THREAD_STATE* pState = AfxGetThreadState();
pState->m_pPrevModuleState = AfxSetModuleState(&afxModuleState);

If similar statements would have run somewhere during the exe module initialization, AFX_MANAGE_STATE(AfxGetStaticModuleState) would have worked equally well there too.

-Ofek

From: Pat
To: Me

…I’ve checked with some other knowledgeable MFC folks and here’s the upshot.

AfxGetStaticModuleState is meant to be used ONLY by DLL functions, and NOT by the main executable. That function will return a NULL pointer when called from the Exe. AfxGetAppModuleState is not documented, and should be, as it is the correct function to call when another MFC module may call back into the exe. There are some very old KB articles and other things that mention this. Normally, the use of the “App” version would not be needed; but when another MFC module has called back into the app (rather than just returning to the app) like this, it would be appropriate to call it to fix the module state. So in your scenario of a static library, you will need to #ifdef your code (just as MFC does for DLL/static) so your code calls the right method.

We will get documentation for AfxGetAppModuleState written so it is no longer undocumented.

Pat Brenner

From: Me
To: Pat


The problem with the #ifdef solution for a static library, is that during the static lib generation there’s no general way to tell whether the resulting lib would be linked into an exe or into a dll. This is not (as far as I understand) the MFC scenario where the same source code can form either an exe or a dll – my source generates a lib either way.
The root issue that I don’t understand is why the need for two different module-state accessors (AfxGetStaticModuleState & AfxGetAppModuleState). As noted, it seems that the code line that makes AfxGetStaticModuleState work, is the line from RawDllMain:
AfxSetModuleState(&afxModuleState);
If a similar line would be called somewhere along the initialization of an exe module (maybe just somewhere more general, say in the initialization of a WinApp), it seems the ..GetStatic.. api would work for both dll’s and exe’s.
Was that a deliberate decision to separate the API’s? is there some design consideration or technical limitation i’m missing?

From: Pat
To: Me
Subject: MFC Module States Question

Hello Ofek,

I’ve checked with another MFC expert, and here is our opinion:

We don’t think there’s a one-size-fits-all solution here.  Changing things at the level of RawDllMain can be very complicated, and have unexpected downstream effects.  Your statement contains the word “seems” and that often gets me into trouble, given the life-span of MFC and the applications written with it.  A change like this might pass all my regression tests and then break some customer after we ship the change.  Additionally, I don’t believe you should call either of the functions when linking into an MFC extension (rather than regular) DLL.  Or maybe one is OK, but the other not—I would have to investigate to know for sure.

The bottom line is that static libraries are probably overstepping their bounds when doing anything that affects module state.  As you said, the static library code doesn’t know the type of module it is linked to, so that makes any module state manipulation dangerous.

On AFX_MODULE_STATE, or – Avoiding afxCurrentInstanceHandle Asserts, Part 2

(Part 1.)

The AFX_MANAGE_STATE macro is an iceberg-tip of some heavy machinery, whose documentation leaves much to be desired. Here’s an attempt to fill in some more blanks.

Mid-Depth Dive

Answering a question raised at part 1, the basic statement -

AFX_MANAGE_STATE(AfxGetStaticModuleState())

couldn’t be made simpler, simply because a broad statement like ‘always use resources from the last called-into user-module’ just won’t hold. You must leave a way for a developer to say stuff like ‘this GUI call, in that DLL, needs to use resources from my main executable’. So some coding must be done, one way or another.

So what alternatives are there to AfxGetStaticModuleState? Basically there are only 4 available module state accessors:

  1. AfxGetModuleState()
  2. AfxGetAppModuleState()
  3. AfxGetStaticModuleState()
  4. AfxGetOleModuleState()

AfxGetOleModuleState() accesses MFC-internal resources, and isn’t really useful to users.

AfxGetStaticModuleState() accesses a per-dll module state. It technically returns the static variable -

_AFX_DLL_MODULE_STATE afxModuleState

- defined in dllmodul.cpp. This file is built into the static-lib component of MFC (such a component exists even when you link against MFC dynamically!), and thus the variable afxModuleState is instantiated separately when every dll is loaded. Hence, this is the module state that you want to switch to and back from (via the AFX_MANAGE_STATE macro), when you need to access dll-specific resources.

AfxGetAppModuleState() accesses a per-executable module state. Technically it returns _afxBaseModuleState, defined as -

PROCESS_LOCAL(_AFX_BASE_MODULE_STATE, _afxBaseModuleState)

- in afxstate.cpp. There’s really nothing ‘process local’ in the PROCESS_LOCAL macro, but since afxstate.cpp is built into mfc[ver][u][d].dll, it is indeed a process-wide singleton (provided that all the user dll’s link to MFC dynamically, and against the same MFC version, as they should).

AfxGetModuleState() accesses the currently active module state. This is actually a tricky concept: the currently active module state is managed per-thread, as the member m_pModuleState in the aptly named _AFX_THREAD_STATE type. The AfxGetModuleState implementation falls back essentially to AfxGetAppModuleState if the thread’s ‘current’ module state is null.

Advanced Scenarios

Suppose a dll calls back into the executable, and this callback needs to access exe resources:

// MyDll.cpp:
void DllFunc( tCallback cb ) { cb(); }

// MyExe.cpp:
typedef void (*tCallback)();
void MyCallback()
{
// What do you declare here to access the proper resource handle?
MyDialog dlg; // this dialog template is an *exe* resource
dlg.doModal();
}

int main()
{
DllFunc(MyCallback);
return 0;
}

As noted the only documented module-state management facility is AFX_MANAGE_STATE(AfxGetStaticModuleState()). One would hope that ‘the root executable’ can be a valid ‘StaticModuleState’ but it turns out that the implementation treats dll’s and exe’s differently and this facility doesn’t hold in this case. As I answered on SO a while ago, the undocumented AfxGetAppModuleState gets the job done here, and the right code is -

void MyCallback()
{
AFX_MANAGE_STATE( AfxGetAppModuleState() );
MyDialog dlg;
dlg.doModal();
}

One can easily form even more exotic scenarios: what if dll1 calls into dll2 which calls into dll3, and this topmost function needs to utilize resources from dll1? What if a function using resources is implemented once in a static lib, linked against both an exe and a dll and is called from both?

There are no off-shelf facilities for such cases, I imagine mostly because it’s very non-obvious what is the expected behaviour for them. Using the ready MFC-generated module states and accessors already mentioned one can generally tailor the module-state behaviour to one’s needs. If worst comes to worst you can technically create and manage your own module states, but I have yet to see a situation where AFX_MANAGE_STATE with AfxGetStatic/AppModuleStates does not suffice.

Final Thoughts

Afx module states are just complex. Much of the complexity stems from the real complexity of the tasks at hand: the already-formidable task of masking away resource-handle management must be implemented to support contexts of executables, dll’s and static libs, all linked either dynamically or statically against MFC.

Still, the documentation is badly outdated. (the main technical article refers to MFC 3 and a windows 3.1 relic called Win32s), and the actual initialization of potential module states and thread states seems much more complicated than it needs to be. I have this nagging fear that even MS engineers tread very carefully around the darker corners of this code.

On AFX_MODULE_STATE, or – Avoiding afxCurrentInstanceHandle Asserts, Part 1

All too often I see one of these asserts fire:

ASSERT(afxCurrentInstanceHandle != NULL)
ASSERT(afxCurrentResourceHandle != NULL);
ASSERT(afxCurrentAppName != NULL);

These are coded in afxwin1.inl, and are part of the MFC high level accessors:

AfxGetInstanceHandle()
AfxGetResourceHandle()
AfxGetAppName()

If these asserts fire for you too, there’s a good chance you’re missing some afx module state management code.

Module States – Purpose

Most Win32 GUI functionality accesses resources in one way or another, and thus requires a resource handle. For example, MFC’s CDialog::DoModal() includes essentially the following calls:

…
hInst = AfxFindResourceHandle(m_lpszTemplateName, RT_DIALOG);
HRSRC hResource = ::FindResource(hInst, m_lpszTemplateName, RT_DIALOG);
hDialogTemplate = LoadResource(hInst, hResource);
…

Where hInst is an HINSTANCE obtained from either the currently running exe or the currently running dll.

Which poses the problem: which one?

I.e., any binary in the address space can contain resources. Should the dialog-creation code search the dialog resource at the executable, or one of the dll’s? How would the code know?

A user of native Win32 API is on his own here, and must explicitly create resource handles from loaded binaries, manage their lifetime and choose the proper one when actual resources are required. MFC pioneers had the noble intention of masking this madness from their users, and thus the AFX_MODULE_STATE apparatus was born: a module state is – roughly put – a wrapper around all binary-specific-data (e.g. resource handle) that is to be switched upon any call from one binary into another.

Note that module states include much more than resource handles (check afxstate_.h and see), but resource management does seem to be the canonical usage – actually the only usage for me personally.

Module States – Basic Usage

Module-data management wasn’t thoroughly hidden away from the developer, but the coding involved was indeed reduced to a minimum. The most succinct description I know is part of the wizard-generated code for an MFC dll:

//TODO: If this DLL is dynamically linked against the MFC DLLs,
// any functions exported from this DLL which call into
// MFC
must have the AFX_MANAGE_STATE macro added at the
// very beginning of the function.
//
// For example:
//
// extern “C” BOOL PASCAL EXPORT ExportedFunction()
// {
// AFX_MANAGE_STATE(AfxGetStaticModuleState());
// // normal function body here
// }
//
// It is very important that this macro appear in each
// function, prior to any calls into MFC. This means that
// it must appear as the first statement within the
// function, even before any object variable declarations
// as their constructors may generate calls into the MFC
// DLL.
//
// Please see MFC Technical Notes 33 and 58 for additional
// details.

So it boils down to a single declaration at the start of a function:

AFX_MANAGE_STATE(AfxGetStaticModuleState());

This is classical RAII usage: the macro generates an object, whose ctor and dtor perform substitution of the module state into a specified slot (more on this slot in later posts). The resulting code is not only compact – albeit admittedly cryptic – but also exception safe.

‘Functions which call into MFC’ is a bit of a broad phrasing (surely CString usage isn’t considered ‘calling into MFC’). I understand this wording as ‘functions which make GUI calls’.

This single line of code is the only documented way of managing state, which makes one wonder why AfxGetStaticModuleState wasn’t baked into the AFX_MANAGE_STATE macro in the first place. More on this – in the next post.

Three F-keys Gotchas

We recently did a small internal app, that had to use all 12 F-keys – which turned out to be surprisingly cumbersome. I hadn’t found this stuff concentrated on a single place, and certainly it could have saved me some trouble.

F1 Gotcha

Quote:

On Win32 systems, the operating system will generate the WM_HELP message when F1 is pressed.”

This isn’t much trouble if you don’t handle the message, as VK_F1 key message is still being sent. On MFC (and other) wizard generated apps, you might have to explicitly disable OnHelp on message maps:

BEGIN_MESSAGE_MAP(CMyWinApp, CWinApp)
…
  //ON_COMMAND(ID_HELP, CWinApp::OnHelp)   ! comment this
…
END_MESSAGE_MAP()

F10 Gotcha

Quote:

If the F10 key is pressed, the DefWindowProc function sets an internal flag. When DefWindowProc receives the WM_KEYUP message, the function checks whether the internal flag is set and, if so, sends a WM_SYSCOMMAND message to the top-level window. The WM_SYSCOMMAND parameter of the message is set to SC_KEYMENU.

So to get proper VK_F10 notifications, you can either bypass DefWindowProc completely – which is unfeasible, or handle specifically the WM_SYSCOMAND message. On MFC apps, that amounts to something like:

void MyWnd::OnSysCommand( UINT nID, LPARAM lParam )
{
  if(nID == SC_KEYMENU) // F10 pressed
    ::SendMessage(m_Child->GetHwnd(), WM_KEYDOWN, VK_F10, NULL);
    // The NULL in LPARAM is kinda sloppy. If you use key-message nuances, invest here a bit further.
  else
    __super::OnSysCommand(nID, lParam);
}

F12 Gotcha

This one is the most obscure – on some developer machines, pressing F12 seems to give a weird error message:

…This may be due to a corruption of the heap…  This may also be due to the user pressing F12…

To cut a long search short, this is a built in Win32 debugging feature. It can indeed be disabled through this registry key:

HKLM\Software\Microsoft\Windows NT\CurrentVersion\AeDebug\UserDebuggerHotkey

But contrary to what the connect page says, I wouldn’t advise to change it into just any nonzero value: this value is the scan code for the key that that would force a breakpoint!  If, for example, you change it to 0×9, you’d have the same problem while using the Tab key (on standard keyboards). I suggest setting the value to 0xFF – looking at some scan code tables around, I saw none that map it to an actual key.

Of course this issue would never manifest itself on customer machines, but solving it can make dev work much easier.

Afterthought

It seems an odd design choice to bake something as basic as F-keys handling so deep into the OS. I’m guessing this was done back when the wildest applications imaginable were word processors and spreadsheets, and the main design goals were to save what was perceived as boilerplate code rather than allow for flexibility. I still think modern app wizards should generate code that lets you opt-in, rather than opt out of these old key handling mechanisms.

AfxIsValidAddress (and Others) Don’t Work as Advertised

MFC exposes a some memory debugging facilities such as AfxIsValidAddress, which (for debug builds) supposedly -

Tests any memory address to ensure that it is contained entirely within the program’s memory space.

Or does it?   AfxIsValidAddress only delegates the call to the undocumented ATL::AtlIsValidAddress, which reads:

// Verify that a pointer points to valid memory
inline BOOL AtlIsValidAddress(const void* p,
                  size_t nBytes, BOOL bReadWrite = TRUE)
{
      (bReadWrite);
      (nBytes);
      return (p != NULL);
}

The first two lines are just no-ops to silence compiler warnings about unused parameters. The promised verification that p is-contained-entirely-within-the-program’s-memory-space, amounts to the test that p is not null.

AfxIsValidString identically does not hold to its word and tests only if the input string is non-null:

inline BOOL AtlIsValidString(LPCSTR psz,
                          size_t nMaxLength = UINT_MAX)
{
      (nMaxLength);
      return (psz != NULL);
}

AfxAssertValidObject  is undocumented, and runs a bunch of redundant AfxIsValidAddress-es. CObject::AssertValid is documented (‘performs a validity check on this object by checking its internal state’!!), and is just as disappointing:

void CObject::AssertValid() const
{
     ASSERT(this != NULL);
}

The reason is probably that somewhere along the way MS realized there is no sane way to keep such promises.

There are several Win32 API similar in functionality: IsBadWritePtr, IsBadHugeWritePtr, IsBadReadPtr, IsBadHugeReadPtr, IsBadCodePtr, IsBadStringPtr. It has been known since at least 2004 that these functions are broken beyond repair and should never be used. The almighty Raymond Chen and Larry Osterman both discuss the reasons in detail, so just a short rehash: IsBad*Ptr all work by accessing the tested address and catching any thrown exceptions. Problem is that a certain few of these access violations (namely, those on stack guard pages) should never be caught – the OS uses them to properly enlarge thread stacks.  In the words of Michael Howard, who first realized this (or so I think – both Larry and Raymond attribute the CrashMyApplication nicknames to him):

You should also not catch all exceptions, but only types that you know about. Catching all exceptions is just as bad as using IsBad*Ptr.

I’m guessing that AfxIsValidAddress – and sisters – worked the same way, until someone realized they too were probably generating more debugging effort than they were saving. However, while the Win32 guys decided to leave their API semantics as are and clearly document them as obsolete and dangerous, the MFC guys turned their API into no-ops and did cleanup work that cannot be described as anything but sloppy. Not only is the documentation wrong, source comments are too:

// AfxIsValidAddress() returns TRUE if the passed parameter points
// to at least nBytes of accessible memory. If bReadWrite is TRUE,
// the memory must be writeable; if bReadWrite is FALSE, the memory
// may be const.

BOOL AFXAPI AfxIsValidAddress( ...

– and even the MFC sources themselves still contain many naive uses of these no-op tests (search around, you’ll find plenty).

Bonus: What If You Really Have To Test Memory for Validity?

This is after all not so far fetched. Years ago I had to work around a bug in nVidia GPU drivers, where occasionally some API (namely IDirect3DVolumeTexture9::LockBox, applied on a huuuuuuuge texture) returned success but gave a bogus memory buffer. The workaround was to VirtualQuery the address, and if not accessible – retry the process on a smaller volume texture. It went something like -

...
D3DLOCKED_BOX box;
HRESULT res = pVolumeTexture->LockBox(0, &box, NULL,
                                     D3DLOCK_DISCARD);
_ASSERT(res==D3D_OK  && box.pBits);
// this should have sufficed, but...

MEMORY_BASIC_INFORMATION meminf;
VirtualQuery(box.pBits, &meminf, sizeof(meminf));

BOOL  bOk = (meminf.State == MEM_COMMIT)  &&
      ( 0 !=  (meminf.Protect &
      ( PAGE_READWRITE | PAGE_WRITECOPY | PAGE_EXECUTE_READWRITE));

if (!bOk)
 //fallback
...

Of course VirtualQuery is innately sluggish, so this is completely unacceptable as mere parameter validation. Use this only when you know you have to, not just to be on the safe side.