1

I have a .NET 3.5 winforms app that calls a method in a VB6 ActiveX DLL, which brings up a dialog box. Everything worked fine for almost 10 years (first as a .NET 1.1 app, then 2.0, then 3.5, all on Windows XP). Now that the customer has upgraded to Windows 7, I am experiencing a System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

The structure of the call is like this:

.NET Code

try {
  VB6Object obj = new VB6Object();
  obj.DoStuff();
}
catch (Exception ex) {  // handle }

VB6 code in VB6Object class

Public Sub DoStuff()
    on error goto errHandle
    RetrieveData();

    Call frmPopup.Show(vbModal)

exitPoint:
    exit sub
errHandle: 
    ' handle error
End Sub

The problem starts in the VB6 code on this line: Call frmPopup.Show(vbModal). The error is propagated to the .NET code (error handlers in the VB6 code are completely ignored) where I handle the error. I thought the problem might be the Load event of the frmPopup, but the code does not ever reach the Load event (i have tracing statements all over the place, so that's how I know).

The interesting part is that the second time I make the call, it does not fail at all, but the form is missing a 3rd party grid (ActiveX DevExpress Quantum Grid), so I am assuming that the grid is probably not compatible somehow with Windows 7.

Everything that I have described happens when I simply run the application as an EXE or from VS2008 IDE, so the VB6 ActiveX DLL is simply a reference. If I use COM indirection and run the VB6 code in the IDE, everything works perfectly fine.

I've tried running things as an Administrator, but nothing changed.

Where can I go next to troubleshoot this issue?

4
  • have you tried to run in compatibility with older version of Windows? Right-click the executable and go to the Compatibility tab; sometimes that works like magic.
    – BeemerGuy
    Nov 19, 2010 at 1:01
  • @BeemerGuy, Yes. I forgot to mention that - yes, I did try the compatibility mode - it did not work. Nov 19, 2010 at 1:04
  • Can you reproduce the failure if you run the VB6 code from within a VB app? If so, you might be able to use the VB debugger to track it down more precisely.
    – Dan Bryant
    Nov 19, 2010 at 2:47
  • @Dan Everything works as expected if I run the ActiveX DLL within the VB IDE (as I mentioned using COM indirection). Nov 19, 2010 at 3:01

2 Answers 2

3

Turn DEP off. Older Ax Controls tend to use funky subclassing thunks (other than ATL/MFC ones) that no one cares to set the NX flag off.

1
  • Dang, you called it. What now? Is there an easy way to fix this or am I replacing a 10 year old grid with another 10 year old grid? Nov 25, 2010 at 0:53
2

Make sure you are compiling for the x86 target instead of anyCPU, especially if you run on 64-bit systems. There is some kind of conflict when you use anyCPU with an older 32-bit DLL on a 64-bit Windows 7 system, and, to quote the old IBM manuals, "the results are unpredictable."

2
  • The results are quite predictable. AnyCPU on a 64-bit Windows system creates a 64-bit process, and the 32-bit DLL then will not be loaded. The age of the 32-bit DLL matters not at all.
    – Ben Voigt
    Nov 19, 2010 at 6:21
  • Win7 is 32-bit in this case, so that's not the issue. Nov 19, 2010 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.