Question for GrayFace

The role-playing games (I-X) that started it all and the various spin-offs (including Dark Messiah).

Moderator: Moderators

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Question for GrayFace

Postby Emjayen » Jun 13 2016, 7:35

Would you happen to know what determines whether or not MM7 will use hardware acceleration for blitting in fullscreen/exclusive mode?

I'm working on a compatibility layer for MM7 (and potentially 6/8) by re-implementing the DDraw/D3D interfaces, however the game seemingly randomly switches between the two implementations and I've been unable to reproduce the environment that triggers it (I figured it would be based on the caps reported by DDraw (ie., IDirectDraw4::GetCaps) however even when expressing via the caps there's no support for blitting it [MM7] continues to attempt to use it)

To be clear, when in "hardware accelerated" mode MM7 uses Blt()/BltFast() elsewise it simply Lock()'s the backbuffer for all its 2D rendering.

(The regkey "2d accel off" seemingly has no affect)

Thanks.

User avatar
GrayFace
Dragon
Dragon
Posts: 1472
Joined: 29 Nov 2005

Re: Question for GrayFace

Postby GrayFace » Jun 13 2016, 9:12

Cool! I seem to have gathered an idea of how D3D things are done in M&M, but It's great that you're working on it.
The registry key is "Use D3D". In MM7 from Mok's patch its value is stored in a bool at 0xDF1A68. It also frequently checks 0xE31AF0, which contains a pointer to a structure only created in D3D mode.
My patches: MM6 MM7 MM8. MMExtension. Tools. Also, I love Knytt Stories and Knytt Underground. I'm also known as sergroj.

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 13 2016, 10:04

Thanks for the prompt reply :)

Do you know anything about the DDraw aspect though? As that's what's really causing some issues at the moment. The game uses DDraw for rendering all of the 2D/UI of which there's two implementations (as mentioned in my first post). It's very likely just some bool in a global data structure somewhere (just like the D3D bool) that selects between the two.

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 13 2016, 23:03

To add some more information:

When using MM7's software implementation of blitting, scene rendering looks like this: (taken from my tool's call tracing):

Code: Select all

<tid:6440> #1: IDirect3DDevice3->SetTexture
<tid:6440> #1: IDirect3DDevice3->DrawPrimitive
<tid:6440> #1: IDirect3DDevice3->SetRenderState
<tid:6440> #1: IDirect3DDevice3->EndScene
<tid:4916> #3: IDirectDrawSurface->IsLost
<tid:6440> #3: IDirectDrawSurface->IsLost
<tid:6440> #2: IDirectDrawSurface4->Lock -- Draws to the backbuffer, ie., overlays the UI on the finished 3D surface
<tid:6440> #2: IDirectDrawSurface4->Unlock
<tid:6440> #3: IDirectDrawSurface->Flip
<tid:6440> #11: IDirectDrawSurface->BltFast
<tid:6440> #5: IDirect3DViewport3->Clear2
<tid:6440> #1: IDirect3DDevice3->BeginScene
<tid:6440> #1: IDirect3DDevice3->SetRenderState


While the path when using hardware accelerated DDraw, it's:

Code: Select all

<tid:5604> #1: IDirect3DDevice3->EndScene
<tid:5604> #9: IDirectDrawSurface->Lock
<tid:5604> #3: IDirectDrawSurface->IsLost
<tid:5604> #9: IDirectDrawSurface->Unlock
<tid:5604> #2: IDirectDrawSurface4->BltFast -- Draws some part of the UI
<tid:4372> #11: IDirectDrawSurface->BltFast -- Draws some part of the UI
<tid:4372> #3: IDirectDrawSurface->Blt -- Draws the mouse-cusor
<tid:5604> #2: IDirectDrawSurface4->BltFast -- etc...
<tid:5604> #2: IDirectDrawSurface4->BltFast
<tid:5604> #2: IDirectDrawSurface4->BltFast
<tid:5604> #2: IDirectDrawSurface4->BltFast
<tid:5604> #3: IDirectDrawSurface->Flip
<tid:5604> #11: IDirectDrawSurface->BltFast
<tid:5604> #5: IDirect3DViewport3->Clear2
<tid:5604> #1: IDirect3DDevice3->BeginScene


Interestingly, when using the hardware accelerated implementation, the game uses a separate thread for rendering the cursor (no idea why) which causes significant performance issues resulting in tearing (manfiesting as "flickering" of the display and/or stuttering). The problem appears to stem from the synchronization between the two threads and from what I can gather via debugging, it may be some timing issue as the worker thread basically does the following:

Code: Select all

for(;;)
{
   Sleep(18); // (18 is hard-coded)

   EnterCriticalSection(&SomeDrawingCS);

   IDirectDrawSurface4->Blt(); // draw the cursor

   LeaveCriticalSection(&SomeDrawingCS);
}


While the main thread also takes the same critical-section before rendering.

A few other anomalies:
- The aforementioned worker thread (that renders the cursor), also copies the framebuffer/frontbuffer (ie., the screen) every frame to some system memory surface and I cannot fathom why (some progressive effect? screenshots? no clue)
- The worker thread isn't always created (atleast not when using the software DDraw path) and furthermore is entirely redundant (the thread can be suspended, if not holding SomeDrawingCS, with no perceivable effects (the cursor is rendered by D3D anyway))

User avatar
GrayFace
Dragon
Dragon
Posts: 1472
Joined: 29 Nov 2005

Re: Question for GrayFace

Postby GrayFace » Jun 14 2016, 18:51

Can your tool output calling addresses? Then I can see what's happening there.
Just in case, here's source code of a dll Bourn made based on his zdraw.dll for Heroes 3 that makes MM6 run in 32-bit mode: https://dl.dropboxusercontent.com/u/447 ... aw_mm6.rar

Emjayen wrote:- The worker thread isn't always created (atleast not when using the software DDraw path) and furthermore is entirely redundant (the thread can be suspended, if not holding SomeDrawingCS, with no perceivable effects (the cursor is rendered by D3D anyway))

What about item attached to cursor, also drawn anyway? That thread does some mouse handling for some reason.
My patches: MM6 MM7 MM8. MMExtension. Tools. Also, I love Knytt Stories and Knytt Underground. I'm also known as sergroj.

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 14 2016, 23:09

Sure, I've added return address outputing:

Code: Select all

<tid:6596> 0x4A3420 #9: IDirectDrawSurface->Lock
<tid:6596> 0x4A3D27 #3: IDirectDrawSurface->IsLost
<tid:6596> 0x4A7872 #9: IDirectDrawSurface->Unlock
<tid:6596> 0x4A7DE7 #2: IDirectDrawSurface4->BltFast
<tid:6596> 0x4A7E28 #2: IDirectDrawSurface4->BltFast
<tid:6596> 0x4A7E70 #2: IDirectDrawSurface4->BltFast
<tid:6596> 0x4A7EC1 #2: IDirectDrawSurface4->BltFast
<tid:6596> 0x4A7F0F #2: IDirectDrawSurface4->BltFast
<tid:5340> 0x46D485 #11: IDirectDrawSurface->BltFast
<tid:5340> 0x46D4BD #3: IDirectDrawSurface->Blt
<tid:6596> 0x4A09FE #3: IDirectDrawSurface->Flip


Unfortunately I can't get a trace for the software implementation since I can't coerce the game into using it (ergo, my current problem)

What about item attached to cursor, also drawn anyway? That thread does some mouse handling for some reason.

Correct, it's drawn by the main thread.

On a side note: one of my original goals apart from compatability was to enable arbitrary display resolutions, unfortunately it's clearly not going to be as simple as I thought: http://i.imgur.com/9tmuYKV.jpg (Although other 'features', such as full-screen anti-aliasing, improved texture sampling, etc are all easily supported as the backend is D3D9)

(Note: I expose a "virtual" primary surface/backbuffer the game can draw to (e.g, UI), and then composite it with the actual video output using a textured quad at the end of the draw cycle (ie., after the game calls EndScene()) -- thus, scaling the UI is not a problem at all since it's done in 3D hardware. This also allows me to set the video mode to 32bpp)

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 14 2016, 23:49

Here's a debug build of the tool itself: http://warcraftgaming.net/ddraw.dll

To install:
- Copy ddraw.dll to your game's installation folder
- Run MM7Setup and select "Might & Magic Compatability Layer" for the hardware device (windowed mode is not currently supported)

Note: No persistent changes are made to the game, ie., no files are modified in any way.

You may find the call tracing interesting; there will be a file written to called mmcl.log containing a dump of all DDraw/D3D calls made.

Naturally, as it's just a current build during development, it's full of bugs (hint: don't tab) however it's functionally correct in most ways. Also, if the cursor is very buggy, that's because the game is using it's hardware accelerated implementation of DDraw.

WARNING: The log file will get very large once you're in-game.

User avatar
GrayFace
Dragon
Dragon
Posts: 1472
Joined: 29 Nov 2005

Re: Question for GrayFace

Postby GrayFace » Jun 16 2016, 7:48

I tried it and it just crashes when I try to load the game. The crash is in IDirect3DViewport3->Release: https://dl.dropboxusercontent.com/u/447 ... p/mmcl.zip
Also, looks like you're using an old mm7.exe, return addresses in your list don't correspond to mm7.exe from my patch (which was taken in turn from Mok's patch).
My patches: MM6 MM7 MM8. MMExtension. Tools. Also, I love Knytt Stories and Knytt Underground. I'm also known as sergroj.

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 16 2016, 8:37

[ERROR (3)] Failed to allocate DxObject

Seems like you hit the pool size limit. Here's an updated build that shouldn't have the same issue: http://wikisend.com/download/288006/ddraw.dll

You're right, I've been testing against v1.1 to start with. Although originally I didn't expect any compatibility issues with your patch, I just did a test using v2.0 and it seems to have introduced a problem: Image

Appears to be an issue with the texture format -- any ideas?

Ck-NoSFeRaTU
Leprechaun
Leprechaun
Posts: 23
Joined: 22 Dec 2009
Location: Russia

Re: Question for GrayFace

Postby Ck-NoSFeRaTU » Jun 16 2016, 11:22

Emjayen wrote:On a side note: one of my original goals apart from compatability was to enable arbitrary display resolutions, unfortunately it's clearly not going to be as simple as I thought: http://i.imgur.com/9tmuYKV.jpg

Actually there had been few attempts of proper resolution changer for mm7.

First of all there was mm7restool which uses only hex patches of exe and you end up with something like this:
http://2.bp.blogspot.com/-zgFIgMoujBo/VmTkyXfyE1I/AAAAAAAAGdo/yCFnB_ktUY4/s1600/MM7%2B2015-12-06%2B10-42-43-88.png

After author of the mm7restool disappeared back in 2009 based on mm7restool patches I made test dll version for 1280x1024 only with more refined UI, in-house movies and so on...
It looked like this:
https://nosferatu.g0x.ru/pub/mm7/screenshots/1.jpg
https://nosferatu.g0x.ru/pub/mm7/screenshots/2.jpg
https://nosferatu.g0x.ru/pub/mm7/screenshots/3.jpg

With this approach there were two problems:
1) too much hassle to hunt all these constants of UI elements which were heavily optimized by compiller and duplicated in many places. If I remember correctly there were about 600 changes already...
2) not portable to mm6/8 as for each exe version it required to start almost from scratch.

So I started making new version from the scratch with the goal to use hooking of rendering pipeline as much as possible so it can be externally manipulated in realtime.
It used this project for hook wrapping.
But it never were never finished as more superior project emerged.

Are you going open source with your wrapper? If yes maybe we can cooperate on this front.

Emjayen wrote:I just did a test using v2.0 and it seems to have introduced a problem: http://i.imgur.com/dKTIjF9.png?1

Maybe NoBitmapsHwl related, anything change if you disable it?

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 18 2016, 3:36

Yeah, I've known about mm7restool for awhile -- very cool.

I'm curious as to how you managed to support 1280x1024 (or any arbitrary resolution) for the game's 3D rendering? The usage of pretransformed vertices, culling for 640x480, etc suggests it's quite hard-coded.

Are you going open source with your wrapper? If yes maybe we can cooperate on this front.

http://www.wikisend.com/download/584014/mmcl.zip -- It's just a hack at the moment to get it working, really needs a total rewrite.

Maybe NoBitmapsHwl related, anything change if you disable it?

Good point, and has resolved the issue with the static world textures however dynamic objects (NPCs, trees, etc) still exhibit the same issue.

User avatar
GrayFace
Dragon
Dragon
Posts: 1472
Joined: 29 Nov 2005

Re: Question for GrayFace

Postby GrayFace » Jun 18 2016, 19:53

Wow, I never thought with HWLs this issue wouldn't appear! It was one thing I encountered and couldn't solve back when I was trying to do 32-bit mode support. I'll see what causes the problem.
Your compatibility mode also messes with mouse position, when I run the game the party is constantly spinning (I'm playing with mouse look).
[edit] Figured it out. IDirectDrawSurface4->GetPixelFormat must return 16-bit color format. It's also important for movies and software rendering.
My patches: MM6 MM7 MM8. MMExtension. Tools. Also, I love Knytt Stories and Knytt Underground. I'm also known as sergroj.

Emjayen
Peasant
Peasant
Posts: 52
Joined: 01 Aug 2013
Location: Sydney, Australia
Contact:

Re: Question for GrayFace

Postby Emjayen » Jun 18 2016, 21:46

Your compatibility mode also messes with mouse position, when I run the game the party is constantly spinning (I'm playing with mouse look).

Interesting -- the mouse cursor has been a headache for awhile now, even though MMCL doesn't actually modify its behavior at all, it must be some indirect consequence, possibly due to assumptions made by the game about timing / somewhere I haven't correctly emulated DX's behaviour.

Originally while testing, when MM7 was using software 2D (before it magically decided to switch to hw), there were no issues with the cursor as the game let DDraw (ie., or, D3D via MMCL) handle the cursor. However in hw 2D, now that it's rendering the cursor itself / whatever else, it's just completely broken.

Based on the cursor behaviour I've observed, the game samples the mouse location (I suspect via DirectInput) and while using MMCL, it's only sampling at a fraction of the rate it should be (maybe due to some synchronization lock).

[edit] Figured it out. IDirectDrawSurface4->GetPixelFormat must return 16-bit color format. It's also important for movies and software rendering.

It does.

Code: Select all

DXAPI(GetPixelFormat)(DxSurface* pThis, LPDDPIXELFORMAT lpDDPixelFormat)
{
   TRANSLATE_INTERFACE(pThis);


   lpDDPixelFormat->dwSize = sizeof(DDPIXELFORMAT);
   lpDDPixelFormat->dwFlags = DDPF_RGB | DDPF_ALPHAPIXELS;
   lpDDPixelFormat->dwRGBBitCount = 16;
   lpDDPixelFormat->dwRGBAlphaBitMask = 0x8000;
   lpDDPixelFormat->dwRBitMask = 0x7c00;
   lpDDPixelFormat->dwGBitMask = 0x3e0;
   lpDDPixelFormat->dwBBitMask = 0x1F;

   return DD_OK;
}


Against 1.1, there's no major issues with rendering (movies, cutscenes, etc all work fine). Only a few problems related to my emulation of direct surface access (e.g, try bringing up the info. panel of an NPC via right-click) and some minor 3D bugs I suspect are related to float precision or some texturing problem (e.g, in the Dragon's Cave on Emerald Isle, you'll notice there's some tiny gaps where some of the polygons connect)

User avatar
GrayFace
Dragon
Dragon
Posts: 1472
Joined: 29 Nov 2005

Re: Question for GrayFace

Postby GrayFace » Jun 19 2016, 8:31

Ah,15-bit color. I should have added support for that, but never encountered it. 16-bit color mode is still better for all purposes.

BTW, with MMExtension and compatibility mode the game crashes immediately after launch.
My patches: MM6 MM7 MM8. MMExtension. Tools. Also, I love Knytt Stories and Knytt Underground. I'm also known as sergroj.


Return to “Might and Magic”

Who is online

Users browsing this forum: No registered users and 10 guests