Widescreen hack and some other fixes aka AiO Patch

Discuss Drakan: Order of the Flame with fellow players and post any technical problems here where an 'unofficial' support team will try and help you. Gameplay help questions can go here too.
User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

Pull request: fixed a broken texture (more specifically - broken alpha map) in the System database. ID of affected texture = 5208 ("Texture24").

The problem was found by unterbuchse while testing his .REC exporter, then I was able to confirm via alternate means that the database entry is in fact corrupt. RGB data was intact (which is why preview looked OK), but when viewed on a model or landscape, it was just a garbled mess, not unlike TV static.

This texture is only used by the medium LOD of the plate armor (both the SP and MP variants), which is probably why it went unnoticed for the last 19 years; especially since the LODs are now disabled by default :roll:

Download link for the fixed System.txd:
https://www.dropbox.com/s/ism0hau4oez3k ... m.txd?dl=0

---------------------------------------------------

EDIT:

2nd pull request: fixed the Alwarren water fountain "puzzle" not showing the water level rising - in fact, no water was visible in the pool at all, because the previous changes to layer visibility have completely messed up the fountain's water layers, making them effectively fully invisible.


EDIT (05/10/2018):

Oops, I just noticed that the "fix" for the water not showing, instead broke the killboxes on the flames.

I've replaced the file with the new version, which has both of those issues properly fixed now.

---------------------------------------------------

EDIT 2:

Just found a bug in dgVoodoo... fast videomemory access breaks when textures over a certain size are encountered.

I haven't established what the critical size is yet, but 2048x2048x32bit is certainly over that limit.
To replicate the problem, add any 2048x2048x32bit texture to any database, put it on the landscape, and load the map in Drakan.

It works perfectly fine in vanilla Drakan (no dgVoodoo), and also in dgVoodoo when fast videomemory access is disabled.
But enabling that option causes the texture to show up as black instead.
Strangely, the performance hit is about the same as when the texture(s) in question would have actually been visible... :?
Attachments
Alwarren.zip
(173.59 KiB) Downloaded 2975 times

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

Thanks! Added updated files to the ZIP. I can see how the texture you fixed in System.txd is messed up when I try to paint it, but can't figure out where the brokenness is encountered during normal gameplay. I did increase LODFactor and gave myself Plate Mail, then flew with the camera away. Should I see the garbled mess then? I must be missing something obvious.

Alwarren was quite troublesome to update with new visibility data using the editor. I took shortcuts navigating the level, so missed that fountain bug. I had no idea there's some trickery involved there. The first revision of the updated level had the bug where on some part of the level, sound changed in a way as if Rynn was in water, even though she was on the ground, due to some water layer being somewhere above. The troublesome water layer belongs to one of the waterfalls.

I've noticed that 2048x2048x32 textures eat RAM and VRAM pretty fast, probably faster than they should. Especially RAM goes quick (shouldn't textures be kept only in VRAM?). Despite the size, they seem to consume SIGNIFICANTLY LESS space on the disk. I always thought that such big textures should be used sparingly, but these readings still seem strange.

With dgVoodoo, resource consumption is even greater. I don't have any problems with Fast video memory access here though, all 26 textures I've replaced showed normally. Fast video memory access just helps with lag spikes when lens flares are on the screen. HD texture replacements shouldn't affect rendering speed either way, at least not on today's hardware. I haven't noticed any further performance degradation with texture replacements, just increased RAM/VRAM usage and loading times.

BTW, I updated the patch a little; FPS limiter now allows more precise limiting of frame rate. Previously, only full integers were accepted, now it's possible to enter values in decimal notation.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

UCyborg wrote: Mon Oct 29, 2018 1:37 amI can see how the texture you fixed in System.txd is messed up when I try to paint it, but can't figure out where the brokenness is encountered during normal gameplay.
It isn't, really - because of the LODs, the affected texture is rendered too small to actually make out any real detail.
Which is probably how it eluded detection for the past 20 years...

There wasn't really any reason to fix that texture - especially with LODs being disabled by default nowadays, too.
I just didn't like the idea of intentionally leaving in something that I know is broken; and it was a "5 minute fix" anyway.

UCyborg wrote: Mon Oct 29, 2018 1:37 amAlwarren was quite troublesome to update with new visibility data using the editor.
Speaking of the Editor: a lot more fixes have been added.
I'll upload the updated files when I release the next CP build; I expect to do that around late November or early December.

UCyborg wrote: Mon Oct 29, 2018 1:37 amWith dgVoodoo, resource consumption is even greater. I don't have any problems with Fast video memory access here though, all 26 textures I've replaced showed normally.
I'll try to consistently reproduce the issue when I can spare some time, and report back.
Maybe it was happening because I was using a pretty huge map to test it with (1024x1024 tiles; almost continuous layer coverage to boot) - but interestingly, that map was only using a total of 2 landscape textures: regular grass texture, and the affected huge texture.

UCyborg wrote: Mon Oct 29, 2018 1:37 amFast video memory access just helps with lag spikes when lens flares are on the screen.
LFE's were quite broken (again...) the last time I checked. Again, something I'll have to look into more carefully.


In other news: I've made a new patch DLL (DSOUND.DLL) with my post-AiO fixes.
As of right now, that only involves some changes to the dedicated server code - but in the near future it will also support redirection of the registry settings to a per-user .cfg file.
Another planned and very much needed functionality is some "helper code" for the patch installer - this is badly needed because of the limitations inherent in how Windows handles privilege elevation, so the installer is not running as the same user who's actually logged in; this breaks the per-user paths.

Also, I'll have a few more updated/fixed .lvl files for the original SP maps; Islands in particular.

Finally - something specific to the dedicated server: I'm also working on a NULL renderer for the dedicated server, so it will be able to run without actually requiring any graphics hardware.
An external "server console" GUI app will receive the redirected "stdio", while also serving as a (re)launcher when the server crashes.
In the future, it will be also used as a framework for the more complex addon functionality (eg. voting system).

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

I just didn't like the idea of intentionally leaving in something that I know is broken; and it was a "5 minute fix" anyway.

True.

Regarding dgVoodoo, can you try selecting "Direct3D 11 MS WARP (software)" for the "Output API" to see if you still get the texture problem then? Forgot to mention, fast video memory access also helps with FPS dip when debug messages are enabled. The reason it's disabled by default is that games must do things certain way for it to work.

Speaking of the Editor: a lot more fixes have been added.

Didn't you say you won't be messing with it again for the foreseeable future? :)

In other news: I've made a new patch DLL (DSOUND.DLL) with my post-AiO fixes.

Replacement DirectSound DLLs are frequently used to restore EAX effects and positional audio in games that refuse to do 3D sound mixing when no hardware accelerated sound buffers are detected. Drakan still tries to do 3D mixing regardless if you enable 3D sound buffers, many old games bother don't due to the fact that Windows XP and earlier have completely and utterly useless software DirectSound3D implementation; driver support is needed for positional sound to work at all unlike on Vista+, where software DirectSound3D implementation is fully functional.

Anyway, EAX is still missing. DSOAL was picked up by someone after being abandoned for several years and is currently probably the easiest solution if you don't already have Creative ALchemy or something similar.
Last edited by UCyborg on Mon Oct 29, 2018 8:39 pm, edited 1 time in total.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

UCyborg wrote: Mon Oct 29, 2018 7:02 pmRegarding dgVoodoo, can you try selecting "Direct3D 11 MS WARP (software)" for the "Output API" to see if you still get the texture problem then?
Ok, I'll try that.
But isn't that just the "software rasterizer" (or the 3D equivalent, whatever that is)?

UCyborg wrote: Mon Oct 29, 2018 7:02 pmDidn't you say you won't be messing with it again for the foreseeable future? :)
True.
I was quite convinced I got most of the really obnoxious issues weeded out - but then I tried using some other features which I have never used previously, and found those to be also broken...

In other words, the foreseeable future turned out to be far shorter than I expected it would be.

Anyway, while I was at it, I also threw in some unrelated fixes.
For example, the invisible textures are now once again visible in the 3D View.
Also I finally (and properly!) fixed the issue of blocking execution while the 3D view is idle.

UCyborg wrote: Mon Oct 29, 2018 7:02 pmReplacement DirectSound DLLs are frequently used to restore EAX effects and positional audio in games that refuse to do 3D sound mixing when no hardware accelerated sound buffers are detected. (...)
Um, ok. Not sure we're quite on the same page here...
My DSOUND.dll implements about as much of DirectSound functionality as your DINPUT.dll implements of DirectInput...

In other words, it's just a fake DLL that exists purely to provide me with an entry point to the Drakan.exe process.

I've also tested it, and it works OK with my Xonar soundcard, which emulates EAX by hooking DirectSound.

The only oddity is that KERNEL32.GetProcAddress fails when there are multiple DLLs of the same name loaded!
Even though it takes a handle, not a name, as the argument... thanks, MegaSh!t.

So my DSOUND.DLL also hooks GetProcAddress in KERNEL32, so it can return the correct values in this particular case.
Otherwise the Xonar drivers fail to load because of the failed GetProcAddress calls, and everything goes to Hell in a handcart.

BTW, I have no intention of actually doing anything with the sound interface.
It's way too far outside of my area of expertise, I don't have the time for it even if I wanted to, and it's a thing I'd rate as "least concern" as of right now.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

The problem with dsound.dll is; whatever your Xonar software does only applies to users of that sound card. I for example use Creative ALchemy, so I need to use its dsound.dll to have EAX. Many people who don't have fancy sound cards will choose to use DSOAL, which is free, open-source, not tied to any particular sound card and uses modified version of OpenAL Soft for sound output. It comes in the form of replacement dsound.dll file.

Yeah, dinput.dll also isn't the ideal way for hooking, there must be some use cases for intercepting DirectInput calls too, I remember some guy ranting why everyone uses dinput proxy DLLs as that prevent usage of real replacement dinput/dinput8 DLLs.

The only oddity is that KERNEL32.GetProcAddress fails when there are multiple DLLs of the same name loaded!

What are you talking about? Then how do all other proxy DLLs out there work successfully without hooking GetProcAddress? Seems like whatever your Xonar does is interfering. The usual method of hooking DirectSound to restore EAX and such is placing replacement dsound.dll in game folder.
Last edited by UCyborg on Mon Oct 29, 2018 9:10 pm, edited 2 times in total.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

Also I finally (and properly!) fixed the issue of blocking execution while the 3D view is idle.

So what trick did you use?

But isn't that just the "software rasterizer" (or the 3D equivalent, whatever that is)?

It's good to rule out driver problems.
Last edited by UCyborg on Mon Oct 29, 2018 9:14 pm, edited 1 time in total.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

UCyborg wrote: Mon Oct 29, 2018 8:56 pmYeah, dinput.dll also isn't the ideal way for hooking, there must be some use cases for intercepting DirectInput calls too, I remember some guy ranting why everyone uses dinput proxy DLLs as that prevent usage of real replacement dinput/dinput8 DLLs.
Originally I started out with using MPR.dll instead: it would have been perfect - except that it doesn't work in Win10.

The main reason why I didn't want to use DINPUT.dll (and then rename your DINPUT.dll to something else) is because I wanted to prevent people from accidentally breaking the Community Patch by manually reinstalling the AiO patch, which would then wipe out my DLL.
Then people would start complaining that it's broken... and even worse, they would be right.

DDRAW.dll was out of the question, because it's already used by dgVoodoo.
All of the other DLLs used directly by Drakan.exe are also off-limits, for the same reason as MPR.dll.

So... any other ideas, then?

Basically I need a solution that:
  1. Doesn't involve modifying the Drakan binary,
  2. Doesn't involve renaming or otherwise altering your DINPUT.dll,
  3. Will work reliably across all of the OSes supported by my patch (so XP...Win10),
  4. Allows executing the earliest user code not later than at the Drakan.exe entry point,
  5. Will not be broken by the user(s) manually reinstalling the AiO patch (which is effectively equal to point 1 AND point 2).
UCyborg wrote:So what trick did you use?
Oldest one in the book: Sleep(1).

The way your version of that "fix" works - well, let's just say that it was suboptimal at best, and caused some unexpected performance issues ("jarring"/skipping frames when resuming from idle state - as well as the unintended "instant stop" when releasing the movement keys).

Breaking up the spinwait with Sleep(1) makes everything run very smoothly, and CPU usage is down to near 0%.

What are you talking about? Then how do all other proxy DLLs out there work successfully without hooking GetProcAddress?
I have no idea what's going on, really.

The symptom was that Drakan was failing to load with my DSOUND.dll, even though everything was in order.
After digging around, I traced the problem down to GetProcAddress - it was returning failure (ERROR_PROC_NOT_FOUND) despite the proc in question being very much listed in the export table...

The failing call to GetProcAddress was issued by HsSrv.dll (a Xonar hooking DLL) - but there was nothing unusual about the call itself; all the arguments were correct. Yet it kept failing for no reason.

So naturally, the obvious solution was to hotpatch KERNEL32.GetProcAddress with a call to my code that first checked for the handle to itself, returning the correct values (or passing the call through to the real GetProcAddress, as appropriate) - and that fixed the problem entirely.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

Wait, so are you trying to tell me that those other soundcards instead require placing a fake dsound.dll in the game folder of each game that needs to have EAX "emulated"? Because that sounds crazy...

Yup. Even though I use ALchemy, I don't actually have Creative's sound card. I have Sound Blaster X-Fi MB3, which is basically software based virtual sound card (it's added as another sound device in Windows). It lets you, among other things, have 3D sound on headphones (HRTF effect) in all programs/games that output multi-channel sound. ALchemy is just a bonus. And contrary to what some people believe, ALchemy isn't tied to neither Sound Blaster X-Fi MB3 or any other real hardware sound card, it just needs valid license files.

Well, it's as crazy as having dgVoodoo DLLs for each and every game. :D The same goes for other DX/Glide wrappers and similar stuff. I personally use NTFS hard links for such files, if I update wrapper DLL in one game folder, it gets updated for all games that use it.

Oldest one in the book: Sleep(1).

Oh, that's simple. Thought it was some complicated trickery. Not sure what goes wrong on your end that my method fails to work properly, I might have missed something. I rely on user pressing Space when the 3D view is in focus to toggle whether I should redirect code flow to GetMessage instead of the usual PeekMessage. Is Space used for something else too besides activating fly mode? Yeah, it shouldn't be redirected when the user is doing something in it. Though on Linux in WINE, this method fails ALL the time, just crashes, so must be disabled there.

Otherwise, interestingly, the editor runs more stable in WINE, at least on my end. Also, on Windows, I haven't managed to make the 3D engine ever crash (dgVoodoo or not), but the editor itself is pretty easy crash. Just starting 3D view on the map with enough various objects, then spamming the key for increasing fog distance.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

UCyborg wrote: Not sure what goes wrong on your end that my method fails to work properly (...)
Basically what happens is this: the instant I release (all of) the movement keys, the 3D View freezes instantly - while the camera should keep moving for a fraction of a second due to "momentum".

Perhaps your code is getting "confused" somehow due to possible extraneous SPACE presses? (eg. I keep pressing SPACE to move up, out of habit, even though it's the wrong key in the Editor...)
I didn't really examine that possibility in any detail, though.

Oh, and one more thing: pressing SPACE doesn't actually release focus - it only releases the MOUSE POINTER; the 3D view does stay in focus... maybe that's the problem?

(...) to toggle (...)
Uh-oh. But pressing any movement key also causes the 3D View to return to "flying mode"...

This starts to sound like an issue of a desync between "expectations" and "reality", due to different methods used by both?


BTW: in my newest version of the Editor, I've remapped most of the keybindings (except SPACE) to more reasonable ones - but obviously it has no relevance in this case; just something I forgot to mention earlier.

UCyborg wrote: Well, it's as crazy as having dgVoodoo DLLs for each and every game. :D
Fair enough...

UCyborg wrote: Also, on Windows, I haven't managed to make the 3D engine ever crash (dgVoodoo or not), but the editor itself is pretty easy crash.
Uh, what?
For me (as well as for Jage), it's almost the exact opposite.

I've almost never had the Editor (2D part) crash, but the 3D view tends to hang (not crash) randomly when viewing 3D models.

I've managed to trace the problem down to the engine getting stuck in a very tight spinwait, apparently waiting for another thread to finish its task, but I have no idea what's going on there. No threads are holding any locks...
I tried breaking up that spinwait with Sleep(1) also, but it did nothing to help (as expected, really).

The issue is not consistently reproducible, in that no particular model is ever guaranteed to cause the hang (or to NOT cause the hang).
Only real way to "reproduce" the hang is to just start viewing a whole bunch of 3D models at random; sooner or later one of them will cause the hang. But closing the 3D View and reopening it, the same model opens just fine...

--------------------------------

Also: I've edited my previous post (just as you replied to it at the same time...), asking for your input about how to go about redoing my DLL hooking; please take a look at that.

--------------------------------

EDIT: A few more observations:

I've finished my NULL renderer for the dedicated server. Ended up being much easier than I expected...
CPU usage is at an off-scale low, and obviously no graphics hardware is needed.


For reference, here's a list of changes to the Level Editor (copied verbatim from the CP changelog for the upcoming release):
  • Level Editor: fixed the DEM Importer's code to behave correctly when a nonzero offset is specified for the import area in the source data,
  • Level Editor: most of the keybindings have been remapped (including those inside the 3D View!); click "Help"/"Shortcut keys..." to see what the new keybindings are now,
  • Level Editor: selecting layer vertices is no longer horribly bugged when dealing with vertices which belong to more than 1 layer; this also allows the "Match vertices" tool to work as expected,
  • Level Editor: selecting vertices located on the very edges of the map no longer causes the Editor to crash with an access violation,
  • Level Editor: the DEM Importer can now handle invalid pixel data data values without throwing a critical error, if the values are only slightly outside of the allowed range of (0...1) - which tends to occur at the edges of the dataset when the DEMs are rescaled with graphical editing tools (eg. GIMP),
  • Level Editor: the "Flatten elements" tool now accurately reports the amount of elements which have been flattened,
  • Level Editor: the "What's new in the Level Editor.txt" file is now actually included in the patch installation (it got accidentally excluded from the patch files in build 153.01),
  • Level Editor (3D View): CPU usage has been reduced considerably (without sacrificing performance) - it should no longer use up 100% of the available CPU time on the CPU core it's running on,
  • Level Editor (3D View): invisible textures (on invisible walls, killboxes, etc.) are now visibly rendered in a visually distinctive way - but note that those "invisible" models which lack any textures (eg. the "Bounding Vertices" model) cannot be rendered at all in any case.

Drakan patching: I've been thinking about this, and it seems like the only reasonable way to do this is to use the InjectDLL Windows compatibility shim.
Or at least, use it as a fallback measure: the primary vector of entry would be through DINPUT.dll (thus renaming your DINPUT.dll to something else instead), but the InjectDLL shim would be used to inject some code that checks for the correct version of DINPUT.dll and throws an error if it detects that it has been overwritten with the AiO DINPUT.dll.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

Oh, and one more thing: pressing SPACE doesn't actually release focus - it only releases the MOUSE POINTER; the 3D view does stay in focus... maybe that's the problem?

Actually, I did miss few quite obvious things. And the code that handled it was pretty monstrous. I see you also went tackling this issue. Guess what; I just managed to make it much better. Code was greatly simplified and the main thread consumes exactly 0,00 % CPU when not controlling camera and it's not lagging anymore when flying and pressing various keys. The trick was to detect whether the renderer is active using the variable 0x49309C and mind the fact that PeekMessage function I'm hooking is also called at some point just to remove keyboard events (this seems to be some sort of hack that prevents input from getting screwed up, something to do with usage of DirectInput I guess). When renderer is active, the old configurable frame-limiter is used that uses Sleep(1), otherwise WaitMessage() is used in hooked PeekMessage().

Now the 2 important updates to Engine.exe itself:

  • Level Editor 3D engine: Fixed hang that happens under certain circumstances when viewing certain models.
Go to 0x438594; the function was moved up a bit to make space for extra conditional check and 6 call instructions to it updated. Sleeping doesn't help here as it's just iterating over the same data over-over and there's no multi-threading going on, the other threads are created by the system, Drakan only creates separate thread for some multiplayer tasks.

This bug rang a bell, I must have known about it at some point, but it's been a while since I was clicking through models. The extra check just uses known pattern to know it should break out of the loop when the situation occurs. That code must have something to do with textures, I got random textures all over the terrain when I messed it up.

  • Level Editor 3D engine: Process all messages in message queue before running a frame, like Drakan.exe.
That's an easy one, just JMP was inserted at 0x41323A (and of course surrounding jumps updated). It makes sure message queue is emptied as quickly as possible and important events processed, should they occur and a lot of messages piled up in message queue for any reason.

Also: I've edited my previous post (just as you replied to it at the same time...), asking for your input about how to go about redoing my DLL hooking; please take a look at that.

The only idea I have besides InjectDLL shim you've mentioned is writing the launcher that injects it. It's quickly described here. Can't copy paste the code directly since he used classes he wrote, but didn't post their definitions, that wrap functionality of some Windows APIs (WriteProcessMemory, VirtualAllocEx, CreateRemoteThread, ...).

Out of curiosity, was there a lot of assembly work in your other fixes?
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

Great, thanks!
I'll remove my version of that "fix" from my patch DLL then; it was just a quick cheap hack anyway.

BTW: I'm leaving my Engine.exe patch DLL as DSOUND.dll; no point rewriting that one for anything else, since the 3D View doesn't actually use sound?
And in any case, lack of EAX wouldn't be an issue in the 3D View anyway, even if it had sound.

Having DSOUND.dll as the patch for the Engine won't affect the Editor itself, because in my CP the 3D View engine sits in \Engine folder - this is so that dgVoodoo can be used with it, as it would otherwise interfere badly with the Editor (the 2D part).

The only idea I have besides InjectDLL shim you've mentioned is writing the launcher that injects it. It's quickly described here.
Unfortunately, the DLL injection method you've linked to is not useful in my case:
I need the DLL to run when Drakan.exe is launched (not some other process which then launches Drakan.exe). This is required for compatibility with existing shortcuts, etc.
Hence the shimming idea.

Remember, any file replacements I do through my installer can be easily (accidentally) undone by misguided users installing the CP, then (re)installing your AiO patch. Yes, it has happened before - although so far, no real damage has been done... yet.

To paraphrase the conclusion of the Discord conversation I've had with Zalasus concerning this issue:
It's a case of "assume nothing, prepare for everything".

BTW: Yes, I do plan to have a launcher / settings editor for Drakan.exe eventually - but it will be invoked from within Drakan.exe itself (if it's started without passing the right commandline param to skip the launcher); in that respect, you could say that Drakan.exe will effectively be its own launcher.
The basic idea is to gather ALL of the settings for Drakan (incl. Arokh.ini, dgVoodoo.conf, etc) in ONE place: the launcher. Currently the settings are located all over the place, and that's causing users unnecessary difficulties.

TL;dr: I'm currently redoing my code as DINPUT.dll; the CP installer will then install your DINPUT.dll as AiO.dll, and my DINPUT.dll has that listed in its import table.
The only function of the InjectDLL shim will be to check if the DINPUT.dll is still the CP version; I don't want to put any other functionality in the shim, in case it doesn't actually activate for whatever reason. (In particular, are those shims even supported in WINE?)

Out of curiosity, was there a lot of assembly work in your other fixes?
My fixes are almost entirely in assembly.
To be fair, I find C/C++ much too confusing for my tastes - additionally, setting up the build environment for that is like being flipped off with a middle finger the size of Burj Khalifa.

I very much prefer easier languages, such as assembly.

Some of the patch logic is instead written in Pascal (Lazarus), mostly because of the conveniences it provides for working with strings, floats, and/or file IO - as well as the forms (although there are of course no forms in the Drakan.exe patches - not yet, anyway).
Currently that means all of my DEM importer code (as well as the GUI for my upcoming layer visibility calculation code); the file parser for the dedicated server player name reservation feature has also been written in Lazarus.

As for the assembly part, I use a combination of 3 approaches: "usual" hooking by overwriting jump/call addresses, overwriting procedure entry points (a'la Detours), or assembling new code directly in place (if it's only a few opcodes and there is enough space for them).

The KERNEL32.GetProcAddress hook instead takes advantage of the hotpatch allowance provided by Microsoft (5 NOPs, then MOV EDI,EDI at proc entry).
MOV EDI,EDI gets overwritten last, to ensure that the proc entry point always contains valid code at all times.

BTW: I plan to release the source for those fixes when I release the next CP build; there's no point in releasing the source earlier, since that'll be the first release of those fixes, and even that initial version is not complete yet.

EDIT: In fact, I've just gone over some of my previous code; in particular, ended up completely redoing the way the keybindings are remapped in the 3D View - my previous approach to dealing with it was so clunky, it was a miracle it worked at all.

Go to 0x438594; the function was moved up a bit to make space for extra conditional check
Hmm, not sure I quite follow what's going on there.

I can clearly see that the circumstances leading to the infinite loop are now accounted for, breaking out of the loop if that's the case - but what's the actual meaning of [EAX + 8]?
And how does that relate to my earlier tests, where manually breaking out of the infinite loop made the engine stubbornly wind up straight back inside it anyway?


In other news: since with this, my patch for Engine.exe is complete for the time being, I've attached the source for it.
Currently it performs 2 functions:
  • Detects invisible 16-bit textures (full transparency) at load time and replaces them with a procedurally-generated visible texture,
  • Allows moving the camera with WASD + left SHIFT & CTRL (hold V to slow down the movement speed); but these keybindings are not really usable in practice without also using the modified Editor binary, though - due to the resulting keybinding conflicts between the patched 3D View and the unpatched Editor.
I've also attached the source for my NULL renderer.
It's originally been written as a straight-up replacement for DDRAW.dll - but since that would have caused other problems, I've modified it to be a standalone DLL.
So instead of replacing whichever version of DDRAW.dll would get normally loaded, my patch code simply hooks DDRAW.dll if Drakan is launched with "-dedicated -norender" commandline params.
I've tested that already, and it works beautifully :D


BTW: I've tried to reproduce the texture loading issues, but failed miserably.
However, I DID notice another problem, where LFE's cause increasing amounts of lag, proportional to the amount of playing time. More verbose description at the link above.
Attachments
Editor 3D View patch source.zip
(12.99 KiB) Downloaded 2873 times
NullRndr.zip
(8.21 KiB) Downloaded 2867 times

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

And in any case, lack of EAX wouldn't be an issue in the 3D View anyway, even if it had sound.

It does have sound (STOMP sequences). EAX can probably be tested correctly only in-game during gameplay, to see if transitions work as expected.

Having DSOUND.dll as the patch for the Engine won't affect the Editor itself, because in my CP the 3D View engine sits in \Engine folder - this is so that dgVoodoo can be used with it, as it would otherwise interfere badly with the Editor (the 2D part).

It wouldn't even with vanilla setup (all binaries in the same folder) since Level Editor.exe uses winmm.dll for sound.

TL;dr: I'm currently redoing my code as DINPUT.dll; the CP installer will then install your DINPUT.dll as AiO.dll, and my DINPUT.dll has that listed in its import table.
The only function of the InjectDLL shim will be to check if the DINPUT.dll is still the CP version; I don't want to put any other functionality in the shim, in case it doesn't actually activate for whatever reason. (In particular, are those shims even supported in WINE?)

Compatibility shims in compatibility layer? Nah, WINE only has compatibility modes (Win95, Win98, etc.) with if clauses altering behavior of certain APIs where applicable. And proxy DLLs to system DLLs such as dinput.dll aren't loaded by default without altering WINE's configuration.

So you'll have 3rd DLL that verifies dinput.dll since InjectDll takes DLL paths/names as parameters?

Hmm, not sure I quite follow what's going on there.

I can clearly see that the circumstances leading to the infinite loop are now accounted for, breaking out of the loop if that's the case - but what's the actual meaning of [EAX + 8]?

Who knows, just guesswork. The actual bug might be somewhere else.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Widescreen hack and some other fixes aka AiO Patch

Post by Mechanist »

UCyborg wrote:It does have sound (STOMP sequences).
Ah, ok then. I haven't been messing around with those yet.

UCyborg wrote: So you'll have 3rd DLL that verifies dinput.dll since InjectDll takes DLL paths/names as parameters?
Yes.
I have already finished it; tested it on Windows and everything works great.

The InjectDLL shim (the Windows feature, not my actual shim .dll) is the best thing ever: not only does it load an arbitrary DLL (or multiple DLLs) into process memory (albeit fairly late in the load order - in fact, appears to always be the last DLL loaded before process entry point), but it fails silently and skips the DLL injection if the specified DLL cannot be found.

Even better - if no path to the DLL is specified (only the filename), the standard Windows search order appears to apply - so no need to put the DLL in \SYSTEM32 or some other fixed location; putting it in the same directory as the .exe works just fine.
As you can imagine, that's a huge boon for deployment, especially since the installer doesn't need to edit the shim .SDB file with the correct path at installation time.

Somewhat unfortunately, the shim loader ignores the return value from the shim's DLLMain; this means that the only way to cleanly terminate Drakan.exe from within the shim DLL is to explicitly call KERNEL32.ExitProcess (which works OK, even though the process creation hasn't even finished yet!); returning 0 from DLLMain does nothing useful, it only unloads the shim DLL and Drakan.exe continues to load as normal.

However, there are 4 minor snags with this shimming approach:
  • When using "Test level" from the Level Editor, CreateProcessA doesn't set the working directory for Drakan.exe (it's still the same as whatever it was for the Editor) - so I use GetModuleFileNameA to retrieve the correct path to where Drakan.exe is located, and switch to that directory before doing anything else; this doesn't affect the Editor, since the new working directory is effective only for the child process.
  • Because by the time the shim code fires, the PE loader already has a lock on DINPUT.dll and has loaded it into process space, the file cannot be deleted (why? the hardcopy is no longer needed...); it has to be renamed instead (which actually works, for no adequately explained reason!).
  • Also for the same reason as above, it means that Drakan.exe has to be terminated to unload the incorrect DINPUT.dll; the "proper" solution would be to somehow silently relaunch Drakan.exe in that case (if the correct DINPUT.dll was successfully restored from backup), in such a way that forces the DLLs to be reloaded - but since that's such a corner case, it just shows a notification message telling the user to restart Drakan after clicking OK (as well as telling them to not overwrite the patch files by manually deploying the AiO patch).
  • As you just confirmed, there isn't a snowball's chance in Hell that it would work on Linux. But there the CP is unsupported anyway, so the user is on their own. There are no plans for it to be supported on Linux before Hell freezes over, at least not unless someone else offers to take care of it; I certainly won't take it upon myself to try and support the CP on an OS which I can't properly use and know next to nothing about. And from what I've seen so far, Linux users appear to be the minority amongst Drakan players anyway.

UCyborg wrote: And proxy DLLs to system DLLs such as dinput.dll aren't loaded by default without altering WINE's configuration.
Interesting. So does it mean that the AiO patch doesn't work in WINE without messing with the configs?

UCyborg wrote: Who knows, just guesswork. The actual bug might be somewhere else.
Whatever works, I guess.
Lol, breaking out of that loop was pretty much my first idea when I realized what's going on - but I gave up on it after seeing what happens the first several times when I did that manually. Guess I was not persistent enough :)

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: Widescreen hack and some other fixes aka AiO Patch

Post by UCyborg »

Because by the time the shim code fires, the PE loader already has a lock on DINPUT.dll and has loaded it into process space, the file cannot be deleted (why? the hardcopy is no longer needed...); it has to be renamed instead (which actually works, for no adequately explained reason!).

Probably because PE modules are memory mapped, not read in a traditional sense and memcpied to newly allocated memory. Memory mapped files cannot be deleted.

Interesting. So does it mean that the AiO patch doesn't work in WINE without messing with the configs?

It works if you import the included registry settings.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

Post Reply