Some reverse-engineering of the Level Editor and TXD stuff

Anything to do with Drakan level editing and modifications, post it here! Also this is the place to tell us about your new levels and get player feedback.

Moderators: Arokhs Twin, mage150, yangez93

User avatar
Mechanist
Dragon
Posts: 303
Joined: Wed Mar 07, 2018 7:27 pm
Location: Poland

Re: Some reverse-engineering of the Level Editor and TXD stuff

Post by Mechanist »

Mechanist wrote:
Tue Aug 21, 2018 9:31 pm
Eg. the texture import codepath has correct handling for importing 32-bit BMPs when it flips the byte order to BGRA, without touching the alpha bytes.
Oops... I take that back. It's actually extremely poorly written! :x

More specifically, it does indeed work as intended... except that it assumes a different source byte order (RGBA) than what is actually used in the 32-bit .BMP files (ARGB)!:evil:

Well, another thing to add to the pile...

--------------------------------

EDIT: Ok, I fixed that mess... removed the part for handling 32-bit cases (which wasn't applicable anyway due to a programming error) and added my own code that BSWAPs the ARGB into BGRA for 32-bit textures, only letting the Editor's byte-flipping code run for 24-bit .BMPs.

Also I took a quick look at texture exports:
First it checks if the texture has an integrated alpha channel, and bails out right away if that's the case.
Then there's a 2nd check after TextureBitnessConverter is called - it bails out if the conversion wasn't successful (makes sense, really).

I disabled these 2 checks to see what happens (tried it with a 16-bit texture with 4-bit alpha): the result is that the Editor spits out a 16-bit .BMP, copying the pixel data verbatim from the texture memory (alpha and all), while also generating an incorrect header for what the file actually contains (due to having some hardcoded values for some of the header fields, and no logic to handle non-24-bit .BMP exports).

EDIT2: It appears that the existing code for converting 16-bit to 24-bit textures (for export purposes) is broken and doesn't work as expected, so that'll also require fixing; the problem was found by unterbuchse while he was trying to make his .REC extractor also export the textures.

--------------------------------

EDIT3:

Ok, I think I've got everything that I wanted working now, including un-breaking my own changes. The updated code and the new .EXE can be found in the AiO patch thread.

Now the next step is to hack in a working DEM (Digital Elevation Model) import.
There's already a "Load heights from bitmap" option - but it's quite buggy and doesn't quite do what I need, even if it weren't bugged in the first place.

Anyway, importing ANY integer grayscale bitmap format is fatally flawed from the start, as it involves mapping 256 values to around 256 thousand possible heights.
So floats (or CSV files) are the only sane answer here... and CSVs can't be readily manipulated by graphical editing utilities, so floats it is.

Luckily, there's at least 1 bitmap format (.PFM) that has all the required properties:
  • Simple header + data format, similar to regular .BMP files,
  • Pixel data is stored as a sequence of 32-bit floats - one per pixel in case of grayscale .PFMs,
  • No compression is used,
  • This format can be easily imported, manipulated and exported using free graphical editing tools (such as GIMP).
--------------------------------

So as usual, it's time for more reverse-engineering of the Editor :D

First up, the way layer data is stored in memory.

Layer metadata (for each individual layer) is stored as a 32-DWORD long struct, with the fields having the following meanings:
  • ID number of layer (?)
  • Unknown (always 0?)
  • Appears to be a flags entry: bit 0 = whether layer is selected for editing (yes=1), bit 1 = set if the layer is "Hidden", bit 3 = member of alternate blending group (yes=1), other bits - unknown,
  • Layer type: 0 = floor, 1 = ceiling, 2 = between,
  • Horizontal size of layer, measured in tiles,
  • Vertical size of layer, measured in tiles,
  • X-coordinate of the layer's upper left vertex,
  • Z-coordinate of the layer's upper left vertex,
  • Layer world height (stored as a 32-bit float!)
  • Unknown (always 1?)
  • Pointer to a struct containing the layer visibility data,
  • Unknown (always 4?)
  • Unknown (always 3?)
  • Pointer to an array containing the tile data structs (details below),
  • Pointer to an array containing the vertex data structs (details below),
  • 32 bytes reserved for a zero-terminated ASCII string containing the layer's name (but note that the Editor limits actual name to at most 30, not 31 characters!)
  • Angle (in radians from the 3 o'clock position; values range from 0 to 2*pi) of the "Lighting/Direction" field (32-bit float!),
  • Angle (in radians; values range from -1/2*pi to +1/2*pi; positive values = light is directed downward) of the "Lighting/Ascension" field (32-bit float!),
  • BGRX value for the "Lighting/Color" field,
  • BGRX value for the "Lighting/Ambient Light" field,
  • Light dropoff type (0 = none; 1 = from north to south; 2 = E->W; 3 = S->N; 4 = W->E);
  • Unknown
  • Unknown
  • Unknown
  • Unknown
The last 4 unknown values are likely related to layer visibility data.

NOTE: This should go without saying, but it is forbidden to increase the dimensions of a layer by writing directly to the metadata struct - doing so will crash the Editor as it tries to read from (or write to) unallocated memory!

Visibility data:
This appears to be an array of structs (as many as there are layers in the map), each struct consisting consisting of a single WORD value (1 for visible and 0 for not visible) followed by a single DWORD value containing the layer ID number.
The contents of that struct reflect the settings in the "Layer visibility" window.
NOTE: Each layer is always visible from itself.
Also note that it's presently unknown whether those are actually layer IDs, or in fact their indices in the layer metadata pointer array.

Tile data (largely copied verbatim from Zalasus' OpenDrakan reference:
26 bytes per face, top left face first, then filled left to right, top to bottom.
The first 2 bytes seem to be flags. Low bit is face division: 0 = /; 1 = \.
Followed by two 4-byte-words which are database references for the face textures.
First is for left triangle, second is right triangle.
The next 16 bytes that indicate texture orientation.
So in a 2x2 tile layer, the tile data is arranged as follows:
12
34

Vertex data:
For each layer, the vertex data is stored as an array of 32-bit structs (NOT a struct of arrays!), with the array indices corresponding to the layer vertices beginning from the upper left corner (vertex 0), left to right, top to bottom - like text on a page.
For example, in a 2x2 tile layer, the vertex data is arranged as follows:
012
345
678

(NOTE: This is the opposite convention to that used in the .BMP and .PFM files, where the ordering of entries is inverted vertically - so it starts at the bottom left corner, instead of top left.)

The vertex data struct:
  • BYTE containing the vertex type (this is the value from the Vertex Properties :arrow: Vertex type dropdown menu),
  • BYTE containing the vertex flags (meaning of the individual bits is unknown at this time),
  • WORD containing the vertex height offset (unsigned, equal to ((vertex height - world height)/2 + 32768)).
--------------------------------

Stuff related to bitmap height data import:
function BitmapHeightImporter (Level_Ed.00431780)
Called from: Level_Ed.00479DD7
Arg_ECX: Pointer to the struct containing layer meta-metadata (details below),
Arg 1: Handle to the calling (main) window.

The pointer passed in ECX is very important, because it points to the structure that (indirectly) holds ALL the information about the layers in the currently opened map - the master key of the layers, if you will.

Here's the meaning of the individual fields:
[*]Unknown (always 1?)
[*]Pointer to the ASCII name of the currently selected layer group (?),
[*]Unknown (always 2?),
[*]Unknown (always 1?),
[*]Unknown (always 5?),
[*]Pointer to an array of pointers to the metadata of individual layers (they appear to be stored in the inverse order, with the most recently created layers first),
[*]Unknown (always 8?),
[*]Total number of layers in the map,
[*]Pointer to a "minimap representation" of the current layer layout (no idea what it's used for),
[*]Horizontal width of the whole map,
[*]Vertical height of the whole map,
[*]Unknown (possibly some flags?).

Post Reply