simple machines forum

Patch #2 (today)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

I think I may have found the reason for random crashing at start in this release in this patch. I also tested the new background loading system today on my KF2 project and found I neglected to test it on an elevation change scenario. It turned out a "shock absorber" effect caused a big glitch. (The crashing I think was caused by the [Number] based extensions in the Ex.ini file, since it is used on the background loading "thread" when it wasn't designed for sharing between threads. There may be other sources of this kind of problem, I now know what to keep an open mind about. Of course SOM isn't a multi-thread system, even though many of its tools are loaded down with "critical sections" because it's written with MFC code that seems to have produced ridiculously bloated applications. When designing the system I had to be careful about what kind of bad interactions could happen. Luckily loading code is pretty isolated.)

Edited: Oh yeah... as for background loading KF2. There's still a hiccup. I don't know if I'll be able to work it out of the demo I'm working on or not. It could very likely have nothing to do with disk access... maybe just memory cache hits. On beefier CPUs it might not hiccup. Hopefully I can profile it some more at some point, or think of some strategies to avoid extra work.

Minor usability patch (SOM_MAP)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

Sometime back in 2017 I noticed when SOM_MAP's palette area moves on its own (after clicking a tile) that it wipes out the information displayed in text boxes under the work area.

But what I somehow didn't notice until yesterday is this also wipes out the stored information used to place copies of the selected tile (instead it reverts to as if you clicked the palette area... forgetting things like elevation and rotation) which is obviously important for work.

So I'm going ahead and putting this out there in case it helps anyone in the meantime. An example of when this would become a problem is if you were using the tile sets packaged with SOM and mix-and-matching tiles from different sections of its KF sample project. It could also happen if the palette just happens to be on the border of the tiles you're working with. I guess it goes to show how little time I spend building game maps with SOM that I didn't notice this for more than a decade of working with it!

Important Patch (Critical)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

Late last night I ran into yet another problem saving layer MAP files. I can't ever seem to complete that project... I really feel like at some point I need to rewrite the MAP file input/output code from scratch, but maybe this is the last time... knock on wood. (What was happening was some code I wrote that juggles some data between the files failed to give READ permissions to the file in question... which explained why some junk would appear at the end of layer MAP files... but the real problem I suppose is I added some error handling code that reported errors, and I thought it would catch my attention in debug modes, but it was incomplete. I think this was the same problem the last time I published a patch the other week... i.e. new error code causing the load/save operation to give up where previously it would've carried on.)

What's a wonder is how this didn't become a problem sooner. (This is a common refrain for bugs... really the problem bugs are the ones that get masked by shear chance instead of backfiring immediately.)

Last night I also found numerous new problems I introduced to x2mdl around textures mainly... and I chanced upon a realization that som_db.exe ignores the last frame in hard (skeletal) animations, and so I've adapted x2mdl to reflect this behavior by clipping the final frame down to 0 time... this means the last frame is shown at the end of the animation but for 0 amount of time. I.e. you won't see it unless you freeze the animation (or set the cursor) on the end. This is actually how animators usually work, so this is a benefit. The only problem is now I have to go back and regenerate all of the hard animation MM3D files from the last release a while ago. (Note, technically this topic/thread is still maintaining that release.)

story time (j/k)

Today was a town/groceries day, but I managed to squeeze in a project to work on programming KF2's drawbridges. It was somewhat eventful (no pun intended.)

How SOM uses switches you could accomplish this a few ways. However in practice it probably wouldn't work for a number of reasons. How I wanted to make it simple, so I set up a global event that is constantly updating. It just checks the state for both drawbridge systems and updates them. Alternatively the event could be triggered by a button press, for each bridge.

The latter approach might require a timer, however the switch animation seems to be synchronized with the drawbridge's, and it seems like SOM doesn't let you flip the switch back until it finishes. Although I haven't looked at the code for this.

Edited: Actually the PlayStation game doesn't work like this. In it the bridges don't begin moving until the switch is fully transitioned, and the bridges move more slowly (and creakily) than the switches do. (I wonder if SOM should work like this... the way it works is more like an analog--1-to-1---switch, whereas the way KF2 works is like a digital switch... both have merit.)

For the approach I chose it wouldn't work because every event frame the animation would reset. So I worked on the object engage (animation) instruction to make sure it finishes playing the animation unless the state is flipped (I should probably add some protection to make sure the animation completes while I'm at it.)

But then it turned out there was another problem, because after the animation plays, the "object" automatically switches over to its resetting animation. For doors that means it closes itself.

This applies to either approach. So in some ways it probably turns out that to have the object remain open that the global event approach is the only way that can work. Of course I had to modify the programming, essentially to have the animation reset the wait timer, which for doors is set to 120. I think at the old 30 fps this holds the doors open for 4 seconds. However a while back I modified doors to behave like KF2's. That's better because they don't close on your face, and the timing is moot because they now will close at an opportune time.

This was a little confusing at first until I figured out that while waiting, it's actually displaying the first frame of the closing animation. I had programmed a strategy when I realized this, and so I knew I should work with the given system and take that into account, and take a new tack.

Then I noticed something, that wasn't a fluke. The drawbridge wouldn't quite open up fully after closing. This was a hint of something I've kind of suspected but never quite understood. It turns out (it seems) that som_db.exe will pretty much ignore the final frame in animations. I never could quite confirm it because usually for a door like animation the final frame isn't that important in the scheme of things... however, for the drawbridge it needed to be level in the final configuration.

This was kind of annoying, since I had to factor all of this into x2mdl. And while I was in the process I found a lot of other recent problems with it... as you do. I'll have to patch it, etc. again tomorrow.

It's actually nice to have the player (som_db) ignore the final frame, because the hard animations use "step" sampling, which is in a way ambiguous on the final frame, and I realized it was causing a problem with "round-tripping" animations between art and runtime files. I figured it was doing the best it could, but I was missing this important piece.

The existing animations will all have to be patched/revised, at least for the hard animations. I'll have to regenerate them. I looked at the player, and it's a little confusing because when the animation advances, it sets it to plus 1... which is what you'd expect. And it rejects it if that value is less than the length of the animation. But what I expect is that "plus 1" is really amounting to rejecting if less than the runtime minus 1, in a roundabout way. For the soft animations the subroutine that gets the runtime actually adds 1 to their length, which is I think a way of bringing both systems into alignment. Soft animations don't use "step" sampling.

If I wasn't clear, this is actually a good development for art work because it ensures the end of the animations are the same. It's a little confusing though because in the player the final frame never appears. It's more or less assumed either it matches the first for looping, or the closing animation will reproduce it.

Another interesting note is SOM doesn't really have a system for KF2's drawbridges and trap doors. What I've used is the double-door model. It doesn't actually capture the incline of these, since it's still working as if it's an upright door. However it pretty much captures everything, except that really you're walking on top of the door, which is assumed to be flat. I could try to improve it some time by factoring in an incline formed by the control-points. But I'm planning to add an even more flexible system, simply by enabling objects to use an MHM like clipping model before very long. I'm thinking this is something I'd like to get into this next demo. A good way to exhibit it would be to be able to climb inside the fountain I think.

I've been able to convert the new map's objects and characters fairly briskly (still very much in progress) but I keep finding myself midday in the middle of fixing some bug or programing this or that, often into the night right up until bedtime. One interesting moment came up last night. I'd been kind of unhappy with how the new map looked for some reason I couldn't place. I thought maybe its per tile ambient lighting isn't set up yet. Then last night I realized there was something wrong with the texture upscaling system (set up because KF2's textures are pretty small by modern standards.) It turned out for some reason (I forget) I'd set it to ignore the sky model's textures (maybe because they're always at a fixed distance that might not qualify, I don't know) and so now that the art system I just set up was sourcing its textures out of the same folder they were having the effect disabled. It looks so much better restored that I felt really good that it was back on track again.

It sounds very simple, but whenever something breaks, and it's for some left field reason, it's kind of stressful for programmers. We have to think on our toes, or dread that wait until every logical possibility is exhausted, and you've been there before, you know it's going to be some curve ball... you just don't know how long until it will reveal itself. Lately I've been having a problem that seemed to creep up when I worked on the MPX preloading system (I'm not sure what to call it) that I thought was a side effect of switching D3D over to multi-thread (which I've recently disabled in the latest patches, it seems to be working now without it.) but it turns out not. What's happening is it's crashing at start up every now and then, always inside an Intel driver. I've been watching it but I haven't got any leads on it. I'm a little worried about it. It's not good. I haven't figured out if it's isolated to my KF2 project. In theory it could be an Intel bug, but anyway, it is something that's bothering me lately. I just don't have any way to approach it. I worry it could be worse (always crash) on other drivers.

Update: I think I figured out the crash, or the basic reason for why crashes started with the introduction of the new background loading system. I wasn't open minded enough to realize some other subsystems could be split across multiple "threads" that aren't designed for that. (The first major such find was the [Number] system that enables some functional programming in the Ex.ini files. I just got lucky that eventually the CPU state post-crash made some sense... or at least gave me an idea.)

Note, the official release location for x2mdl is now found at http://csv.swordofmoonlight.net/x2mdl.dll/ (find the newest ZIP file and unzip.) I've added this to the large font download linklinks. I'll still update the other from time to time, but for downloading x2mdl.exe the official is best. Plus x2mdl.exe is now updated in the TOOL folder when updating x2mdl.dll. If you're looking to do custom model work I recommend downloading both (at least for the notes) and ask for help.

Edited: While I'm here... yesterday I found a mistake I made with scaling animations. They need to be remade, unless perhaps if a model only has one animation in it (which all of SOM's scaling animations do.) I've also been making lots of improvements to it lately since I've been doing modeling work on my KF2 project. (I've also been working a lot on MM3D. It's even come to dominate my work lately.)

off-topic

(I also had a good idea to make A and S and E set elevation values in SOM_MAP. A for add, S for subtract, and E for equal/elevation. That way you can do it left handed! Unfortunately I still have to work on this...)

Sorry this really isn't on-topic, but just as a note... I worked on this this morning, which will be in the next patch, whenever that is.

I first looked at SOM_MAP's hotkeys. They're a little unusual, but they all work. I've never learned them I guess because they're not intuitive. I remember seeing them in the manual. Here are some of the more odd ones...

Ctrl+1 through Ctrl+3 choose between the 3 different views or tabs. This also works in SOM_PRM.

Ctrl+I opens the events menu. This one seems odd, so does putting hotkeys on the right side of the keyboard. I wonder why I of all letters? Maybe I should add a Ctrl+E option come to think of it. Maybe I is for Japanese something?

Ctrl+T opens a game (a test).

Ctrl+W opens the MPX compiler menu, but doesn't compile it.

Ctrl+P is the same as Ctrl+Click used to be, which copies the tile into the current "pen" and syncs the palette viewer.

Ctrl+Left (and Right) does horizontal paging. Up/Down don't, but I think PageUp/Down does. I guess I should add those come to think of it.

new SOM_MAP hotkeys

New ones I've added (mainly oriented to using a mouse with left hand on keyboard) are A to add 0.5 to elevation, S to subtract. E to assign elevation. Note, these 3 don't currently work with Undo, but I will probably fix that if it ever starts to annoy me...

C does same as Ctrl+P. V does same as Enter/Space but doesn't depend on the Ctrl key or CtrlLock button. I recommend using V now, but I also made Ctrl+Enter/Space work with CtrlLock off, which it hadn't previously.

Ctrl+D now deletes tiles. F and G flip the tile, equivalent to 2 turns. They're the same actually, except you can press Shift to make them do 1 turn, like Z and X. Useful to avoid moving your finger between them. Plus F and G are closer to V.

EDITED: For 1hr the following patch was bad (I hate when this happens) because I forgot I disabled the fix to try to test it before realizing it wouldn't work with the art system absent.

Patch Part 2

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (see EDITED in bold)

The SomEx.dll link is because I botched the earlier patch, and I'm merging the warning I posted into this post.

As for x2mdl, I noticed a problem with my wall writing decals in my KF2 project and was convinced x2mdl was outputting bad colors in the textures somehow, and so I finally did something I've been meaning to do to optimize how x2mdl outputs textures from back in the day when they needed to be downscaled to 256x256 in 16-bit color for MDL files. (Originally it was actually using D3D to do this color conversion, but it failed once on a GPU or device that didn't support it.)

Well, it turned out at some point when I had lost the original texture, and only the TXR file was correct. After I started using the new art system that TXR was no longer used. So IOW my scramble to fix x2mdl was for nought there. However, I also realized it was still downscaling to 256x256 and MDO files actually support much larger textures than this. I'm not sure how many mipmap levels som_db allocates, but with do_mipmaps it doesn't even matter. Moratheia has 2048x2048 textures that I know of, so at least that big (which is too big) probably works without SomEx.dll.

So I've fixed that problem. The only textures in SOM's larger than this is 3 of its 4 sky textures I enlarged to 512x512 in the original release for this release. I didn't realize they were getting hacked back down to 256x256. To see that benefit it's necessary to clear out the shortcut/lnk files in the map/model folder for sky files (called sky) or delete that folder from the temporary cache.

I've also updated these files in the TOOL folder. I meant to do that earlier, but I'm glad I forgot to now.

I think x2mdl should run a little faster now when converting models. It actually was pretty optimal already when updating textures by themselves, and didn't have this 256x256 problem then (actually that's probably why I missed the sky kneecapping come to think of it) however, in practice it still generates the textures when converting the models, and usually that comes first unless the model's texture(s) already exist with shortcuts. Anyway, now it's almost as good, but still allocates a texture in video memory in case it needs to be downscaled for the MDL file... although I can fix that too come to think of it (next patch.)

Important Patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

This patches a serious SOM_MAP bug that might be in the current build/patch which will fail to save your MAP file if encountered. It's down to random chance, so you want to update this to not lose work.

I'm not 100% certain this needs patching, but I can't test it because my map that encountered the bug depends on the new art system, which isn't available in the current build/patch. (The bug happens when the buffer happens to end at the last tile in a string of blank tiles at just the right spot... probably because of a sanity check I recently added.)

Another thing I happened to notice today is bugs in a few places around the vertical limits I added to event triggers/activation a while ago. (I really messed up in multiple places here: 1) was a rounding mistake with negative values; 2) monsters were checking an NPC parameter to determine if they're alive or not; 3) square triggers were using the round trigger's parameters.)

enhancements

Everything seems pretty stable right now, so also in this patch is support for the last two model formats under the new art system, which I've been using lately and it's really a vast improvement for building maps with custom art.

I haven't yet added some new advanced features to these formats, but they're as good (better) than the existing x2msm/x2mhm option. One thing to look out for is untextured models are still treated as wall-to-wall CPs, except for when converting to MM3D, which I've added some code to its path to tease them apart in order to be able to convert MHM files. If you put x2mdl into MHM mode it won't do this, but the art system doesn't do that, so models either need to be textured, or make the MHM geometry slightly transparent to defeat the CP detectors. (I don't know if I can fix this really, the best that can be done is to only treat lone triangles as CPs, but sometimes low-poly art might have long triangles too... luckily MSM files should be texture mapped because they don't have materials even if you wanted to do something more like the original KF.)

Another thing it doesn't do (yet) is split models up into MSM and MHM parts. IOW the art system will just output both an MSM and an MHM file, so you don't just use one of them if you just need it for MSM or MHM for time being.

Edited: Another secret I added today (I guess SOM can hold secrets if KF does) is holding the Alt key when clicking in the workshop tools (PrtsEdit, ItemEdit, etc.) will open up the model in your 3D editing software associated with your art file's file format. I intend to make this available from outside the workshop tools too. (I also had a good idea to make A and S and E set elevation values in SOM_MAP. A for add, S for subtract, and E for equal/elevation. That way you can do it left handed! Unfortunately I still have to work on this...)

status update

On the second map, all of the tiles are now texture mapped, including palette swaps (there are a more than just the first map's Harvine room, which I also finally finished yesterday.)

It's pretty short work, but it's a mountain of work. It's definitely going much faster than before, but I'm not sure the quality of the texture mapping is as good. I intend to go back and tweak the maps after most everything is finished, some along the way. There's probably no perfect mapping because the PlayStation address model is so different, and the tiles have a like a cm of extra space that overlaps, that makes it hard to say what's correct.

(Edited: FWIW, for the most part, especially for tiles, I'm not having to make any adjustments to the UVs this time around. I added some simple shortcuts to my MM3D software, but I realized after a few tiles I was doing the same thing, and so I should just put that in the TMD conversion code as my starting point.)

There are some "objects" that need to be converted to tiles. I actually thought the game had a sophisticated system here it would plug in static tile models until things like secrets (and traps) are engaged, and only then replace them with the animated models, in order to hide their internals from the PlayStation's no depth-buffer approach... except yesterday I was in the Harvine castle area and every single secret there was obvious because the walls would glitch around them. I'm starting to wonder if the false walls in this area were just not given the full treatment. I don't remember the secrets being so obvious as these. I suppose secret panels are somewhat different on the PlayStation if they give multiple tells through various glitches. All it takes is a keen eye then.

I haven't noticed that many new items on this map. Hopefully I won't have to work on any more weapon animations.

random

I've spent a lot of time the past two days working on MapComp diagnostics. Somehow the error code forwarding (to SOM_MAP) seems to have broken down, which I've fixed, but I also found myself in a weird situation where somehow the DELETE character (127) was injected into the MHM field of one of my PRT files, by Microsoft's text box.

The MHM field was supposed to be blank. So MapComp was trying to open up an MHM file called character-code 127 .MHM. Of course that didn't go well. What it did do however is reveal how brittle MapComp/SOM_MAP are in this regard, since all MapComp generates in this case is a single error code... and (IOW) it's up to you to figure out (assuming you know what it means) which PRT file is the offending one... and of course that's a literal software "hell" since there are thousands of such files to comb through.

Another fun quirk is there is no error code if a PRT file itself doesn't open. In theory that might leave a hole in the MPX file, but in practice MapComp seems to not be able to abide by this and will crash. I guess if running it via SOM_MAP then SOM_MAP would have likely reported the missing PRT file, but I'm not sure these days since I've relaxed that some so that you can at least open and repair a MAP file with missing PRT references (it does raise a box.)

So I've invented a PRT error code (which hopefully isn't already taken) and have added a pop-up box to give a hint about which PRT the MHM or MSM error codes refer. Oh yeah, another point of failure is the do_missing_file_dialog_ok extension wasn't working. Otherwise it would tell you the missing file's name. In practice an MHM or MSM file may have a very similar file name to a PRT file, but this isn't a certainty (so maybe you could reason backward to it.) One reason it was failing is Microsoft's PathFileExistsA was lying about the file's existence. This was because (I think) it was being given a fake path with a > in it that I use to fake SOM into supporting Unicode folders. I fixed some code that was allowing that to happen. A second problem is do_missing_file_dialog_ok can be disabled (that's what it does) and maybe it shouldn't be so for tools, but for MapComp I changed the code to ignore it since MapComp will fail to make the MPX file, so it needs to give a reason. I opted to ignore missing objects (it reads every object MDO file so it can use their CPs for lamp flames, but this is a bit extravagant since it really ought to just read the lamp MDO files when the maps actually use lamps!! I should fix that sometime as an optimization.)

I have a database inside the site here of all of the files and their references (and back references) which was essential at one point, butt less vital these days now that the DLC tools like PrtsEdit are built into SOM proper. Unfortunately the new art system is going to outmode this database since it largely dispenses with all of these file formats (by making them transient files) so it's very likely to remain a time capsule going forward until I disconnect it or something happens. It's a shame because it has a cool WebGL based 3D model viewing system (largely an experiment at the time) that I'm not sure I want to try to salvage with the new extra level of indirection required. It gives me a head ache to think about it. But anyway, the only way to rectify it would be to build x2mdl into this website to convert art files into runtime files, and then redirect the WebGL system to a second cache level to convert those to its format. I think it's probably too tall an order to do anything with.