simple machines forum

Please login or register.

Login with username, password and session length
 

News:

Remember to make your own backup of posts before submitting.

 
 
Pages: [1] 2

Author Topic: Map piece tinkering ("AA" worthy)  (Read 27671 times)

Verdite

  • ***
  • Offline Offline
    • View Profile
Verdite says,
« on: January 12, 2016, 08:47:25 PM »

Welcome to the map piece tinkering thread.

I put this under projects as it's a joint effort between Holy and I.
Holy at the helm with ideas and scrutiny, and myself bearing the monotony of conversions*, modifications, etc. :coffee:

So rather than bounce emails back and forth about tinkering with the map models in order to achieve map pieces that'll work with the new AA system, I've opened this thread to hopefully improve workflow and discussion. I'm expecting him to make edits to this (top) posts when I've missed something, and to the Required modifications section. I don't mind editing in his ideas for convenience.

I'll place a :smug: beside my posts in the ideas section.

EDITED: I guess I am :saint: then. *The "monotony of conversions" should cease to be a thing before the end of 2016, if I meet all of my goals for the year.

Project summary
Verdite says: :smug:
Modification of the original SOM pieces is required in order to achieve an accurate representation of the new AA system (and show it off.)

Required modifications
Holy answers RFC: :saint:
Eliminate all ("AA"-related) cracks for tiles that obviously mate with one another. Details in Replies.

Ideas and brainstorming
 :smug: 12/01/16
Uniform separate step model to apply to models of same height. Same will apply for all map pieces in other sets.

:saint:
I cannot grasp what this image is demonstrating (it's a little small for me to be honest) but I am automatically against tiles that bury themselves into the ground like potatoes or carrots. I am not especially crazy about SOM's original stock artwork, but it is very sacrosanct, if only because it represents From's original product. I honestly do not think we should invest too much into it right now. It will not produce the best showcases for SOM. I want to get defects out, but after that, I'd rather you focus on your own artwork, or help with the project to port KF2 to SOM. It's high time for SOM to make an impression on outsiders, one way or another.

17/1/16 :smug:
The image depicts a 'foot' model to go below map pieces in order to create steps. The extrusion of the roof edges is required, else there will be an unsightly gap.
Absolutely agree, I hope we can get this project done and dusted soon. I'd like to work on finishing up Moratheia after this. Hopefully with a new x2mdl in the pipeline to give me a boost.

17/1/18 :saint:
I'm not good at imagining these things. There won't be a new x2anything, you'll just save/export to DAE and the player/editors will automatically do any necessary conversion/caching, probably even without stop/restarting the running programs. Just remember, if you make new tiles for the original sets, they must have proper icons, that are easy to make sense about.


Public and joint effort files
17/01/16
Set 1 ready for AA testing. Not yet stepped.
Set 1 msm files.

12/01/16
Pristine, converted parts 0-78 are available to review here.
File is saved as an .mqo. This will require Metasequoia, or MetasequoiaLE to view.
Please make sure you place the Texture folder into your Metaseq directory, then merge it with the existing Texture folder.
« Last Edit: January 18, 2016, 06:33:41 AM by Holy Diver »
Author of Moratheia. Lurker, for now.

Verdite has 59 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #1 on: January 13, 2016, 07:31:16 AM »

Here is my advice about the "AA" cracks (if you want to do other changes you can pursue that, but this is the most pressing issue as far as I am concerned)

First it's refreshing to have a thread here. I've been feeling very irrelevant lately...

SOM_MAP can show you where cracks will appear. There are NEW buttons that have upright triangles on them, that when pressed show where the vertices are. Of course you can see this in your modeling software, but it's probably easier to compare tiles side-by-side in SOM_MAP.

Usually the cracks appear between two tiles, because their vertices do not match. But in some tiles there are like intra-tile cracks, caused by the tile's own vertices not matching. Without do_aa this doesn't yield cracks because you can think of the edges meeting in a T junction.

You see in this T, if the top crossbar of the T doesn't have a vertex in the middle, where the itersection is, then it will crack.

Two places where cracks are likely to appear:

Consider stoop tiles. I want to call them steps, but they are steps for giants, or terraces if anything. They just form a zigzag each one being placed usually 1/2 meter higher than the previous. These will always crack. I'm not sure what logistics are required to prevent cracking, but it seems like just adding extra cuts to their walls at every 1/2 meter interval would do the trick.

I think we would benefit from developing some standard measurements, but ultimately it depends on the tile sets.


Second place cracks may appear:

What a crack is, is a place in the model where you can see through its veneer. It's okay for this to happen if there is part of the model that lies behind an open section such as this.

Consider for example a teacup. The handle of the cup doesn't have to mate with the cup, in terms of vertices and edges. It can simply disappear into the cup. Or be lined up perfectly with the exterior of the cup. It will not crack because the cup part is behind the handle.

A crack appears when there is nothing visibly backing the parts that do not mate exactly. The reason for the cracks is the vertices are always moving slightly, and so a visible seam will open up, unless every vertex is matched. If the vertex moves, and there is no matching vertex, its only a problem if that opens up a view to "the void"; or the empty space inside of 3D models, or a view into the backdrop, or a compartment that should not be visible.


EDITED: I also want to add. That some tiles may have protrusions, like little windowsills or shelves. I'm not sure what to call them. Fluting. Anyway, I don't know how From's artists modeled these. But I reckon it's more than likely that they don't have anything behind them. In that case, it's probably better to make these parts separate from the wall behind them, because otherwise they introduce complicating vertices along the edge of the wall. So if you have a wall like this:

-------
A
====
B
-------

And A and B are separate in the model. My advice is to break the ==== part out, and make A and B one piece (a single quadrangle) and just have ==== sit over that piece. It's probably not a problem if ==== doesn't run into the outer edges of the tile, but I do not know what is better practice to be honest.


PS: Please do not try to fix the black parts of the textures that manifest as holes. I want to fix those myself so I can ensure that the procedure is mathematically sound, since this is not as simple as it seems, because it isn't merely pure-black parts that form holes, and neither is it any pixel where one of the color components is pure-black (basically it's any one where all components are between 0 and 7, and so care must be taken to select the correct replacement color, with respect to adjacent pixels)

[To be honest I haven't even decided. It's very tempting to say only pure-black is a hole, and repair the fallout from that decision instead. It isn't merely tempting. I realize that this is probably what I will decide to do. The only reason I hesitate at all is it will break all of the non-tile model textures, and will be a breaking change from classic SOM. But the second problem doesn't concern me so much, because that's a dead end regardless.]
« Last Edit: January 13, 2016, 08:03:01 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Verdite

  • ***
  • Offline Offline
    • View Profile
Verdite says,
« Reply #2 on: January 17, 2016, 10:41:54 PM »

Checked over the whole of set 1, and amended 3 major offenders. The worst offender was 0030, which had to be rebuilt from scratch along with pixel perfect UV replication.
Files up for testing in the Public and joint effort files section, to make sure we're 100% on the same page.
I'll step up set 1 after checking over set 2 on my next bout.

Cheers,
:smug:
Author of Moratheia. Lurker, for now.

Verdite has 59 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #3 on: January 18, 2016, 06:56:05 AM »

There will definitely be growing pains around the AA technology.

Ultimately it might make sense to automatically insert the extra cuts in the polygons when generating MPX files. Still internally the models must be free of cracks. And I don't want to automate this unlessuntil we get to a point where per-vertex-lighting is never to be used again.

I was thinking about this while waking up today. There are a few different ways to approach the MSM files. I'll leave it up to you to decide what is easiest.

Strikeout: I thought there were more, but I realized one wouldn't work. So there's probably only two approaches.

The trouble is for detached pieces of a model, unless the triangles dig into each other, there is always the theoretical possibility of a a visible crack if you can line the view up so you are looking perfectly along the separation between the parts so that the perspective is cancelled out...

And having the triangles dig into one another is an option, but I worry it will cause Z-buffer artifacts, and will not produce lines that are as clean as they'd otherwise be. That is because the Z-buffer will determine the formation of the line--not Z-fighting, which is a separate kind of artifact. I don't have enough experience with this to say one way or another.



That said, the simplest way I could think of to do the terraces, would be to make the walls separate from the floors, and extend the edges of the walls until they touch the bottom of the steps on one side, and the top of the eave on the other side. This would introduce no new polygons, but might result in UV-mapping issues since the part of the texture below the wall might bleed into it. And since the wall is separate, in theory if you can get the view lined up directly along the wall, there's a small chance of seeing between the wall and the floor.

EDITED: I tried to think of another way, but I couldn't, other than cutting every 1/2 meters along the side as already suggested. That introduces more polygons, but x2msm does the same more or less. It may be more work to balance all of the new UVs unless your modeling software is very good at that, or is able to manage vertices without UVs (which is something I do not know if DAE can natively support or not.)

PS: I will try to setup a test project to try these new tiles out today or tomorrow. I kind of hoped you'd just understand what the problem is and work on it until all the cracks go away in the player. But I'm curious to see, if only because always working alone takes its toll on me. Reviewing others' work makes me feel like I'm in some kind of company.
« Last Edit: January 18, 2016, 07:16:49 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #4 on: January 18, 2016, 12:09:44 PM »

Update

I setup a test project. I can see at least one place where you've altered a tile. What you've done is in accordance with what I described as "the simplest way" in the above post. However you only did the alteration on the top. The bottom half of the wall remains. To do the bottom in the same way part of the wall will be hidden behind the step portion, and the triangles cannot be cut...

I don't like hidden sections of polygons, but if you want to do it this way that is fine in this case, as it is a difficult one. Polygons could be added so the part that would be hidden could be minimized, but I worry how it might affect vertex-based lighting to have skinnier sections like that.

The other approach described is likely to yield more even vertex-based lighting, since it would simply carve the walls up into regular grids. What to do is up to you.


PS: I don't know how x2msm's splitting algorithm works. The only way I know to avoid it for certain is to make the polygons small enough that it will not try to split them further. As it splits it may introduce new vertices. So if the splits are regular, then they should be mirrored in the other tiles, but I won't be surprised if this becomes a complicating factor.

PPS: About half of your tiles I looked at no longer match the rotation of their icons/originals. They will have to be spun around at some point. Please don't lose faith.
« Last Edit: January 20, 2016, 08:41:53 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Verdite

  • ***
  • Offline Offline
    • View Profile
Verdite says,
« Reply #5 on: February 04, 2016, 10:15:46 PM »

Hello, I'm pretty much back to normal so I felt like dropping a thought off here before I set about making fixes...

I've been thinking over where I went wrong on the lower half of the model, as I followed the same design / AA fix principals for the whole model. I've attached a comparison image to show where I made amendments, if they are wrong please let me know.
Author of Moratheia. Lurker, for now.

Verdite has 59 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #6 on: February 05, 2016, 12:32:19 AM »

Hello, I'm pretty much back to normal so I felt like dropping a thought off here before I set about making fixes...

I've been thinking over where I went wrong on the lower half of the model, as I followed the same design / AA fix principals for the whole model. I've attached a comparison image to show where I made amendments, if they are wrong please let me know.

I am not sure. In the case of this model, the windowsill like pieces may cause a problem. I cannot tell if they are separate or not, but if they are, then the full height of the wall can be a single section. They are clearly detached from the wall that forms the back of the niche.


However, I get the impression that maybe you think my reply above was in regard to this tile. If so. Just know that it wasn't. My main concern is the walls that form the terraces. They pose the biggest source of do_aa related cracks. So maybe you'd like to post those pieces instead?

I expect the proper way to do this is to cut the walls every 1/2 meters. So if you can separate the windowsill-like sections on this piece on display, that would ensure that they would not interfere with the 1/2 meter cuts.


BTW: I do not know if the triangles are built into the MSM files or not. They are capable of storing quadrangles and larger polygons. Unfortunately even though the Assimp library used by mdo2x is supposed to support polygons, I could not get it to accept the output of the loaders I developed for SOM, so they had to be triangulated. I think actually the problem was that Assimp only supported "verbose" models, and I wasn't about to output such models, and so they had to be run through a post-process to convert to "verbose" and that process did not support non-triangle data. These are things I am presently working to overcome, by taking the Assimp code-base into my own hands on behalf of SOM/the open-development 3D scene at large (edited: MDO cannot store non-triangle data. MDL can store quadrangles, but I don't think polygons, or I don't think I've ever come across a file with polygons anyway. It won't matter after converting to DAE, since these proprietary formats will only exist in temporary folders. And that will make it much easier to iterate on the formats, since the temporary cache can just be cleared if new features make old files incompatible. DAE will never be incompatible)

But what I mean to say is, even though it's extra work, assuming the pristine files you've cataloged so far will be the future art files for these tile sets, it's good to have the unnecessary diagonal edges taken out of the picture as you have done here.


I realize it's not easy, but neither is it impossible. You just have to reason about the AA. Imagine every way the tiles can be arranged (don't consider arrangements that introduce missing sections) and make sure there are no unmatched vertices for any of those arrangements. If you cannot learn its requirements you simply won't be able to produce models that SOM is able to use ever again :evil:

In order to make this compatible with vertex-based lighting, generally it will be necessary to add vertices, and then propagate those vertices further into mating tiles. Which is why cutting at every 1/2 meters is best, since this will produce more even lighting patterns (until we ultimately transition away from vertex-based lighting. The pressures are too great not to)
« Last Edit: February 05, 2016, 12:39:22 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #7 on: August 06, 2017, 09:27:49 AM »

Update

I've come back to SOM after a long break, working on COLLADA. And it may be only a short break from COLLADA :sweatdrop:

I've been looking at this problem again. The landscape hasn't really changed. I can't really use COLLADA to get SOM's models into Blender. But after a break, and having forgotten the challenges involved, it occurred to me to use the old mdl2x tool, ran through Assimp. And luckily there's an export function in Assimp's command-line tool now, so I didn't have to make a program. Which might've been a nonstarter.

I used Assimp to convert from X to OBJ, and imported the OBJ into Blender. Then I remembered there were problems getting non-triangle polygons out of Assimp. I wanted to make a collection of SOM's models not in its proprietary formats. Eventually COLLADA would be used. But I thought X was good starting point. I added a trick to mdl2x so it can communicate to the SOM loaders that it wasn't doing post-processing. Assimp had required exploded triangles just for its validator post-process to work, and I think its loaders all outputted such triangles; not bothering to preserve polygons.

But then I realized that it was for naught if the X->OBJ step triangulated the models, but surprisingly it didn't. So maybe that problem finally got addressed. I certainly didn't have the sway to move the Assimp gatekeepers back in the day.

Also, x2msm would not work with the models produced by mdl2x. I didn't know in the past if I couldn't make x2msm (and friends) work or if it just silently ignored certain kinds of X files. I remembered I had some X files in a folder that were given to me long ago. They worked. So I sat about trying to figure out what the difference was. And someone had asked me to make a workaround to get text X files out of mdl2x, and that made it practical to hand edit the text until it would work...

So what I determined is x2msm requires a single mesh to be global and not inside the scene graph. And it ignores any additional meshes. I guess embedded meshes appear to it as if the file is empty. Modern day apps would use D3DXLoadMeshFromX, which is a horrible API that shouldn't exist, since it encourages this kind of ignorance. But I'm not sure that's the API that x2msm would've used. Possibly so. Possibly it depended on the retained-mode DLL in those days.

It also refuses to acknowledge files that have names that don't limit themselves to a-z A-Z 0-9 and _.

I think the biggest benefit of this may be that I can use mdl2x to mess around with x2mhm now. I intend to do that before long. I've always been mildly curious about MHM and now it's time to really figure it and MPX out. An impediment to that is I could never quite make out what was what in the MHM files, and I didn't have systematic way to make tweaks to the X files. I guess if I'd thought about it I could've used those same contributed X files and hand edited them, but I seem to have no shortage of blind spots.

So in the second attachment is the result of bringing the MSM files into Blender. The diamonds I added to note a few models that don't have PRT files and so don't appear in SOM_MAP. These have names following the same pattern as the rest of this set...

The theory I formed about the two doorways (after I made this screenshot) is I think they are same doorways, that would've been arranged back-to-back. My guess is it might more closely resemble how secret passages are done in the PlayStation games, but that it was scrapped because it was too complicated for a consumer product. (6 whole tiles dedicated to secret doors does seem like bad presentation. Really the door should've been built-into the whole apparatus and not treated as an object, so it would match the lighting.)

There are two ceiling pieces that seem to have receptacles for ceiling hung traps. Just a theory. And 4 pieces that I thought was a tunnel system, but I see that they are transitional only, and I'm not sure what they would connect to except for each other or possibly one or more of the other sets.

These missing pieces might have analogues in King's Field that didn't make it into the port in order to streamline the editor itself.


Sadly this morning I learned that the split-level pieces that transition to different elevations are impossible to fix for do_aa with x2msm. I played with them a lot to try to get x2msm to subdivide them so they would match the regular walls. I learned in the process how x2msm's subdivision algorithm works:

What x2msm does is not clever at all. It doesn't subdivide based on anything at all, except for a 1x1x1 meter grid that has an origin at 0,0,0. My guess is it has a multilevel subdivision tree system only to repair the cuts made by this initial subdivision. I can't recall if it subdivides down to triangles or not.

This is not a good system for attractive per-vertex lighting. Or shadowing as it were.

But it means that two tiles that differ in elevation by 1/2 a meter can never line up on that 1x1x1 grid.

This means for do_aa that I must develop something for generating new MSM files. I'm already planning on doing something that does this, so I might just get on with that, or I might use x2mdl. I think I'm more likely to do the former, unless I already added MSM to x2mdl at one point. Then I might consider it.

On the plus side, I'm somewhat relieved that there's no cause for me to have to replicate x2msm's subdivision algorithm for any reason. It's not a good algorithm, so if anything I should create a new one and run all of the MSM files through that. But right now, in the immediate term, if I work on an MSM outputting tool, I will want to use it to fix these problem tiles, and to manually subdivide them. So the new MSM tool would not require any subdivision step. Not for now. The only long term value to subdivision I think, is that it avoids baking the high-resolution vertices into the model that artists work with. Which would just be clutter for them. That may very well become a non-issue shortly, if artists will be advised to switch to using COLLADA.


I've added a second image. Only as reply to the images provided by Verdite before, two posts up.

I've illustrated the necessary changes (I made a day or two ago) to the first set of SOM's tiles. The number of changes are relatively small in truth. It's possible the other sets will require more changes. But it's also very possible that this is the most involved set with regard to changes.

I think based on recent emails that Verdite has a better understanding of the do_aa extension today. I put the niche front and center to show how it needed to be changed. It can only be seen from behind. I didn't keep before images. So I've drawn diagrams around the changed areas.

In the niche the back wall has been made into a single piece, so that its edges match adjoining walls whether they are niche walls or plain walls.

For the bridge piece in the back, the floor part is extended all the way across, so that it matches adjoining floor pieces.

The other piece is the split-level wall. This is the hard case that can't be addressed with x2msm. Because its subdivision step spoils it. Its wall must match the other walls. That's complicated by the walls meeting at a different height.

I've changed it by making one of the walls seem to have a webbed connection to the floor, that is out of view. I regret that this is not so attractive from inside of SOM_MAP. I tried a different arrangement with two disjoint flat panels also, while trying to make sense of x2msm. It looks a little better, but is still confusing I think. A better way would be to split it up. But if you do that either A) all connecting walls must be split, which would be a lot of changes. More than I can do. Or B) You have to hope that x2msm would do the splitting for you. But it's now known that cannot be the case. Maybe an alternative subdivision algorithm could do something like that. But it can get tricky because adding uniform splits can be like unraveling knitting.

EDITED: I also did not mention that, theoretically it's possible to create degenerate polygons. But I don't know if modeling software would be happy with that. It's something that an x2msm replacement could do with instructions of some kind embedded in the models. Another possibility to make SOM_MAP looks better, is similar, but just short of degenerate triangles. To do this the webbed parts could be cut back. But that disrupts the UVs and lighting, and so is very hard to manage with UV maps, and might not look so good with per-vertex lighting. I'm sure I theorized this in earlier posts.

So, in conclusion. I've put at least two full days into this. And I'm beginning to be able to do things with Blender. I'm by no means impressed with Blender. But I'm relearning it. Which is something I'd hoped I'd have the opportunity to do. Only last week or so I threw up my hands with working with Blender, and regretted the possibilities. But after I gave up trying to import COLLADA, I made some progress. If only short term.

My work on an alternative to Blender for SOM is taking longer than I hoped. So to try to accomplish some things, including do_aa, sooner than later, I am just trying to make ends meet again.


P.S. The diamond marked 5 looks like it might connect to the lofted ceiling beside it. But it's not. It's like the ceiling beside it. Recessed for some reason. But seemingly of cruder construction.
« Last Edit: August 10, 2017, 06:13:10 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #8 on: August 11, 2017, 09:15:51 PM »

This afternoon I've tried to figure out how to interpret a COLLADA file as one of SOM's file formats. Blender's built-in COLLADA exporter is not so hot. I've been asked before to come up with a system for not having to set MSM tiles at 0,0,0 to export them...

I first tried to use a joint, or bone, or whatever Blender calls them; but everything I tried produced laughable results with the COLLADA exporter, that wouldn't be usable. Probably it's completely useless for animated MDL models. I don't know how the hell Second Life or anything could use it...

But what seems to work is actually pretty nifty for MSM management. Blender has an "Empty" widget. And it can be used as a parent to the tiles. I've attached an image of it. I've made it use the Image mode, and set the tile's icon to be the image. The color is the same always, and must be changed in the theme Preferences. It is black at first, which is pretty much impossible to see.

EDITED: Also set them to show Axes, a Lable, and X-Ray mode.

Because the tile has a container, it can be moved around. And the container here should even remember which direction is north for the tile, unless I misunderstand it.

(I think the tiles need standard directions. And some of the legacy tiles may have to be fudged to achieve this.)

The real problem is I would like to be able to drag-and-drop the DAE files, without providing any context. I also don't want to label things with MSM and MPX and so on. Because the goal of this exercise is to abstract away from runtime/proprietary formats. And those are just awkward acronyms that people shouldn't have to memorize.

Blender has next to nothing in terms of frills. So its COLLADA exporters can't provide any real information even if it wanted to. Not unless they cooperate with Add-ons.

If MDL can be eliminated, that just leaves MDO, MHM, MSM, and MPX. My strategy for MSM is to look for the word "tile" in all permutations in the name of a node, and if it doesn't have model data, assume it's a container. For MHM: anything that doesn't have a texture is automatically MHM. Textures are about the only reliable feature. Textures are required...

MDO/MDL is everything else except MPX. Hopefully "tile" doesn't clash with names. Maybe "plan" can be used as an MPX container. And something else for other objects, that trumps both "tile" and "plan"? Content?

Strikeout: I'm actually considering emitting MDO files for tiles too, so that MPX can eventually have transparent elements and lighting. I may keep MHM as a non animated MDL format, so objects can use MHM too. That just means figuring out a way to mark tiles for emitting MSM.

EDITED: The icon file is not exported. It's just for reference. I'm not sure if it could be used if it was, since the icon is more of a PRT thing.
« Last Edit: August 17, 2017, 02:29:02 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #9 on: August 13, 2017, 07:53:04 PM »

*Exacerbated smiley* 1 set down. 4 or 5 to go. Then 4 rounds of copies. (Apparently at least the final set isn't a straight palette swap. So I can't assume anything.) Of course, the work is small potatoes to the technical hurdles.

I'm so tired of manipulating Windows Explorer :grumble:

Either Blender or its COLLADA exporter outputs relatively small floating-point errors, but not minuscule. On top of the dragging and dropping I would drop into the text editor and edit these errors away. Of course they could very well be in the mesh data itself if Blender's math is just sloppy.

I wonder if they are enough to open up seams themselves. If it becomes a problem the new conversion process will have to round the data off. It's features like this that modeling software always seems to lack.

I worked a new component that I'm calling COLLADA-SOM.dll for the time being. I use it by dropping DAE files onto SOM.exe. It saves them to a folder like C:\Users\Michael\AppData\Local\Temp\Swordofmoonlight.net\file----C--Users-Michael-Sword_of_Moonlight-data-map-msm if the file comes from the map/msm folder (or anywhere on your computer. Theoretically anywhere on the Internet.)

Historically CSV files are dropped on SOM.exe to trigger the updater. This extends the idea of updating to the 3-D models themselves. Eventually I'll either change its Open function to accept DAE files in the filter, or add a new menu item. That will include a COLLADA viewer also.

This workflow is going to make everything much more manageable. It's very good, but will be disruptive. I guess some of the little side projects I've done in the past that expected to use SOM's files and file tree won't make a whole lot of sense after all this.

The image shows these problem tiles. They come pre-divided up. For now I'm just modifying the MSM files. This will overwrite files in the DATA folder. If you know someone with a modified DATA folder, remind them to make their own Data folders with the new SOM_EDIT Settings function. I know it's not so simple because some of the accessory tools expect to use the DATA folder.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #10 on: August 17, 2017, 04:50:17 PM »

The second tile set doesn't seem to have any do_aa issues. It broadly works on a 1m basis or if it has any that aren't so, they are so irregular as to not mate with any other tiles.

The third only has the 1/2m rise (technically a fall) pictured here. I actually did this one last night; but I simplified the wall so that it was like a plain flat wall, since that's what it is. But I realized afterward that that might clash under a lamp light. So I went back and just did the subdivision as MSM would if it understood elevation transitional tiles.

It looks crazy, but I reckon just let the chips fall where they may. Vertex-based lighting is always a bit of a crap shoot. The level walls would look pretty similar fully subdivided.


I added little handles to this Blender file, while importing the MSM models. It's a pretty fast process. Not completely painless because Blender links the icon when it is copied. So it has to be un-linked every time. It would be nice if selecting multiple files in the import menu would import multiple files, but it doesn't.

The best tool for these split-level tiles is the Bisect tool. It's basically completely useless. I thought Blender was poorly designed. But I realize now that its developers are just lazy bastards. There's no craftsmanship to Blender whatsoever. And so I guess it's hard to criticize something so poorly constructed, because you can never know if it's because of deprivation or genuine ineptness and carelessness. In any case, it doesn't inspire confidence.

Bisect would be perfect if I had a system for determining the plane normal to bisect along. But Blender won't even tell you the value of a face's normal. Bisect shouldn't even be expecting a hand inputted normalized value in the first place. Which is half of why it's an awful, half-assed piece of work.

In trying to see a normal I came across this (https://wiki.blender.org/index.php/User:Mont29/Foundation/Split_Vertex_Normals/Custom_Split_Normals_Manual) and this (https://docs.blender.org/manual/en/dev/modeling/modifiers/modify/normal_edit.html) which might just possibly be capable of softening things like the outdoor set's tiles. A plot of grass shouldn't look like a pyramid. But there's really so little data to it, that I reckon From's staff just gave up on the surface normals.


I didn't finish bringing in the rest of the cave set, because it's actually spread out all over the place MSM wise. It suggests that the staff started with the cave set. It's very disorganized. I'm trying to organize them into Blend files for future modeling work. I would like to tackle the lighting normals as soon as possible, and the UV maps scream for extensive work.


I realized that MSM files should probably be MDO files too. They need transparency information. I can't think of any good reason for the artificial division. I thought that MHM could be a lightweight MDL file going forward. That means if a model isn't animated, it can make do with MHM. This means that the new system only needs to know if a model is a map tile or not. And that is just so you can drag and drop it to get an MSM file back.

This simplifies the present use scenario. So instead of using words like "Tile" to mark things. I think I've decided to get away from English, and use symbols instead. But the problem is that COLLADA 1.4.1 names are NCName (this was a mistake) meaning that they can't include ASCII symbols. But they can include most Unicode symbols. So I think I've chosen to use WHITE LARGE SQUARE as a tile symbol. This means you can use any name you want to. (This symbol can just say, make a MSM file, and so it's probably the only one we'll need. Eventually something like <keyword>msm</keyword> can be used instead. But I'm trying to make this easy to do with only Blender.)

I would've put the square in the model's name, so it's not visible in the little fuchsia handles. But it's convenient to be able to copy the handle, without having to copy-paste the square into the names. Since the model names come from the exporter.


Set 4 has many tiles that require attention. And 5 is going to need some creativity too. It has its inset plaques and paneling. Then comes the outdoor set. It doesn't look too work intensive.

I expect to have this done by next week. After I am going to write a disposable little program that reads in instructions to generate palette swapped MSM files. Eventually this practice will be obsolete. But I'm writing the program because I think it will need to be run again in the future. If I do it by hand, I'll have to do it again and again, so I'm not going to.

The same program will fill in the holes in the bitmaps and output TXR files. It will be usable enough for other's projects to use.

« Last Edit: August 17, 2017, 05:14:32 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #11 on: August 17, 2017, 09:34:50 PM »

Some "porn" for SOM freaks :hearton:

I couldn't do this without the model database inside this website from years ago. It seems to be broken since my gracious host moved the website onto a different server earlier in the year. I need to work on it. I think it shows models that were viewed before, but it won't generate new models on demand. Fortunately I can get by on icons for now, as long as I'm doing tile work.

EDITED: Some of these islands (pictured) are not lined up on the grid correctly. The Viaducts actually connect to the Arches. I later put them together.
« Last Edit: August 18, 2017, 02:27:48 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #12 on: August 18, 2017, 02:34:03 AM »

 :confused: Hit another crazy snag.... *drum roll* (:drummer:) x2msm doesn't do its subdivision thing on polygons below 0!

I'm unsure how to proceed. I will get the problem cases for sure. I might not bother with cliff like pieces that seem designed to fall off of.

The Balcony tiles (pictured previously) are 2.5m below 0.
« Last Edit: August 18, 2017, 11:57:45 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #13 on: August 18, 2017, 06:03:00 PM »


Had a pretty awful half workday today.

I learned about Blender's "Auto Smooth" setting; confirming that Blender is truly the devil. I'm guessing you don't so much learn about it, as be driven to the brink of madness before offhandedly figuring out about this thing you never asked for and is nowhere to be found anywhere in the vicinity of any of the lighting related functions. (And the cherry on top is once you learn about it you get to spend the rest of your life disabling it and or reencountering it on a mesh by mesh basis as near as I can tell.)

Blender is anti-software. I want to take in its pain and learn from it how not to do things ever. I hope I can turn the pain into something good...


This problem manifested as messed up lighting where these edited models were concerned. Inside SOM and inside Blender.

There a second problem rolled up into it though, and at first they seemed like the same problem...

Actually this turned out to be a relatively minor problem. It was an MSM. The wall with a niche in the 4th set. But the problem is either not in the second copy of that set, or is somehow not as apparent. It would come out when the shadows are produced by a lamp light source.

I didn't realize the problem was in the original MSM. Because I was comparing it to its twin. Which didn't have the problem, or not so badly.

Let's just say that it looks so bad that if you tried to park a light near this tile, you'd either have to change the tile or remove the light.


Anyway, I thought this was a larger problem with my workflow. But really there were two bad vertices in this MSM's model. One was nearly on the 1m grid cut mark. So x2msm would merrily create a sliver of a triangle between it and the cut. That really throws off the shadow system it would seem.

The discrepancy is actually not very small. It's off by like 0.005, which is not simply rounding error. And it's probably too large to plug too. You're talking about rounding to the centimeter. The MDL files can do millimeter precision. But still, the effect was dramatic.

The other vertex was off by considerably less. Like 0.00005. It was not on the grid. But should've been equal to a sister vertex. It seems that the sister casted an inescapable shadow on the off vertex, or vice versa.


I did not expect any problems in the lighting department. But here were two. That seemed related, or the same, but were not.


Good news is the 4th set is finished. Although some of its twins are not strictly identical, like the 5th set. I will have to be very careful that I don't obliterate these differences. One example of small differences is that wall with a niche.

I did change the lighting for wall with a column set in it. It looks round instead of octagonal now. Which I think is probably the desired effect. In other cases, it's hard to justify straying from flat lighting.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #14 on: August 22, 2017, 12:36:00 AM »

For the record, I ran into a new problem days ago (I've been in town for the weekend.)

This problem is because 1) Blender is highly inaccurate. And 2) mapcomp.exe's shadow generation system doesn't have any kind of threshold. So that a number like 0.000001 will cast a shadow on a 0.000000. It probably has some sense of an epsilon (very small threshold) by some mechanism, but it is very small if so, and it really should be treating these numbers as equal.

Blender introduces major errors when you use its Parent->Child relationship system. You can create this relationship, and then you will have errors, that cannot be seen in the numbers. You can then set these numbers equal to themselves to erase the errors. (A typical Blender WTF??? moment.) But that's impractical.

I don't know what artists do to make custom made SOM art pieces ... but it seems like we've been very lucky up to now. Call it some kind of providence, even if nothing much has come out of SOM user communities thus far.

I've thought from the beginning that something like this would be in order. I didn't expect this though.

My naive approach to rounding didn't work. It would produce acceptable shadows, but was insufficient for do_aa. Although it might have worked if every file had the technique applied to it, but here I am mixing existing MSM with new MSM.

Thankfully though I was able to research and learn about less naive ways, that do seem to do the trick.

So I've implemented this rounding for the new pathway hosted by SOM.exe (COLLADA-SOM.dll) and also for mdl2x, since I'm using it to convert DAE files to X files, for x2msm. Except in the latter case it's designed to work with Visual Studio 2010 which didn't have a rounding function, so I had to improvise one, that I assume will work.

So for now, I'm back on track.

Actually I ran into this problem with the 4th set's split-level tiles. The tile that serves as a middle piece for those, has a texture that is completely mismatched. So I'm going to try to repair that texture; to map it to the right section of the image, or change the image if it's using the wrong image.

I didn't intend to do texture work. But I think if I find these total mismatch problems, I should try to address them immediately.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2730 posts
Pages: [1] 2