Page 1 of 2


PostPosted: Tue Nov 13, 2018 7:21 pm
Hi folks, so..... its been an exceptional amount of time now since I have last posted an update (4 years almost!) and its about time I made one.

Mapscape 3

So mapcape is still in development, and is taking much longer to develop than I originally anticipated. I would like to state that I have only had my free time in between real "life/work" to work on mapscape. However I have only recently found myself unemployed for first time in 12 years, so I am putting all my effort into mapscape whilst I find alternative work.

An alpha version of mapscape was released back at the end 2016 and has had several updates since then. You can try the mapscape alpha here >

Its a bit of a buggy mess as of the moment and will be sorted in due time. I would also like to state that changes will be made that might make the current save format obsolete, so don't get too carried away with making levels with the current version. I do get asked often for an ETA on when mapscape will be available to buy, to this I have no idea as some things may take longer to add than anticipated. But now terrain and culling is in and most of the hard to do "stuff" is over and done with. Purchasing mapscape will allow access to exports to which are *.Dbo *.Fbx *.x *.Obj *.Ent.

Work in progress:

Terrain texture improvements
So the limited number of texture layers (currently five) has been a bit of an issue when it comes to making nice and varied terrain. So I have spent a ridiculous amount of time trying to come up with a better texturing method with a half decent number of texture layers. Originally I invested some time with texture atlases, but a big issue with atlas maps is seems inbetween textures to which cannot really be fixed due to texture filtering across the texture atlas on lower mipmaps. An alternative way was to use cube maps with each face individually fetched. This would allow up to six textures per texture stage getting around the pesky 8 texture stage limit. However, this resulted in a large pixel shader with branching used to optimize and minimize texture lookups. This adds additional computation and now took to long to load and compile the effect. As a last ditch effort and continuing on from the cubemap idea I decide to try pre capturing the terrain layers into a massive cubemap, with each cubemap face captured form an ortho view from above the terrain (rendered as a 2d plane). This in effect works similar to how our cascade shadow maps function, so a closer render is used for the fist split then gets progressively larger untill the whole terrain has been captured. There were some doubts due to resolution qulity and memory consumption but this works surprisingly well, and without a doubt this has been the best way so far to tackle this problem.


At range imposters will replace your high poly mesh with a very low poly representation. The most common way is to render a 3d sprite with a 2d texture that can represent you original object. However, this will not work if your object has to be rotated or has non uniform scaling, so an alternative method is going to be needed here and I now have a pretty good method on how I can add imposters whilst getting around the above mentioned issue. This is done by captureing the object from all angles into a cubemap, then projecting this onto a cross section mesh. Rotation and scale is no longer an issue here when using this approach. but each imposter will now unfortunately consist of 6 polys instead of usual 2 used in a simpler "sprite" method.


Still to be added:

Terrain meshes/decals
This will be used for small details such as rocks and grass. This will combine multiple object instances in to a single mesh, using a vertex shader to position the mesh to the surface of the terrain. Hopefully this will be pretty straight forward to add, so I am hoping it wont take to long to implement as most of the work is done within the vertex shader. My only concern as of the moment is culling and making sure to draw only what is needed as meshes will be batched resulting in a single very large poly object, one for every potential mesh layer. Since I now have pre capture terrain texture layers, decals or stamps can easily be added to the render allowing for additional detail to the terrain surface with out impacting performance.

Media refresh
This was something actually being worked on for the last update, but was put on hold. This is actually going to be incredibly tricky to implement as I need to write in a new media management system that keeps track of all loaded assets. Any assets will need to be removed if the original source material no longer exists.

Im not to sure how to add water to the advanced lighting system due to the way things are now rendered in a deferred manner. I will most like try and copy the old system over then make a few changes to get it functioning properly. The issue I am facing is how to produce the refraction effect without adding an additional render. The current distortion system which is used for simple refractive surfaces such as glass will not be able to produce the correct distortion effect that is needed to make water waves look like... well real 3d water waves.

Possible features still to come

Smooth groups rework
the current smooth faces will only work across faces that are assigned to the same csg brush. Im going to redo the smooth system so smoothing can be apply across different faces even when they are not sharing the same csg brush.

Redundant poly removel
When editing vertex you can set individual vertices position belong to the same polygon at the same position. So when multiple vertices are ovelapping, this will actually make polygon redundant and no longer drawn. I will need to add a system here that will go thorough and check each vertex and remove any polygons where a member of its vertex are occupying the same position.

CSG instancing
This will collapse selected csg brushes in to one object and will lock the selection to act like a regular loaded object from the entity system, this will allow to create instances where the same csg brush may have been duplicated multiple times. This will reduce the memory footprint as well as hopefully increasing performance. Csg instances will be unlockable any time allowing modification of its individual csg components. I also want to add a way of applying level of detail (lod), As all csg brushes have a fixed polygon count with no real method of supporting lods. This is a bit of a draw back to csg brushes and probably one reason as to why most modern level editors now use a mesh biased approach. So I am hoping you could have the potential to build an entire house from just csg brushes, lock it as csg instance and then create multiple duplicates with no memory implications with lod support resulting in a decent performance boost.

Link entities with inputs outputs
This will create a link between entities with the ability to share keys and values between the link. This is going to be tricky to add but this is something that is needed to make entities more programable, especially when making something that consists of triggers/switches etc. I also may look at scrips also but mapscape will remain a level editor and wont be a game engine as such.

Tree it

So tree it has received several updates recently, although tree it is officially unsupported I do occasionally make additions that I personally want or required. I'm currently looking at adding 2 more additions at some point, but I'm not too sure when this will be available.

An .fbx export was added at the beginning of the year, not something I needed but was requested so often I ended up adding it. Other than a few other additions such as twist and presets its pretty much has remained the same tool.

The latest update focus was to make improvements to texturing. Texture mapping has been added allowing to export your model as a single mesh by assigning all individual textures into one texture atlas. Texture blend has changed from a shader effect to within the texture allowing the texture blend on exports. You can now also change the texture format you wish to export too within the options window.


Future improvements

Split trunk/branches
So I have actually alredey started to work on adding spliting of trunk and branches. although this will unfortunately only be able split the branch or trunk once, this will give least five growth layers to the tree structure where we currently only have 3 (trunk>branch>branchlet). I was hoping to add multiple splits but the current method Im using to create tree geometry would need a complete rewrite and that would take a lot of work. Hopefully this will be part of the next update and I will come back to working on implementing this once the next update for mapscape is out, but so far I can already tell is going to be one of the more tricky things to add, I also need to take into account joint editing with ability to modify and break them independently after the trunk/branch has been split.

So this will allow you to load custom meshes and attach them to your tree model, this could be anything from spheres to represent fruit/coconuts, or maybe plant pot for a small plant to sit in. I'm hoping this will open up the door to add your own custom leaf meshes. I also want to add a mesh editor allowing to make cutom leaf shapes that can help minmize alpha testing. As you can imaging this is going to be a lot off work and will require a new mesh tab to hold mesh properties such as mesh count, scale, position, and rotation, but I do believe this will be easy enough to add as this is mostly quite straightforward stuff. the only concerning thing is setting up correct vertex colors that are used for handling wind animations.


I think thats it for now, there are other things I could add, such I'm thinking of making texture tool 2.0 for mapscape but this is so far down the road there is no point in talking about it.

I would like to personally thank those who have donated since I recently put the donation link backup (on the treeit page).


Re: 13/11/2018

PostPosted: Wed Nov 14, 2018 9:04 pm
by just1
Nice to see you here again, Evolved! :D
Great work!Will be waiting your next release!
Few words:
Terrain - Still use one of your previous demos - with atlas system and 8 separate textures.
It works fine and I don't remember any glitches or artifacts with seams.They must be visible only with special view angles?
Instances - You should definitely check new version of jGfx plugin it has new powerful instance system.
Shadows - You move from VSM shadows to some new technique?Is it better?
TreeIt - It is propably best free tool for tree creating.Still it will be great if you fix cropped render to texture window.My post about this is here:

Re: 13/11/2018

PostPosted: Fri Nov 16, 2018 8:06 am
The older terrain did use a somewhat of an atlas approach, but would still produce seems on lower mipmaps at a distance or at a grazing angles (hard to see but still there). The approach Im now using works pretty well and is similar to the method used and developed by Josh described here >

I'll take a look at jGfx istanceing.

Im using shadow masking, this basicly separates the shadow output into its own render target making it available to blured in screen space. Good explanation here >

The main reasoning behind the change from evsm, is vsm produces light bleeding when shadow penumbras overlap and evsm will not fix this completely. There is also another shadowing issue where shadows are lost on smaller casters thanks to brighter pixels overriding neighbouring pixels when post bluring the shadow map.

I did briefly change the shadowing approach to a more conventional pcf with linear interpolation between neighbouring samples, essentially the same result as what hardware shadows produce. But this approach relied on large number of samples and complex code with not so good results. The current mask approach is mainly used to tackle shadowmap shimmering by applying dithering to the shadowmap. But this can make shadows look horrible (gta 4?), but dithering artifacts can be greatly reduced with a screen space blur resulting in a decent soft shadow.

I will see if I can make the render to texture cropping guidelines more accurate when I start working on the next update.


Re: 13/11/2018

PostPosted: Wed Nov 21, 2018 8:24 am
little update

Ive been working on implementing percentage closer soft shadows with some pretty good results so far. Performance is issue with this technique due to large number of samples used. I have optimize the method as best as I can and now works ok but is something reserved for ultra high end gpus.


Full res image here>

Re: 13/11/2018

PostPosted: Sun Nov 25, 2018 5:31 pm
by Chulain73
Glad to see you are still working on this Evolved, the introduction of imposters is exciting to say the least. I hope to change my project over from the old AL to the new version once you get it fleshed out, good luck with the changes.

Re: 13/11/2018

PostPosted: Sun Dec 02, 2018 5:58 pm
So I've now made several updates to the Advanced lighting sys with the latest includs parallax mapping applied to terrain surface. This will now most likely be the last update for some while.

Parallax on:

Parallax off:

As performance is now something of a concern I have decided to start work on implementing terrain mesh surface detail used for flora as well as some form of an ocean shader. Once these two performance hitting additions are added I can then start see where Im at with regard to overall performance and look at where I can add crucial opermization.

Re: 13/11/2018

PostPosted: Thu Dec 06, 2018 11:13 am
So I have some good news, Ive decided to implement physically based rendering as part of the main lighting module. So far I have a new directional light which so far looks considerably better than the basic current diffuse and phong specular setup. Cook-torrance BRDF is used which plays a key roll in making this look as good as dose, but this will all come at a performance cost unsurprisingly. I'm using the metallic workflow, so the texture used are albedo + normals + third texture that stores metallic (0 or 1), roughness and ambient occlusion. The current issue is there will be no option to have pbr on or off as texturing so specialized to the pbr work flow and looks a bit bizarre when used under conventional lighting methods. Im currently moving form cube maps back to dual paraboloid mapping, hoping that this will ensure the mip chain give similar results to prefilted cubemap which is used to ensure all light is distributed equally across all cubemap faces.


Not sure as to when this will become available but I'm hoping this wont take to long to convert, once I have reflections sorted should be quite straightforward.

Re: 13/11/2018

PostPosted: Tue Jan 08, 2019 12:20 pm
Just want to give a little update,

what an absolute nightmare this is turning out to be, I have all the shaders done etc and was battling for weeks with performance. but after butchering the shader code a bit (optimizing it Ha) Ive managed to get this to run just as good if not better than the current available system.

I do have some big issues with terrain that Im still in the process of trying sort out which is the cause of the current delay. Hitching was an big issue but seems to be sorted now as the virtual terrain texture update is spread across a larger number of frames rather than the ridiculously large 1750x1750 texture on every single frame. Also improved the mip filter that was causing an anisotropy effect at steep viewing angles resulting in shimmering. As of now Im working on blending the current and previous virtual terrain texture updated to prevent a popin effect as a result of spreading the texture update.

Thats it for now,

Re: 13/11/2018

PostPosted: Thu Jan 10, 2019 6:42 pm
by just1
Nice that you keep coding Evolved!
1)How you will implement Vertical Mapping using your new terrain technique?
2)Can you provide any code from CubeMap tiles terrain tryouts?

Re: 13/11/2018

PostPosted: Fri Jan 18, 2019 7:25 pm
To hell with it.

After going round in circles I've decided to upload what I have done so far to the advanced lighting system. A large number of changes have been made here and not everything has been fully tested so bugs present.

You can still download the older version here>

Vertical Mapping is working with the new terrain method, This is done in a similar manna to how its done with geometry by projecting textures along the XY and ZY world position, with Y coordinates generated from the terrain heightmap in the pixel shader. You can open up the "LayersA.fx" to see how texture layers uv coord are generated.

I no longer have any of the older code when using cubemap to texture terrain, it was pretty useless as texture sampling became quite substantial resulting performance hit.

However I individually fetch cube map faces and repeat them by using something like this.

float3 FacePosX=texCUBE(YourCubeSampler,float3(1,frac(V)*2-1,frac(U)*2-1));
float3 FaceNegX=texCUBE(YourCubeSampler,float3(-1,frac(V)*2-1,frac(U)*2-1));
float3 FacePosY=texCUBE(YourCubeSampler,float3(frac(U)*2-1,1,frac(V)*2-1));
float3 FaceNegY=texCUBE(YourCubeSampler,float3(frac(U)*2-1,-1,frac(V)*2-1));
float3 FacePosZ=texCUBE(YourCubeSampler,float3(frac(U)*2-1,frac(V)*2-1,1));
float3 FaceNegZ=texCUBE(YourCubeSampler,float3(frac(U)*2-1,frac(V)*2-1,-1));

Hope that helps.