SceneryDesign.org
After the recent move of the FSDeveloper site to another server some of the download links to my tools as reported here on the tools pages were broken. I have now fixed them all.
All of the files are now hosted on this website directly, so you browse all old stable releases in one place if you want.
Today I have finally deprecated most of the pre-OGR importers. So that means that IMPORTSHP, IMPORTOSM and IMPORTKML are no longer available in scenProc as steps. You have to use IMPORTOGR from now on to read those (and many more formats).
One of the main reasons to deprecate them is to get rid of the dependency on the Microsoft Access database driver, which was used by the shapefile library used by IMPORTSHP. This dependency gave a lot of crashes, since it seems not everybody read the manual to check what other dirvers need to be installed.
With the switch to OGR this problems should be less. However I am seeing some crashes now from people that are related to the OGR libraries as well. These seem to be caused by wrong .NET framework versions. Make sure you have .NET framework 2.0 and 3.5 installed when using scenProc.
One of the issues with creating autogen is that vegetation polygons might suppress buildings. When the vegetation overlaps the building the building will not be shown. This can be a bit annoying, since you would like the building to suppress the vegetation normally.
To work around this problem I have added a new step to scenProc. With this step you can subtract features from each other. So this allows you to subtract the buildings from the vegetation polygons. That way both of them will show up correctly in the autogen. As you can see on the picture the step also allows you to add a small buffer zone that removes vegetation that is near the building.
The syntax of the step is as shown below. The first argument is the filter to select the polygons you want to subtract from. In this case I’m selecting forest polygons. The second attribute is the filter to select the features that should be subtracted. In this case I select my buildings. The third argument is the scaling that should be applied to the features that are subtracted. So in this case I double their size to get a buffer area. The last argument is optional. If you provide MINAREAPOLY there the algorithm will subtract the minimum area polygon of the feature (the best fitting rectangle), instead of the feature itself. For buildings that is often what you want, since the autogen buildings are created from this minimum area polygon as well.
SUBTRACTFEATURES|type=forest|type=building|2.0|MINAREAPOLY
From now on you will find a PDF manual inside the scenProc downloads. At the moment not all sections of the manual are finished, but I will try to complete it as soon as possible.
With all the recent changes to scenProc and with some of the functions being quite complex, I think it is important to have an up to date manual with the tool. I will for example describe in detail how the feature detection works.
This PDF manual will replace the manual on the wiki. I generate it from LaTeX source code that I keep together with the scenProc source code. So I’ll try to directly update the manual when I make some changes. That way you should always find up to date information.
A few weeks ago I made two versions of scenProc available on the development releases page. One for 32 bit operating systems and one for 64 bit operating systems. Since both of these files are rather big, due to the inclusion of all the GDAL/OGR libraries, it means that downloading an updated version can take a while. Therefore I have now added a third version, called scenProc update. This version only includes my own files and not the external libraries. So once you have installed either the x86 or x64 version, you can use the update version to patch it to the latest development release every time.
The autogen SDK shows us how the texture sheet that is used for generic autogen buildings is structured. There are parts for houses, medium/large building and warehouses. See the images below that comes from the SDK.
But the SDK doesn’t tell us which type of building get’s which part of the texture sheet. On obvious parameter determining which part of the texture is used is the height of the building and that depends on the height distribution that you set for the autogen file. But the length and the width of the texture also influence which part of the texture is used.
So I set up a test with buildings of different sizes and I made a table of the resulting texture that was shown. I have used the same colours as shown in the SDK picture below. So houses are pink, small/medium buildings are green, large buildings are blue and warehouses are green.
Below is the resulting table for buildings with 1 or 2 floors. The numbers shown in the headers are the width and the height of the building in meters. As you can see the small buildings all get the house part of the texture sheet, but once the buildings get bigger the medium buildings part is used (of course only the first two floors of that sheet). And even bigger buildings get the warehouse part of the sheet.
Below a similar table is shown, but than for buildings with between 3 and 5 floors. As you can see it is quite similar to the previous table. The white part indicates buildings that do not show with this height. These buildings do show with only 1 or 2 two floors and therefore follow the table above.
Below you find the table for the highest two building categories, 6-8 and 9-12 floors. The table is almost the same as the one above, expect that the large buildings part is used instead of medium buildings.
The scenProc IMPORTGDAL step to import imagery data assumes that the imagery you use is georeferenced in some way. So for example as GeoTIFF or as BMP file with an accompanying world file. But that is not always the case of course. Some developers use unreferenced images and provide the needed coordinates in the INF file for resample.
Therefore I have now added a new step to scenProc that will read these INF files: IMPORTINF. This will go over the INF file and for each image defined in there add it as a feature to scenProc with the correct georeferencing. Afterwards you can use the image in the DETECTFEATURES step for example.
The scenProc feature to detect holes from imagery often returns polygons with holes, especially in area with dense forests and some clearings in the forest. When the ProcessHoles option is set to true the holes will be processed for the output of the detect feature step as well.
But I have to give a warning, for big and complex polygons the output is sometimes corrupt. I will try to fix that issue, but for now you can improve the output by first applying a split grid step to slice your imagery into chunks. When the detect features step is run on those chunks the output is better in general. See an example configuration file below:
IMPORTGDAL|..\geo\imagery\gtif_wgs84_4band\321778.tif # SPLITGRID|0.001 # DETECTFEATURES|FTYPE=RASTER|filter_ndvi_test.tfc|OBJ|FOREST # MERGEGRID # EXPORTSHP|OBJ=FOREST|trees_test
As you can see I used quite a small grid (even smaller than the autogen grid). In the end I use a merge grid step to combine all the cells again and export them as one shapefile. That shapefile I can then use in my further processing to autogen.
An added benefit of the SPLITGRID step is that the performance of the feature detection gets better. When I ran it on the image as a whole it took around 4000 seconds, but now with the small chunks it takes only 300 seconds. So for better performance using split grid is also a good advice.
After all the recent experimentation with running scenProc in batch mode I was curious how many autogen objects I had created in the end. So I made a little tool that will count the objects in a folder full of AGN files. This tool I called agnstats and will come with the next agntools development release.
I know this tool is not really useful for development. But maybe you can impress your friends with the numbers or else you can use it to brag a bit about the amount of autogen you made in your documentation. Below is an example of the output:
Autogen statistics for folder: texture
Generic buildings 505028 5ae04eb6-934c-4f63-bb48-5e7dee601212 Roofs Gabled _ALL_ 21562 311de8c7-e596-4e28-a94d-da09d0252ce4 Flat roofs for medium short buildings Total: 526590
Polygonal vegetation 93168 db9149ee-bb9d-4da6-89c8-2c4900eed8fa Terrain Autogen Class Deciduous Broadleaf Forest 6118 a4a30975-075c-49ec-87fb-7e0931cb5004 Terrain Autogen Class Coniferous Forest 9349 c9dc45ae-f240-42a9-a137-b7617452a308 Terrain Autogen Class Mixed Forest Total: 108635
Rectangular vegetation 299482 c9dc45ae-f240-42a9-a137-b7617452a308 Terrain Autogen Class Mixed Forest 343208 2903412b-69a1-4a63-94c9-a799b91afdbf tb Bush General 3-8 ft Total: 642690
Library objects Total: 0
When you want to process bigger quantities of data with scenProc the graphical user interface is not always the easiest way to do this. In that case it is much easier if you can call scenProc from a script, for example a batch file. This is already possible for a while, but yesterday I have made some changes to make this mode of running scenProc even more powerful.
Let’s just start with some details on how to run scenProc from the command prompt. Below you see the syntax. The first argument you provide is the configuration file you want to use. The /run argument tells scenProc to directly run the configuration after opening, so that you don’t have to press any buttons to start it. This will also ensure that scenProc automatically closes when it is finished. The /log is optional, but when you provide it the messages from the event log are written to the indicated file when the processing is done. This allows you to check what happened later on.
scenProc myfile.spc /run /log mylog.txt
So far for the batch options that were already available (only the argument to generate the log has changed). What I have added now is that you can also pass additional variables into the scenProc configuration. This makes it possible to use one template scenProc configuration and customize using the variables exactly what will happen.
Let’s give an example. Let’s assume you have a folder with different shp files and you all want to process them with the same scenProc configuration. Then you should call scenProc each time with a different filename as variable. For example:
scenProc myfile.spc /run /log mylog.txt input13.shp
In your configuration file you will also have to make a change. Your configuration file would become like this:
# @0@= test.shp
#
IMPORTOGR|@0@|*|*|NOREPROJ
#
SPLITGRID|AGN
CREATEAGNPOLYVEG|FTYPE=POLYGON|{guid}
WRITEAGNFILES|FSX|texture
As you see in the IMPORTOGR step we don’t provide a filename, but the user variable number surrounded by @. You should always start counting from zero and the number is the order in which you provide the variable from the command prompt. While running scenProc will substitute this special code with the value you passed.
As you can also see I put a special comment at the top of the file. In this comment I also assign a value to the special batch variable. This value in the comment is only used if you run from the GUI, so that way you can still test your configuration. But when running from the command prompt it is ignored.
But of course you can use this to pass other things than file names. You could also use this to process your data in parts. Let’s take this configuration file as example:
IMPORTOGR|country.osm|@0@;@1@;@2@;@3@|*|NORPROJ
#
SPLITGRID|AGN
CREATEAGNPOLYVEG|FTYPE=POLYGON|{guid}
WRITEAGNFILES|FSX|texture
In this case the user variables are used to specify a different bounding box for the data to read. So you can specify different extends every time you call scenProc and thereby process a big area in pieces, without having to preprocess all your data.
I hope this feature is useful and I’m sure some of you will come up with even more interesting ways to use this.