BGLComp SceneryObject flags

A few days ago I added support to ModelConverterX to read the additional options like <NoShadow /> or <NoAutogenSuppresion /> from the object placement code in the BGL file. Initially I had some trouble to find out where these options ended up in the binary code, since it was not documented in the otherwise excelent description that Winfried Orthmann made a while ago.

After the latitude, longtide and altitude the description says there is a short that stores if the altitude is AGL or not. But actually this short can store much more than that. It more acts like a set of flags for multiple options. Here is the list of possible flags:

0x01 AltIsAgl
0x02 NoAutogenSuppression
0x04 NoCrash
0x08 NoFog
0x10 NoShadow
0x20 NoZWrite
0x40 NoZTest
Just wanted to share this, in case other tool developers are struggling with the same in the future.

Tree detection – part 3

I have made some more progress on the tree detection algorithm in the last days, but I am not there yet. So some more testing is needed before the functionality can be put in scenProc.

In my previous post I showed some results from the algorithm running in a test application. What I have done now is put it in scenProc itself. This means that scenProc can now read raster files through GDAL, run the histogram based filtering on the image and create vectors from the results (thanks to GDAL and OGR again).

Here are my first findings of putting the algorithm in scenProc:

  • It is still very slow. Running the whole island of Nantucket through the tree detection algorithm takes more than half a day. So I guess it would be a good moment to get the profiler tool again and find out where performance gains can be made.
  • The screenshots I showed last time were from images at a lower resolution. I had down sided them from the 30 cm resolution to make testing easier. It seems the filter that worked well on those, does not necessary work well on the full resolution as well. It seems resizing them made the trees slightly darker. So I will have to do some more testing to see which filter settings work well with the full resolution.
  • With the algorithm you get a lot of relatively small polygons. Most polygons cover a few trees. Only in a dense forest you would get bigger polygons. Creating polygonal autogen vegetation from these small polygons does not work well, quite often no trees show at all in FSX. Using rectangular autogen vegetation seems to work much better. Even for a small rectangle at least one tree will show up. So it seems rectangular vegetation would be the way to go with this algorithm.

And now back to some more testing…

Tree detection – part 2

I am travelling for my work this week, but when I had an few hours in the evening to spend in my hotel room I did some more experimentation with the tree detection algorithm.

I have now modified my test tool so that you can click on the trees in the image to determine the rules that are used for the detection. Just click on a dozen different trees or so, and the tool will figure out which rule should be used to find the trees in this kind of image.

From the tests I have done it seems to work quite well. Below are two screenshots of the Nantucket imagery where I used to same rules to find the trees in both images. I am quite happy with the result till now.

tree_detect1

tree_detect2

Next step will be to add this algorithm to scenProc, so that I can test it on a bigger area and see if the results still look convincing. I am curious to see how this autogen will turn out. But that will have to wait till I am back home, I don’t have FSX here at the moment.

Tree detection

One of the first problems I encountered when I tried to make autogen for Nantucket is the lack of vector data for the vegetation. OpenStreetMap contains hardly any vegetation and also the GIS department of Massachusetts has not much usable data.

So therefore I started to experiment with detecting the vegetation in the imagery.  And while playing with this idea, a similar tool popped up at the FSDeveloper forum.

The approach I am testing now is to check the histogram of a small part of the imagery and based on the average value and the standard deviation decide what is forest and what not. I have made a small test tool to see which algorithm works best. Below you see a screenshot of the imagery and the detected vegetation.

Image2013-06-02 2016.37.672

Image2013-06-02 2017.14.604

Of course it is not yet perfect. As you can see some vegetation ends up in the water for example. I have already seen that different images sometimes need different settings to get the right result. I am thinking about turning my test tool into some kind of configure tool. Hopefully if as a user you click on some areas that are vegetation and some areas that are not, I can have the tool figure out the best filter for that type of imagery.

Once I find an algorithm that works OK, I plan to put this functionality in scenProc as well of course. But for now there is more testing and exploring to do.

Below is a quick screenshot of the result I got now in FSX. I think it is not too bad, but there are still some bugs left. And I might have to pick a different type of tree to match the Nantucket area better.

Image2013-06-02 1139.18.253

 

Nantucket photo scenery

As I mentioned in my previous post I would write a bit more details about how I created the photo scenery for my Nantucket test project.

From the website of the Massachusetts GIS department I could download 30 cm imagery of the entire island. But they came in 101 tiles and in the SID format. So how to make them ready for a FSX photo scenery?

My normal approach would be to make a mosaic of the images into one big GeoTIFF and then it is quite easy to make a photo scenery with resample. But for the entire island at this resolution that would result in a file of around 20 GB. So that is not something that resample can handle.

So I took another approach here. First I used GDAL to convert all the SID files to GeoTIFF and I made sure they are reprojected to geodetic WGS84 as resample requires. I did this from the command prompt:

for %f in (*.sid) do gdalwarp -t_srs "+proj=latlong +datum=WGS84" %f %~nf.tif

So now I have a whole bunch of GeoTIFF files in the projection that resample can handle. But there are more than 100 hundred files. So that makes writing the INF file quite some work. To tackle that I made a simple BAT file that will scan all GeoTIFF files in a folder and create a INF file that contains them all. Here is the code of the batch file:

@setlocal EnableDelayedExpansion
@echo [Source]
@echo Type = MultiSource
@echo NumberOfSources = 101
@set icount=0
@for %%f in (..\geo\imagery\gtif_wgs84\*.tif) do @set /a ICOUNT+=1 & @echo [Source!icount!] & @echo Type = GeoTIFF & @echo SourceDir = "..\geo\imagery\gtif_wgs84" & @echo SourceFile = "%%~nf.tif" & @echo Layer = Imagery & @echo Variation = All & @echo NullValue = 0,0,0
@echo [Destination]
@echo DestDir = "..\..\scenery"
@echo DestBaseFileName = "Nantucket"
@echo DestFileType = BGL
@echo LOD = Auto
@echo SplitFileLOD = 10
@echo Compression = 85

As you can see there are still some paths and numbers hardcoded. If I would use this kind of script often, I would probably better make a tool for it. Maybe I can add the functionality to resample.

Then the last step was just to run the INF file through resample and wait a few hours. And there it is, a photo scenery made from a whole bunch of files that could easily be downloaded from the USGS.

Image2013-06-01 1652.32.735

Of course there is more to do. For example add a watermask to get the dynamic water. And add a blend mask to get a gradual transition at the edges. But those tasks I still have to do. Once I have done them I will write about my experiences again.

Test project Nantucket

I was looking for a way to help me develop the scenProc tool and explore the possibilities of autogen further. In the end I decided to start a little scenery project where I could play around with all aspects of autogen.

I decided to pick an area in the US, because there is a lot of data available from the USGS. That makes it easier to do a photo scenery. An island is always a nice location for a test area, because you have a natural border of the area you are working on, I decided to use the Nantucket island in Massachusetts. The state of Massachusetts also has a nice website of their GIS service where you can find a lot of data.

So I got 30 cm imagery of the island and processed that into a photo scenery. I will make another post about the techniques I used for this soon. Below are some pictures of the photo scenery in FSX.

Image2013-06-01 1652.32.735 Image2013-06-01 1652.53.864

So now I can start the challenge to populate the island with autogen. I’ll start with the vegetation and the buildings. And after that I’ll see what other autogen elements I can still add. Hopefully I can use the autogen to represent the atmosphere of the real island, as shown on the picture below.

nantucket1

By the way, I don’t know at this moment if this scenery will be an addon that I will publish later on or not. For me it is really a test area to help me develop the tools further. And making it ready for a release does not have priority for me. And the island also has an airport, so maybe I can test some new ground polygon wizard ideas there as well later on.

Blinking lights

In the attached object editor of ModelConverterX you can add lights to your object. On export to FSX these lights will be converted into effect (FX) files. Besides the colour you can now also specify a blink duration for those lights. This means you can create flashing lights.

If you enter a blink duration of 0 seconds you get a static light as before. If you enter 1 second you will get a light that is on for 0.5 seconds and then off for 0.5 seconds. If you enter 3 seconds for the blink duration you will get a light that is on 1.5 seconds and then off 1.5 seconds. I guess you’ll get the pattern. The smaller the number, that faster is blinks.

Image2013-05-26 2050.36.443

Dot vs. comma again, argh…

This must one of the most annoying “features” of the FS SDK tools. Many of them only work correctly when the decimal character is set to a dot. For example MakeMDL or XtoMDL refuse to work correctly when the decimal character is a comma. Many developers bump into this problem, since a lot of countries have the comma by default. I found this nice map on Wikipedia to show which country uses which decimal character.

But it would be too easy to only complain that Microsoft made their SDK tools in the wrong way. When developing my own tools I also often encounter this problem. Especially when working with file formats like SCASM or X files, that should always use a dot. When parsing or writing such files it is very easy to forget to add the instructions that make the code always use the dot.

My aim is to let me tools work well with both settings. But I have my own PC set to use the dot, due to the SDK tools that only work in that case. So therefore I tend to do less testing with the comma. So tonight I made some changes again and now the ground polygon wizard should work fine with both settings. Only when the SDK tools are needed, like when exporting MDL files, the user will have to change his settings. But in all other cases I try to let my tools work fine whether you use a comma or a dot.

But it remains an annoying issue, wouldn’t it be much easier if the entire world used the same symbol as decimal character?

Having fun with autogen – part 1

A while ago I started to explore the possibilities of custom autogen objects a bit more. I wanted to see what I could do with custom vegetation groups for polygonal vegetation. In this post I want to share some of my finding until now.

Using the Annotator configuration editor it is not so hard to add new objects to the AutogenDescriptions.xml file. Instead of adding a tree model, I decided to add some models of cows and see how well that works as “vegetation” in a field.

One obvious benefit of this approach is that it allows the cows to be seasonal. I can setup the autogen so that the model does not show during the winter. With normal placement of the models in FSX that is much harder to achieve.

But I ran into a couple of problems when I used my new vegetation group in my autogen.

First my objects did not show at all. After some trial and error I found out that for autogen you can only use MDL files that have drawcall batching enabled. If that is not the case, nothing will show at all. So if you made your model in GMax it will not work, you first need to change the MDL to use drawcall batching (you can do this in ModelConverterX).

The second problem is shown in the picture below.

Image2013-05-13 2118.17.096

You might wonder what it is, but I got all kind of weird effects on my screen. It looked like huge polygons with random textures from the surrounding. After some more trial and error I found at that if I reduced the complexity of my cow model these effects are gone.

I think it is related with too many polygons in a single drawcall, but I can’t be sure of course. From my experimentation I have concluded that it is best to keep autogen models below 200 triangles, then these issues don’t seem to happy with the maximum autogen density. If the vegetation class is less dense (by adding a zero guid with enough weight) the higher triangle count models work fine as well.

So now I have some cows standing around in my field and they are placed with autogen. I quite like the effect and will continue to explore other ways to make more use of autogen.

Image2013-05-13 2120.23.864

scenProc crashes should be fixed

A little while ago I changed how the autogen configuration is read into scenProc. This change resulted in some unexpected crashes for some users. These seemed to happen mainly when FSX was installed within the Program Files folder and Windows did not give the right permission to read the files.

I have now made the code to read the configuration more robust and also worked around these permission issues. So if you download the current version these issues should be fixed. So if you downloaded scenProc in the last week or two, please update to the latest version to save yourself some trouble.