Be careful for narrow houses

narrow house 6Two users recently reported the same problem to me with autogen files made with scenProc. In Annotator they could not see all the buildings and after saving in Annotator many of the buildings were gone.

It turned out this problem was caused by very narrow buildings. So I want to warn all scenProc users of this problem. There are two causes for narrow buildings:

  1. Your data already contains them
  2. The SPLITGRID step made them

For the first case there is not much I can do, so just check your data. But to be honest the second case is most likely the cause of this issue in most cases. Normally the SPLITGRID step does make sure all features are divided along the borders of the autogen tiles. So if a polygon crosses the border it is split into two polygons.

Now what would happen if you house just crossed the border a tiny little bit? That would result in that tiny bit being sliced away from the house and being represented as a separate polygon (and thus seen as another house). This can result in very narrow houses in some cases.

The obvious solution for this is to not slice the houses, but instead just put the entire footprint polygon of the house in the terrain tile where the centre is. This option is already possible with scenProc, it is the second argument you can give the step.

Let me show you two examples of how to use it. The first case assumes that you load all your building footprints from a separate file, let’s assume it is called buildings.shp. In that case you can tell scenProc not to slice the buildings by using the following command:

SPLITGRID|AGN|FROMFILE=buildings.shp

Another case would be where you loaded OpenStreetMap data. In that case the buildings are not in a separate file. So you would need to filter on some attribute. For example:

SPLITGRID|AGN|building=*

Tree detection – part 4

It’s a while ago that I posted about the progress of my tree detection efforts. So here is a quick update. Some other things kept me busy, so there hasn’t been too much progress recently.

I was able to run the tree detection algorithm on the entire island of Nantucket now. It took around 3.5 hours. Below you see a screenshot of the result. I have to be honest that I took the screenshot from a good angle. There are areas where there are quite a few false detections, which results in trees on locations where they should not be. Mainly in the water. So my efforts will be focussed on finding a way to eliminate those.

In one of my earlier posts I mentioned that polygonal autogen vegetation did not work well, since trees not always show up. But that was wrong. I ended up added a buffer to my detected areas to make them a few pixels bigger and that was enough to get the trees displaying. I now I conclude that the polygonal vegetation is probably still the best way to go.

Image2013-06-16 1715.10.962

SPLITGRID? Check!

check-markIn my bug reporting system I can see that it relatively often happens that people run into trouble (crashes) because they remove the SPLITGRID step from their configuration file. If you then try to create autogen features scenProc will crash. This is because I assumed that the SPLITGRID step will be done first. This step will make sure all features are divided correctly in the terrain grid used by FS.

So to prevent such crashes from happening, I have now added an extra check to the validation of your configuration file. If the SPLITGRID step is not present before using the steps to create the autogen you will get an error.

I hope this will help to prevent this common mistake from happening.

I flipped it

Yesterday I fixed a bug in the scenProc step that imports AGN files. There was an issue in this step that resulted in part of the autogen features being displayed wrong. They were vertically flipped.

So if you are using the import AGN step in your workflow, please update to the latest version and run your configuration file again. Else you can have wrong results.

BGLComp SceneryObject flags

A few days ago I added support to ModelConverterX to read the additional options like <NoShadow /> or <NoAutogenSuppresion /> from the object placement code in the BGL file. Initially I had some trouble to find out where these options ended up in the binary code, since it was not documented in the otherwise excelent description that Winfried Orthmann made a while ago.

After the latitude, longtide and altitude the description says there is a short that stores if the altitude is AGL or not. But actually this short can store much more than that. It more acts like a set of flags for multiple options. Here is the list of possible flags:

0x01 AltIsAgl
0x02 NoAutogenSuppression
0x04 NoCrash
0x08 NoFog
0x10 NoShadow
0x20 NoZWrite
0x40 NoZTest
Just wanted to share this, in case other tool developers are struggling with the same in the future.

Tree detection – part 3

I have made some more progress on the tree detection algorithm in the last days, but I am not there yet. So some more testing is needed before the functionality can be put in scenProc.

In my previous post I showed some results from the algorithm running in a test application. What I have done now is put it in scenProc itself. This means that scenProc can now read raster files through GDAL, run the histogram based filtering on the image and create vectors from the results (thanks to GDAL and OGR again).

Here are my first findings of putting the algorithm in scenProc:

  • It is still very slow. Running the whole island of Nantucket through the tree detection algorithm takes more than half a day. So I guess it would be a good moment to get the profiler tool again and find out where performance gains can be made.
  • The screenshots I showed last time were from images at a lower resolution. I had down sided them from the 30 cm resolution to make testing easier. It seems the filter that worked well on those, does not necessary work well on the full resolution as well. It seems resizing them made the trees slightly darker. So I will have to do some more testing to see which filter settings work well with the full resolution.
  • With the algorithm you get a lot of relatively small polygons. Most polygons cover a few trees. Only in a dense forest you would get bigger polygons. Creating polygonal autogen vegetation from these small polygons does not work well, quite often no trees show at all in FSX. Using rectangular autogen vegetation seems to work much better. Even for a small rectangle at least one tree will show up. So it seems rectangular vegetation would be the way to go with this algorithm.

And now back to some more testing…

Tree detection – part 2

I am travelling for my work this week, but when I had an few hours in the evening to spend in my hotel room I did some more experimentation with the tree detection algorithm.

I have now modified my test tool so that you can click on the trees in the image to determine the rules that are used for the detection. Just click on a dozen different trees or so, and the tool will figure out which rule should be used to find the trees in this kind of image.

From the tests I have done it seems to work quite well. Below are two screenshots of the Nantucket imagery where I used to same rules to find the trees in both images. I am quite happy with the result till now.

tree_detect1

tree_detect2

Next step will be to add this algorithm to scenProc, so that I can test it on a bigger area and see if the results still look convincing. I am curious to see how this autogen will turn out. But that will have to wait till I am back home, I don’t have FSX here at the moment.

Tree detection

One of the first problems I encountered when I tried to make autogen for Nantucket is the lack of vector data for the vegetation. OpenStreetMap contains hardly any vegetation and also the GIS department of Massachusetts has not much usable data.

So therefore I started to experiment with detecting the vegetation in the imagery.  And while playing with this idea, a similar tool popped up at the FSDeveloper forum.

The approach I am testing now is to check the histogram of a small part of the imagery and based on the average value and the standard deviation decide what is forest and what not. I have made a small test tool to see which algorithm works best. Below you see a screenshot of the imagery and the detected vegetation.

Image2013-06-02 2016.37.672

Image2013-06-02 2017.14.604

Of course it is not yet perfect. As you can see some vegetation ends up in the water for example. I have already seen that different images sometimes need different settings to get the right result. I am thinking about turning my test tool into some kind of configure tool. Hopefully if as a user you click on some areas that are vegetation and some areas that are not, I can have the tool figure out the best filter for that type of imagery.

Once I find an algorithm that works OK, I plan to put this functionality in scenProc as well of course. But for now there is more testing and exploring to do.

Below is a quick screenshot of the result I got now in FSX. I think it is not too bad, but there are still some bugs left. And I might have to pick a different type of tree to match the Nantucket area better.

Image2013-06-02 1139.18.253

 

Nantucket photo scenery

As I mentioned in my previous post I would write a bit more details about how I created the photo scenery for my Nantucket test project.

From the website of the Massachusetts GIS department I could download 30 cm imagery of the entire island. But they came in 101 tiles and in the SID format. So how to make them ready for a FSX photo scenery?

My normal approach would be to make a mosaic of the images into one big GeoTIFF and then it is quite easy to make a photo scenery with resample. But for the entire island at this resolution that would result in a file of around 20 GB. So that is not something that resample can handle.

So I took another approach here. First I used GDAL to convert all the SID files to GeoTIFF and I made sure they are reprojected to geodetic WGS84 as resample requires. I did this from the command prompt:

for %f in (*.sid) do gdalwarp -t_srs "+proj=latlong +datum=WGS84" %f %~nf.tif

So now I have a whole bunch of GeoTIFF files in the projection that resample can handle. But there are more than 100 hundred files. So that makes writing the INF file quite some work. To tackle that I made a simple BAT file that will scan all GeoTIFF files in a folder and create a INF file that contains them all. Here is the code of the batch file:

@setlocal EnableDelayedExpansion
@echo [Source]
@echo Type = MultiSource
@echo NumberOfSources = 101
@set icount=0
@for %%f in (..\geo\imagery\gtif_wgs84\*.tif) do @set /a ICOUNT+=1 & @echo [Source!icount!] & @echo Type = GeoTIFF & @echo SourceDir = "..\geo\imagery\gtif_wgs84" & @echo SourceFile = "%%~nf.tif" & @echo Layer = Imagery & @echo Variation = All & @echo NullValue = 0,0,0
@echo [Destination]
@echo DestDir = "..\..\scenery"
@echo DestBaseFileName = "Nantucket"
@echo DestFileType = BGL
@echo LOD = Auto
@echo SplitFileLOD = 10
@echo Compression = 85

As you can see there are still some paths and numbers hardcoded. If I would use this kind of script often, I would probably better make a tool for it. Maybe I can add the functionality to resample.

Then the last step was just to run the INF file through resample and wait a few hours. And there it is, a photo scenery made from a whole bunch of files that could easily be downloaded from the USGS.

Image2013-06-01 1652.32.735

Of course there is more to do. For example add a watermask to get the dynamic water. And add a blend mask to get a gradual transition at the edges. But those tasks I still have to do. Once I have done them I will write about my experiences again.

Test project Nantucket

I was looking for a way to help me develop the scenProc tool and explore the possibilities of autogen further. In the end I decided to start a little scenery project where I could play around with all aspects of autogen.

I decided to pick an area in the US, because there is a lot of data available from the USGS. That makes it easier to do a photo scenery. An island is always a nice location for a test area, because you have a natural border of the area you are working on, I decided to use the Nantucket island in Massachusetts. The state of Massachusetts also has a nice website of their GIS service where you can find a lot of data.

So I got 30 cm imagery of the island and processed that into a photo scenery. I will make another post about the techniques I used for this soon. Below are some pictures of the photo scenery in FSX.

Image2013-06-01 1652.32.735 Image2013-06-01 1652.53.864

So now I can start the challenge to populate the island with autogen. I’ll start with the vegetation and the buildings. And after that I’ll see what other autogen elements I can still add. Hopefully I can use the autogen to represent the atmosphere of the real island, as shown on the picture below.

nantucket1

By the way, I don’t know at this moment if this scenery will be an addon that I will publish later on or not. For me it is really a test area to help me develop the tools further. And making it ready for a release does not have priority for me. And the island also has an airport, so maybe I can test some new ground polygon wizard ideas there as well later on.