It is currently Wed, 12-08-20, 0:37 GMT

All times are UTC




Post new topic Reply to topic  [ 196 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8, 9 ... 14  Next
Author Message
 Post subject:
PostPosted: Tue, 25-09-07, 19:39 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
DW,

I confirm both for the "dwarf data set" the the full-sized 85k data: error=0!
under Linux on my x86 P4.

At last....

The timing for the 85k full-data set was in case of Example1

9 min 37.86 sec

for my Desktop (3.2 GHz P4/ 3 GB RAM) under Linux
Without the fastfloor(), fastnint() and 2-pass mods in tx2pow2, the time was

12 min 56.17 sec

So under Linux the speed increase is somewhat less than under Windows with my Core 2 Duo.

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Tue, 25-09-07, 23:04 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 438
Location: South Korea
t00fri wrote:
Now what do you think: I would be inclined to post a .pre3 version ASAP. I think it would make the code much more transparent if we commit ourselves to only the fastfloor() and fastnint() solutions that are now known to work on all three OS. Also I think we can substitute your two-pass version in tx2pow2 for good. I checked it in various environments with nvimgdiff and it always agreed with the one-pass. What do you think??


Great, I'll clean up the files and send them to you later in the day.
**Update: I sent you the files.


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 27-09-07, 13:57 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
DW,

please look in your email. I did not get the file yet...

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 27-09-07, 16:31 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 438
Location: South Korea
I did send it yesterday, but anyway I mailed you again today and also tweaked the code again while I was at it, so make sure you're getting the most recent version (it should be a tar.gz attachment).

I also spent some of the afternoon applying similar optimizations to nmtools too. Those changes are in a separate mail. I'm cc'ing the other adsl address that you gave me just in case I get tagged again as a spammer.


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 01-10-07, 21:31 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
Hi all,

here is an impressive performance AND correctness test of the new nmtools-1.5pre1 release that I posted yesterday for download and testing:

First I calculated with the old nmtools-1.0.1 release and the 85k monster input file the following commandline, which practically involves ALL nmtools (apart from nmtiles):

Code:
nmtools-1.0.1
-------------

time bash -c 'gzip -dc < srtm_ramp2.world.86400x43200.bin.gz | resc2pow2 86400 1 | halfsize 65536 | halfsize 32768 | nms 6378.140 16384 5.0 > nm16k-old.ppm'

total time: 19 min 06.28 sec

next I timed the identical commandline (apart from renaming) for the new release:

Code:
nmtools-1.5pre1
----------------

time bash -c 'gzip -dc < srtm_ramp2.world.86400x43200.bin.gz | nm2pow2 86400 1 | nm2half 65536 | nm2half 32768 | nms 6378.140 16384 5.0 > nm16k.png'


total time: 7 min 18.6 sec

You first of all see the impressive speed-up of the new release ( it's a factor 2.61 faster, despite the new conversion to PNG format!!!

Next, I converted the old .ppm output to .png and then applied the image comparison tool 'nvimgdiff' from the new nvidia-texture tools to both resulting files:


> nvimgdiff nm16k-old.png nm16k.png

Image size compared: 16384x8192
Total pixels: 134217728
Color:
Mean absolute error: 0.000000 <=========================
Max absolute error: 0.000000 <=========================
Root mean squared error: 0.000000 <=========================
Peak signal to noise ratio in dB: 999.000000

------------------------------------------
Trara!! the differences are NULL.
------------------------------------------

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 01-10-07, 23:25 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 438
Location: South Korea
That's great. In the mean time, I've compiled F-TexTools pre3 on my Mac but I'll need some time to test.
**Update: I sent you the compiled binaries.


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 03-10-07, 15:16 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
As suggested by DW, I did also some extensive testing for grayscale (bpp=1) and RGBA (bpp=4) modes of the F-TexTools, both on my Desktop (Linux) and my Core2Duo (Win XP). Here is a summary of the results which were just perfect!

Bpp = 1 (grayscale) Tests:

Test 1)
Code:
time bash -c 'tx2pow2 1 86400 < world.watermask.86400x43200.bin | tx2half 1 65536 | tx2half 1 32768 | bin2png 1 16384  > watermask16k.png'


3.2GHz P4/ 3 GB RAM          (Linux): 1:55.27 total
Core2Duo 2.0GHz / 2GB RAM (Win32 XP): 2:27.20 total


The result is a beautiful 16k improved specular (grayscale) map.

Comparing the output files between Linux and Win32 gives NULL:

Code:
> nvimgdiff watermask16k_win32.png watermask16k_linux

Image size compared: 16384x8192
Total pixels: 134217728
Color:
Mean absolute error: 0.000000    <================Trara!
Max absolute error: 0.000000
Root mean squared error: 0.000000
Peak signal to noise ratio in dB: 999.000000



Bpp = 4 (RGBA) Tests:

Test 2) = Example 2): generate first the improved 85k specmap file, specmap.bin:

Code:
time bash -c 'gzip -dc < srtm_ramp2.world.86400x43200.bin.gz | specmap 86400 world.watermask.86400x43200.bin 0.12 1 > specmap.bin'

3.2GHz P4/ 3 GB RAM          (Linux): 4:20.73 total


Test 3) generate a 16k RGBA Earth texture, with specmap.bin as alpha channel:

Code:
time bash -c 'gzip -dc < world.200406.3x86400x43200.bin.gz | tx2rgba 86400 specmap.bin | tx2pow2 4 86400 | tx2half 4 65536| tx2half 4 32768 | bin2png 4 16384 > out_RGBA_16k.png'

3.2GHz P4/ 3 GB RAM          (Linux): 14:47.36 total


and here are the Celestia displays of the generated 16k RGBA texture (out_RGBA_16k.png) with "unified" RGB base texture (BMNG) and specmap:

Image

click to enlarge
Image

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 14:22 GMT 
Offline
User avatar

Joined: Sun, 02-09-07, 16:06 GMT
Posts: 346
Location: Rome, Italy
Hello Fridger, sorry for this post, but I fear I found a bug.
I observed it with the first release of F-TexTools, but I was thinking it could heve been due to some mistake from mine, so I waited for the further releases to check it again.
Now I have installed the new textures, carefully obtained with the last F-TexTools-1.0pre3, and the results are the same.
I use WIN XP SP2, Celestia 1.4.1, nVidia 163.69, and all VT textures are in .png format.
Here is a Tunisia image obtained at sunset

Image

and at noon

Image

As you see there are many whitish spots that beforehand were invisible.
Let’s go on with another example, now we are in USA, Gran Canyon- Lake Meade area, at local sunset (00:00 UTC)

Image

and at local noon 19:00 UTC)

Image

To understand where was the problem, I tried the VT normalmaps obtained with nmtools together with the Jmii’s 64k VTs, and here are the results

same Tunisia image obtained at sunset, this from Ft-tools image

Image

and this with Jmii’s 64K Earth

Image

The same at noon, Earth VT with Ft-tools

Image

and Jmii’s 64K Earth

Image

Now I think that if the problem is not due to my mistakes, and given that the white spots appear only with the Sun very high, and never when it's closer to sunrise-sunset, IMHO the problem could be in the specmap or watermask map, when coupled together and linked as Alpha channel to the Earth image.
Am I wrong? :oops:
If yes, where did I make a mistake?
Thank you for your check and your reply.
Bye

Andrea :D

_________________
Andrea
WorldPoss Sn Team Member
Core 2 Quad Q6600 G0 3.6 GHz- 4 GB DDR2- DELL 2709W 1920x1200-WinXP 32 SP3- VISTA 64- ASUS P5K-E- 8800 GTX 768MB- SATA II 320-300 GB-IDE 1 TB- 169.21- Celestia1.4.1_patch3- 1.5.1-LUA Edu Tools


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 15:27 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 438
Location: South Korea
I think Andrea is right - if you follow the examples when generating tiles then the alpha channel in some tiles has holes over land. These holes become perfect specular reflectors (white spots). Maybe this happens with parts of land that are below sea level?


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 15:33 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
Hi Andrea,

thanks for your careful report! No you did NOT make a mistake, I was already aware of this phenomenon as well as it's origin:

What you identified as white spots/regions in Tunesia are actually reflective areas directly from the ORIGINAL, published BMNG watermap. Here is the relevant piece that I cut out for you from a 16k version:

Image

So these regions are supposed to be water (oasis?) according to the official watermask!! The amazing fact is however that these "lakes" are located in the desert!

You best have a look yourself in the published watermask from the arctic NASA server.
There is a small 5400x2700 PNG file of the watermask, where you can confirm the above "lakes" immediately.

http://snowy.arsc.alaska.edu/nasa/landm ... 0x2700.png

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 15:36 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
There is only one way out, namely to turn to the much better and higher quality next_next generation that is already available, however only as TILES! So again a tile-composer tools is badly needed!

Also the main purpose of the tools is to quickly and reproducably generate a new texture set
when improved imaging data become available!

Cheers,
Fridger

_________________
Image


Last edited by t00fri on Thu, 04-10-07, 17:07 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 16:17 GMT 
Offline
User avatar

Joined: Sun, 02-09-07, 16:06 GMT
Posts: 346
Location: Rome, Italy
t00fri wrote:
There is only one way out, namely to turn to the much better and higher quality next_next generation that is already available on snowy, however only as TILES! So again a tile-composer tools is badly needed! Also the main purpose of the tools is to quickly and reproducably generate a new texture set when improved imaging data become available!
Cheers, Fridger

Thank you fridger, happy to know I didn't make some strange mistake.
Regarding this problem and your suggested solution, i.e. the tile-composer tools needed to use the next_next generation watermap already available, have you put in your schedule such a further enhancement?
I'm obviously NOT pressing you, but IMO all the great stuff you did to obtain outstanding Earth textures needs such further enhancement, in order to show our Earth in all its glory (topographic only, alas, nothing can be done this way for all the big Earth problems!). :cry:
BTW, what is the exact name of such next_next tiles?
The snowy site is filled-up of files, but I didn't find this one.
Bye and thank you for the very quick reply.

Andrea :D

_________________
Andrea
WorldPoss Sn Team Member
Core 2 Quad Q6600 G0 3.6 GHz- 4 GB DDR2- DELL 2709W 1920x1200-WinXP 32 SP3- VISTA 64- ASUS P5K-E- 8800 GTX 768MB- SATA II 320-300 GB-IDE 1 TB- 169.21- Celestia1.4.1_patch3- 1.5.1-LUA Edu Tools


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 16:55 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
Adrea,

here is the relevant reference

ftp://e0srp01u.ecs.nasa.gov/srtm/version2/

srtm wrote:
Version 1 of the SRTM data consists of the original Digital Elevation Models produced
by the SRTM project with data from the STS-99 mission in Feb., 2000, and delivered to
the National Geospatial-Intelligence Agency (NGA.) These data are unedited and
contain spurious data points in area of low radar backscatter such as water bodies.

Version 2 is the results of a substantial editing effort by the NGA and exhibits well
defined water bodies and coastlines
and the absence of spikes and wells (single pixel
errors), although some areas of missing data (‘voids’) are still present. The Version 2
directory also contains the vectorized coastline mask used by NGA in the editing, called
SRTM Water Body Data (SWBD), in shapefile format.

Version 2 is a superior product and it is recommended for most users.


We are talking about SRTM3 obviously that you find in the version 2 directory!

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 17:26 GMT 
Offline
User avatar

Joined: Sun, 02-09-07, 16:06 GMT
Posts: 346
Location: Rome, Italy
t00fri wrote:
Andrea, here is the relevant reference
ftp://e0srp01u.ecs.nasa.gov/srtm/version2/
We are talking about SRTM3 obviously that you find in the version 2 directory!
Cheers, Fridger

Thank you Fridger, I found it and, my God, there are zillions files!.
I downloaded the N34E007.hgt that probably contains the Tunisia region shown in my above shown images (at N36.5 and E07) , but obviously it's in data format, whose here is an example :shock:

   & ' ) * * + / 0 1 2 4 4 8 : = B G H M P R T U U T T S L E G J Q U Z ` f u z } } z ~ ‘ â„¢ Å“ ¤ ª ³ ¼ Ä ÃŒ Ãâ€â€

_________________
Andrea
WorldPoss Sn Team Member
Core 2 Quad Q6600 G0 3.6 GHz- 4 GB DDR2- DELL 2709W 1920x1200-WinXP 32 SP3- VISTA 64- ASUS P5K-E- 8800 GTX 768MB- SATA II 320-300 GB-IDE 1 TB- 169.21- Celestia1.4.1_patch3- 1.5.1-LUA Edu Tools


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 04-10-07, 17:32 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4675
Location: Hamburg, Germany
Andrea wrote:
t00fri wrote:
Andrea, here is the relevant reference
ftp://e0srp01u.ecs.nasa.gov/srtm/version2/
We are talking about SRTM3 obviously that you find in the version 2 directory!
Cheers, Fridger

Thank you Fridger, I found it and, my God, there are zillions files!.
I downloaded the N34E007.hgt that probably contains the Tunisia region shown in my above shown images (at N36.5 and E07) , but obviously it's in data format, whose here is an example :shock:

   & ' ) * * + / 0 1 2 4 4 8 : = B G H M P R T U U T T S L E G J Q U Z ` f u z } } z ~ ‘ â„¢ Å“ ¤ ª ³ ¼ Ä ÃŒ Ãâ€â€

_________________
Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 196 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7, 8, 9 ... 14  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group