It is currently Wed, 23-08-17, 19:39 GMT

All times are UTC




Post new topic Reply to topic  [ 40 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject:
PostPosted: Sat, 27-10-07, 12:51 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 991
Location: NE PA, USA
Missed that line underneath.


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 13:04 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4478
Location: Hamburg, Germany
After browsing a bit through the SRTM docs, it seems that we are involved in the wrong kind of task right now:

There is already a very advanced new tile set of waterbody data available that was derived from the 1 arcsec data. These 12229 .zip files are to be found in the SWBD directory (SRTM Water Body Data) on the NASA srtm server:
ftp://e0srp01u.ecs.nasa.gov/srtm/version2

After the processing was done, the shorelines from the one arc second (approx. 30-meter)
DTED 2 were saved as vectors in ESRI 3-D Shapefile format.

---------------------------------------------
So the proper task seems to be to convert that vector ESRI 3-D Shapefile format into a bin format of the appropriate resolution. No idea yet how to do this.
---------------------------------------------
There is a technical description of that format, however in the server's doc directory.

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 15:14 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Hi all,
I know it could be cleaned up more but so far this is what i know of writing code. Go easy on me. here you go :
Code:
@echo off
SET CMDIR=0
echo.
echo  Credit is hereby given to Fridger Schrempp for his fantastic set of tools,
echo  without which  this script would not exist. Also
echo  Abramson, whom when I saw his script in the Shatters forum inspired me to create
echo  this script. 
echo  Also John Van Vliet who helped me awhile back with virtualtex. And all the
echo  other celestia users whom posts have been most informative. 
echo                              Thanks to  all of you!
echo.
echo usage: nvcompress [options] infile [outfile]
echo.
echo Input options:
echo  -color        The input image is a color map (default)
echo  -normal       The input image is a normal map.
echo  -tonormal     Convert input to normal map.
echo  -clamp        Clamp wrapping mode (default).
echo  -repeat       Repeat wrapping mode.
echo  -nomips       Disable mipmap generation.
echo.
echo Compression options:
echo  -fast         Fast compression.
echo  -nocuda       Do not use cuda compressor.
echo  -rgb          RGBA format
echo  -bc1          BC1 format (DXT1)
echo  -bc2          BC2 format (DXT3)
echo  -bc3          BC3 format (DXT5)
echo  -bc3n         BC3 normal map format (DXT5nm)
echo  -bc4          BC4 format (ATI1)
echo  -bc5          BC5 format (3Dc\ATI2)
echo.
echo.
echo If you have an Nvidia GPU and installed the cuda toolkit, remove the "-nocuda" option 
echo.
echo If you want "png" format instead of "dds" simply type "rem" in front of both "for" commands for all levels
pause   
mkdir BMNG
cd BMNG
mkdir textures
cd textures
mkdir hires
cd hires
mkdir Srtm
mkdir Blue_Marble
cd Blue_Marble
mkdir level0
mkdir level1
mkdir level2
cd ..\..\..\..

mv srtm_ramp2.world.5400x2700.bin.gz BMNG\textures\hires\Blue_Marble

mv world.watermask.5400x2700.bin BMNG\textures\hires\Blue_Marble

mv world.200406.3x5400x2700.bin.gz BMNG\textures\hires\Blue_Marble

cd BMNG\textures\hires\Blue_Marble

gzip -dc < srtm_ramp2.world.5400x2700.bin.gz | specmap 5400 world.watermask.5400x2700.bin 0.12 1 > specmap.bin

gzip -dc < world.200406.3x5400x2700.bin.gz | tx2rgba 5400 specmap.bin | tx2pow2 4 5400 > level2\world.200406.3x4096x2048.RGBA.bin                   
 
tx2half 4 4096 < level2\world.200406.3x4096x2048.RGBA.bin > level1\world.200406.3x2048x1024.RGBA.bin
 
tx2half 4 2048 < level1\world.200406.3x2048x1024.RGBA.bin > level0\world.200406.3x1024x512.RGBA.bin

cd level2
txtiles 4 4096 2 < world.200406.3x4096x2048.RGBA.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc1 %%x
for %%x in (tx*.png) do del %%x
del world.200406.3x4096x2048.RGBA.bin

cd ..\level1
txtiles 4 2048 1 < world.200406.3x2048x1024.RGBA.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc1 %%x
for %%x in (tx*.png) do del %%x
del world.200406.3x2048x1024.RGBA.bin

cd ..\level0
txtiles 4 1024 0 < world.200406.3x1024x512.RGBA.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc1 %%x
for %%x in (tx*.png) do del %%x
del world.200406.3x1024x512.RGBA.bin
cd ..

del specmap.bin 
mv world.watermask.5400x2700.bin ../../../..
mv world.200406.3x5400x2700.bin.gz ../../../..
mv srtm_ramp2.world.5400x2700.bin.gz ../Srtm

cd ../Srtm
mkdir level2
mkdir level1
mkdir level0

gzip -dc < srtm_ramp2.world.5400x2700.bin.gz | nm2pow2 5400 1 > level2\srtm_ramp2.world.4k.bin
 
nm2half 4096 < level2\srtm_ramp2.world.4k.bin > level1\srtm_ramp2.world.2k.bin
 
nm2half 2048 < level1\srtm_ramp2.world.2k.bin > level0\srtm_ramp2.world.1k.bin
 
cd level2
nmtiles 2 6378.14 4096 2.5 < srtm_ramp2.world.4k.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc3n %%x
for %%x in (tx*.png) do del %%x
ren *.dds *.dxt5nm
del srtm_ramp2.world.4k.bin

cd ..\level1
nmtiles 1 6378.14 2048 2.5 < srtm_ramp2.world.2k.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc3n %%x
for %%x in (tx*.png) do del %%x
ren *.dds *.dxt5nm
del srtm_ramp2.world.2k.bin

cd ..\level0
nmtiles 0 6378.14 1024 2.5 < srtm_ramp2.world.1k.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc3n %%x
for %%x in (tx*.png) do del %%x
ren *.dds *.dxt5nm
del srtm_ramp2.world.1k.bin
cd ..
mv srtm_ramp2.world.5400x2700.bin.gz ../../../..


tips and suggestions are appreciated. I'm going to work on one for the "Big 85k's" now).

Bye,

Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Last edited by Ozark on Mon, 29-10-07, 19:56 GMT, edited 2 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 16:06 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4478
Location: Hamburg, Germany
Ozark,

that looks like a great start. Generally you should best introduce variables at the beginning of the script for configuring nvcompress etc. This way one can always adapt the script easily and transparently to different conditions/requests.

----------------------------------------------
It might be useful to discuss the best way of implementing the new generated tile sets into the Celestia directory hierarchy.
----------------------------------------------

For people (including myself) who want to use the tiles as their permanent preference, a standard structure is this
Code:
extras -> <all the other extras stuff>
          -> solarsys.ssc
          -> textures -> hires
                      -> medres
                      -> lores
                      -> tiles -> <all the required tile dirs>
 

and then start all entries in solarsys.ssc with the Modify keyword, like so
Code:
Modify "Earth" "Sol"
{
    Texture "earth.ctx"
    NightTexture "earth-night.ctx"
    NormalMap "earth-normal.ctx"                         
    ...
}

This structure allows to work always without modification of the solarsys.ssc of the distribution!

Bye Fridger

_________________
Image


Last edited by t00fri on Sat, 27-10-07, 16:24 GMT, edited 7 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 16:12 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Hi Fridger,

Thanks. Will edit my previous post. Should be a quick fix.

Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 16:17 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4478
Location: Hamburg, Germany
Good,

have a look I edited my previous post adding some further issue.

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 16:22 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Code fixed! :wink:

Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 16:32 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
If anyone runs my above script, can anyone let me know what the issue is with the times recorded by BMTIME.txt and SRTMTIME.txt.

apparently, the srtm portion of the script is starting before the RGBA portion of the text ends! :? I don't get it.

Thanks,
Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 18:14 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
does this look right? The output looks kind of funny here! :?

Code:
[tx2rgba]: Mo
u[nttxi2npgo wa2 ]1:x 8R ebdiutc isnpge ctuol anre agrreasytm appo
w e r - o f - t w o   saibzoev:e  the 3 x685 b5i3t6  RxG B3 2b7a6s8e
 texture           => 4x8 bit RGBA image
rgba[  1024 rows of 43200 ->  19.42 s]


here is the code for this output:

gzip -dc < world.200408.3x86400x43200.bin.gz | tx2rgba 86400 specmap.bin | tx2pow2 4 86400 > level2/world.200408.65k.RGBA.bin


Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 18:21 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4478
Location: Hamburg, Germany
Ozark wrote:
does this look right? The output looks kind of funny here! :?

Code:
[tx2rgba]: Mo
u[nttxi2npgo wa2 ]1:x 8R ebdiutc isnpge ctuol anre agrreasytm appo
w e r - o f - t w o   saibzoev:e  the 3 x685 b5i3t6  RxG B3 2b7a6s8e
 texture           => 4x8 bit RGBA image
rgba[  1024 rows of 43200 ->  19.42 s]


here is the code for this output:

gzip -dc < world.200408.3x86400x43200.bin.gz | tx2rgba 86400 specmap.bin | tx2pow2 4 86400 > level2/world.200408.65k.RGBA.bin


Ozark


Ozark,

that
Quote:
[tx2rgba]: Mo
u[nttxi2npgo wa2 ]1:x 8R ebdiutc isnpge ctuol anre agrreasytm appo
w e r - o f - t w o saibzoev:e the 3 x685 b5i3t6 RxG B3 2b7a6s8e


looks unfortunately crazy. How did you generate specmap.bin and is it located in the SAME directory where you started this command line?

I never had any problems here.

It is much better to take the time routine out of the script. Instead you start your script named 'script' simply as

> time script

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 27-10-07, 18:25 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Fridger,

here s the code:
Code:
mv srtm_ramp2.world.86400x43200.bin.gz BMNG/textures/hires/Blue_Marble
mv world.watermask.86400x43200.bin.gz BMNG/textures/hires/Blue_Marble
mv world.200408.3x86400x43200.bin.gz BMNG/textures/hires/Blue_Marble
cd BMNG/textures/hires/Blue_Marble

gzip -dc < world.watermask.86400x43200.bin.gz > world.watermask.86400x43200.bin

gzip -dc < srtm_ramp2.world.86400x43200.bin.gz | specmap 86400 world.watermask.86400x43200.bin 0.12 1 > specmap.bin

gzip -dc < world.200408.3x86400x43200.bin.gz | tx2rgba 86400 specmap.bin | tx2pow2 4 86400 > level2/world.200408.65k.RGBA.bin

mv world.watermask.86400x43200.bin.gz ../../../..

cd level5


Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 29-10-07, 3:47 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Hi Fridger,

I finished a script for the 85k files today. And did some testing of the TEST script that was giving me wierd output during the tx2pow2 stage.

I don't have a clue as to why this is happening! :?

here is a portion of the script in question:
Code:
mkdir BMNG
cd BMNG
mkdir textures
cd textures
mkdir hires
cd hires
mkdir Srtm
mkdir Blue_Marble
cd Blue_Marble
mkdir level0
mkdir level1
mkdir level2
cd ..\..\..\..

mv srtm_ramp2.world.5400x2700.bin.gz BMNG/textures/hires/Blue_Marble

mv world.watermask.5400x2700.bin BMNG/textures/hires/Blue_Marble

mv world.200406.3x5400x2700.bin.gz BMNG/textures/hires/Blue_Marble

cd BMNG\textures\hires\Blue_Marble

gzip -dc < srtm_ramp2.world.5400x2700.bin.gz | specmap 5400 world.watermask.5400x2700.bin 0.12 1 > specmap.bin

gzip -dc < world.200406.3x5400x2700.bin.gz | tx2rgba 5400 specmap.bin | tx2pow2 4 5400 > level2/world.200406.3x4096x2048.RGBA.bin                   
 
tx2half 4 4096 < level2/world.200406.3x4096x2048.RGBA.bin > level1/world.200406.3x2048x1024.RGBA.bin
 
tx2half 4 2048 < level1/world.200406.3x2048x1024.RGBA.bin > level0/world.200406.3x1024x512.RGBA.bin

cd level2
txtiles 4 4096 2 < world.200406.3x4096x2048.RGBA.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc1 %%x
for %%x in (tx*.png) do del %%x
del world.200406.3x4096x2048.RGBA.bin

cd ../level1
txtiles 4 2048 1 < world.200406.3x2048x1024.RGBA.bin
for %%x in (tx*.png) do nvcompress -nocuda -bc1 %%x
for %%x in (tx*.png) do del %%x
del world.200406.3x2048x1024.RGBA.bin


I fixed the code a bit and ran it four times in a row without further modifications. 1st run showed strange output:

Code:
[specbin]: Improving the watermap from height=0:   5400 x  2700
spec[  1024 rows of  2700 ->   0.33 s]
spec[  2048 rows of  2700 ->   0.63 s]

[tx2rgba]: Mounting a 1x8 bit specular grayma
[tx2pow2]: Reducp
           aboveing to nearest pow  the 3x8 bit RGB er-of-two size:  4base texture
     096 x  2048
      => 4x8 bit RGBA image
rgba[  1024 rows of  2700 ->   0.69 s]
pow2[  1024 rows of  2048 ->   1.25 s]
rgba[  2048 rows of  2700 ->   2.78 s]
pow2[  2048 rows of  2048 ->   4.19 s]
[tx2half]: Input size:  4096 x  2048 => New size:  2048 x  1024
half[  1024 rows of  1024 ->   0.16 s]
[tx2half]: Input size:  2048 x  1024 => New size:  1024 x   512
[txtiles]: Input file is a  4x8 bit RGBA texture:  4096 x  2048
               1st run


second and third run went fine, no problems, again no modifications

Code:
[specbin]: Improving the watermap from height=0:   5400 x  2700
spec[  1024 rows of  2700 ->   0.33 s]
spec[  2048 rows of  2700 ->   0.63 s]

[tx2rgba]: Mounting a 1x8 bit specular graymap
           above  the 3x8 bit RGB base texture
           => 4x8 bit RGBA image

[tx2pow2]: Reducing to nearest power-of-two size:  4096 x  2048
rgba[  1024 rows of  2700 ->   0.70 s]
pow2[  1024 rows of  2048 ->   1.02 s]
rgba[  2048 rows of  2700 ->   2.41 s]
pow2[  2048 rows of  2048 ->   3.63 s]
[tx2half]: Input size:  4096 x  2048 => New size:  2048 x  1024
half[  1024 rows of  1024 ->   0.14 s]
[tx2half]: Input size:  2048 x  1024 => New size:  1024 x   512
[txtiles]: Input file is a  4x8 bit RGBA texture:  4096 x  2048
                                        2nd run

=================================================

[specbin]: Improving the watermap from height=0:   5400 x  2700
spec[  1024 rows of  2700 ->   0.34 s]
spec[  2048 rows of  2700 ->   0.64 s]

[tx2rgba]: Mounting a 1x8 bit specular graymap
           above  the 3x8 bit RGB base texture
           => 4x8 bit RGBA image

[tx2pow2]: Reducing to nearest power-of-two size:  4096 x  2048
rgba[  1024 rows of  2700 ->   0.69 s]
pow2[  1024 rows of  2048 ->   0.91 s]
rgba[  2048 rows of  2700 ->   1.39 s]
pow2[  2048 rows of  2048 ->   2.42 s]
[tx2half]: Input size:  4096 x  2048 => New size:  2048 x  1024
half[  1024 rows of  1024 ->   0.14 s]
[tx2half]: Input size:  2048 x  1024 => New size:  1024 x   512
[txtiles]: Input file is a  4x8 bit RGBA texture:  4096 x  2048
                                         

the fourth run no modifications:

Code:
[specbin]: Improving the watermap from height=0:   5400 x  2700
spec[  1024 rows of  2700 ->   0.33 s]
spec[  2048 rows of  2700 ->   0.63 s]

[tx2pow2]
[tx2rgba]: : Reducing to nearMeosutn tpionwge ra- o1fx-8t wboi ts iszpee:cular gr a y4m0a9p6
  x      2 0 4 8
  above  the 3x8 bit RGB base texture
           => 4x8 bit RGBA image
rgba[  1024 rows of  2700 ->   0.69 s]
pow2[  1024 rows of  2048 ->   1.14 s]
rgba[  2048 rows of  2700 ->   2.64 s]
pow2[  2048 rows of  2048 ->   3.59 s]
[tx2half]: Input size:  4096 x  2048 => New size:  2048 x  1024
half[  1024 rows of  1024 ->   0.16 s]
[tx2half]: Input size:  2048 x  1024 => New size:  1024 x   512
[txtiles]: Input file is a  4x8 bit RGBA texture:  4096 x  2048


is this something with my computer or what?

thoroughly baffled,
Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 29-10-07, 9:26 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4478
Location: Hamburg, Germany
Ozark,

this really looks strange, noone has so far reported anything like that under Windows.

But apart from the missing 'p' in my name ;-), I noted that you used a mixed UNIX-DOS notation in your script. No idea how well this is supposed to work! Note if you copied the commands partly from scripts of mine, this might have been for Linux!

In DOS/Win32 directories are ..\..\..\ NOT ../../../
and also other commands might be incompatible partly.
You can always find the correct command names and usage by typing help in the dos console.

The other possibility might be that there is some malfunction corresponding UNICODE font implementations on your system?

But really I am blindly guessing.

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 29-10-07, 21:07 GMT 
Offline
User avatar

Joined: Sun, 21-10-07, 5:43 GMT
Posts: 21
Location: Minnesota, USA
Fridger,


t00fri wrote:
Quote:
But apart from the missing 'p' in my name

heh heh! Sorry bout that. All fixed :wink:


t00fri wrote:
Quote:
I noted that you used a mixed UNIX-DOS notation in your script. No idea how well this is supposed to work! Note if you copied the commands partly from scripts of mine, this might have been for Linux!


Very strange indeed :shock: it seems i can run the script just fine with "/" OR "\". Except for the last line, that has to be opposite otherwise the script fails.
While i did not "cut and paste" your code I did rewrite mine the way you have it written at the begining of this topic. I have had problems in the past with "line endings" and such while copying, so i choose to type it out myself. Besides your way was more efficient than the way I had it written. :wink: Well, since this seems to be an issue with my system, its probably best to put this to rest.

I suppose I should spend a little more time learning bash or something a little more "universal" before i post anymore scripts. Thanks for you help though :)

as to your earlier post:
Quote:
----------------------------------------------
It might be useful to discuss the best way of implementing the new generated tile sets into the Celestia directory hierarchy.
----------------------------------------------

For people (including myself) who want to use the tiles as their permanent preference, a standard structure is this
Code:

extras -> <all the other extras stuff>
-> solarsys.ssc
-> textures -> hires
-> medres
-> lores
-> tiles -> <all the required tile dirs>


and then start all entries in solarsys.ssc with the Modify keyword, like so
Code:

Modify "Earth" "Sol"
{
Texture "earth.ctx"
NightTexture "earth-night.ctx"
NormalMap "earth-normal.ctx"
...
}


This structure allows to work always without modification of the solarsys.ssc of the distribution!


This sound good to me! I prefer vt's as default textures as well. Currently, Celestia only looks in lores, medres, or hires, for textures, Correct? Is this something that users could remedy, or does this have to be implemented in Celestias source code ?

As Always Thanks.

Ozark

_________________
Celestia 1.6 4GHZ E6550
4GB DDR3 ram
Windows XP
512 GeForce GTS 250 - NVIDIA


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 13-12-07, 18:31 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 8:49 GMT
Posts: 120
Location: Oxford UK
Okay chaps... what sort of frame rates are you getting with these superb textures?


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 40 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group