It is currently Mon, 18-12-17, 3:16 GMT

All times are UTC




Post new topic Reply to topic  [ 15 posts ] 
Author Message
 Post subject:
PostPosted: Wed, 12-09-07, 11:19 GMT 
Offline
Site Admin
User avatar

Joined: Thu, 30-08-07, 22:52 GMT
Posts: 2726
Location: France, South, not far from Montpellier
DW, any way to do for osX a Tool kind SquishDDS based on Fridger's codes?


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 12-09-07, 11:21 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
Yes, I made SquishDDS, a gui front-end for the squish library (the same version used by the latest nvidia tex tools).
Improvements include mip map generation, color matching (Mac only), DXT5nm generation, and batch operation (e.g., for virtual tex tiles). The mip map and dxt5nm generation code is cross-platform and while done entirely on the cpu, is robust and seems to work pretty fast. If you (Fridger) are interested in any of these ideas or code for F-Tex I'll be happy to help.


Last edited by dirkpitt on Wed, 12-09-07, 11:31 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Wed, 12-09-07, 11:28 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
ElChristou wrote:
DW, any way to do for osX a Tool kind SquishDDS based on Fridger's codes?


Sure, since SquishDDS is pretty lightweight, I've been thinking the F-Tex and even the nmtools features could be possible additions to SquishDDS for a lightweight, convenient "one stop shop". Of course I'd also have to think of a snazzy new name too...


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 12-09-07, 11:42 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
Excellent, DW, thanks.

I spent quite a bit of thinking what might be the best strategy concerning the multi-platform availability and use of the new OpenSource nvidia-texture tools.

They are based of course on the high-quality squish DXT compress algorithm and read PNG format as input. For that reason all my new F-TexTools and the upgraded nmtools write PNG format directly now (instead of PNM format). This requires the use of libpng in my tools, however.

Originally, I also coded some custom version of the nvidia-texture tools and also contemplated to create a "one stop shop" with everything included. Later I realized, however, that the /official/ DXT-compress code is still strongly evolving. Hence the multi-platform updating efforts for such /custom/ versions would be kind of tedious.

For now I rather tend to prefer leaving the nvidia-texture-tools archive in it's /original/ (OpenSource) form, but offer binaries with high-performance flags activated (sse2, mms,...). They are NOT set in the official binaries for Windows, although they strongly boost the performance.

Moreover, since Ignacio Castano is a very active maintainer/developer of the nvidia tools, I rather thought of inviting him over to this forum to further enhance our contacts. So perhaps we can convince him to implement the few changes we'd need for Celestia "officially". Like a .dxt5nm file ending in case of DXT5nm compression.

How does this sound?

For Linux and Win XP I can produce easily such specially compiled binaries.

Of course, I cannot make any statements about MAC-OS.

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 13-09-07, 5:12 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
Yes, short answer: I think it'd be great if Ignacio were to be made aware of F-TexTools development.

I tried compiling the nv tex tools today - as you said it doesn't do any SSE or AltiVec optimizations by default but specifying the compile option worked right away. My SquishDDS already uses SSE2/Altivec optimized squish, but I may switch to using the nv tools to take advantage of its more numerous normal map options.

Normal maps are definitely an area where the nv tools could become really powerful. Large, detailed images generated with your tools are obviously excellent input data for the nv tools and Ignacio should probably be kept in the loop as much as possible.

These items on the nv tools TODO list look promising:
- Add support for accurate normal map mipmap computation. Using lighting integrals.
- Add support for YCoCg and JPEG-LS color transforms. Add high quality DXT5-RGB compression.

I think Ignacio would appreciate it if we could act as testers and provide feedback. I have both a PPC Mac and a CUDA-capable PC, and profiling/compiling optimized builds is something I do all the time.

BTW, I found out that the ".dxt5nm" extension can just be specified in the output filename.
For example, nvcompress -normal -bc3n input.png output.dxt5nm works as expected, and nvddsinfo output.dxt5nm parses the file correctly.


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 13-09-07, 23:17 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
dirkpitt wrote:
These items on the nv tools TODO list look promising:
- Add support for accurate normal map mipmap computation. Using lighting integrals.
- Add support for YCoCg and JPEG-LS color transforms. Add high quality DXT5-RGB compression.

For VT tiles mipmaps are superfluous, of course. But the second issue looks interesting to me.

Quote:
I think Ignacio would appreciate it if we could act as testers and provide feedback. I have both a PPC Mac and a CUDA-capable PC, and profiling/compiling optimized builds is something I do all the time.


Well Ignacio has already kept quite regular contact with me/us in the shatters.net developer talk forum. So I will certainly try to make him visit here.

Quote:
BTW, I found out that the ".dxt5nm" extension can just be specified in the output filename.
For example, nvcompress -normal -bc3n input.png output.dxt5nm works as expected, and nvddsinfo output.dxt5nm parses the file correctly.


Well but that does not seem to help when you have to print out 2048 level 5 tile names. ;-)
Within a shell-script one can do it however.

My custom version just does generate the 2048 correct .dxt5nm tile names from the .png tiles upon the single command

find . -name "*.png" -exec nvcompress -normal -bc3n \{\} \;

In case of the official nvcompress version, you need to be more fancy:
Code:
-------------------------- doit.sh -------------------------
#! /bin/sh

name = "`basename $1 .png`"
nvcompress -normal -bc3n  $name.png $name.dxt5nm

-------------------------------------------------------------


and calling subsequently

Code:
find . -name "*.png"  -exec doit.sh \{\} \;


Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 14-09-07, 0:53 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
I love shell scripting! The bourne shell is especially great, the same thing could be done in Windows with wsh (VBScript, JavaScript, etc) but would look so much more verbose.

P.S. Sorry for the nitpick, but your script has a couple of bugs. "name" gets assigned "file.png.png",
the spaces in "name = ..." won't work, files in sub directories won't work, etc.
How about this?
Code:
#!/bin/sh

name=${1%.*}          #full path to file specified by $1, minus extension
nvcompress -normal -bc3n $1 ${name}.dxt5nm


Or may be just this one-liner:
Code:
#!/bin/sh

nvcompress -normal -bc3n $1 ${1%.*}.dxt5nm


I guess what I really should be saying is, since ".dxt5nm" is not really an "official" file extension (only Celestia uses it, no?) it might be hard to convince Ignacio to change the nv tools code just for this.


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 14-09-07, 7:19 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
Sorry DW,

while I was in a hurry and didn't test my trivial script, the ONLY reason why it does not work, was the /typo/ name = "`....`" instead of name="`...`".

The same script without the accidental " = " spaces works clearly and I have also tested it now:

Code:
-------------------------- doit.sh -------------------------
#! /bin/sh

name="`basename $1 .png`"
nvcompress -normal -bc3n  $name.png $name.dxt5nm
-------------------------------------------------------------


Note there is a blank between basename $1 and .png.

Bye Fridger

_________________
Image


Last edited by t00fri on Fri, 14-09-07, 13:59 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Fri, 14-09-07, 8:14 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
Ah ha, I didn't know that .png was a 2nd argument to basename! My bad.
This is getting even more off topic, but just for the sake of anyone trying these scripts out - the script as it is probably has issues if the file isn't in the current directory; for example if $1 is ./level0/xxx.png, which can happen since 'find' does a recursive search by default. My example ought to avoid this issue.


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 14-09-07, 9:21 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
DW,

clearly ${1%.*} is a more elegant and versatile option. I just had 1 minute to write that post and was not sure anymore off hand whether the notation ${1%.*} was also recognized in the simple Bourne shell. So I used the "safe" basename ... way.

I am a bit more concerned about Windows, since wsh is not installed by default. Of course people having CYGWIN installed, will have a zsh or bash available anyway under Windows.
In general, many Windows users are kind of reluctant to install further command line programs.

Since I reserved a whole scripting thread, we can find out together what the best approach will be for all three platforms.

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 14-09-07, 13:53 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
DW,

how about the default availability of libpng in MACS?
I use version 1.2.8 for Linux AND Windows. It's a stable, reasonably modern version.


Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 15-09-07, 2:45 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
No problem, that's the same version I use when compiling Celestia.
The libpng I use is a shared library, which means it needs to be distributed
along with the F-Tex binaries - this will be ok I hope?


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 15-09-07, 9:42 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
dirkpitt wrote:
No problem, that's the same version I use when compiling Celestia.
The libpng I use is a shared library, which means it needs to be distributed
along with the F-Tex binaries - this will be ok I hope?


I use shared libpng throughout. For Windows, I have included the *.dll's to the archive's Win32_PC.bin directory, where the WIB32 executables are. But for Linux, libpng.so is virtually a STANDARD part of any distribution, so I have renounced to including it once more. And you say that this is different for MAC OS?

After all in UNLIX-like installations one wants to install the programs in some general executable directory (/usr/local/bin). The programs then should also refer to standard installation directories for shared libs, like /usr/lib, /usr/local/lib. In Linux libpng.so is always in /usr/lib.

Bye Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 15-09-07, 11:12 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
It's true, there is no shared libpng that's installed as part of Mac OS X.
OS X does read PNG's just fine, but it doesn't use libpng - at least not as a shared library nor in any other way that makes it available to developers. Installing the library in /usr/local/lib works fine, but it requires the user to sudo and I think it'd just be simpler to distribute it in the same directory as the F-TexTools binaries.


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 15-09-07, 11:39 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4514
Location: Hamburg, Germany
dirkpitt wrote:
It's true, there is no shared libpng that's installed as part of Mac OS X.
OS X does read PNG's just fine, but it doesn't use libpng - at least not as a shared library nor in any other way that makes it available to developers. Installing the library in /usr/local/lib works fine, but it requires the user to sudo and I think it'd just be simpler to distribute it in the same directory as the F-TexTools binaries.


OK, distributing libpng also has advantages, since the lib has a number of 'pecularities' sometimes. By distributing a well-tested runtime version, we can be sure to have always the same settings internally. In Windows compilation with VS++ 200x.net, it is crucial for example, to use exactly the same compile options for libpng as for the program. Notably single/multi threadedness is a critical option. This is a familiar problem to MS. They have released even respective knowledge base articles.

Actually, I compiled my libpng in multi-threaded mode!

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group