It is currently Thu, 29-06-17, 2:05 GMT

All times are UTC




Post new topic Reply to topic  [ 9 posts ] 
Author Message
PostPosted: Thu, 20-09-12, 12:35 GMT 
Offline

Joined: Thu, 20-09-12, 11:23 GMT
Posts: 8
Hi folks!

I'm trying to use a normal, specular and watermask map for my Cinema 4d Earth Scene to get the best visual result. Hence, tiled versions are not usable for me. I even thought about putting the tiles together in Photoshop but then I tried the tools from Fridger:

Unfortunately I am unable to get correct results with Nm and F-texTools when I try to create big maps on my system.
The test file (4k) for Normal worked fine but when I try to create 64k maps the resulting Normal map looks disastrous - I did not get any error messages...

The specular map looks incorrect as well.

Gzip was also not registered in registry but that is the least of problems because I have Winrar - so I can and did it manually.

I am using W7 ultimate x64 with 14gb ram along with 2x physical Quads.

I have the watermask by cartrite in 84k so I actually only need the Normal and the specular Map to improve my scene in C4D....

The command prompt was new territory for me so I invested about 8-10h so far... without a usable result
-__-

Is someone, please!, willing to share a Normal and Specular Map in 64k, UNTILED!
I'm registered with 4shared - they have 15GB of free space.

Help will be much appreciated!

regards

Martin[/i]


Last edited by skunkworks on Thu, 20-09-12, 17:23 GMT, edited 1 time in total.

Top
 Profile  
 
PostPosted: Thu, 20-09-12, 14:33 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4447
Location: Hamburg, Germany
skunkworks wrote:
Hi folks!

I'm trying to use a normal, specular and watermask map for my Cinema 4d Earth Scene to get the best visual result. Hence, tiled versions are not usable for me. I even thought about putting the tiles together in Photoshop but then I tried the tools from Fridger:

Unfortunately I am unable to get correct results with Nm and F-texTools when I try to create big maps on my system.
The test file (4k) for Normal worked fine but when I try to create 64k maps the resulting Normal map looks disastrous - I did not get any error messages...

The specular map looks incorrect as well.

Gzip was also not registered in registry but that is the least of problems because I have Winrar - so I can and did it manually.

I am using W7 ultimate x64 with 14gb ram along with 2x physical Quads.

I have the watermask by cartrite in 84k so I actually only need the Normal and the specular Map to improve my scene in C4D....

The command prompt was new territory for me so I invested about 8-10h so far... without a usable result
-__-

Is someone, please!, willing to share a Normal and Specular Map in 64k, UNTILED!
I'm registered with 4shared - they have 15GB of free space.

Help will be much appreciated!

regards

Martin


Welcome at CM, Martin!

Your report sounds depressing...

What do you mean precisely with " the 64k Normal map looks disastrous" ??
If it looks somehow "scrambled" then most probably you made a mistake related to the "Endianness" that becomes relevant when dealing with 16bit binary numbers (elevation map!!).

As clearly discussed in my tutorial:

My Tutorial wrote:
First of all, you need to download the 1.2 GB gz-packed input heightmap of Earth from the NASA Arctic server. Here is the link, ready for clicking.

Remember, unlike our 4k warming-up input heightmap above, this file uses 'big endian' storage mode of its 16 bit integer height values.


Hence if you copied the command line parameter of the 4k example straight into your 64k task with only the widths values adapted, your result will look scrambled!!!

In conclusion, for your present 64k task, all 'little endian' machines (Windows PC, Linux PC, Intel Mac) need to insert the byteswap = 1 option in the first nmtools command of the "pipeline".

The first nmtools command is concerned with a reduction of the 85k elevation map to the nearest power-of-two size, i.e. to 64k. Adding the endedness compensation flag, the correct command line for a Windows PC looks like so:

nm2pow2 86400 1 < srtm_ramp2.world.86400x43200.bin | nms 6378.140 65536 2.5 > normalmap64k.png

I here have assumed that you unpacked the input heightmap separately and that the 64k normalmap file is called normalmap64k.png.

For a 'big endian' PPC MAC, the byteswap=1 compensation flag above would NOT be necessary, of course!

If you used a different command line from the one above, please let me know what was so difficult in writing this correct one? I have emphasized the Endianness issue even in two places, in order to hopefully avoid people stepping into this "trap":

1) Detailed discussion in the Tutorial!
2) Upon running nms without any parameters, a help text is first printed and then at the bottom:

Your computer uses +++ Little-endian (Intel-PC, Intel-MAC,...) +++ byte order!
If different from byte order of input file, use byteswap = 1


I have never heard of a Celestia user who would want to consider 64k UNTILED textures. The upload takes hours due to the asymmetry in up/download speeds.
A 64k RGB Normal map in (stage 9 packed) PNG format will be close to 10 GB!
That's why I wrote my tools: to make people save a lot of upload / download time and to hopefully make them understand a bit better how some things work...

In summary: To me it looks highly probable that you made some kind of error (Endianness!?) when dealing with the 64k Normal map amd Specmap tasks. In my tools there is NO reason why 4k jobs should work while 64k jobs give nonsense. The tools have actually been written for handling 'monster' textures! If you made the presumed Endianness mistake it would also have affected the 64k Specmap generation.

Of course I can only help specifically, if I see the command lines that you ACTUALLY used as well as the links and other operations for getting hold of the input elevation files.

Good luck,
Fridger


Last edited by t00fri on Sat, 22-09-12, 13:27 GMT, edited 2 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu, 20-09-12, 15:13 GMT 
Offline

Joined: Thu, 20-09-12, 11:23 GMT
Posts: 8
Thanks so much for your effort and time, Fridger....

Let me just try the above and I'll check in about 1/2h if there's a better result.
back soon...


Top
 Profile  
 
PostPosted: Thu, 20-09-12, 17:20 GMT 
Offline

Joined: Thu, 20-09-12, 11:23 GMT
Posts: 8
Quote:
Your report sounds depressing...


It indeed was! I tend to feel like a 14year old, in those situations! :mrgreen:

nm2pow2
86400 1 < srtm_ramp2.world.86400x43200.bin | nms 6378.140 65536 2.5 > normalmap64k.png

You hit the nail's head with that - I realized I was lacking this module > newer NM tools version :idea:
Now it seemed to work like a charm!
YES!

PNG is less than 1GB btw.
I tried to stick to the endian issue in the first place, indeed!
My problem was - I assume! - that the ReadMe only shows the command lines with the gzip module.
Hence my gzip module did not work and my files were already unzipped I tried to do it without.
I think it might be a good idea to add command lines for unpacked data - just like above!

In addition, I assume that the outcoming PNG format was another positive factor.

Now there is a whole dimension between understanding and applicating....
what's wrong with my improvisation (?): :|

tx2pow2 86400 1 > srtm_ramp2.world.86400x43200.bin | specmap 86400 swbd-watermask.1x86400x43200.bin 0.12 > specmap64k.png

again I'm trying to create the specmap without gzip - because it did not add to the registry, as mentioned earlier.
Almost there!

again: a million thx!

rg
Mart


Top
 Profile  
 
PostPosted: Thu, 20-09-12, 18:21 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4447
Location: Hamburg, Germany
Martin,

fortunately, this report of yours sounds much more optimistic!! ;-)
skunkworks wrote:
what's wrong with my improvisation (?): :|

tx2pow2 86400 1 > srtm_ramp2.world.86400x43200.bin | specmap 86400 swbd-watermask.1x86400x43200.bin 0.12 > specmap64k.png


Unfortunately, this commandline shows that you don't seem to have understood a number of VERY essential things: almost everything is wrong in that line.

1)
  • > filename means write the file named filename to STDOUT
  • < filename means read the file named filename from STDIN


What you want is READ not WRITE the file srtm_ramp2.world.86400x43200.bin

2) All tools from F-TexTools including tx2pow2 ONLY work on 8bit binary format which implies that Endianness issues are irrelevant! So for tx2pow2 there is no byteswap parameter. When you read in srtm_ramp2.world.86400x43200.bin you are reading a file composed of 16bit integers (height values!) and it indeed makes a crucial difference how the 1st and 2nd byte of the 16bit integers is stored relative to each other in your machine and in the elevation file. Obviously, you need the command nm2pow2 from the Nmtools package and NOT tx2pow2. So the first part of the pipeline must read

nm2pow2 86400 1 < srtm_ramp2.world.86400x43200.bin |

But it is MUCH simpler if we arrange the tasks in the pipeline differently:

We don't reduce the 85k filesizes to 64k initially. Since always the watermask file size (85k) and the elevation map file size (85k) must be equal, we write

specmap 86400 swbd-watermask.1x86400x43200.bin 0.12 1 < srtm_ramp2.world.86400x43200.bin > specmap85k.bin

This is the first step generating a 85k specmap file with oceans, rivers and lakes, but still in 8bit binary raw format.

Suppose the 85k BMNG surface texture is called world.200406.3x86400x43200.bin corresponding to June of 2004. Then it is clever to write

tx2rgba 86400 specmap85k.bin < world.200406.3x86400x43200.bin| tx2pow2 4 86400 | bin2png 4 65536 > earth64k_rgba.png

This way you have mounted the generated specmap as "fourth color" on top of the rgb surface map, reduced it to the nearest power-of-two size (64k) and converted the binary raw format to PNG.

If you don't want to use this fast performing 4 color approach, you can proceed as follows with your generated specmap85k.bin grayscale file

tx2pow2 1 86400 < specmap85k.bin | bin2png 1 65536 > specmap64k.png

Of course you can combine these two lines also into one pipeline:

specmap 86400 swbd-watermask.1x86400x43200.bin 0.12 1 < srtm_ramp2.world.86400x43200.bin | tx2pow2 1 86400 | bin2png 1 65536 > specmap64k.png

This spares the storage for the large specmap85k.bin file that you don't really need.

Try to precisely UNDERSTAND the command lines I wrote down for you!!! Find possible typos I might have introduced in writing all this during just a few minutes ;-)

Good luck
Fridger


PS: Note since we were not using Path names, ALL files that are read for input must reside in the same directory!


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 24-09-12, 17:34 GMT 
Offline

Joined: Thu, 20-09-12, 11:23 GMT
Posts: 8
Tank you again for your patience, Fridger!

I've integrated the new texture into my scene and done some un-post-edited renderings - straight out of cinemas physical renderer.
I've worked on the base texture though: coast lines were "cleaned" and the ocean was color corrected.
The reflective light needs much more tuning and the cloud map is still a mess!
anyway... take a look:

Image

Scene's too dark I think...
The land-surface on the specmap turned out to be way too reflective in Cinema but it was fairly easy to correct that by just making it darker in Photoshop. This was especially visible were the normalmap produced a lot of geometry.

opinions/critique appreciated!

thankful regards!

Martin


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 24-09-12, 18:08 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4447
Location: Hamburg, Germany
Hi Martin,

great that you seem to be making a lot of progress! The Cinema image looks very nice and "intriguing" as well. Intriguing because the Caribbean is breathtakingly golden. But some sunsets there are indeed supposed to be out of this world.

Image

As to the reflectivity of the specular landmap, this can be changed by decreasing the <light> value in my specmap routine.

Regards,
Fridger


Top
 Profile  
 
 Post subject: re
PostPosted: Wed, 26-09-12, 9:08 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 749
Location: N 42.38846 W 83.45456
glad to see you are getting things working

on a side note
you WILL find errors in the spec map( the watermask)
it DOSE NOT 100% line up with the earth map

a good example is Hawaii .
the Big isl. is off by about 5 pixels

and the direction of the error is a bit random .So there is no "easy fix" .

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 26-09-12, 12:40 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 991
Location: NE PA, USA
http://forum.celestialmatters.org/viewt ... =3549#3549

Some more examples of errors. From what I remember, it seemed to be confined to islands. Maybe something wrong with original data or the conversion process. I never found out.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group