It is currently Sat, 21-09-19, 23:57 GMT

All times are UTC




Post new topic Reply to topic  [ 101 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next
Author Message
 Post subject:
PostPosted: Thu, 15-05-08, 15:27 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
BobHegwood wrote:
I have but one more Brain-Dead question for you geniuses here. :wink:

Is there a way to correct the Gamma of the tiles using GDAL (or some other) command-line utility? I have managed to convert Andrea's beautiful new Night Light tif images into PNG tiles, but I will now have to do the same thing that I did for each and every one of the resulting tiles I got in the exercises above unless there is a way to do this via a command-line utility...

In other words, I had to use a Windows program (IrfanView) to adjust the gamma values of ALL of the resulting tiles (each and every one by hand) in order to use them properly on my system, and I really do NOT wish to do this again.

Again, is there some way to adjust Gamma Offsets using one of these utilities? Have looked at the help layouts for most of the GDAL commands, but didn't see anything which would work here.

Any other suggestions?

Thanks again, Brain-Dead

EDIT: Hah... Never mind. I found out how to do this with IrfanView's very own batch converter.
Thanks. Great stuff, this command line feature. :D

I still can't see why you don't correct the gamma on the 43200x21600 image. Then there is no need to do every tile. The Gimp can do this with the Tools => Color Tools => Levels. The Input Levels are at the top and the center adjustment is for Gamma adjustment.
I'm not aware of any Gamma correction with GDAL. IrfanView's very own batch converter is a way to do it but it probably has a way to do the entire 43200x21600 png image which will make doing it to every tile produced unnecessary.
PS It looks like you got a much better system than I got. You should be able to process large images easier than me.
cartrite


Last edited by cartrite on Thu, 15-05-08, 16:25 GMT, edited 2 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 15:33 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
cartrite wrote:
I still can't see why you don't correct the gamma on the 43200x21600 image. Then there is no need to do every tile. The Gimp can do this with the Tools => Color Tools => Levels. The Input Levels are at the top and the center adjustment is for Gamma adjustment.
I'm not aware of any Gamma correction with GDAL. IrfanView's very own batch converter is a way to do it but it probably has a way to do the entire 43200x21600 png image which will make doing it to every tile produced unnecessary.
cartrite


Very simply, I CANNOT do it because my system apparently doesn't have enough RAM. That was hard for me to believe, but it keeps crapping out whenever I try to edit the 43200x21600 images. Don't know why...

Perhaps I'm not getting the cache large enough? Dunno... :roll:

The following is my setup:

Code:
OS Name   Microsoft® Windows Vistaâ„¢ Home Premium
Version   6.0.6001 Service Pack 1 Build 6001
Other OS Description    Not Available
OS Manufacturer   Microsoft Corporation
System Manufacturer   HP-Pavilion
System Model   GQ498AA-ABA m9047c
System Type   X86-based PC
Processor   Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz, 2394 Mhz, 4 Core(s), 4 Logical Processor(s)
BIOS Version/Date   American Megatrends Inc. 5.07, 8/8/2007
SMBIOS Version   2.5
Windows Directory   C:\Windows
System Directory   C:\Windows\system32
Boot Device   \Device\HarddiskVolume1
Locale   United States
Hardware Abstraction Layer   Version = "6.0.6001.18000"
Time Zone   Eastern Daylight Time
Installed Physical Memory (RAM)   3.00 GB
Total Physical Memory   3.00 GB
Available Physical Memory   2.06 GB
Total Virtual Memory   6.22 GB
Available Virtual Memory   5.27 GB
Page File Space   3.29 GB
Page File   C:\pagefile.sys


You tell me. Why does my system end up with "Program Not Responding" when I try to edit the file?


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 16:19 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
I'm not sure about this "Available Physical Memory 2.06 GB "
Vista uses 1 gb of memory?
Since you have "Processor Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz, 2394 Mhz, 4 Core(s), 4 Logical Processor(s) "
Maybe you can try to set the Number of Processors to use to a higher level.
Maybe you can try 2 or 4?
Here is a screenshot of the Preference menu from the Gimp.

Image

I recommend trying to set tile cache size to 1 gb.
Underneath the environment settings are folders.

Image

Make sure the tmp and swap folders are on a disk partition that has plenty of disk space. 15 gb +. The more the better. If your C drive has plenty of free space you can leave them in the default position. Then all you need is patience. This is a big file and it takes time. I need to wait a long time for tasks to finish on my system too.
I was able to work with the 84k bmng files with the Gimp when I only had 2 gb of ram and a 2 ghz processor. I was also able to use the Gimp on my old computer which had 512 mb of ram and a 900 mhz processor. It just takes time and disk space. If I try to interrupt the process by doing something else while the Gimp is running, I too got a "Program not Responding". You just got to leave the computer be while it is working and give it a chance to finish.
PS It looks like you got a much better system then me. You should find it easier to process large images then I do.
cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 18:22 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
Wow...

I simply had no idea that you could configure the Gimp as you have displayed above. (BIG surprise there I'll bet. :roll: )

As always, many thanks for your help here Steve. Is there anything you people don't know? :wink:

I'll give this a shot and see what happens. Seriously though, it never even occurred to me to see if the Gimp could be configured to use RAM the way you have explained it above. Almost certain that this is my problem.

Again, many thanks my very patient friend. You sure you don't want a Brewsky?
Hell, I'd be willing to MAIL you a case of them now. :D


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 18:35 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
I've noticed something about setting Tile cache size on my Linux system. When I increased my RAM to 4 gb, I increased the Tile cache size to 3 gb. This caused all kind of problems. My system monitors showed that it used all ram available and then started duplicating it on my swap drive. When I set it to 2 gb, it seems to run perfectly and allows the OS to remain in RAM. Half the ram seems to be the best setting for the Tile cache size setting. That is why I suggested 1 gb in your case since your specs showed 2 gb available.
Bob wrote:
You sure you don't want a Brewsky?
You know that sounds like an idea. Unfortunately for me, my body can't handle it anymore. You see, when I start drinking brewskies, my eyes tend to be larger than my stomach. :shock: Hangover city.
cartrite :shock:


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 19:59 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
cartrite wrote:
You know that sounds like an idea. Unfortunately for me, my body can't handle it anymore. You see, when I start drinking brewskies, my eyes tend to be larger than my stomach. :shock: Hangover city.
cartrite :shock:


Well Hell, I'll send you some good news then. :wink:

When I applied the settings as you described, my system went BLOOEY!

However, after some patient investigation and playing around with values, I arrived at a scenario which will allow me to edit up to (and including) a 3.5 GB file...

First, I'm using the Gimp 2.4.4 for Windows on my Vista Home Premium, 32-bit system.
I changed my settings to the following, and all is now well with the world:

Minimum Number of Undo Levels=1
Maximum Undo Memory=250MB
Tile Cache Size=1024MB
Max New Image Size=1024MB
Number of Processors to use=4

I also had to create a temp folder on my C:\ drive before the program worked as I had hoped. I then directed the Gimp to use this folder for the Temp and Swap files.

All now works well, so thanks again. :D

I do still prefer, however, to use the batch file alternative provided by IrfanView.

Why? Because it is not nearly as taxing to my system, and it executes almost immediately compared to the Gimp. Also, I have not yet found any way (in the Gimp) to adjust Gamma values. I am very happy now though, so thanks again.

Rule Number Seven for the Brain-Dead trying to create Virtual Textures:
Investigate all of the options for your software. Learn from MY mistakes. :roll:


Last edited by BobHegwood on Fri, 16-05-08, 13:45 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Thu, 15-05-08, 21:24 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
BobHegwood wrote:
When I applied the settings as you described, my system went BLOOEY!


BobHegwood wrote:
Minimum Number of Undo Levels=1
Maximum Undo Memory=250MB
Tile Cache Size=1024MB
Max New Image Size=1024MB
Number of Processors to use=4
Not sure why it would go BLOOEY. I recommended 1 gb and 1024 mb is 1 gb. The settings in the preference window I posted were my settings. Minimum Number of Undo Levels=1 and
Maximum Undo Memory=250MB reduce the working file size. If these are set too high, the working file that Gimp stores in memory will increase in size with every operation performed. If it is set to zero, the file will not grow but you if a mistake is made you can't undo it.

BobHegwood wrote:
Also, I have not yet found any way (in the Gimp) to adjust Gamma values.

In the Gimp you can adjust the gama by clicking Colors->Levels. When the Levels window appears, Look at the Input levels. There are 3 adjusters. The center one adjusts gama.
cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 17-05-08, 20:57 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
cartrite wrote:
BobHegwood wrote:
Also, I have not yet found any way (in the Gimp) to adjust Gamma values.

In the Gimp you can adjust the gama by clicking Colors->Levels. When the Levels window appears, Look at the Input levels. There are 3 adjusters. The center one adjusts gama.
cartrite


Ah, but it does NOT adjust the Gamma values in the way I wish to adjust them, and it takes approximately 35 minutes to finish a single pass. If you don't know what value to place in the slider, you are screwed. I simply used the batch facility from within IrfanView to adjust my final tiles in less than 5 minutes. :D

Another note here...
It took me over 4 hours to do what I'm about to describe in the next post using the Gimp. Even though I have it working to the best of my ability, it is extremely slow for use on very large images. This is also due (I'm almost certain) to the fact that I am also using Windows Vista. Just FYI... :wink:


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 17-05-08, 21:34 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
Okay, here's what it took for me to create a pleasing Night Map VT... :roll:

1. Changed all Vista settings (via Control Panel) to place emphasis on performance, rather than quality.
2. Ended ALL unnecessary programs and started the Gimp.
3. Loaded the 43200 x 15600 nightlights.png texture (which I had already resized from 43201 x 15601.)
4. Took the Gimp approximately 15 minutes to load this image and to get ready for commands.

Important Note: On my system, I had to make certain that each step was finished before I went on with the the next step. I watched my disk in use light until it quit shining.

5. Enlarged the image's canvas size to 43200 x 21600 and centered the existing 43200 x 15600 image.
6. Flattened the resulting image and saved it to disk. By the way, this image was taking up 4.49 GB in RAM, so now you know why it took soooo long. Kept swapping into and out of temporary disk file used for Virtual RAM.
7. Closed image and then loaded the stable_lights.png image I got from NASA originally.
8. Changed view to 1:8 (12.5%) so that I could accurately select a rectangular region precisely.
9. Selected the upper 3000-byte region from this file and saved it to a buffer named "top."
10. Closed stable_lights.png (without saving) and the reopened the nightlights.png file I saved in step 6.
11. Changed view to 1:8 so that I could select the same 3000-byte area (in this image) which I copied in step 9.
12. Pasted named buffer (top) into the selection.
13. Anchored the resultant floating layer, then Flattened the image. (Took about 45 minutes.)
14. Saved the nightlights.png to disk and closed. (Just the image, NOT the Gimp.)
15. Repeated steps 7-14 for the LOWER 3000 bytes. The only difference was that I copied the lower 3000-byte section to a buffer called "bottom." Copied lower 3000-byte section from stable_lights.png into nightlights.png.

Note: There may not have been any image information in this section, but I wanted to make certain, so I did this in order to insure that I got everything I wanted.

After finishing with these processes, (Took about 4 hours) I then proceeded with the commands needed by the Good Doctor's F-TexTools in order to create levels 0 through 4 of the Earth Night Map VT.

Please note that I used IrfanView's batch facility in order to adjust the Gamma and sharpness of each tile to the levels I wished. The entire IrfanView process took less than 5 minutes. So?

Rule Number Eight for the Brain-Dead trying to create Virtual Textures:
Get IrfanView HERE.

That's it. I'm finished, worn out, and ready for a Brewsky.
Unfortunately, I don't drink either, Steve. (I'm an insulin-dependent diabetic with neuropathy and rheumatoid arthritis. Got enough problems.)

That's it for me. 8) Hope this helps other Brain-Dead users in the world.
Thanks, Bob


Top
 Profile  
 
 Post subject:
PostPosted: Sun, 18-05-08, 18:13 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Hey Bob,
Glad to see you got this worked out. Good job. :D
cartrite
Quote:
Unfortunately, I don't drink either, Steve.

I fell of the wagon yesterday and today I climbed right back on. I guess every now and then I have to remind myself why I gave it up. Hopefully this next dry spell will last much longer.
The Gimp can be challenging with big files. Important thing is to let it finish each step. Best to start a step and walk away for a while and let it finish.
Quote:
(which I had already resized from 43201 x 15601.)

I resized mine by leaving the top row and last column off. I couldn't find any reason why there was an extra row and column in the documentation, but NASA usually adds an extra row at the top and the extra column at the end.
Here is what I did.
Opened the F152003 file which was 43201x15601.
If you notice the Gimp window I posted above, Under the word File is a little rectangle. If you click on it it brings up the rectangle select dialog and the different settings available. I used a fixed size of 43200x15600 and had it set to "Replace the selection". I viewed the image at 1600 % so I could be sure to start the selection on the right pixel. Then I positioned the image to the upper left corner. I positioned the cursor on pixel 0,1 and clicked the selection. This selected 43200x15600 pixels leaving off the top "0" row and column" 15601" . I copied this selection to the clipboard. Then I opened the rad-cal-nightlights and selected a rectangle that was 43200x15600 starting at pixel 0,3000. Then I pasted what was copied from the clipboard into the selection, anchored it, then saved. Before I anchored it I played with the opacity of the pasted layer to "Blend" the 2 layers. I set it to 75% opacity with the Layers, Channels, Paths window.
cartrite


Top
 Profile  
 
 Post subject: re
PostPosted: Tue, 20-05-08, 20:50 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 847
Location: N 42.38846 W 83.45456
hi bob just a note on my old dell with 512 meg ram and a 2 gig virtual ram i can open a 32k map , in .v ( vips format ) in about 10 sec. and a 32k .png map in about 2 min 10 sec and a 23k map of Dione in 30 sec.( in comparison i can open the 32k map.bz2[1min 5 sec] it . using vips/nip2 also all operation's like gamma correction are done when saving
a 32k .png at compression 4 takes about 6 to 8 min
and a 32k.v saves in 1min 45sec
and from your post
Posted: Sat May 17, 2008 9:34 pm
--
Enlarged the image's canvas size to 43200 x 21600 and centered the existing 43200 x 15600 image.
--
it dose it fine
--
Flattened the resulting image and saved it to disk. By the way, this image was taking up 4.49 GB in RAM, so now you know why it took soooo long. Kept swapping into and out of temporary disk file used for Virtual RAM.
--
there is no alpha chan support ( no transparency)
but the "insert " function dose it most of the time for me
--
8. Changed view to 1:8 (12.5%) so that I could accurately select a rectangular region precisely.
9. Selected the upper 3000-byte region from this file and saved it to a buffer named "top."
10. Closed stable_lights.png (without saving) and the reopened the nightlights.png file I saved in step 6.
11. Changed view to 1:8 so that I could select the same 3000-byte area (in this image) which I copied in step 9.
12. Pasted named buffer (top) into the selection.
--
the crop works great for this
nip is what i wsed in making my night lights and adding the north/ south 30 deg to the map.
also for adding the lower res image to the southern area missing for the mars hires data
there is a VERY BIG learning curve for nip2 because it is NOT like Gimp or PhotoShop but more like Excel .

Quote:
Quote:
Unfortunately, I don't drink either, Steve.

I fell of the wagon yesterday and today I climbed right back on. I guess every now and then I have to remind myself why I gave it up. Hopefully this next dry spell will last much longer.

i don't do AA ( i am an atheist) and use Rational Recovery
a one day slip is onething just don't start falling back into old ways and good luck

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6


Last edited by John Van Vliet on Wed, 21-05-08, 11:15 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Wed, 21-05-08, 1:42 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
Well again, I'm thankful for all of the help and advice here. Hopefully, this topic will serve to get others through the (rather intimidating) learning curve. Many thanks to Doctor Schrempp for allowing this topic, and to Steve for all of his help during the process. Thanks to you too, John. I always like your stuff.

Please note, however, that your quote was not accurate.
Quote:
Quote:
Unfortunately, I don't drink either, Steve.

I fell of the wagon yesterday and today I climbed right back on. I guess every now and then I have to remind myself why I gave it up. Hopefully this next dry spell will last much longer.


This came from two different sources, and not just one as your post implies. The first line was mine. The rest was NOT. :wink:


Top
 Profile  
 
 Post subject: re
PostPosted: Wed, 21-05-08, 11:11 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 847
Location: N 42.38846 W 83.45456
i meant the end for cartrite and steve oops!

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6


Top
 Profile  
 
 Post subject:
PostPosted: Tue, 24-06-08, 14:25 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
One more thing I had to do here in order to create (at least to me) very pleasing night textures...

I finally figured out how to use IrfanView's batch converter in order to increase the contrast of my night textures, so that they look really great now. Again, this is just what I wished for... Obviously, others might prefer different settings for their night textures, but the reason for this post is two-fold...

First, IrfanView is a very valuable freeware tool for people who live in the GUI world.

Second, I don't know why the original images I used ended up in such a mess, but (at level 4 especially) the night lights were drawn incorrectly on my machine. They featured very hard-edged (for lack of better terminology) lights on the surface of the Earth. As you can see below, however, these lights are now VERY pleasing to me. I used IrfanView in order to increase the color-depth to 16.7 million colors, to 24 BPP, and to increase each image's contrast by a factor of 50 in IrfanView. I have absolutely no idea what percentage this is, but it made an extreme difference in the output. See image below.

Image

The night textures I had prior to executing this procedure extended out into the oceans and seas, and were just plain ugly. They were partially opaque, and did not blend into the overall texture of the Earth as they should have.

Again, this is just for everyone else's information here... :wink:

EDIT: If you care to view the improvement this made, have a look at THIS post to see the difference it made.


Top
 Profile  
 
 Post subject:
PostPosted: Tue, 24-06-08, 15:53 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
Better yet, here's a comparison for the (almost) same area both before and after my conversion as described in the previous post:

BEFORE:
Image

AFTER:
Image

Worth doing... Yes? :wink:


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 101 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group