It is currently Tue, 19-06-18, 8:32 GMT

All times are UTC




Post new topic Reply to topic  [ 75 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject: Re: A Windows Story
PostPosted: Sat, 22-06-13, 0:18 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Thanks chuft-captain,
I've had a look and I'm gonna look into it. Working with big images needs customized code.

I downloaded IM because I wanted to automate things but was disappointed with IM's performance. So I wrote the rgb program and was a little freaked out when I seen the above results. I'd still like to know why using stdout seemed to be the reason for the errors. Hardware fault? I've been pushing this thing lately.
cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sat, 22-06-13, 4:50 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
This is 6/22/13 and I'm still being reminded to update my graphics driver to the older version. What a great OS.


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sat, 22-06-13, 6:28 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 780
Location: N 42.38846 W 83.45456
Quote:
This is 6/22/13 and I'm still being reminded to update my graphics driver to the older version. What a great OS.


That used to drive me BATTY
and not the good kind that Jack Nicholson to Michael Keaton's batman


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sat, 22-06-13, 11:47 GMT 
Offline
User avatar

Joined: Fri, 03-04-09, 8:21 GMT
Posts: 219
To get you started...

Here's a bit of a hacked together processing "sketch" to combine your 3 test images into a single RGB:
Code:
PImage r = loadImage("red.jpg");
PImage g = loadImage("green1.jpg");
PImage b = loadImage("blue1.jpg");

size(512, 512);
PImage rgb = createImage(512, 512, RGB );
rgb.loadPixels();

int w = r.width;
int h = r.height;
for ( int i=0; i < w; i++)
{
  for (int j=0; j < h; j++ )
  {
    color c;
    int p;

    p = j * w + i;
    c = color( r.pixels[p] >> 16 & 0xFF, g.pixels[p] >> 8 & 0xFF, b.pixels[p] & 0xFF ); 
    rgb.pixels[p] = c;
  }
}
rgb.updatePixels();
image(rgb, 0, 0);
save("rgb.png");


... producing:
Attachment:
rgb.png
rgb.png [ 105.11 KiB | Viewed 2706 times ]

FYI:
This will process a 2048x2048 version in ~ 1.7 secs on my machine, .... and a 10240 x 10240 version in 28.618 secs. --- I ran out of memory trying 20480 x 20480 :wink:
I don't know what size images you ultimately want to produce, but for very large ones, you'll clearly need a more sophisticated approach, to avoid running out of memory.


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Tue, 25-06-13, 4:04 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 780
Location: N 42.38846 W 83.45456
bypassing IM , as most know i like gmic ( though there is a 32 bit mem issue in Cimg.h max file size of 4 gig )
--- for gray .pgm images
Code:
gmic red.pgm green.pgm blue.pgm -append[0,1,2] c -o RGB.ppm

RGB.ppm is a 24 bit ( 8 bit /layer)

for RAW uchar 2048x2048

Code:
gmic -type uchar red.raw,2048,2048 -type uchar green.raw,2048,2048 -type uchar blue.raw,2048,2048 -append[0,1,2] c  -type uchar -o RGB.ppm


for a 2k image
it ran to fast to measure
less that 0.25 sec.

A 16384x8192 red,green,blue took 13 sec.


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Tue, 25-06-13, 13:43 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
With the program I wrote, it did up to 11 4800 x 4800 in less than a second. The processing sequence is done in less than a minute. See below.
Code:
for %%I in (*sur_refl_b01.tif) do gdal_translate -a_nodata -28672 -scale -100 16000 0 255 -ot Byte -of png %%I ..\png-files\b01\%%~nI.png
copy ..\png-files\b01\*.xml ..\rgb
for %%I in (*sur_refl_b04.tif) do gdal_translate -a_nodata -28672 -scale -100 16000 0 255 -ot Byte -of png %%I ..\png-files\b04\%%~nI.png
for %%I in (*sur_refl_b03.tif) do gdal_translate -a_nodata -28672 -scale -100 16000 0 255 -ot Byte -of png %%I ..\png-files\b03\%%~nI.png
cd ..\png-files\b01
for %%I in (*sur_refl_b01.png) do png2bin < %%I > ..\%%~nI
cd ..\b04
for %%I in (*sur_refl_b04.png) do png2bin < %%I > ..\%%~nI
cd ..\b03
for %%I in (*sur_refl_b03.png) do png2bin < %%I > ..\%%~nI
cd ..\..\png-files
for %%A in (*.sur_refl_b01) do rgb 4800 4800 %%~nA.sur_refl_b01 %%~nA.sur_refl_b04 %%~nA.sur_refl_b03 ..\rgb\%%~nA.sur_refl_b01.bin
cd ..\rgb
for %%I in (*.bin) do bin2png 3 4800 4800 < %%I > %%~nI.png

Then it takes about a half an hour to do this.
Code:
for %%I in (*.png) do gdal_translate %%I %%~nI.tif
for %%I in (*.tif) do gdalwarp -r cubicspline %%I ..\png-file\Geo-world-a1.tif

The hard part is the extreme northern latitudes and probably the southern latitudes too. They are all different sizes and the Modis Reprojection Tool does not scale the images properly. An image at 500 meter with a size of 301 x 95 is scaled to 600 x 192 at 250 meter. This leaves a border line around the image. So I have to rescale them before rgb and bin2png can be run. This also stops me from running "Identify" from IM to extract file sizes.
cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Wed, 17-07-13, 22:24 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
I'm still being harassed about updating my driver to the older version. :shock: Starting to get on my last nerve. :evil:

Another note, :wink: been working with Modis MOD09 images. Seems like an impossible task to get a new good quality texture of the earth with this data. A lot of data to download. But I'm gonna take my time and keep updating areas with poor quality data. There were a couple of good thing that came of this project. A northern ice map / mask and another mask of Antarctica. Here are some images. The first is a raw mosaic without any mask. The next is a mask I used on it. A combination of data from the spec map form SWBD data, GLFC Water Mask, and the state band from the MOD09 images which is what the northern ice mask was created with. Eventually I'm gonna use the SRTM height data to create a continental and island mask. That will be combined with a mask / map of Antarctica derived from zero values at sea level. This is a file called ramp200dem_osu_v2.bin.gz This is a citation at the site that acknowledges the data contributors.
Quote:
Liu, H., K. Jezek, B. Li, and Z. Zhao. 2001. Radarsat Antarctic Mapping Project Digital Elevation Model Version 2. Boulder, Colorado USA: National Snow and Ice Data Center.
http://nsidc.org/data/nsidc-0082.html

It is a 200 meter map and I'm gonna combine it with SRTM-3 data to create the land mask and also a height map to generate a new normalmap.

Here the MOD09 mosaic from an 8 day composite release for day 185, 2012. Most of the files were downloaded from the USGS using Earth Explorer. They are the NASA LPDAAC Collections, Land Surface Reflectance Data set.

Attachment:
small-world.jpg
small-world.jpg [ 159.51 KiB | Viewed 2625 times ]


This the mask I used described above. A combination of snow and ice flags, and a flag for cloud cover from the MOD09 files. It also used the specmap produced with data from the SWBD and GLFC Watermask. Antarctica is from the SRTM map used for some time for everybody's normal maps.

Attachment:
small-mask.jpg
small-mask.jpg [ 167.98 KiB | Viewed 2625 times ]


The result from using above mask. I added the color value the BMNG used for deep oceans. The actual rgb of Antarcticais data from day 355, 2009, using MOD09GA daily level 2 lite files.

Attachment:
small-test4.jpg
small-test4.jpg [ 162.77 KiB | Viewed 2625 times ]


Here is the new mask I'm working on using Radarsat Antarctic Mapping Project Digital Elevation Model Version 2 and the MOD09A1 files using the 500 meter state band. A 16 bit file with flags for fire/smoke, ice/snow, clouds, aerosol quality, etc. Notice the difference in Antarctica.

Attachment:
small-south-ice.jpg
small-south-ice.jpg [ 58.64 KiB | Viewed 2625 times ]


cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Fri, 21-11-14, 20:35 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
I was trying to build qt5 and the build was failing. The install directions were saying that "windows sdk 7.1" was needed but every attempt to install failed. After some research on the web and the readme file supplied with qt5, I uninstalled msvc sp1. After that I still couldn't install the sdk 7.1. So I uninstalled msvc 2010 and tried to reinstall it. But no header files were installed. :X Finally, today I came across the answer.
https://connect.microsoft.com/VisualStu ... re-missing
A Library key had to be deleted from the registry after msvc 2012 express was installed. I think that is what was being said. Anyhow after backing up the key and deleting it. Msvc 2010 express installed correctly with all the header files. At least I think it did. :clap: I haven't tried a build yet. ;)
cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Fri, 21-11-14, 20:57 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 780
Location: N 42.38846 W 83.45456
windows is not the only OS with qt4 VS qt5 issues

i know there is at least one other here that uses SUSE
do a new clean install of opensuse 13.2
there is a nasty qt issue if one upgrades 13.1 to 1.32

but this is a MS windows thread so only a few small lines

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Fri, 21-11-14, 21:04 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
I've had so many problems with qt5 and opengl, I don't think this forum is big enough to hold a thread like that. ;) I posted the issue I'm having with cosmographia at the qt5 forum. Haven't got a reply yet. Go figure.
cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Fri, 21-11-14, 22:58 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Instead of using sdk7.1, I installed sdk 8. And now I'm using that debugger. Wouldn't you know it, I actually get debugging info that makes sense. I spent a lot of time trying stuff that didn't need trying. I'm almost completely convinced that the issues I have been seeing with cosmographia are a qt5 problem. In other words, what I'm seeing in the debug info is saying the size is right and it is a power of 2. The two first things to look for when glTexImage3D is used and causes a problem.
cartrite


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Fri, 19-12-14, 22:35 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Possible Windows story here. To make a long story short, I thought I was going thru another computer melt down. The thing wouldn't start. After I tried many things, it finally started after about a half an hour of boot time. So finally, after uninstalling many windows updates, it is almost back to normal again. Par for the course. ;) What a great company. :wall:


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sat, 20-12-14, 1:10 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 780
Location: N 42.38846 W 83.45456
so on "black" update Tuesday
i take it you installed the update KB 3004394
the one that is causing so many problems

MS did remove it and has a new update to fix (??? we will see ) the mess of the old one

turn off AUTO install of updates
set it to just inform you
then ignore the two WARNING messages

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sun, 21-12-14, 0:47 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
John Van Vliet wrote:
so on "black" update Tuesday
i take it you installed the update KB 3004394
the one that is causing so many problems

MS did remove it and has a new update to fix (??? we will see ) the mess of the old one

turn off AUTO install of updates
set it to just inform you
then ignore the two WARNING messages
Actually, no. That's not the update that did it. KB something and I don't give a f. That was it. But in all fairness to Microsoft. They just got so many.

Here is the thing. I just can't afford the time to try and figure this stuff out anymore. I long for the days when the major technology of the day was common sense. Figure that one out.


Top
 Profile  
 
 Post subject: Re: A Windows Story
PostPosted: Sun, 21-12-14, 2:56 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 780
Location: N 42.38846 W 83.45456
figuring out what a MS update does .....

well you really have to pull them apart and decompile it
and do a diff on two VM's to compare what is being updated
and / or

read 20 tech support and security sites

that is one thing i like with linux
the bug it fixes IS listed
and almost all the time the bugzilla address for the report IS given
or a pointer to the kernel-dev mailing list

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 75 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group