It is currently Thu, 20-09-18, 14:42 GMT

All times are UTC




Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next
Author Message
 Post subject:
PostPosted: Thu, 16-02-12, 17:35 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
... and now for the bad news:

After 1.5 days of testing and experimenting, I am afraid that the new Chang'e 2 surface map of the moon is NOT the best option for Celestia(.Sci)!

Instead, my recommendation is to use the 64k LRO-WAC surface VT set, as adapted by John Van Vliet from the Nov 2011 data:

http://www.celestiamotherlode.net/catal ... on_id=1108

If I had a wish free about JohnVV's VTs, I'd love to see the mares in higher contrast (i.e. darker), and a computer-colored surface using the UVVIS 5 band colored moon imaging as a template...

Anyway, the LRO-WAC surface texture is throughout in beautiful alignment with the official 32k LRO-LOLA normalmap VT release of March 2011!

http://www.celestiamotherlode.net/catal ... on_id=1549

Only after I had created the high-resolution 32k Chang'e 2 surface map yesterday and carefully inspected the maps for misaligned regions etc. it became obvious that the LRO-WAC surface data are preferable for Celestia. While both surface maps are very valuable in general, they do have different key features, as reviewed recently by Phil Stooke

http://www.planetary.org/blog/article/00003372/

For rendering with Celestia, a perfect matching of surface map and normalmap at high resolution has high priority, though. Therefore my preference.

Let me just illustrate a couple of issues.

  • Misalignment of Chang'e 2 versus LRO-WAC:
    Here is a overview shot over the remarkable Mare Orientale. In this smaller zoom view, both surface textures do well when combined with the LRO-LOLA normalmap. Here is JohnVV's LRO-WAC texture, as an example:

    [Click twice (!!) on image by all means!]
    Image

    But now let me zoom in and look at the alignment of individual craters:

    Chang'e 2
    ========
    Image

    LRO-WAC
    ========
    Image

    While I kept the same size for both images, one clearly observes a considerable disturbing misalignment in case of the Chang'e 2 surface texture! The LRO-WAC image beautifully cooperates with the LRO-LOLA normalmap. Note that the apparently inferior surface quality of the Chang'e 2 image is mainly due to the lower resolution used here (32k for Chang'e 2 compared to 64k for LRO-WAC).
  • Another issue are remaining stripes in Chang'e 2 (dark mare regions!).
    ==================================================
    Image

    While these stripes may be eliminated in principle by combining a low-pass and a high-pass filter, it's not so straightforward for >= 32k texture sizes. Here ISIS3 .cub files exceed the hard 2GB size limit and the powerful destriping routines of ISIS3 cease to work.



So, altogether: despite spending already a considerable amount of time on this project, I decided to stop further work on the Chang'e 2 surface texture for the time being. Sigh...

If anyone is interested to do some further experiments with the 16k or 32k Chang'e 2 textures, let me know. I am happy to offer them for download, if there is demand.

Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 16-02-12, 19:35 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
EDIT
Quote:
China's space agency has released an amazingly detailed map of the entire lunar surface this week. The new moon map is made up of many high-resolution photos snapped by China's second lunar probe — the Chang'e 2 orbiter —and stitched together into a complete equirectangular map.


Sorry, I missed that. But where did you see that? I couldn't find any details./EDIT

I just started looking into this and couldn't find any details of how the change2 map was created. Main thing is what projection they used. The LRO map may be using a different projection?

ISIS3 can redo do the projection with map to map. I think there is also a way to remap the change2 map using the lro shape model in ISIS3. This may correct the misalignment problems.

It's been years since I used these programs or had a computer to use them. Still stuck with a laptop using Window 7. So.........

Anyhow someday soon I hope to start using Linux again.

cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Thu, 16-02-12, 22:44 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
Hi Steve,

nice to read you again after a while!

cartrite wrote:
EDIT
Quote:
China's space agency has released an amazingly detailed map of the entire lunar surface this week. The new moon map is made up of many high-resolution photos snapped by China's second lunar probe — the Chang'e 2 orbiter —and stitched together into a complete equirectangular map.


Sorry, I missed that. But where did you see that? I couldn't find any details./EDIT


Well, here is one source from which I borrowed some of my introductory sentences.
http://www.space.com/14536-china-moon-m ... mages.html

And sure enough you need to be fluent in Chinese to exploit this site
http://159.226.88.30:8080/CE2release/cesMain.jsp
which has a very handy browsing feature...and many details I cannot understand ;-)

But if you google with chang'e 2 moon you find a LOT of info about Chang'e 2. Notably also the report that Chang'e 2 has reached the Sun-Earth L2 Lagrange point in fall of 2011. Hungry4Info first displayed the link to the Change'e 2 data archive at shatters, etc.

Then of course, there is Phil Stookes comparison of Chang'e 2 with LRO-WAC
(see my discussion above this post)

Quote:
I just started looking into this and couldn't find any details of how the change2 map was created. Main thing is what projection they used. The LRO map may be using a different projection?

I wouldn't have claimed it to be in equirectangular projection in this thread without authoritative confirmation ;-) . The huge original .jp2 texture Global-50m-sc.jp2 comes with an accompanying .xml descriptive file. It is read out with gdalinfo as you will remember:

> gdalinfo Global-50m-sc.jp2

Code:
Driver: JP2KAK/JPEG-2000 (based on Kakadu)
Files: Global-50m-sc.jp2
       Global-50m-sc.jp2.aux.xml
Size is 218527, 109166
Coordinate System is:
PROJCS["sc",
    GEOGCS["GCS_Moon_2000",
        DATUM["Moon_2000",
            SPHEROID["Moon_2000_IAU_IAG",1737400.0,0.0]],
        PRIMEM["Reference_Meridian",0.0],
        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Equirectangular"],                 <=====================
    PARAMETER["False_Easting",0.0],
    PARAMETER["False_Northing",0.0],
    PARAMETER["Central_Meridian",0.0],
    UNIT["Meter",1.0]]
Origin = (-5461989.407011999748647,2729076.540000000037253)
Pixel Size = (50.000000000000000,-50.000000000000000)
Corner Coordinates:
Upper Left  (-5461989.407, 2729076.540) (179d52'30.49"E, 89d59'57.03"N)
Lower Left  (-5461989.407,-2729223.460) (179d52'30.49"E, 90d 0'14.47"S)
Upper Right ( 5464360.593, 2729076.540) (179d47'48.98"W, 89d59'57.03"N)
Lower Right ( 5464360.593,-2729223.460) (179d47'48.98"W, 90d 0'14.47"S)
Center      (    1185.593,     -73.460) (  0d 2'20.75"E,  0d 0'8.72"S)
Band 1 Block=2048x128 Type=Byte, ColorInterp=Gray
  Overviews: 109264x54583, 54632x27292, 27316x13646, 13658x6823, 6829x3412, 3415x1706, 1708x853, 854x427, 427x214, 214x107


In the output I marked the specified projection with a long arrow: <==============

Quote:
ISIS3 can redo do the projection with map to map. I think there is also a way to remap the change2 map using the lro shape model in ISIS3. This may correct the misalignment problems.

Not required, see previous paragraph.

Quote:
It's been years since I used these programs or had a computer to use them. Still stuck with a laptop using Window 7. So.........

Anyhow someday soon I hope to start using Linux again.

cartrite

I seriously hope we'll be lucky seeing more of you here, these days...

Cheers,
Fridger


Last edited by t00fri on Fri, 17-02-12, 22:35 GMT, edited 2 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 10:24 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Hey Fridger,
Sorry I haven't been active lately. I'm either too busy or when I got the time, I don't have the equipment to really get back in the game.

I never downloaded this file but I suspected there would be an xml file with some of the details. I overlooked that gdal output you posted. Sorry about that.

Anyhow, I hope I didn't overlook this too. Did you know why this file is not a power of 2? Is there any duplication of data columns? ISIS may be a better way of reducing the file. That way it could crop out any duplicate data before resizing. That is if there is any duplicate data. It seems an odd size.

cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 11:37 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
cartrite wrote:
Hey Fridger,
Sorry I haven't been active lately. I'm either too busy or when I got the time, I don't have the equipment to really get back in the game.

I never downloaded this file but I suspected there would be an xml file with some of the details. I overlooked that gdal output you posted. Sorry about that.

Anyhow, I hope I didn't overlook this too. Did you know why this file is not a power of 2? Is there any duplication of data columns? ISIS may be a better way of reducing the file. That way it could crop out any duplicate data before resizing. That is if there is any duplicate data. It seems an odd size.

cartrite


Indeed, the Chang'e 2 file size is a bit of a mystery. Apart from NOT being a power of two, the texture has a strange aspect ratio as well. This could well be a reason for the partial mismatch with the LRO-LOLA normalmap. Unfortunately, most of the info on the Chang'e web site is in Chinese...

Isis3 has its limits for really big size textures: 2GB for a RGB cub file is not much in this game. I was trying to offer a colorized (RGB) texture using the UVVIS-500m, 5 band images as color template. Expanding from a grayscale .cub to a rgb .cub already exceeds 2GB for a 32k texture size.


Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 11:52 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Here is something I seen at the ISIS support forum.

Quote:
Hi Brian,

The maximum file size in ISIS2 is 2GB.

In ISIS3 there is certainly some limit imposed by the operating ofr file system. I know of people who have created over 120GB images in ISISI3.

Jeff


https://isis.astrogeology.usgs.gov/Isis ... ml#msg8027

I still think I remember there being a limitation on file size when converting a .cub file to std (isis2std). But a work around for that would be to cut up the main .cub file and convert them in sections. Still there is much I forgotten. I haven't worked with ISIS for some time now.

EDIT If I remember correctly, :wink: I don't think there is a file size limit on the isis2raw program. That way you could use your tools to convert to png. /EDIT

cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 12:56 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
t00fri wrote:
While these stripes may be eliminated in principle by combining a low-pass and a high-pass filter, it's not so straightforward for >= 32k texture sizes. Here ISIS3 .cub files exceed the hard 2GB size limit and the powerful destriping routines of ISIS3 cease to work.


I'm not sure why this would happen. But I did see in the ISIS support forum that file sizes can be set in ./ISIS/ISISPREFRENCES or something to that effect. Maybe this is why?

I can remember doing a lot of HiRISE images in color which were well over 2 GB and I can't remember having any problems running hipass or lowpass. I believe I did work on them 1 band at a time though? Not sure now.

But this is in the ISIS docs.
Quote:
Please note: there is currently a 2GB filesize limit on the output image. This limit does not apply to output JPEG2000 files. JPEG2000 files can be output as 8-bit, signed 16-bit, or unsigned 16-bit files.


cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 16:17 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
Steve,

cartrite wrote:
t00fri wrote:
While these stripes may be eliminated in principle by combining a low-pass and a high-pass filter, it's not so straightforward for >= 32k texture sizes. Here ISIS3 .cub files exceed the hard 2GB size limit and the powerful destriping routines of ISIS3 cease to work.


I'm not sure why this would happen. But I did see in the ISIS support forum that file sizes can be set in ./ISIS/ISISPREFRENCES or something to that effect. Maybe this is why?

Indeed, now that you reminded me of it: I read that the 2GB value may be modified.
Thanks, I'll investigate...

Incidentally, my last IRIS3 activities are also from quite a while ago. Do you happen to remember, how to make certain stretch adjustments (brightness, contrast) in qview permanent such that they may be exportet?
Quote:
I can remember doing a lot of HiRISE images in color which were well over 2 GB and I can't remember having any problems running hipass or lowpass. I believe I did work on them 1 band at a time though? Not sure now.

But this is in the ISIS docs.
Quote:
Please note: there is currently a 2GB filesize limit on the output image. This limit does not apply to output JPEG2000 files. JPEG2000 files can be output as 8-bit, signed 16-bit, or unsigned 16-bit files.


cartrite


When I got an error from working with too large a texture, the wording sounded as if there was a hard limit, since some library (forgot which) was unable to handle 6GB sized files...

Fridger


Last edited by t00fri on Fri, 17-02-12, 22:36 GMT, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 16:49 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
t00fri wrote:
Incidentally, my last IRIS3 activities are also from quite a while ago. Do you happen to remember, how to make certain stretch adjustments (brightness, contrast) in qview permanent such that they may be exportet?

Wow, I think can remember making something permanent but I don't think it was stretch values. I think I was able to change special pixels, null, his, lis, etc. to a value and then there was an option to save the changes.

I think there was a way of viewing the image and manually setting the stretch values but these were not saved. This would give you an idea of the results. But I just noted the final values and used the values manually in isis2std.

I can't remember if I ever did anything bigger than 6gb. Not sure. That's close though. Could be why I never had a problem? But I am pretty sure I didn't work with more than a band at a time. Multi band images were combined as a last step.

I think I also remember reading something at the ISIS forum that some qt libs want to load the entire image into ram during some program runs. So it may depend on how much ram your computer has.

A lot of the work I did on larger files was via scripts. This may be the reason I never had a problem. I think the qt libs are called when using the gui's? but not command line scripts?

cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 17-02-12, 23:31 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
cartrite wrote:
t00fri wrote:
While these stripes may be eliminated in principle by combining a low-pass and a high-pass filter, it's not so straightforward for >= 32k texture sizes. Here ISIS3 .cub files exceed the hard 2GB size limit and the powerful destriping routines of ISIS3 cease to work.


I'm not sure why this would happen. But I did see in the ISIS support forum that file sizes can be set in ./ISIS/ISISPREFRENCES or something to that effect. Maybe this is why?

I can remember doing a lot of HiRISE images in color which were well over 2 GB and I can't remember having any problems running hipass or lowpass. I believe I did work on them 1 band at a time though? Not sure now.

But this is in the ISIS docs.
Quote:
Please note: there is currently a 2GB filesize limit on the output image. This limit does not apply to output JPEG2000 files. JPEG2000 files can be output as 8-bit, signed 16-bit, or unsigned 16-bit files.


cartrite


Steve,

I had a look for the config file IsisPreferences. There is nothing in $HOME. But in /opt/isis3/isis. Actually the cube size was set (default) to 12 GB! But actually, my error message referred to a Qt lib that was not able to handle more than 2GB.

Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 18-02-12, 1:06 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Quote:
A lot of the work I did on larger files was via scripts. This may be the reason I never had a problem. I think the qt libs are called when using the gui's? but not command line scripts?

Maybe?
cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 18-02-12, 11:09 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
cartrite wrote:
Quote:
A lot of the work I did on larger files was via scripts. This may be the reason I never had a problem. I think the qt libs are called when using the gui's? but not command line scripts?

Maybe?
cartrite


Steve,

Unfortunately this suspicion of yours is not right. At least not for my isis3 version 3.2.0.

Meanwhile I reproduced the error in isis2std, trying to convert a 64k grayscale .cub (~8.5 GB!) to .png with the command line:

> isis2std from=moon_64k.cub to=test.png

**USER ERROR** Cube exceeds max size of 2GB. Qimage cannot support that much raw data. Your cube is 2.0 GB.

The same error I get when I try to use the isis2std GUI.
Note that my cube size is 8.5 GB NOT 2 GB...

Given this fact, I looked at the isis3 forum where the same problem was discussed. Here is the authoritative answer by isis support team member ssides

ssides wrote:
ISIS is using the Qt interface for the output of standard file formats (png, jpeg, tiff, ...). Qt is limited to 2GB files. With a 9GB input, it is very possible that no amount of compression will allow you to save the file in less than 2GB. We are planning to find a better answer than using Qt for this, but it is not on the schedule yet.


One might think there was an easy workaround in this case: gdal_translate
BUT under Linux (at least)

> gdal_translate -of png moon_64k.cub moon_64k.png

GDALOpen failed - 4
`moon_64k.cub' not recognised as a supported file format.


This contradicts the statement from

> gdal_translate --formats

inquiry that the isis3 .cub format is supported in read-only mode (not in creation mode, though!). It might be that Windows works, since the GDAL version there is a bit more recent.

Sigh...

Sure enough, there is "light at the end of the tunnel" ;-)
Unlike isis2std, isis2raw works (being not based on a Qt lib)
This matches also your earlier suspicion:

> isis2raw from=moon_64k.cub to=test.bin bittype=8bit

Since my F-TexTools also work for "arbitrarily" large files, one uses my
bin2png command subsequently, to achieve the PNG conversion for monster-sized ISIS3 cube files.

Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 18-02-12, 14:07 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
cartrite wrote:
EDIT If I remember correctly, :wink: I don't think there is a file size limit on the isis2raw program. That way you could use your tools to convert to png. /EDIT

cartrite


I'm not senile after all :D

But still, this doesn't explain why the filters wouldn't work on a large cub file :? Maybe a bug?

cartrite


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 18-02-12, 14:23 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4576
Location: Hamburg, Germany
cartrite wrote:
I'm not senile after all :D
cartrite


Right, as I wrote in my previous post

t00fri wrote:
Unlike isis2std, isis2raw works (being not based on a Qt lib)
This matches also your earlier suspicion


Fridger ;-)


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 18-02-12, 14:41 GMT 
Offline
User avatar

Joined: Thu, 25-10-07, 15:20 GMT
Posts: 992
Location: NE PA, USA
Concerning gdal, I used to build my own from source using a bigTiff lib. I do remember having problems using gdal with big files and that is what I used as a work around. Not sure if your case is the same though.

cartrite


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group