Good morning,

I'm new here so I'd like to say hi first. We're Parasol Island an animation and vfx studio in Düsseldorf Germany which is why and how wo found interest in Nasa's Blue Marble data and photographs. Of interest for us is the fact that the data is in high res, freely available and that everything aligns nicely.
To process the binaries from Nasa we initially found an old version of Fridgers tools which didn't work at all so I contacted him which was when he informed me about the existence of this forum, about the download link for the latest versions of his software and about his nmtools tutorial.

After being able to actually run all the tools on both of our platforms (we use fast Macs and PCs) and understanding the piped workflow (we're using node based compositing software which follows the same principle ) we're now stuck with always the same problem. No matter (seemingly) which command from which tool we run the result is always a 0kb sized, correctly positioned and named output file and a "commandname: command not found" in the terminal or command prompt acompanying it.

I searched this forum for this error but failed to imediatly find a comparable thread. One thread was similar but was due to older compiles which thanks to a recompile by Fridger is now fixed.

Any ideas would be greatly appreciated.

Wilfried

(Edit: Maybe to complete this post, our "larger goal" is to extract the data in the different bins as high-res, high-bit depth sRGB images.)

Hello Wilfried,

welcome at CelestialMatters!

In order to have a chance of localizing the reason for your 0byte output, I urgently need the precise command line and the link to your input data that you used.

Despite many users of the Tools, it's the first time I hear about your kind of problem.

In any case I stongly recommend practising the commands with a relatively small BMNG test data set that you can download here:

http://forum.celestialmatters.org/viewt ... c&start=11

Please study also the examples given there! Notably, the fact that for 16 bit input you must take the correct endedness settings into account. The 16 bit BMNG data are stored in 'big endian' format. Hence on 'little endian' PCs and Intel MACs you got to add the option byteswap=1 as also explained in the tutorial.

Good luck,
Fridger

_________________

Hi,
I solved the issues which were all related to rights management.
Running as an admin everything worked flawlessly.
In the end it was actually very simple, I used nms.exe on a little endian pc workstation in combination with the command
"nms 6378.140 86400 2.5 1 < srtm_ramp2.world.86400x43200.bin > nm86k_wK_byteswap.ppm".

Thanks so much for your support and for the app Fridger!

Wilfried



Edit: Actually I'd still like to achive a depth (greyscale) map in combination to the normals map the command I posted above created.... researching this now.

Wilfried,

I am very happy to see your progress!

Fridger

Another nice to have would be 32bit color depth in the normals but I guess know I'm getting too greedy

RGB normalmaps or also surface textures have: 3 * 8 bit = 24 bit, i.e. 8bit per color.

RGBA images have 4 * 8bit = 32 bit.
A= alpha channel.

Fridger

the 16 bit signed raw image you started with IS a Height map
the pixel values ARE in meters( well 0.5 M) above/ below sea level


there are many tools that can convert the SINGED 16 bit raw image into a unsigned 16 bit tiff or a 32 bit float tiff

for images UNDER 4 gig as a 32 bit float
-- that is a file size max for a program i like , it is still a work in progress
G'Mic
http://gmic.sourceforge.net/
only the terminal version works with 16 and 32 bit images

it is a 32 bit float command line image tool
like ImageMagic but using 32 bit float to work with images

or

one tool developed FOR Geo-spatial data
GDAL
http://www.gdal.org/
GDAL is used a lot in this forum

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6

Thanks for the answers ;D
Its a pitty that non of Fridgers tools can "free" the hight map data into a greyscale image out of that Nasa Bin.
Does any of you happen to know any links to a specific guide or tutorial to do just that using GMic? (Its time consuming to learn yet another command line tool)
At the studio we're looking into ways to create a DEM out of the NMS created normalmap but Im not sure this will be successful because normals tell only about angle to camera, not distance and our lead compositing td told me yesterday to have more than just 8 bit per channel would be very advantageous to have. I told him that on something that is as unspecific as an organic landscape we might be able to get away with this data but when I open the normals png in Photoshop and look at the 8bit the ranges are also very compressed, so even the 8 bit per channel seem to be poorly used (that is with the normals calculated at Fridgers suggested 2.5 value, not the 20 he used in his tutorial)

Wilfried

what ?
that *.bin file IS a raw 16 bit singed integer HEIGHT map DEM , it is only in a RAW format .


Photoshop should be able to use it ????
but PS can also import a raw image

but
for LARGE images - i work with some very big ones
-- my Venus map is 12 GIG --

I use Nip2 ( dose NOT yet support RAW images)
http://www.vips.ecs.soton.ac.uk/index.php?title=VIPS
it is a gui to the VIPS image lib
( this program IS explicitly designed to work with images that can be larger than 24 or 50 or 100 GIG in size
and on multi-layer images

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6

that is working BACKWARDS
and will get you VERY CRAPPY bad image .
( see image at bottom )

turning a 24 bitRGB TANGENT ( it is a Tangent Normal map) into a 8bit GRAY Height map
FROM the srtm_ramp2 HEIGHT map


srtm_ramp2.world.86400x43200.bin
that IS A DEM !
nms converts it into a TANGENT Normal map

now you want to change the normal map BACK INTO A HEIGHT MAP DEM ????

WHY ???
use the original DEM !!!
srtm_ramp2.world.86400x43200.bin


but if you want to
?? why ??? i do not know ???
there is an OLD tool for windows called
njob.exe
http://charles.hollemeersch.net/njob
-- exaamples --
http://www.game-artist.net/forums/gener ... -tool.html
but i GUARANTY you will DISLIKE the output
-------------
srtm_ramp2.world.86400x43200.bin to a normal
srtm_ramp2.Normal.png to a srtm_ramp2(8bit-height).bmp

use the original

here is an example that "srtm_ramp2.world.86400x43200.bin "
resized down to 4096x2048
from left to right to below
the 16 bit DEM
The NormalMap ( made using NMS )
then the 8 bit bmp Normal to height ( made with njob)

so USE the Original DEM

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6

I think it would be great if Wilfried told us in more detail what exactly he and his colloeagues are planning to do? Sounds kind of confusing so far...

Fridger

_________________

Good morning guys,

we're an animation studio as I told you in my little intro so we'd like to use DEMs to displace flat geometry to result in a detailed Earth surface in 3D that we can use for Animations. Instead of modelling the whole planet which would take 13 years or so

Im also very aware that getting a DEM from a Normals map is sluggish at best and working backwards but my starting point of this whole topic (I never worked with a bin before) was fridgers tools and they are for normals so thats how I got them

Im gona check out some of your links which I appreciate you posting.

Wilfried

the 7 gig image "srtm_ramp2.world.86400x43200.bin"
is a height map

At 7 gig you can not use gmic
but GDAL is needed

http://www.gdal.org/
the Windows binarys are listed here
http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries

make a TEXT file ( notepad.exe or BETTER YET Scite - google it )
save it with this name
srtm_ramp2.world.86400x43200.hdr

and put this text into it


then with the bin and hdr file in the same folder
run this line of code
the name of the header "hdr" file MUST match the raw "bin" file)


that will give you a 14 GIG 32 bit tiff image
with pixel values from -415 to 7356

the lowest point on earth to the highest point
at a 1 M elevation scale

for a single image that is 14 Gig in size you will want to cut it up into usable tiles
i use Nip2 for BIG images
http://www.vips.ecs.soton.ac.uk/index.php?title=VIPS


blender has a plugin to read into the program the MOON topo data
with a bit of editing of the earth topo data , it can be imported into Blender

but at a very low resolution
a 3d mesh of the height data is going to be very BIG
http://blenderartists.org/forum/showthr ... 1-06-25%29

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE Leap 15 & CentOS 7.6

Photoshop cant accuratly use the RAW format though. Nasa even admits that on the blue marble site for the bin: "Although it is possible to import the file into Photshop CS and later, Photoshop will not read the signed data correctly: elevation values above sea level start at 0, while negative elevations "wrap" to values near white (65535), which is sea-level."

This was the first thing I tried but the result is indeed very messy.

Wilfried,

as I emphasized already in our early PM conversations, grayscale DEMs represent the INPUT data for my tools. It seems to me that your task is rather a mere format conversion of the published 16bit DEM data file (BMNG) into a grayscale TIFF or PNG, for example. My NM tools could be helpful to handle e.g. size reductions at the full 16bit DEM level.

For such format conversion tasks of 16bit 'monster' files I usually employ the ISIS 3 tools from USGS astrogeology (=> raw2isis | isis2std) that are however NOT ported to Win 7. They all have a nice Qt GUI and --like my tools-- have no fixed size limitation for multi-GB 'monster' files:
https://isis.astrogeology.usgs.gov/Isis ... /index.php
http://isis.astrogeology.usgs.gov/Appli ... index.html

The purpose of my NM tools is really to generate highest quality normalmaps from DEMs, which is not what you are after. I fully agree with John that you should NOT try to proceed via an intermediate normalmap. The latter involves numerical gradients and hence is a sort of "sensitive" object numerically.

Fridger