It is currently Sun, 19-11-17, 21:47 GMT

All times are UTC




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Displacement mapping
PostPosted: Tue, 03-05-16, 2:24 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
I implemented displacement mapping in celestia.Sci.
While this is a prototype and there are still lots of problems, here are some early results from my experiments:
(Top: Stock Moon with normal mapping. Bottom: Displacement mapped Moon with ray traced shadows)
Attachment:
2004-12-18-before2.png
2004-12-18-before2.png [ 290.17 KiB | Viewed 2567 times ]
Attachment:
2004-12-18b2.png
2004-12-18b2.png [ 276.31 KiB | Viewed 2567 times ]

Detail of Montes Apenninus, slightly brightened in Gimp to match the brightness of the photo reference:
Attachment:
2004-12-18b-detail.png
2004-12-18b-detail.png [ 109.95 KiB | Viewed 2567 times ]

Photo reference (source):
Attachment:
2004-12-18.jpg
2004-12-18.jpg [ 221.01 KiB | Viewed 2567 times ]

Shadowed crater (artifacts on the horizon are a known issue):
Attachment:
crater_shadow2.png
crater_shadow2.png [ 77.03 KiB | Viewed 2567 times ]

Real-time topographic shading:
Attachment:
topo2.png
topo2.png [ 290.75 KiB | Viewed 2567 times ]


Recently I have been using our lab's stereo display wall to view the surface of the Moon in Celestia.
One thing that immediately struck me was, despite using a high-resolution 128 pix/deg LOLA normal map, that the surface terrain looked incredibly flat when viewed in stereoscopic 3D, despite the realistic lighting coming from the normal map.

I realized that there were 3 big problems:
  1. Terrain features lacked parallax, i.e., does not change as it should when the viewpoint changes. I should be able to gradually view the side of a crater rim when rotating around it, but without parallax the crater rim looks pasted in place.
  2. Despite dark areas being rendered nicely by normal mapping, there were no shadows that moved with the Moon phase.
  3. The limb of the Moon is a smooth sphere. But as we know, terrain causes the silhouette of the Moon to look jagged.

These are caused by the fact that we are using normal mapping, which only moves the normals of the surface while the terrain geometry is flat. The solution is to really displace the geometry. In computer graphics this is called displacement mapping.
There are two main displacement mapping techniques: vertex and fragment.
Vertex displacement uses vertex texture fetch (VTF) to read a DEM texture inside the vertex shader, then moving vertices according to texture samples. VTF gained hardware support way back in 2004 so nearly all GPUs have it now. This technique is easy to understand and to implement, but high levels of detail require a large number of polygons. Typically this means a level of detail (LOD) scheme is required to reduce the number of polygons to draw.
I have been playing around with the cdlod algorithm (warning: PDF).
CD LOD is one of the better LOD schemes (others include Celestia/.Sci's own LOD scheme, geo clip/mipmapping, ROAM, and projective grid mapping) because it is relatively simple, explicitly avoids cracking, and smoothly morphs between successive detail levels. However, very high polygon densities are still required at distances close to the observer, and shadows have to be generated via shadow volumes (which require clipping on concave spherical surfaces like planets and penumbra are complicated to generate) or shadow mapping (which suffers from aliasing and high storage requirements).
Celestia and .Sci does not use VTF, but does apply a LOD scheme to planetary spheres. However a very simple scheme is used; the same LOD is applied to the entire sphere. This results in an excessively high polygon density at far distances, and too few polygons close up.
I also posted previously here on CM about projective grid mapping (PGM), which projects a mesh in screen space; this naturally maps less polygons to far distances. However PGM suffers from shimmering (the terrain appears to swim around like a mirage as the observer moves; this is caused by the geometry "swimming" above the height map), and a very high polygon density is still required when looking straight down on the terrain.

This caused me to look at fragment level displacement techniques. These typically use ray tracing to calculate intersections between rays shot from the observer and a height map. One immediate advantage is far fewer polygons than VTF are required; fragment displacement is limited only by screen size (pixel fill rate). In this example from Smits et al., 1997 (PDF), only 20 triangles are stored:
Attachment:
smits97b.jpg
smits97b.jpg [ 13 KiB | Viewed 2567 times ]

Another advantage is that accurate shadows can generated by tracing shadow rays, and the technique can be extended to path tracing for global illumination.
In that paper from 1997, this was before the development of consumer-level GPU shader hardware so the rendering times quoted are on the order of tens of hours per frame using Monte Carlo path tracing. But using GLSL fragment shaders we can do much better. From 2001 to 2005, several important fragment displacement techniques were published starting from Parallax Mapping, Relief Mapping, Parallax Occlusion Mapping (POM), sphere tracing, etc (Szirmay-Kalos and Umenhoffer, 2006 gives a nice overview).
I focused on relief mapping and POM, because the rendering quality is high and complicated and imperfect preprocesing requiring additional time and storage is not required. For example, POM was later extended to generate prisms to extrude surfaces up to enable silhouettes. This is not required if we borrow the curved ray tracing technique used in Relief Mapping (Oliveira and Policarpo, 2005 [PDF]) and first introduced in Kajiya, 1983.
The problem is that all displacement mapping techniques assume a flat base terrain. However, all planetary surfaces have a small, but significant curvature. Instead of dealing with both an arbitrary displaced terrain and a spherical curvature, Kajiya and Oliveira and Policarpo noticed that we could still treat the base terrain as flat, and curve the light rays that we trace instead (Szirmay-Kalos and Umenhoffer, 2006):
Attachment:
curved_tracing.png
curved_tracing.png [ 34.5 KiB | Viewed 2567 times ]


Frame rates range from 30~60 fps using six 1440x1440 height maps forming a cube. This is with no LOD scheme at all. Currently remaining challenges include eliminating thin artifacts just above the horizon (limb), eliminating flattening of terrain at extreme grazing angles, removing discontinuities at texture borders, and improving shadow quality. Many or all of these may be caused by my naively applying a quadratic approximation (as used by Oliveira and Policarpo) to the sphere curvature.


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Tue, 03-05-16, 3:30 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 773
Location: N 42.38846 W 83.45456
Quote:
despite using a high-resolution 128 pix/deg LOLA normal map,


as has been talked about before the lola data suffers a bit from spline interpretation

the 100Meter or 256 WAC stereo imaging height data is a bit better
http://lroc.sese.asu.edu/data/LRO-L-LRO ... AC_GLD100/
( the poles are filed in with the lola data )


a displacement on a mesh is what i use for the Ceres " image of the day" renders
https://goo.gl/photos/iAtFUPmE4QVxpcY29

while these are Shape from shade

i find that i need to ADD a bit of Gaussian noise ( 0.01 % on a 32 bit float heightmap)
this adds just a bit of roughness to the image

as you can see a small amount of noise to the heightmap adds a bit of texture to the mesh
an example from "low altitude mapping orbit " day 77
Image

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Tue, 03-05-16, 6:48 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4508
Location: Hamburg, Germany
dirkpitt wrote:
I implemented displacement mapping in celestia.Sci.


Let me be more precise. Displacement mapping is an option that dirkpitt = DW is currently studying. This project is accompanied by a substantial amount of scepticism from my side. Therefore it remains to be seen whether displacement mapping may become a permanent feature in celestia.Sci.

Fridger

_________________
Image


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Tue, 03-05-16, 21:35 GMT 
Offline

Joined: Tue, 04-09-07, 9:17 GMT
Posts: 15
Location: Regensburg, Germany
Yet it seems to be an interesting option.
The shadows don't look bad.
Scepticism may still be justified, but nevertheless I appreciate reading about new ideas. Thanks, dirkpitt. :)


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Wed, 04-05-16, 4:35 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
@Fridger: Yes, this is only an experiment at the moment but I did implement it starting from .Sci code. It would be nice to see if I can solve most/all of the major remaining issues and arrive at something that everyone can use.

@JVV: Yes, LOLA has big problems especially at the equator. But to this date it's not clear to me that there is a "winning" DEM product. GLD100 is nice, but is low resolution and has its own problems. SLDEM 2013 and 2015 attempt to improve on LOLA, GLD100, and Selene (Kaguya), but end up having problems of their own.


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Wed, 04-05-16, 7:28 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4508
Location: Hamburg, Germany
dirkpitt wrote:
@Fridger: Yes, this is only an experiment at the moment but I did implement it starting from .Sci code. It would be nice to see if I can solve most/all of the major remaining issues and arrive at something that everyone can use.

Which also means that dirkpitt's available time as a .Sci developer, with plenty of more urgent coding tasks ahead, will shrink to zero... This in turn will cause a considerable further delay of the public release of celestia.Sci...Personally, I also love getting involved with hires rendering tasks (e.g. all textures in the official hires folder of the Celestia distribution have been done by me; or think of my fast normal map and texture tools). But this sort of fun is clearly for LATER times, when there is a public .Sci release etc.!

Below I just pulled 2 images over from our non-public dev area, as a reminder of what the displacement map challenge really looks like against the present optimized normal maps ! At this high level of detail and dynamical shading via normal maps the fps rate is still big on current laptops.

[By all means, click on image and then hit your browser's fullscreen key (F11)]
Moon (Schrödinger crater), zoom, normal map ONLY!
Attachment:
Moon_zoom_32knmap_allfixed_5.6.jpg
Moon_zoom_32knmap_allfixed_5.6.jpg [ 222.66 KiB | Viewed 2526 times ]


[By all means, click on image and then hit your browser's fullscreen key (F11)]
Evening scene on Mars with normal map
Attachment:
Mars2_allfixed_5.6.jpg
Mars2_allfixed_5.6.jpg [ 253.61 KiB | Viewed 2526 times ]

_________________
Image


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Fri, 13-05-16, 14:25 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
As a partial answer to your challenge, here are side-by-side comparisons with normal mapping and my most recent displacement mapping code. Note that these are with relatively low-resolution maps (only 1/16 deg/px LOLA DEM was used since virtual textures are not working yet). I think the results are quite nice, and I think, will take a temporary break from this rather fun (and very challenging) experiment.

I also took the opportunity to implement a proper Hapke-Lommel-Seeliger BRDF. The existing lunar lambertian function in Celestia and .Sci has two problems: only Lommel-Seeliger is implemented (limb brightening) and the terminator is not sharp enough. Based on Sato et al. 2014, I implemented a Hapke-Lommel-Seeliger function with a more realistic terminator and the shadow-hiding opposition effect (SHOE), which is the halo effect seen in Apollo pictures like this one:
Attachment:
21634076726_e9aa434f3a_m.jpg
21634076726_e9aa434f3a_m.jpg [ 13.27 KiB | Viewed 2464 times ]


A big remaining task is to implement virtual textures. This unfortunately requires implementing an LOD mesh that maps to the virtual texture. While not difficult in theory, in practice this will require a lot of testing. Also planet silhouettes are still not correct.

Latest results (YouTube video):
Top: normal mapping only, Bottom: displacement mapping with realtime shadows
Attachment:
shadow3-normonly.png
shadow3-normonly.png [ 244.56 KiB | Viewed 2464 times ]
Attachment:
shadow3-fullshadows.png
shadow3-fullshadows.png [ 230.7 KiB | Viewed 2464 times ]


Top: normal mapping only, Bottom: displacement mapping with realtime shadows
Attachment:
south-normonly.png
south-normonly.png [ 336.41 KiB | Viewed 2464 times ]
Attachment:
south-fullshadows.png
south-fullshadows.png [ 306.27 KiB | Viewed 2464 times ]


Lunar South Pole video:
https://www.youtube.com/watch?v=K7YmpT9-3LY
Attachment:
vlcsnap-southpole.jpg
vlcsnap-southpole.jpg [ 26.26 KiB | Viewed 2464 times ]


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Fri, 13-05-16, 17:42 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4508
Location: Hamburg, Germany
With my selfmade 16k Lola normalmap & Chang'e2 texture set I reproduced your Shackleton based scene rather closely: Shackleton is seen as the dark crater centered near the bottom. The sun is low.
[Click on image, followed by fullscreen key (F11 for FF)]
Attachment:
shackleton2.png
shackleton2.png [ 1 MiB | Viewed 2456 times ]


Distance from Moon: ~ 111km
Date/Time: 2008 Nov 10; 20h:23m
Despite my much higher texture resolution, I get fps = 62 for the 1826 x 765 .Sci window.
Why did you skip quoting the fps values for your comparison?

While such type of scene (low distance from Moon surface & low Sun position) is toughest for normal map rendering, I wouldn't want to renounce this sort of rendering qualiy and performance in exchange of displacement mappings.

However, given that KARI plans a moonshot and sending a rover to the moon in a second stage, your focus on moon-related projects is perfectly understandable ...

While your displacement map shot of the Shackleton region may be considered to be encouraging, since shadows are frequent and conspicuous, I must say that the landscape just doesn't look like the Moon. Sorry, but the terrain is far too smooth, compared to the apparent depth of the craters. ==>
Image

For this crater depth, the surface requires a far higher sampling rate I'd say. While DMs do remain in focus at very near distances, according to my understanding one must not work with a constant sampling rate /texture resolution when the distance decreases significantly. In case of normal maps, one usually implements mipmaps for compensation or perhaps even better just VT tiles that automatically embody kind of a mipmap effect.

More soon, when I am less busy.

Fridger

_________________
Image


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Fri, 13-05-16, 22:07 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
Thanks for your comments as always Fridger. As I mentioned, I am using a very low resolution DEM (LDEM_16.IMG, which is only 1/16 deg/px) which is why the terrain looks very smooth.
The lunar south pole region indeed contains a lot of small craters that are missed by this DEM.
With a high-resolution virtual texture, the result should be even more impressive. This requires more coding work however, since I am not using the existing lodspheremesh class. I will leave this as an exercise for the future.


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Fri, 13-05-16, 22:10 GMT 
Offline
User avatar

Joined: Mon, 07-01-08, 13:30 GMT
Posts: 339
Location: Switzerland
Looks like an interesting feature to add, but would also be interested in the performance. I do know that I see a fairly noticeable slowdown in Elite: Dangerous when I get close enough to planets for the terrain mapping to kick in, but I don't know how well optimised their implementation is.


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Fri, 13-05-16, 23:47 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
True, it could be faster. I've hardly done any optimization, so I am seeing ~20 fps on my MacBook Air with Intel HD 3000 graphics from 2011.
There is certainly a lot of room for improvement, e.g., texture streaming, LOD, frustum culling, shader optimization, etc.
But put in other words, it's quite amazing that even with none of these standard optimization techniques I am seeing 20 fps.

Incidentally on another machine with dual GTX 970M in SLI (this is the one I used for the YouTube video), I'm consistently seeing 50~60 fps even at 4k resolution ;-)


Top
 Profile  
 
 Post subject: Re: Displacement mapping
PostPosted: Sat, 14-05-16, 17:36 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 2:32 GMT
Posts: 430
Location: South Korea
I couldn't resist creating an animated gif that demonstrates the Hapke shader.
Notice how as the viewing angle approaches the Sun incidence angle, the lunar surface brightens dramatically (you can also see the shadows getting hidden).
The gif plays only once, so shift-reload this post to view again.

Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group