It is currently Mon, 24-07-17, 8:34 GMT

All times are UTC




Post new topic Reply to topic  [ 33 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject:
PostPosted: Wed, 10-09-08, 20:04 GMT 
Offline
Site Admin
User avatar

Joined: Thu, 30-08-07, 22:52 GMT
Posts: 2726
Location: France, South, not far from Montpellier
Héhé... 287 millions unique objetcs...


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 10-09-08, 21:34 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
ElChristou wrote:
Héhé... 287 millions unique objetcs...


Certainly, and O(10 TB) storage on the servers!

Here is another handy SQL query script that I used yesterday:

Code:
SELECT s.z as redshift, p.ra/15.0, p.dec, p.modelMag_r, p.isoA_g, p.isoB_g, p.isoPhi_g, p.type   
FROM SpecObj s, PhotoObj p, plateX w
WHERE u between 0 and 20.0
      and p.ObjID=s.bestObjID and w.plateID=s.plateID
      and p.ra between 0 and 45.0
      and s.z between 0.0001 and 1.0
      and s.zConf > 0.95
      and s.specClass = 2


Here ra is directly converted from the default decimal degrees to decimal hours. Moreover, a number of useful galaxy parameters are returned in this query besides z,ra,dec from my first example:

  • p.modelMag_r, a corrected best fit red filtered magnitude, which is handy for reducing the huge sample of >330 000 resulting galaxies.
  • p.isoA_g, p.isoB_g are the bet fit major/minor galaxy axes in pixels (green filtered)
  • p.isoPhi_g the position angle in the skyplane (green filtered)


Again one usually has to split a query over the full range of ra , 0<=ra<= 24h into several queries for smaller ra windows.

Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 10-09-08, 21:54 GMT 
Offline
Site Admin
User avatar

Joined: Thu, 30-08-07, 22:52 GMT
Posts: 2726
Location: France, South, not far from Montpellier
Imagine we could enter a query from Celestia, wait a minute for the download and see the result in 3D! Would be great, no?


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 10-09-08, 22:07 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
ElChristou wrote:
Imagine we could enter a query from Celestia, wait a minute for the download and see the result in 3D! Would be great, no?


Certain simple downloads are certainly possible in that manner. Some examples:

  • Satellite position updates ( need to be VERY fresh for good accuracy)
  • Weather clouds ...
  • DSO's as "dots" or simple extended sprites


Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 10-09-08, 22:51 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
Find below a very neat Cosmology Tutorial by Prof Ned Wright/UCLA
That's him (PhD@Harvard)
http://www.astro.ucla.edu/~wright/intro.html


Ned Wright's Home Page & Tutorials
Ned Wright's Cosmology Tutorial
Ned Wright's online Cosmology Calculator

There is even a French and an Italian version of his well-known tutorial!!. You also find on his homepage slides of nice public talks...

That could be fun. Let me know, so I can normalize my searches correctily!

Here is a compact summary on the Universe and BigBang Theory from NASA:
http://map.gsfc.nasa.gov/universe/

Enjoy,
Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 10-09-08, 22:58 GMT 
Offline
User avatar

Joined: Sat, 13-10-07, 18:18 GMT
Posts: 373
t00fri wrote:


Wow...
Thanks very much here Doctor. Will be gone for a couple of days while I explore these thoroughly. :wink:
Great stuff here. Thanks again.

EDIT:
Sorry, my enthusiasm was short-lived. If you compare the site I suggested with the sites listed here,
you'll see the problem. I simply cannot fathom what is trying to be explained to me on the sites listed
here. Apparently, it is in a language which I simply cannot follow. This language is "GobbledyGook."
The reason I liked the site I suggested earlier is because it is in plain English. Sorry, but I did try.

Another EDIT:
Please understand here, that this is just MY brain on drugs. The reference here may well be of use to
others here, but I'm afraid that I simply do not understand it. Remember too, that I have absolutely no
training in mathematics and this site relies heavily on math for its explanations. What I need are plain old
English explanations. :wink:


Top
 Profile  
 
 Post subject: re
PostPosted: Tue, 13-10-09, 6:14 GMT 
Offline
User avatar

Joined: Tue, 04-09-07, 21:55 GMT
Posts: 757
Location: N 42.38846 W 83.45456
don't be to hard on your self bob .
not all of us were taking a lecture series ( u of m - Jim Loudon's " Astro fest " ) when they were 13 .

My dad , the Valedictorian, still has a very hard time with Quantum mechanics
and some Cosmology. Even though he took me to the lectures and the astronomy club meetings when i was a kid.
--------------- added to -------------
Quote:
Unfortunately after a lot of work, a really STUPID conversion error of parsec to light years

so i am not the only one who dose this . I have a very bad habit of it .One of those " I should of had a "V8" " moments .
-- added ---
-- well i do like the new firefox 3.5.3 .I just had a major kernel crash and after reboot what i typed is still here . Cool .

_________________
"I don't pitch Linux to my friends, I let Microsoft do that for me."
Using OpenSUSE 42.1 & Scientific Linux 6.7


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 26-10-09, 11:19 GMT 
Offline
User avatar

Joined: Fri, 03-04-09, 8:21 GMT
Posts: 211
Fridger,

I'm personally not familiar with this schema, however there appears to be a discrepancy between the numbers in your dataset derived from PhotoObj, and the Galaxy VIEW as defined by SkyServer.

Have a look at the following...

This is essentially identical to your original SQL on page 1...
Code:
SELECT count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
WHERE u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2

and today returns 401955.

Then following SkyServer's optimization advice ("if all you care about are galaxies, use the Galaxy view in your FROM clause, instead of PhotoObj"), I ran the following query....
Code:
SELECT count(1)
FROM galaxy p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
WHERE u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2

This returns only 400598.

This means that some 1357 objects in your PhotoObj dataset have been excluded from the Galaxy VIEW.
The following query will identify those objects...
Code:
SELECT p.ObjID -- count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
left outer join galaxy g on p.ObjID = g.ObjID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2
and    g.ObjID is NULL


I haven't done enough analysis to identify the exact distinguishing features of these excluded objects.
I have however noticed that there are other object types other than galaxies in your PhotoObj dataset, but only 353 of the 1357 excluded objects are NOT galaxies:
Code:
SELECT count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
left outer join galaxy g on p.ObjID = g.ObjID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2
and    g.ObjID is NULL
and s.objType != 0


and those 353 have the following distinct s.objTypeName's:
Quote:
QSO
REDDEN_STD
ROSAT_D
SERENDIPITY_BLUE
SERENDIPITY_DISTANT
STAR_BHB
STAR_WHITE_DWARF
which means that there's another 1004 objects with s.objTypeName = "GALAXY" which are also excluded for some reason from the Galaxy VIEW.

You are more familiar with galaxies than I am :wink:, so I leave it as an exercise for you to determine the exact nature of the remaining 1004 excluded galaxies, and therefore why they've been excluded from SkyServer's galaxy VIEW.

CC


Top
 Profile  
 
 Post subject:
PostPosted: Tue, 27-10-09, 21:00 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
CC,

thanks for your observation! It's quite a while ago that I did these queries. So please give me a bit of time to investigate. In my above investigation I wasn't all that keen to discern the nature of below ~ 1000 objects out of > 400000 in my particular investigation above.
It really didn't matter all that much.

Still, it's good to understand what's going on here...

Fridger


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 28-10-09, 8:16 GMT 
Offline
User avatar

Joined: Fri, 03-04-09, 8:21 GMT
Posts: 211
You're welcome,

As you say it's always good to be sure of your data.
I've done some more analysis on your behalf, with the following conclusions.
In this case I believe that your dataset includes a small number of points (375) which are actually STARS, and also excludes some 8849 galaxies.

The problem appears to arise from the use of specClass to identify galaxies.
I don't know what your reasoning for that approach is, but while it does appear to be a reasonable approximation, it appears that there are a small number of stars which also have specClass = 2):
Code:
-- Some Stars have specClass = 2
--
SELECT distinct t.name, count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
join PhotoType t on p.type = t.value
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2
and p.type != 3
group by t.name

t.Name,COUNT
STAR,375



Possibly of more concern is the exclusion of 8849 galaxies because they have specClasses other than 2:
Code:
-- Some GALAXIES are not specClass 2
--
SELECT distinct s.specClass, count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass != 2
and p.type = 3
group by s.specClass

specClass,COUNT
1,811
3,7948
6,90



All of this assumes of course that PhotoObj.Type is the correct attribute to identify a particular object's type. If that is the case, then I believe the query to get the data you want is:
Code:
SELECT count(1)
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
and p.type = 3



As you can see, this get's all galaxies (p.type = 3) regardless of specClass, and excludes those pesky stars.

This should return 410429 galaxies.
(401955 - 375 + 8849 = 410429)

Although the differences only represent about 2% of the full data, it would be interesting to see if this results in any 'significant' differences in the appearance of your plots.
(What if galaxies of a particular *excluded* spectral class were primarily located in the filamentary regions for example Wink)

CC

PS. There's still one discrepancy that I haven't been able to explain as yet....which is the fact that the SDSS's galaxy VIEW still returns 995 less objects than PhotoObj:
Code:
SELECT count(1)
FROM galaxy p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN plateX w ON w.plateID=s.plateID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
and p.type = 3



This returns 409434 ( cf. 410429 from PhotoObj )


Top
 Profile  
 
 Post subject:
PostPosted: Wed, 28-10-09, 21:16 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
CC,

Quote:
You're welcome,

As you say it's always good to be sure of your data.
I've done some more analysis on your behalf, with the following conclusions.
In this case I believe that your dataset includes a small number of points (375) which are actually STARS, and also excludes some 8849 galaxies.


It is a well known issue in the context of rather small galaxies that in any galaxy catalog there is some "noise" from stars. Happens e.g. also in the NGC/IC catalog. More complicated query criteria can exclude most of them but my above study was rather meant to be quick exploratory shot with a specific aim:.

My study was mostly about (spacial) density questions associated with LARGE numbers of galaxies. Hence I mainly took care that misinterpretations etc could only concern a small proportion of the whole set. But for finalizing such a study it is certainly good to take your more sophisticated considerations into account.

Thanks!
Fridger

PS: But isn't it fun to be "in command" of so many galaxies at one's "fingertips"??


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 30-10-09, 13:21 GMT 
Offline
User avatar

Joined: Fri, 03-04-09, 8:21 GMT
Posts: 211
t00fri wrote:
It is a well known issue in the context of rather small galaxies that in any galaxy catalog there is some "noise" from stars. Happens e.g. also in the NGC/IC catalog. More complicated query criteria can exclude most of them but my above study was rather meant to be quick exploratory shot with a specific aim:.

My study was mostly about (spacial) density questions associated with LARGE numbers of galaxies. Hence I mainly took care that misinterpretations etc could only concern a small proportion of the whole set. But for finalizing such a study it is certainly good to take your more sophisticated considerations into account.

Thanks!
Fridger

PS: But isn't it fun to be "in command" of so many galaxies at one's "fingertips"??

Don't flatter me ... my query's neither more complicated, nor sophisticated...
... just more accurate than your ad-hoc criteria, as I'm making use of the available foreign key in the SDSS database. (This really isn't "rocket science"! :wink: )

Incidentally, you should always choose a foreign-key type relation (if one is available which satisfies your query) in preference to other criteria, ... because Foreign Keys are more likely to have indexes which help the query optimizer, and lead to better query performance. (Of course, this won't always be an option.)

As for this being "fun", perhaps if the database wasn't so slow!! :shock:

In the interests of trying to get some insight into what (if anything) is significant about the galaxies excluded by the SDSS from the galaxy VIEW... I created a couple of Hubble Charts...just for fun!

The first is just a sample from the general population...
Code:
select top 2000 objid,ra,dec,modelmag_g,z
from specphoto

Image

The second is a plot of the 372 (of 995 excluded galaxies) which have measured spectra in the DB ...

Code:
SELECT sp.objid,sp.ra,sp.dec,sp.modelmag_g,sp.z --
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
JOIN specphoto sp on p.ObjID = sp.ObjID
left outer join galaxy g on p.ObjID = g.ObjID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and s.specClass = 2
and    g.ObjID is NULL

Image

Hmmm...interesting. 8)


Top
 Profile  
 
 Post subject:
PostPosted: Fri, 30-10-09, 18:26 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
CC,

thanks for your interesting plots... I definitely have to get back to this. One forground task on my waiting list is to code an explicit correction algorithm for the "Finger of God" effect that is very evident from my above plots....While the physics of it is very simple (i.e. the neglect of peculiar velocities that can be substantial in big galaxy clusters) it is less easy to arrive at a workable AND transparent algorithm that compresses the "Fingers of God" to roughly spherical galaxy clusters.

I suppose you know what all this is about?

Moreover, I must confess that my above queries were my first attempts for reading out galaxies from the SDSS database. So independently of my chronical lack-of-time syndrome, your query is simply superior.

Cheers,
Fridger

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Sat, 31-10-09, 2:21 GMT 
Offline
User avatar

Joined: Fri, 03-04-09, 8:21 GMT
Posts: 211
t00fri wrote:
CC,

thanks for your interesting plots... I definitely have to get back to this. One forground task on my waiting list is to code an explicit correction algorithm for the "Finger of God" effect that is very evident from my above plots....While the physics of it is very simple (i.e. the neglect of peculiar velocities that can be substantial in big galaxy clusters) it is less easy to arrive at a workable AND transparent algorithm that compresses the "Fingers of God" to roughly spherical galaxy clusters.

I suppose you know what all this is about?

Moreover, I must confess that my above queries were my first attempts for reading out galaxies from the SDSS database. So independently of my chronical lack-of-time syndrome, your query is simply superior.

Cheers,
Fridger
Well, although I have an amateur interest in cosmology (I did a couple of undergraduate courses in astronomy and astrophysics, and I've read "A Brief History of Time" :lol: ) I'm certainly no expert.

I assume you're talking about the fact that distance measurements using Hubbles Law of individual galaxies can be skewed by the peculiar velocities imparted by the "local" gravitational affect of the cluster?
That's about all I know about it.

I would think that "The Finger of God" would be a term more likely appropriated by the Intelligent Design proponents to explain the "Cosmological Constant Problem". :wink:

One thing I wondered about your plots and other conformal plots is whether you/they are plotting the "observed" distances without extrapolating for "lookback time" (the time-machine effect :wink:).
ie. As you observe more and more distant objects, you're actually observing them where they were billions of years ago, rather than where they actually are positioned now. (I guess as you look further and further away/back, the observed universe also begins to look smaller and smaller, until you observe the actual big bang itself!! :mrgreen: )

So basically what I'm asking is...
Do these plots show simply the "observed" positions, or are you extrapolating from the observed positions in order to plot their expected present day positions?
(I assume that this is the purpose of your "Comoving distance" calculations, but just checking! :)

CC

PS. Have you tried the Image List Tool?
You can view the imagery of these galaxies by using this query to fill the Image List form:
Code:
select s.specobjid as name, s.ra, s.dec
FROM PhotoObj p
JOIN SpecObj s ON p.ObjID=s.bestObjID
left outer join galaxy g on p.ObjID = g.ObjID
WHERE p.u between 0 and 20.0
      and p.ra between 0 and 360.0
      and s.z  between 0.0001 and  1.0
      and s.zConf > 0.95
      and p.type = 3
and    g.ObjID is NULL

:)

One final thing....

Can you explain why I can't find the Sombrero Galaxy as an object in this database?
See here:
http://cas.sdss.org/astro/en/tools/chart/navi.asp?ra=189.9975&dec=-11.62305556
This is certainly looking in the right place, but it's not recognized as an object in the database!
ie. A SQL query with appropriate RA and DEC constraints will not find it.


... and additionally, you may be interested to know that it appears to be incorrectly inclined in Celestia. Appears to be inclined at something like 45degs to the Solar System, whereas in photos (such as the one in the SDSS database) it appears almost edge on. (I'm assuming that we haven't yet sent cameras far enough away to introduce such a large difference in viewpoint !! :lol: )


Top
 Profile  
 
 Post subject:
PostPosted: Mon, 02-11-09, 21:25 GMT 
Offline
Site Admin
User avatar

Joined: Fri, 31-08-07, 7:01 GMT
Posts: 4466
Location: Hamburg, Germany
CC,

CC wrote:
I assume you're talking about the fact that distance measurements using Hubbles Law of individual galaxies can be skewed by the peculiar velocities imparted by the "local" gravitational affect of the cluster?
That's about all I know about it.


...and it's also basically correct. Since velocities are 3-vectors, it is intuitive that vast misinterpretations of the spacial locations of galaxies arise, if the measured values of the (spectroscopically determined) redshift z are interpreted as arising exclusively from a radial velocity component associated with the Hubble expansion. It is easy to see that this (rough) simplification then leads to a distorted spacial profile of members of galaxy clusters (via Hubble's law) in such a way that instead of the true spherical cluster shape, the clusters appear as strongly elongated galaxy "jets" that always point towards the observer. Hence the "Finger of God" nomenclature. You can easily notice this effect in my SDSS plots above.

In case of interest, there are various very good scientific papers about this point...let me know.

A more popular and short discussion with typical examples is here:
http://web.ipac.caltech.edu/staff/jarre ... /fgod.html

While the effect is easy to understand, the coding of an automatized "finger-of-God" correction algorithm is no so easy. That's one of the tasks that I am tackling in this field from time to time...

As to your question about "lookback time". Well, since the underlying framework here is general relatiivity (GR), one has -- first of all-- to define what is meant with the notion of distance in a GR setting. It so happens that the (measured) redshift z, is most directly related to the co-moving distance that makes sense relativistically, (of course, provided peculiar velocities can be neglected)!

Since GR-based distances include the "lookback" effect naturally, there is no problem, once one calculates co-moving distances in terms of z. Indeed, that's why I chose the co-moving distance. Actually, in order to calculate it, one needs to input one's favorite cosmological model (in terms of the standard parameters) . I wrote a Perl script for this purpose and used the parameters from the latest Wmap data.

No idea why you can't find the Sombrero galaxy in SDSS. It's certainly included in all other relevant catalogs.

As to the incorrect inclination of M104, it's quite unavoidable within my "mass-rendering" framework in Celestia. It's a limitation of my computer-based rendering that -- as a matter of principle-- only uses the given catalog info. M 104 is archived with Hubble type Sa but actually shows a highly non-canonical shape (that's why it's so famous ;-) ) . So the given angles (PositionAngle and inclination) in the scientific NGC/IC catalog are interpreted in a way by my code that appears incorrect due to inherent morphological ambiguities. Of course one day I may well include some special treatment for a few famous galaxies into my code...

Along the same lines, I could also quote to you many galaxies in my galaxy.dsc catalog, where the spiral rotates the wrong way around. Again this is unavoidable, since unfortunately, in NO galaxy catalog there are data about the rotational sense of the spirals. Since the task was to render 10000+ galaxies, individual comparison with photographic imaging would keep me busy for the rest of my life ;-)


Fridger


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 33 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group