Difference: Web2p0Demonstrator (1 vs. 5)

Revision 52015-11-02 - NigelHambly

Line: 1 to 1
 
META TOPICPARENT name="DeliverablesGenius"

GENIUS Web2.0 Demonstrator

Line: 23 to 22
  Once the results set from the query above are presented in summary form, you can invoke interactive plotting by pressing the "Plot Results" icon. In this case, selecting mu_alpha and mu_delta in the attribute selection box (again, auto-completion is available for convenience) and selecting "Plot type: Scatter" produces a browsable plot - zoom in to see the group of cluster proper motion members offset to (+20,-40) from the general field population near (0,0) - these are Pleiades proper motion members. N.B. when plotting larger row sets, it is advisable to use "Plot type: Mixed Scatter/Density" or even "Select Mode" -> static for reasonable response times in the present implementation.
Changed:
<
<
For cross-querying, note that crossmatch neighbour tables are present in the ESO VST southern hemisphere optical survey ATLASDR1 DB only, and that these do not include "best" match tables or "mates" fields (as specified in the Gaia DPAC CU9 schema) but otherwise are similar with master and slave unique IDs pointing between any two source catalogues within a predefined neighbourhood radius of 10 arcsec. For example, load in the table metadata for ATLASDR1 and TWOMASS by clicking on "Load database(s)" -> "More" and click on the "+" beside each followed by "OK". Clicking on "Show loaded Metadata" will now display ATLASDR1 and TWOMASS in addition to GACS (click on "Hide metadata" to collapse the view when not required) and expand these up to and including the specific table resource required for querying to load the metadata for auto-complete functionality.
>
>
For cross-querying, note that crossmatch neighbour tables are present in the ESO VST southern hemisphere optical survey ATLASDR1 DB only, and that these do not include "best" match tables or "mates" fields (as specified in the Gaia DPAC CU9 schema) but otherwise are similar with master and slave unique IDs pointing between any two source catalogues within a predefined neighbourhood radius of 10 arcsec. For example, load in the table metadata for ATLASDR1 and SDSS DR9 by clicking on "Load database(s)" -> "More" and click on the "+" beside each followed by "OK". Clicking on "Show loaded Metadata" will now display ATLASDR1 and BESTDR9 in addition to GACS (click on "Hide metadata" to collapse the view when not required) and expand these up to and including the specific table resource required for querying to load the metadata for auto-complete functionality.
  An example crossmatch query is given here that checks the g-band photometric consistency between the VST ATLAS DR1 survey photometry and that in SDSS DR9:
Deleted:
<
<
 
select gAperMag3, psfMag_g
from atlasSource as a, photoprimary as b, atlasSourceXDR9PhotoObj as x
where gAperMag3 > -99 and psfMag_g < 99 and distanceMins < 1.0/60.0
Line: 34 to 32
  In this case the crossmatch is limited at query time to 1 arcsec via the predicate on "distanceMins". Other predicates remove default rows, and provide the relational join between the two source catalogue tables and the crossmatch neighbour table.
Changed:
<
<
Other data holdings presented via the demonstrator include SDSS DR9 ("BESTDR9"), Vista Hemisphere Survey DR1 ("VHSDR1") and the WISE source catalogue (not surprisingly, "WISE"), and all have crossmatch tables present in ATLASDR1 following the naming convention
>
>
Other data holdings presented via the demonstrator include 2MASS ("TWOMASS"), Vista Hemisphere Survey DR1 ("VHSDR1") and the WISE source catalogue (not surprisingly, "WISE"), and all have crossmatch tables present in ATLASDR1 following the naming convention
 
atlasSourceX...

viz.

Changed:
<
<
atlasSourceXDR9PhotoObj
>
>
atlasSourceXtwomass_psc
 atlasSourceXvhsDr1Source atlasSourceXwise_allskysc
Line: 61 to 57
 

Comments


<--/commentPlugin-->
Changed:
<
<
>
>
 
META FILEATTACHMENT attachment="Demo_Documentation.doc" attr="h" comment="Draft documentation for the Web2.0 Demonstrator" date="1428484769" name="Demo_Documentation.doc" path="Demo_Documentation.doc" size="43008" user="NigelHambly" version="1"

Revision 42015-04-08 - NigelHambly

Line: 1 to 1
 
META TOPICPARENT name="DeliverablesGenius"

GENIUS Web2.0 Demonstrator

Line: 61 to 61
 

Comments


<--/commentPlugin-->
\ No newline at end of file
Added:
>
>

META FILEATTACHMENT attachment="Demo_Documentation.doc" attr="h" comment="Draft documentation for the Web2.0 Demonstrator" date="1428484769" name="Demo_Documentation.doc" path="Demo_Documentation.doc" size="43008" user="NigelHambly" version="1"

Revision 32015-04-01 - NigelHambly

Line: 1 to 1
 
META TOPICPARENT name="DeliverablesGenius"

GENIUS Web2.0 Demonstrator

Line: 7 to 7
 CONTEXT

The Web2.0 concept is one of enhanced end-user experience via useful functionality embedded within the browser session. In this case, we have

Changed:
<
<
  • auto-completion when typing freeform ADQL, according to data resources available when loaded as part of the session;
  • embedded plotting/browsing;
  • transparent cross-querying of catalogue resources via underlying Distributed Query Processing (DQP) infrastructure.
>
>
  • auto-completion when typing freeform ADQL, according to data resources available when loaded as part of the session
  • embedded interactive plotting/browsing and filtering through query results
  • transparent cross-querying of catalogue resources via underlying Distributed Query Processing (DQP) infrastructure
  • VOTable/Fits/CSV/HTML download options
 BRIEF INSTRUCTIONS

From the home page linked above, click on "Freeform SQL" in the left-hand navigation bar. This will take you to a window into which you can type ADQL queries in plain text. Be aware that your session has a "context" in that data resources can be selected for auto-completion and archive metadata browsing - by default, the demonstrator loads the ESAC Gaia archive database via the GACS TAP service, and auto-completion options displayed as you type will be restricted to that metadata unless you load further resources into the session (see later). To see the catalogue metadata currently available within your session, simply click on "Show loaded Metadata" and expand the treeview as required.

Line: 19 to 21
  As you type, auto-completion options can be displayed in a pop-up by pressing ctrl-space.
Changed:
<
<
Once the results set from the query above are presented in summary form, you can invoke interactive plotting by pressing the "Plot Results" icon. In this case, selecting mu_alpha and mu_delta in the attribute selection box (again, auto-completion is available for convenience) and selecting "Plot type:" Scatter produces a browsable plot - zoom in to see the group of cluster proper motion members offset to (+20,-40) from the general field population near (0,0) - these are Pleiades proper motion members.
>
>
Once the results set from the query above are presented in summary form, you can invoke interactive plotting by pressing the "Plot Results" icon. In this case, selecting mu_alpha and mu_delta in the attribute selection box (again, auto-completion is available for convenience) and selecting "Plot type: Scatter" produces a browsable plot - zoom in to see the group of cluster proper motion members offset to (+20,-40) from the general field population near (0,0) - these are Pleiades proper motion members. N.B. when plotting larger row sets, it is advisable to use "Plot type: Mixed Scatter/Density" or even "Select Mode" -> static for reasonable response times in the present implementation.
  For cross-querying, note that crossmatch neighbour tables are present in the ESO VST southern hemisphere optical survey ATLASDR1 DB only, and that these do not include "best" match tables or "mates" fields (as specified in the Gaia DPAC CU9 schema) but otherwise are similar with master and slave unique IDs pointing between any two source catalogues within a predefined neighbourhood radius of 10 arcsec. For example, load in the table metadata for ATLASDR1 and TWOMASS by clicking on "Load database(s)" -> "More" and click on the "+" beside each followed by "OK". Clicking on "Show loaded Metadata" will now display ATLASDR1 and TWOMASS in addition to GACS (click on "Hide metadata" to collapse the view when not required) and expand these up to and including the specific table resource required for querying to load the metadata for auto-complete functionality.

Revision 22015-04-01 - NigelHambly

Line: 1 to 1
 
META TOPICPARENT name="DeliverablesGenius"

GENIUS Web2.0 Demonstrator

Line: 9 to 9
 The Web2.0 concept is one of enhanced end-user experience via useful functionality embedded within the browser session. In this case, we have
  • auto-completion when typing freeform ADQL, according to data resources available when loaded as part of the session;
  • embedded plotting/browsing;
Changed:
<
<
  • transparent cross-querying of catalogue resources via underlying Distributed Query Processing infrastructure.
>
>
  • transparent cross-querying of catalogue resources via underlying Distributed Query Processing (DQP) infrastructure.
 BRIEF INSTRUCTIONS

From the home page linked above, click on "Freeform SQL" in the left-hand navigation bar. This will take you to a window into which you can type ADQL queries in plain text. Be aware that your session has a "context" in that data resources can be selected for auto-completion and archive metadata browsing - by default, the demonstrator loads the ESAC Gaia archive database via the GACS TAP service, and auto-completion options displayed as you type will be restricted to that metadata unless you load further resources into the session (see later). To see the catalogue metadata currently available within your session, simply click on "Show loaded Metadata" and expand the treeview as required.

Line: 23 to 23
  For cross-querying, note that crossmatch neighbour tables are present in the ESO VST southern hemisphere optical survey ATLASDR1 DB only, and that these do not include "best" match tables or "mates" fields (as specified in the Gaia DPAC CU9 schema) but otherwise are similar with master and slave unique IDs pointing between any two source catalogues within a predefined neighbourhood radius of 10 arcsec. For example, load in the table metadata for ATLASDR1 and TWOMASS by clicking on "Load database(s)" -> "More" and click on the "+" beside each followed by "OK". Clicking on "Show loaded Metadata" will now display ATLASDR1 and TWOMASS in addition to GACS (click on "Hide metadata" to collapse the view when not required) and expand these up to and including the specific table resource required for querying to load the metadata for auto-complete functionality.
Added:
>
>
An example crossmatch query is given here that checks the g-band photometric consistency between the VST ATLAS DR1 survey photometry and that in SDSS DR9:
select gAperMag3, psfMag_g
from atlasSource as a, photoprimary as b, atlasSourceXDR9PhotoObj as x
where gAperMag3 > -99 and psfMag_g < 99 and distanceMins < 1.0/60.0
and a.sourceID = x.masterObjID and b.objid = x.slaveObjID

In this case the crossmatch is limited at query time to 1 arcsec via the predicate on "distanceMins". Other predicates remove default rows, and provide the relational join between the two source catalogue tables and the crossmatch neighbour table.

 Other data holdings presented via the demonstrator include SDSS DR9 ("BESTDR9"), Vista Hemisphere Survey DR1 ("VHSDR1") and the WISE source catalogue (not surprisingly, "WISE"), and all have crossmatch tables present in ATLASDR1 following the naming convention
atlasSourceX...
Line: 32 to 39
 atlasSourceXvhsDr1Source atlasSourceXwise_allskysc
Changed:
<
<
respectively etc. These are hosted locally on the SQL database back-end DBs serving the demonstrator
>
>
respectively etc. These are hosted locally on the SQL database back-end DBs serving the demonstrator, so in general crossmatches between them are fast, being executed entirely within the DB server itself via a relation join operation. The exception is for 2MASS, where we have configured DQP to query this as if it were hosted remotely on the wide area network with provision through TAP. Hence crossmatch queries involving 2MASS will be considerably less speedy than those on catalogue pairs not including 2MASS, and are more representative of the current performance of DQP in general.
  KNOWN ISSUES
Line: 44 to 51
 
  • Once loaded, table metadata are presented in auto-complete pop-ups even if the DB resource is subsequently deleted in the "Load database(s)" view
All of these issues are presently being worked on and will be improved in future versions of the prototype.
Changed:
<
<
Nigel Hambly - 2015-04-01
>
>

Nigel Hambly - 2015-04-01
 

Comments

Revision 12015-04-01 - NigelHambly

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="DeliverablesGenius"

GENIUS Web2.0 Demonstrator

The GENIUS Web2.0 prototype demonstrator is now available. This page gives some context and notes for users test driving this service.

CONTEXT

The Web2.0 concept is one of enhanced end-user experience via useful functionality embedded within the browser session. In this case, we have

  • auto-completion when typing freeform ADQL, according to data resources available when loaded as part of the session;
  • embedded plotting/browsing;
  • transparent cross-querying of catalogue resources via underlying Distributed Query Processing infrastructure.
BRIEF INSTRUCTIONS

From the home page linked above, click on "Freeform SQL" in the left-hand navigation bar. This will take you to a window into which you can type ADQL queries in plain text. Be aware that your session has a "context" in that data resources can be selected for auto-completion and archive metadata browsing - by default, the demonstrator loads the ESAC Gaia archive database via the GACS TAP service, and auto-completion options displayed as you type will be restricted to that metadata unless you load further resources into the session (see later). To see the catalogue metadata currently available within your session, simply click on "Show loaded Metadata" and expand the treeview as required.

For a test query, type in (or cut-and-paste for speed) the following query:

select mag_g, mu_alpha, mu_delta from GACS.igsl_source where alpha between 55.75 and 56.75 and delta between 23.5 and 24.5

As you type, auto-completion options can be displayed in a pop-up by pressing ctrl-space.

Once the results set from the query above are presented in summary form, you can invoke interactive plotting by pressing the "Plot Results" icon. In this case, selecting mu_alpha and mu_delta in the attribute selection box (again, auto-completion is available for convenience) and selecting "Plot type:" Scatter produces a browsable plot - zoom in to see the group of cluster proper motion members offset to (+20,-40) from the general field population near (0,0) - these are Pleiades proper motion members.

For cross-querying, note that crossmatch neighbour tables are present in the ESO VST southern hemisphere optical survey ATLASDR1 DB only, and that these do not include "best" match tables or "mates" fields (as specified in the Gaia DPAC CU9 schema) but otherwise are similar with master and slave unique IDs pointing between any two source catalogues within a predefined neighbourhood radius of 10 arcsec. For example, load in the table metadata for ATLASDR1 and TWOMASS by clicking on "Load database(s)" -> "More" and click on the "+" beside each followed by "OK". Clicking on "Show loaded Metadata" will now display ATLASDR1 and TWOMASS in addition to GACS (click on "Hide metadata" to collapse the view when not required) and expand these up to and including the specific table resource required for querying to load the metadata for auto-complete functionality.

Other data holdings presented via the demonstrator include SDSS DR9 ("BESTDR9"), Vista Hemisphere Survey DR1 ("VHSDR1") and the WISE source catalogue (not surprisingly, "WISE"), and all have crossmatch tables present in ATLASDR1 following the naming convention

atlasSourceX...

viz.

atlasSourceXDR9PhotoObj
atlasSourceXvhsDr1Source
atlasSourceXwise_allskysc

respectively etc. These are hosted locally on the SQL database back-end DBs serving the demonstrator

KNOWN ISSUES

Please be aware of the following issues and limitations when test driving the service:

  • Cross-querying is not yet available between the GACS tables and other catalogues (cross-neighbour tables are not yet available);
  • Array fields (stored as VARBINARY types in backend databases) are not yet supported in the interface layers;
  • Full Distributed Query Processing is not yet optimised so some cross-querying will be sluggish and/or time out;
  • Auto-completion is limited to those resources loaded AND expanded in the "Show loaded Metadata" view, i.e. to load the metadata for a given table to make them available for the auto-complete functionality, expand the treeview up to, and including, that table
  • Once loaded, table metadata are presented in auto-complete pop-ups even if the DB resource is subsequently deleted in the "Load database(s)" view
All of these issues are presently being worked on and will be improved in future versions of the prototype.

Nigel Hambly - 2015-04-01

Comments


<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback