Summary of activities March 2015 - May 2015
WP1 - Management
- First GENUS trimestral telecon on 12 of March 2015 to complement monthly telecons CU9 - GENIUS.
WP2 - User community
- Costigan gave a talk on Gaia at University College Dublin (7th April). Also meet with Parameter Space (UCD, Dublin), who have been awarded the Geo Return contract to build a Gaia added value interface portal (GAVIP) for the archive.
- Costigan attended the GAVIP System concept workshop at ESAC (14th April). Also in attendence were the Finish group (Space Systems) and Portuguese group (Fork Research) who will build advanced archive tools within the GAVIP framework. Costigan has also worked on identifying further tools that could be incorporated into the GAVIP.
- YY attend Gaia CU3 AGIS meeting at Heidelberg, (http://wiki.cosmos.esa.int/gaia-dpac/index.php/CU3:Core_Processing:Meetings:AGIS_22#Agenda) and have a talk on Nano-JASMINE status. Gaia and Nano-JASMINE members share the situation.
- YY organized the Nano-JASMINE data analysis software review meeting, 11 Mar 2015. SH (Shunsuke Hozumi) also attended the meeting. Reviewers are Ken Ebisawa (ISAS/CSODA), Issei Yamamura (ISAS / Akari-Spica), Mitsuru Soma (NAOJ, astrometry), Takafumi Takata(NAOJ Subaru data analysis), and Hisashi Miyashita (Cybernet co., software engineering). we reported the current status of software development including the collaboration with Gaia DPAC team, and reviewers suggested possible problems on the way of future development.
- YY also attend the system development review of Nano-JASMINE satellite and ground station including the anaysis system 17 Mar 2015.
WP3 - System design
T3.1 Technical Coordination
T3.2 Aspects of archive interface design
- Web2.0 demonstrator enhanced to load (and keep loaded) archive metadata in the background as user adds archive to session context
- Working on connecting up the "error bus" through interface layers to the end user to provide meaningful error messages on DQP, ADQL parser errors etc. and also to provide "heartbeat" indicating in-progress (for long queries) and provision of cancellation button.
T3.3 VO infrastructure
- Continuing to lobby IVOA for improvements and enhancements to ADQL
- Working to get enhancements and bug fixes to ADQL Parser merged back from forked version to the main branch version used by ESAC
T3.4 Data Centre collaboration
- DQP components now compartmentalised into Virtual Machine containers ready for deployment (e.g. at ESAC)
T3.5 Data mining environments
- Experiments and practical implementations with Docker seem to be providing a good Virtualised environmental infrastructure - planning Astronomy & Computing paper on this
WP4 - Data Exploitation
T4.2 - Visualization
Object Server
- Designed and implemented new baseline approach for levels of detail. Has brought an important increase in fluidity, lower storage and optimized data transfer.
- Optimised data structures for indexing and storage (e.g. used in the server octrees)
- Capable of serving 100 million points for interactive visualisation. Tested with hardware at U. Lisbon
- Already installed at ESAC. Current version capable of efficiently serving Tycho2 for 3D visualisations (thus, OK for TGAS if it arrives)
- Tested creation of a full 2B rows GUMS-MW visualization and ensured that it can be done with a moderate amount of system RAM (~8GB is enough for one visualization, does not scale linearly but more is better as it masks low disk bandwidth). Tested with hardware at U. Lisbon.
- Confirmed that pre-computations for each full GUMS-MW visualization (=a set of variables to be visualised) will take at least 270GB of space to be persisted (for 2B rows, if more than X,Y,Z are defined more space is needed)
- Confirmed that disk I/O is a bottleneck (lots of Random Reads and Writes needed) for the pre-computations.
3D Client
ESAC Related
- Created TAP vs JDBC test case, which simulates a some of the workflow (includes, Sequential and Random access tests).
- Validated ESAC VM changes (16GB ram plus 1TB nfs share, I/O at 1Gbps)
Other
- Updates to the documentation in the WP980 wiki.
- End user tests of the whole client-server visualisation capabilities with U. Wien
T4.3 - Data Mining
- Firsts tests as a demostration of a possible procedure to extract data from the Gaia Archive to be used be the Data Mininng framework.
- Analisis on newly implemented MlLib algorithms on Spark, tested on non-Gaia data.
- Work Package coordination telecons and 2 days meeting at ESAC facilities to agree on a WP development plan, tasks and responsibles.
- New WP definition and responsibles.
T4.4 - VO tools and services
- VOSA access to Gaia photometry (Initial Gaia Source List at Gaia Archive Core System). TAP is used as access protocol
- New VOSA distributed environment final tests.
- First tests on parallelized copmuting
- First test on asynchronous jobs.
WP5 - Validation
T5.1 - Technical coordination
- Participation to Genius telecon T2 (March 12) - Arenou
- Organisation of 7 telecons with T5.2 - Arenou
- Telecon with X-matching for a use of validation environment - Arenou
- Participation to CU9/Genius telecon (April 13) - Arenou
- Participation to CU9/Genius telecon 10 (May 19) - Arenou
- Participation to telecon with GACS (May 22) - Arenou
- Preparation of all the environment (management, software, auxiliary data) for the reception of TGAS preliminary solution
T5.2 - Definition of problem cases, validation scenarios and tools
- Specified several tests to be implemented after the TGAS release - Findeisen
- Made WP942 validation tests more portable between development and release environments - Findeisen
- Implementation, testing, and debugging of TGAS validation tests, using input data in Gbin format - Findeisen
- Organised development of StatisticalTools library to support validation - Findeisen
- Implementation of CU9-Validation-Tools-IOC common tool - Findeisen
T5.3 - Simulation versus reality: from models to observables
- Implementation of tests for validation of parallax and improvement of previously implemented tests.
- Comparison of mock catalogue generated from a new version of BGM adapted and improved for modelling of bright stars with 9 < G mag < 13 (R.Moor, Barcelona Univ., A. Robin) with Tycho, RAVE, and AGISLab catalogue with the aim of optimizing validation methods and criteria.
- Significant modification of validation code to implement magnitude and latitude dependent validation criteria.
- Preparation and publication of the Software Release Note for WP943 validation package, GAIA-C9-TN-IUOB-HZ-001: delivery of prototype of model-based validation tools D5.2, May 19)
T5.4 - Confronting Gaia to external archives
- Developed complete internal structure of WP944: crossmatches, tables and objects formats and management, global tools, global running,... - Babusiaux, Ruiz-Dern
- Interaction and discussions with WP946 to make internal tools developed in WP944 available to other WP in Validation - Blanco-Cuaresma, Babusiaux, Ruiz-Dern
- Subdivision, after discussion, of WP944_Val_030_005 (parallax Zero-Point) into two tests: WP944_Val_030_005 (distant stars) and WP944_Val_030_006 (very distant stars), because of the different analysis complexity needed for first releases of TGAS/Gaia and later releases - Arenou, Babusiaux, Ruiz-Dern
- Implemented code to run R scripts from Java - Babusiaux
- Preparation and selection of new small catalogues and groups of stars to confront with Gaia data, such as Hipparcos LMC/SMC (among others) - Turon, Babusiaux
- Pritority 1 tests for TGAS ready and operational: Comparison to Hipparcos astrometry (WP944_Val_030_003), Parallax Zero-Point using very distant stars (WP944_Val_030_006) - Babusiaux, Ruiz-Dern
- Developing tests of priority 2 and others: WP944_Val_010_003 (Issues for specific stars), WP944_Val_030_004 (high proper motions) - Babusiaux
T5.5 - Data demining: outlier analysis
T5.6 - Transversal tools for special objects
- Variability and time series
- WP946 presentation at the 20th CU7 meeting (20–22 May 2015, Lisbon) - S. Blanco-Cuaresma, L. Eyer
- Provided a list of Hipparcos variable stars with Tycho identifiers for the validation WP - S. Blanco-Cuaresma
- Tests adapted to the validation tools version 5.1 - S. Blanco-Cuaresma
- Tests improvements to re-use WP944 code - S. Blanco-Cuaresma
- First tests for TGAS where we cross-match with Hipparcos, calculate difference in parallax, separate constant from variable stars and test that: - S. Blanco-Cuaresma
- Both groups have a mean parallax difference of zero (Student's t-test)
- Both groups have the same mean parallax difference (Student's t-test for 2 samples)
- Both groups have the same variance for parallax difference (F-test)
- Both groups have the same distribution (Kolmogorov–Smirnov test)
WP6 - Support
T6.2-Simulated Catalogue
Simulation work is momentarily on stand-by, as explained in the 1st year review, due to the changes in the needs of the mission. The second large simulation expected for this year has been postponed and the simulation resources at CSUC have been moved to data mining (also agreed in the 1st year review). The latest update from the DPAC consortium indicates that a new simulation will be needed towards the end of the year (TBC) and the activity would then resume.
T6.3-Science Alerts testbed
- Improvements have been made to the alerts pipeline, in particular continued progress towards the integration of the spectro-photoemtric classifier has been carried out.
- It is anticipated that, with significant improvements to the upstream Gaia cross matching, to be implemented in June 2015, that the release of alerts will reach a nominal level towards the end of 2015.
- The Gaia alerts continue to be published to the community, via the publisher at http://gaia.ac.uk/selected-gaia-science-alerts. This is now transitioning to act as the focal point for distribution of alerts to the amateur and school community. meanwhile the main point of access to the science community is the Gaia Alerts Publisher at http://gsaweb.ast.cam.ac.uk/alerts/alertsindex
- The Gaia followup community is providing confiormation of most alerts, and these are returned to the above alerts page. In order to improve coordination of ollowup activities a Gaia Marshall will shortly be released.
- Normal activities have included participation in GENIUS management telecons.
WP7 - Dissemination
T7.1 - Coordination of dissemination activities
Monthly meetings for the final design of Gaiaverse by Todojunto.
T7.2 - Community portal infrastructure
The development of the new community portal, named Gaiaverse, has been in the spotlight. Thus, four different meetings at CSUC facilities have been arranged during this period of time under the objective of getting ready the portal as soon as possible: March 3, March 10, April 9 and May 29. The meetings provided the basis for a proper design and implementation of Gaiaverse and the website implementation follow-up with the team of developers, who has incorporated the Editorial Board feedback over time.
According to the editorial board comments and following Eduard Masana instructions, multilingual content has been introduced.
T7.3 - Community portal, outreach and academic activities