Summary of activities Dec 2015 - March 2016

WP1 - Management

• Amendment sent in Dec. Accepted in Feb 12. CSIC termination, INTA inclusion as a partner
• Two contracts made by UB/FBG for INTA to overcome administrative delays Mauro Lopez and Francisco Jimenez-Esteban work on VO tools.
• 2nd period financial report submitted Feb 17.
• GENIUS-CU9 monthly telecom MT#11, 2 March 2016
• Request for clarification on the 2nd period financial report: CNRS, UL, CSUC, INAF and KU expenses to be clarified.

WP2 - User community

• Developement of ideas on the long term preservation of Gaia and related software. Contact made with CU leaders for feedback on feasiblility in different areas. Technical note will be completed once this is recieved in March. (Costigan)
• Task 2.2: Assessment of user requirements and assessment of the pre-lease Gaia archive. Beta testing group initiated, with betat testing underway. First assessment report due Apr 2016. Beta testing results captured at teh Gaia wiki, with issues arising raised in the Gaia issue reporting system (JIRA).
• Continued assistance in the development of the beta testing of the archive (Costigan).
• Task 2.4: The activities for this task are closely connected to CU9 cross-match activities in the near-infrared/optical/UV range and benefited from that work and developments. We started the multi-wavenlength cross-match work in july 2015 when M.Fabrizio joined the INAF/ASDC/OATo group. He started from the radio domain by making census of the existing and future radio surveys (GAIA-C9-TN-ASDC-MFA-001). The work was continued in sub-millimiter/infrared and high-energy (x- and gamma-ray) domains. He prepared two additional technical notes (MFA-003 and MFA-004). The quoted notes describe the more popular and populous surveys available in literature, concerning unresolved and/or compact sources. For each survey, he collected the relevant information about the instrument(s) adopted and the observed sources, especially those important for the XMatch. Moreover, he gave a detailed description of data model for every catalogue. During this period, he also made a census of the cross-match techniques (MFA-002) and we are in the process of implementing, testing and comparing different algorithms. Indeed, the goal of Task 2.4 is the seamless archive access (i.e. access to several catalogues as if it was a single one), combining Gaia with both large and small catalogues in the different wavelength domains. The work M.Fabrizio is doing is on this trail.
• Finished the first public version of BEANS software for data analysis of huge datasets, paper submitted to MNRAS (Hypki)

WP3 - System design

T3.1 Technical Coordination

• Continuing to liaise between the UEDIN development team and ESAC-SAT (Salgado, Gonzalez, Segovia) to coordinate contribution of GENIUS-developed components to the Gaia Archive Core Systems Interface
  
• Liaising with Enrique Solano of INTA (formerly CSIC) in order to define work under WP3 Task 3.3 (VO infrastructure) for recently hired developer

T3.2 Aspects of archive interface design

• Further contributions to the Gaia CU9 published catalogue data model (definition of indexes/views)

T3.3 VO infrastructure

• Further contributions to the Gaia CU9 published catalogue data model (refinement of UCD1+)
• INAF-OATs continued development of VODance:
  • Refactoring TAP code to align with the developing TAP 1.1 draft
  • Refactoring the publishing architecture to a more cloud-based solution
  • fixing an authentication and authorization layer on top of the current TAP implementation

T3.4 Data Centre collaboration

• UEDIN have created a dynamic ADQL autocomplete library in preparation for inclusion in the GACS interface; now working with ESAC-SAT to integrate in to the GACS system (which employs Google Web Tools)

T3.5 Data mining environments

• UEDIN have drafted (see dockerpaper.pdf) a paper for Astronomy & Computing concerning use of Docker for iease of containerisation and virtualisation of system components in the context of deployment of components at astronomy Data Centres
• Continued work on Docker containerisation of DQP and interface components

WP4 - Data Exploitation

T4.2 - Visualization

• System design of the visualisation client.
• Implementation of first versions of the visualisation webapp
• Integration of the webapp at ESAC. Demonstration presented to GACS operations team via webex.
• Ingestion of the Gaia Rehearsal data gbins in the visualisation database.
• Creation and serving of visualisations based on the Gaia Rehearsal dataset
• Design of new more scalable algorithms for computing visualisations and serving levels of detail at much higher speeds than the previous approach.
• A new physical machine is about to made available by ESAC/GACS for visualisation. This will allow computing and serving visualisations of the scale of the full Gaia catalogue. Migration to new machine is starting just after the end of this reporting period.
  Several telecons with the GACS team at ESAC.
  
  

T4.3 - Data Mining

• Validation of data conversion tool from ASCII to Parquet to be used in all catalogues to be ingested into the Data Lake
• Final results on the Grand Challenge Use case (IMF-SFR characterization).
• Progress meeting at Madrid (UNED premises) with key members of the WP.
• Definition of specific use cases to be delivered in the GENIUS final report as platform demonstrators
• General infrastructure with basic services completely set-up.
• New services being tested.

T4.4 - VO tools and services

• INTA-UB. Clusterix: Beginning of the design and development phase of Clusterix 2.0, a web-based application to calculate the grouping probability of a list of objects using proper motions. The tool also takes advantage of the Virtual Observatory to gather parallaxes, radial velocities and proper motions from VO services and to estimate temperatures, gravities and metallicities using VO tools like VOSA.
• INTA-UB. Clusterix: Preparation of a preliminary prototype version for the progress meeting to be held at Barcelona on April 22th.
• INTA. VOSA:VOSA is a tool to estimate physical parameters from the fitting of the photometric SED to a grid of theoretical models. In the period of time covered by this report we have started to work on the definition of a robust criterion to estimate the goodness of the fitting in order to avoid the current need of visually inspection.
• INTA. VOSA. Optimization of the Filter Profile Service (FPS). The Filter Profile Service is a key module of VOSA. It contains hundreds of filter collections from the most important photometric systems. The heterogeneity of these systems both in terms of units (energy counter vs photon counter) and components (filter/detector/optics/atmosphere) is very large and may have an impact on the obtained results. We are reviewing the available documentation (and contacting the owners when necessary) for each set of filters to homogenize all this information.

WP5 - Validation

T5.1 - Technical coordination

• telecons with WP managers on Dec. 14 and Feb. 8
• full participation of main work packages to DR1 rehearsal
• 11 telecons with GACS for DR1 rehearsal
• organisation of validation of TGAS_01.00 data

T5.2 - Definition of problem cases, validation scenarios and tools

• 6 telecons for progress meetings
• Erika Antiche will leave the project end February, and Sergio Soria will replace her, starting mid January.
• We have updated our tests to make sure they can run on more than 2000 million sources.
• We have updated our tests to make them more robust to changes in the Gaia data model.
• We have added new tests to look at the photometric data (magnitude and fluxes).
• We have added new tests to look at small numbers of observations or missing data.
• Implemented 5 validation tests.

T5.3 - Simulation versus reality: from models to observables

• Sékou Diakité is now our of developper in place of Houri Ziaeepour.
• Paola Di Matteo and Misha Haywood have made progress on other galaxy models for release 2.
• Improvements in the handling of the output files and scripts to extract latex tables and gnuplot figures done.
• Tests on new simu from Michalik also done.

T5.4 - Confronting Gaia to external archives

• New tests for DR1 photometry (hipparcos, HST, colour-colour relations) and TGAS (HST plx, Kepler distances) added.
• For the rehearsal, direct access to the data allowed early identification of several bugs, both from GACS and from WP944.
• Documentation started.
• Run on TGAS 1.0 done and under redaction.
• APOGEE spectrophotometric distances computed.
• VLBI proper motion issue understood, cannot be used

T5.5 - Data demining: outlier analysis

• Two statistical analysis now implemented as VTS_020_100: statistical summery and VTS_02_110: robust multi-parameter analysis
• Both very created in order to be able to sustain in reasonnable time and resources the management of billion stars
• Proved very efficient for TGAS_01.00 analysis, though very simple

T5.6 - Transversal tools for special objects

• Clusters: The proper motion test (010-010) now runs on lists of reference proper motions taken from several catalogues.
• Clusters: The parallax checks (010-030) are currently performed with photometric distances as references, but we are now looking into using other reference distances.

WP6 - Support

T6.2-Simulated Catalogue

• New Universe Model catalogue (GUMS) simulated. 2 billons objects.
• Complete Source catalogue simulated with the new error models

T6.3-Science Alerts testbed

• Science alerts testbed released for operational use from Jan 2016: see http://www.cosmos.esa.int/web/gaia/gaia-data and http://gsaweb.ast.cam.ac.uk/alerts/home

WP7 - Dissemination

T7.1 - Coordination of dissemination activities

• Gaiaverse Editorial Board coordination activities.
• Contacts to add new languages to Gaiaverse
• Contacts with CU9 - DPAC outreach work package.

T7.2 - Community portal infrastructure

• Slovenian, Japanese, Macedonian, Croatian and Greek have been added to Gaiaverse during this period. The portal has now 11 languages available on it. In coordination with other project members, who became editors of the Gaiaverse portal as well, the work consisted of setting these languages into the platform by incorporating the translation packages distributed to them.
• Numbers for Twitter look good. On January 11, this Twitter profile broke the barrier of 250 followers, having now (March, 2016) 312 followers. See the evolution below:
  • Dec.15: tweets (13), tweet impressions (8,567), profile visits (994), mentions (14), new followers (22), total number of followers on December 2015 (240).
  • Jan.16: tweets (16), tweet impressions (9,791), profile visits (985), mentions (7), new followers (30), total number of followers in January 2016 (270).
  • Feb.16: tweets (15), tweet impressions (10,900), profile visits (1,115), mentions (12), new followers (31), total number of followers in February 2016 (293).
  • March 16: tweets (10), tweet impressions (7,864), profile visits (837), mentions (8), new followers (24), total number of followers in March 2016 (312).

T7.3 - Community portal, outreach and academic activities

• Preparation of outreach activities for the Gaia Data Release 1