Summary of activities December 2014 - February 2015

100 - Management

  • First annual review report Brussels, 16 Dec 2014 (XL, NB, AB, FA, NH, LB)
  • Reviewer report received 19 Jan 2015
  • After many technical problems, final submission of economic report 28 Jan 2015 (PP beta-version opened on 19 Dec 2014).
  • Participation in a coordination (ESA-GENIUS-CU9) meeting for cross-match activities, Rome 26 Feb. 2015 (XL)

200 - User community

  • Luri/Smart/Marese/Hambly and others carried out a cross match priorities meeting in Rome 26/27 Feburary
  • INAF-ASDC to produce a flexible user friendly cross match tool concentrating on figure of merit.
    • Large catalog matching procedures
    • Developer/tester of figure of merit procedures required for early March
  • Brown particpated in the CU9 telecons and continued his supervision of the work of Costigan and Hypki.
  • Costigan continued research into Living archive, gathering information on what is already available in the community.
  • Costigan attended Archive review meeting at ESAC (Jan 21st)
  • Costigan made contact with others (Vogt, Jordan, Madana) to develop idea of a simple, user friendly interface for archive.
Hypki undertook the following activities
  • working on a design of a software for user access to the archive
  • choosing and evaluating the proper technologies for the user access libraries
  • examining Apache Spark as a higher level language for data analysis of big datasets
  • working on the preparation of the Galaxy simulation done by Prof. S. Portegies Zwart in order to compare this model against the catalogue data

YY and RN organize Nano-JASMINE science workshop (Feb 16th)

  • gathering user needs
  • Members are required to bring more concrete plan for collabration between Nano-JASMINE and Gaia.
  • Japanese advantage is collaboration of Radio on ground and optical in space astrometry group.
  • See attached agenda. (Sorry it is in Japanese).
  • Requirements of Gaia catalogue
    • Spectrum type and mass may be useful for science application.

300 - System design

T3.1 Technical Coordination

  • Represented WP3 at first year review meeting, EC Brussels
  • Local meetings with development group December 11th and February 5th
  • Representing WP3 interests in wider CU9 telecons and provided input to crossmatch activities
  • CU9 required enhancements to DPAC MDB Dictionary Tool refined and merged back into trunk and release made

T3.2 Aspects of archive interface design

  • In the process of setting up a "Web2.0" demonstration prototype, incorporating our latest UI functionalities and DQP demonstration (linked to GACS TAP in the first instance)

T3.3 VO infrastructure

  • Enhancements to ADQL parser (branched from Gregory Mantelet's "CDS trunk") have been completed and benchmarked against a set of archive standard SQL queries
  • Now in contact with Mantelet to get these enhancements merged back into trunk so that all, including ESAC SAT, can get the benefit
  • INAF-OATs team continuing development of VO-Dance for generic TAP functionality
    • Now features authentication on search endpoints
    • SSL request/response transactions
    • Standalone (command-line) version of the web application
    • First draftversion of a VMWare virtual machine ready for TAP
    • Experimenting on ODA (Oracle Database Appliance)

T3.4 Data Centre collaboration

  • Working on OGSA-DAI internal SQL grammar parser to enhance in line with ADQL developments noted above
  • Testing DQP infrastructure against archive standard SQL queries, including pass-through logic for optimum performance given locality of resources

400 - Data Exploitation

T4.2 - Visualization

  • Several optimizations on the server code in terms of memory usage, multi-threading safety, cpu-usage and bandwidth usage between the server and the client are being implemented (December to March).
  • First prototype of the Object Server deployed at a Virtual Machine at ESAC in December, but unusable due to security restrictions. After iterations with ESAC's CSG, the necessary ports and authorizations for external access of the machine was obtained in late January. The VM is still not able to cope with the entire archive volume, but it is enough for the first tests we are performing at the moment (10^7 points). (December to February).
  • Received Toni Sagristà for three days in early february to discuss the adoption of the Object Server API in Gaia Sandbox. Sandbox is being adapted at this moment for this. (February)
  • Participation to CU9 Validation Workshop (2-4 March 2015, Meudon)

T4.3 - Data Mining

  • Meeting with the active members of the Work PAckage in Madrid (23rd December).
  • A more detailed work package definition has been done, with an estimate schedule and responsible from 2015 to Spring 2017. This has resulted into a set of 11 WP covering the different aspects of the tasks to perform, from design and implementation point of view to divulgation activities at a later phase.
  • A Data Mining prototype is being implemented and it's about 70% of its completion. This prototype should be the basis for the future framework.
  • New cluster has been requested to CSUC which should be in place during February-March.
  • Big scientific use case is being studied and its initial prototyping is ongoing using the base technologies selected for this WP (Spark, Hadoop). It will be finished during this year 2015.

T4.4 - VO tools and services

  • VOSA is a tool to gather photometric information from VO-compliant catalogues, information that is compared to different collections of theoretical spectra to estimate physical parameters. In this sub-WP we will adapt VOSA to the requirements defined by Gaia users. In particular, the following tasks have been accomplished since December 2014.
    • File upload:
      • Upgrade of the file upload process to allow for big files (thousands of objects).
      • Allow several values of user photometry (for instance, photometry taken at different epochs) for the same filter.
    • VO photometry:
      • Implementation of an asynchronous for big files.
      • The problem that external services stop answering if they receive too many "simultaneous" requests has been fixed. The number of paralelized processed has been fine tuned and the retrials are now handled in a much better way.
      • Implementation of the option to cancel a search for photometry at any stage of the process.
    • Infrared excess: Implementation of a new algorithm.

500 - Validation

T5.1 - Technical coordination

  • One important point for this trimester is the announcement of the TGAS exercise which is changing completely our agenda: tests for R1 which are being delayed, while new tests not planned before R2 (published fields such as parallax were not planned before R2) which now have to be done. This may or may not change the delivery date of the planned GENIUS deliverable.
  • Participation to CU9 telecon T2 (Dec. 5)- Arenou
  • Participation to GENIUS meeting (Dec. 16)- Arenou
  • WP validation manager telecon 8 (Dec. 19) - Arenou
  • Participation to CU9 telecon T3 (Jan. 9)- Arenou
  • Participation to CU9 validatio nintegration telecon (Jan. 12)- Arenou
  • Participation to CU9 telecon T4 (Feb. 6)- Arenou
  • First release of the Validation Test Specification document (Feb. 19)

T5.2 - Definition of problem cases, validation scenarios and tools

  • Standardized VTS document - Findeisen
  • Added many new tests for TGAS release, and fleshed out specifications of existing tests - Findeisen
  • Added specifications of tools used for multiple validation tests - Arenou, Findeisen
  • Created semi-empirical simulation of TGAS output - Arenou
  • Added infrastructure for testing statistical properties of Gaia catalogue(s), based on TGAS simulation, above - Findeisen

T5.3 - Simulation versus reality: from models to observables

  • Test and debug of WP943 software and its installation on the ESAC Hudson system (HZ, AR, CR))
  • Development of a special version of Besancon Galaxy Model for bright stars and generation of mock star catalogues to compare with Tycho data and preparation for TGAS validation (AR and RM)
  • Development of data analyzing utilities for quantification of validation criteria.
  • Update of VTS documents according to implementation of test procedures.
  • Addition of validation procedures relevant for TGAS validation to VTS document and improvement of validation procedures for parallax.
  • Addition of validation procedures for testing scale-dependent anomalies in the distribution of sources using decomposition to spherical harmonic to VTS documents. Development of software for implementation of these tests is undergoing.

T5.4 - Confronting Gaia to external archives

  • Adaptation of already developed tests to the new VOTAP, called ValidationTools (December 2014) - Ruiz-Dern, Barache, Babusiaux
  • Update VTS document to include new and adapted tests for TGAS (December 2014/January 2015) - Arenou, Babusiaux
  • Tested new "share tables" functionality of v0.5 GACS version, from website and from command-line: not sure if useful in Validation group (January 2015) - Ruiz-Dern, Shih
  • Discussions with ESAC (Jesús Salgado) and Mantis reported 0031714, due to crossmatch issues with new GACS version 0.5 (January 2015) - Ruiz-Dern
  • Developed creation of Healpix Maps within VTS WP944_Val_010_001: the test needs to access full Gaia/IGSL catalogue, which is not possible through TAP (January 2015) - Ruiz-Dern
  • Requested use of gbins instead of TAP for these cases where full catalogue is needed for tests. Working to adapt this option within the ValidationTools (January 2015) - Shih
  • Changes in ivy.xml of WP944 to have coherent dependencies: ValidationTools must be compiled and added in WP944 classpath when compiling WP944. Therefore, dependencies on ValidationTools are automatically added within the WP classpath. Working to have this safest configuration in both Java platforms NetBeans and Eclipse (January/February 2015) - Babusiaux, Shih, Arenou, Ruiz-Dern
  • Developing VTS WP944_Val_030_005, Parallax Zero-Point and precision using distant stars, for TGAS. Use of simulated TGAS catalogue by Arenou, only available in GEPI database by the moment (January/February 2015) - Ruiz-Dern
  • ValidationTools have been included within the ivy.xml, so no need anymore to have this project in the BuildPath of the working package (March 2015) - Babusiaux
  • Participation to CU9 Validation Workshop (2-4 March 2015, Meudon)
  • Discussions with T5.6 (WP946) to agree about common and generic tools needed for both working packages (as crossmatch, cleaning data,...) (4 March) - Babusiaux, Ruiz-Dern, Blanco-Cuaresma, Barache
  • Created generic CrossMatch class and VTSUtils to be used for all tests within WP944 (maybe WP946 too). Improvements are still in development, as the use of Stil library to work with StarTables. VTS and Junit tests codes are being adapted in parallel (March 2015-) - Babusiaux, Ruiz-Dern
  • "Jobs Over Quota" detected in GACS interface (common user vcu9), impeding codes to be executed ("Insufficient Storage"). List of jobs deleted by hand. Need to be discussed with the ESAC to better manage already finished jobs (March 2015) - Ruiz-Dern, Shih, Babusiaux

T5.5 - Data demining: outlier analysis

  • Most of the activity is being done in T5.2 and can be found above

T5.6 - Transversal tools for special objects

  • Variability and time series
    • Exploration of potential validation scenarios for TGAS from WP946 (Eyer, Blanco-Cuaresma)
    • Participation to CU9 Validation Workshop (2-4 March 2015, Meudon)
    • Compilation of test propositions from the CU9 workshop in Paris (Blanco-Cuaresma, Arenou, Soubiran)
    • Discussion about the development of tests linked to AGNs using LQAC-3 catalog (Blanco-Cuaresma, Taris, Eyer)
    • Initiated development of OGLE-IV cross-match in GACS (Blanco-Cuaresma)
    • Identified common cross-matching methods to be shared with T5.4 / WP944 (Babusiaux, Blanco-Cuaresma)
  • Validation tests SSOs
    • Definition of validation criteria of SSO-ST data
    • Definition of SSO classes for testings
    • Implement external asteroid orbital elements database (to crossmatch with Gaia-based orbits of selected groups of asteroids)
    • Participation to CU9 Validation Workshop (2-4 March 2015, Meudon)
    • Development of JAVA software with DPAC rules to implement the defined tests

600 - Support

T6.2-Simulated Catalogue

  • CSUC simulation resources being moved to data meeting as agreed in the 1st year review meeting. Technical personnel to be reallocated to new tasks.

T6.3-Science Alerts testbed

700 - Dissemination

T7.1 - Coordination of dissemination activities

T7.2 - Community portal infrastructure

  • Still working on the design and implementation of the new community portal, focused on being fully intuitive, dynamic and easy to use. During this period, two different meetings have been carried out following user's demands on the community portal. The first took place on January 21, where a first version was presented, whereas the improved version based on the feedback collected from last meeting was introduced on February 25. Definitive version might be ready by the beginning of March.

T7.3 - Community portal, outreach and academic activities

  • Gaia outreach WebConf on January 29.
Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf 201502_pres_IGSLwarnings2.pdf r1 manage 338.0 K 2015-03-13 - 08:43 RichardSmart  
PDFpdf NJScienceWorkshop2015_2_-_PukiWiki.pdf r1 manage 344.4 K 2015-02-26 - 07:15 YoshiyukiYamada Agenda of Nano-JASMINE science workshop on Feb 2015, (Japanese, sorry)
PDFpdf crossmatchmeeting.pdf r1 manage 51.7 K 2015-03-13 - 08:33 RichardSmart  
More topic actions...
Topic revision: r31 - 2015-03-19 - XaviLuri
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback