Edit Raw edit Attach


Practical Work with GUMS and GOG

For setting up the Virtual Machine on your own laptop, please follow the instructions written for the last GREAT ITN School held last January 2012 in Leiden at http://great.ast.cam.ac.uk/Greatwiki/GreatItn/ItnSchoolJan2012/setup.

Once the Virtual Machine is up and running, please have a look at another GREAT ITN School wiki page (http://great.ast.cam.ac.uk/Greatwiki/GreatItn/ItnSchoolJan2012/HistogramGenerationFramework) for the rationale of the framework built on top of the new Big Data processing paradigm (Map Reduce), meant to easily generate n-dimensional histograms. All the examples contained in those pages are for GUMS10 dataset ("ideal" error-free catalogue, up to G=20). The examples dealing with GOG (actual catalogue simulation including Gaia-like observations, up to G=17) will be shown below.

There is a third page worth mentioning (http://great.ast.cam.ac.uk/Greatwiki/GreatItn/ItnSchoolJan2012/RunningOnAmazonEMR) which explains how to deploy this histogram generation workflows on Amazon Elastic Map Reduce for processing the whole datasets (binary and compressed with hundreds of GB in size).

Examples for GOG dataset

Raw View |  Raw edit |  Edit Attach More topic actions...
Topic revision: r2 - 2012-02-22 - DanielTapiador
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2021 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback