Project 4

Stars meme


Data lives in /bigdata/data/pan-starrs1 on delenn (not in HDFS). Find the 100 largest stars according to pixel area and report their location in RA/Dec coordinates. The RA/Dec coordinates are calculated as:

ra  = ra_for_this_image  + (0.25/3600) * ((img_width-x)-img_width/2)
dec = dec_for_this_image + (0.25/3600) * ((img_height-y)-img_height/2)

where ra_for_this_image and dec_for_this_image are found in the file /bigdata/data/pan-starrs1/radec.csv. If RA<0, add 360. RA/Dec coordinates can be entered on this website in the form ra,dec e.g. 206.126,7.15499.

Use Spark. Likely use OpenCV. Consider using the GPU. Do not use R. Your final output (top-100) must be produced by Spark.

Run Python OpenCV or g++ on delenn as follows:

Boost_DIR=/opt/boost-1.63.0-gcc4.9/ OpenCV_DIR=/opt/opencv2.4-gcc4.9/ PKG_CONFIG_PATH=/opt/libzip-1.2.0-gcc4.9/lib/pkgconfig PATH=/opt/openexr-2.2.0-bin/bin:/opt/protobuf-3.2.0-gcc4.9/bin/:/opt/python2.7-gcc4.9/bin:$PATH PYTHONPATH=/opt/dlib-19.3-gcc4.9/:/opt/opencv2.4-gcc4.9/lib:/opt/opencv2.4-gcc4.9/lib/python2.7/site-packages/ LD_LIBRARY_PATH=/opt/openexr-2.2.0-bin/lib:/opt/python2.7-gcc4.9/lib/:/opt/boost-1.63.0-gcc4.9/lib/:/opt/opencv2.4-gcc4.9/lib/:/opt/protobuf-3.2.0-gcc4.9/lib:/opt/glog-gcc4.9-bin/lib:/opt/gflags-gcc4.9-bin/lib:/opt/snappy-gcc4.9-bin/lib/:/opt/cuda/lib64/:/opt/libzip-1.2.0-gcc4.9/lib/ [YOUR COMMAND HERE]

Use g++-4.9 compiler not the g++ compiler. To compile C++ code, run this after that sequence above:

g++-4.9 -Wall -g `PKG_CONFIG_PATH=/opt/opencv2.4-gcc4.9/lib/pkgconfig/ pkg-config opencv --cflags --libs` -o myprog myprog.cpp
CINF 401 material by Joshua Eckroth is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. Source code for this website available at GitHub.