DS 625 Big Data Architectures and Systems
The term ‘Big Data’ has been coined to describe massive systems; these might include, for instance, files of videos, images, handwriting, etc. that cannot be accommodated on a single server. Such systems introduce problems of scale: how to store vast quantities of data, how to be sure the data is of high quality, how to process that in efficient ways, and how to derive insights that prove useful. These topics include a range of techniques typically used in addressing the problems of scale. Such systems can be complicated, and so consideration is also given to software support for Big Data applications.
Covered are the following topics: problems of scale and the implications of Big Data on computation requirements; theoretical and methodological issues employed in the context of Big Data; appropriate algorithms to harness the processing power of the cluster; approaches to simplifying the programming interface used in developing Big Data applications.