Nowadays the amount of data produced and collected every day is exponentially rising; a phenomenon that provides new challenges for data mining and database management. The traditional tools are hard to scale for such problems and the advancement of today's hardware cannot keep up with the rapidly growing expectations.
Solving these problems require fundamentally new methods and thinking; topics that BigData attempts to solve. Most of today's internet services heavily rely on tools provided by BigData, thanks to which the sector is experiencing an explosive growth.
The purpose of this thesis is giving a fundamental understanding of the topic, through which the topic of cluster computing is explored, followed by the demonstration of the basic structure of a modern search engine and recommender system working with Twitter data.
Throughout the chapters several of the newest BigData technologies and some of the most important algorithms fundamental to the internet are presented. The complete implementation is demonstrated using a widely used web framework, attempting to simulate the real-world use of such applications.