The domain name system with which we can descriptively name different machines or services on a computer network is an integral part of our modern internet. As a consequence of this a sizeable amount of information is avaliable of these names. However acquiring any of this data can prove rather challenging as there is no single well defined way to download them.
I spent this semester designing and implementing a system which is capable of automatically downloading the aforementioned data in a structured way. One of the main goals of this project is to in the future build a freely avaliable and searchable database. Apart from the database it consists of three major components. The registry is the central information hub. Its task is to store every setting and state variable the system requires. Perhaps the most important component is the node which handles the actual data download. Its job also entails the upload of all this information in a well structured way to the central database. The last component is responsible for keeping every connected instance of the downloader busy with new tasks and recording the currently assigned tasks in case a downloader becomes unresponsive.
After the initial implementation I began testing the system. At first I tested each component separately, after which I assembled the complete system and kept adding new downloaders to it.