A Multi-Level WEB Based Distributed Processing System

Abstract

Over the past few years, a number of efforts have been exerted to build distributed processing systems that utilize the idle power of LAN’s and PC’s available in many homes and corporations. The main advantage of these approaches is that they provide cheap distributed processing environments for those who cannot afford the expenses of supercomputers and parallel processing hardware. However, most of the solutions provided are not very flexible in the use of available resources and very difficult to install and setup. In this thesis, a multi-level web-based Distributed System (MWDS) is designed and implemented. MWDS is based on the idea of volunteer computing, very reliable, easy to setup and easy to use. MWDS allows three types of subscribers: simple volunteers (single computers), super volunteers (full networks, mainframes …etc) and users. All of these entities are coordinated transparently through a secure web site. Volunteer nodes provide the required processing power needed by the system users. There is no limit on the number of volunteer nodes, and accordingly the system can grow indefinitely. Both volunteer and system users must register and subscribe. Once, they subscribe, each entity is provided with the appropriate MWDS components. These components are very easy to install. Super volunteer nodes are provided with special components that make it possible to delegate some of the load to their inner nodes. These inner nodes may also delegate some of the load to some other lower level inner nodes …. and so on. It is the responsibility of the parent super nodes to coordinate the delegation process and deliver the results back to the user. MWDS uses a simple behavior-based scheduler that takes into consideration the current load and previous behavior of processing nodes. Nodes that fulfill their contracts within the expected time get a high degree of trust. Nodes that fail to satisfy their contract get a lower degree of trust. MWDS is based on the .NET framework and provides the minimal level of security expected in distributed processing environments. Users and processing nodes are fully authenticated. Communications and messages between nodes are very secure. The system has been implemented using C#. MWDS may be used by any group of people or companies to establish a distributed processing or grid environment.