SWARM: Cooperative Reinforcement Learning for Routing in Ad-hoc Networks
Year: 2003
Type: dissertation
Abstract: Existing ad-hoc routing protocols are based on a discrete, bimodal model for links between nodes: a link either\nexists or is broken. This model usually considers only the most recent transmission as determining the state of the\nlink. Unfortunately, this model cannot distinguish transmissions which fail due to interference or congestion from\nthose which fail due to their target being out of transmission range.\nThis thesis presents a new ad-hoc routing protocol based on a continuous (rather than discrete) model for links\nwithin the network. We use a statistical measure of link performance over time to represent the quality of the link.\nWe propose that such a model is required for efficient operation within real-world wireless networks.\nIn order to define optimal routes in a network with links of variable quality, we model ad-hoc routing as a\ncooperative reinforcement-learning problem. Cooperative reinforcement-learning describes a class of problems\nwithin machine learning in which agents attempt to optimize it?s interaction with a dynamic environment through\ntrial and error and information sharing. We assign a value to routes that is representative of the costs to the agents\nusing that route. The ad-hoc routing problem is thus expressed as the optimization of the value of routes.\nOur model of link quality is a statistical one and requires data to be gathered over time. We devise a learning\nstrategy that gathers information about available routes and the quality of their links over time. This learning\nstrategy operates in an on-demand manner, gathering information only for the traffic flows which are in use, and\nin proportion to the amount of traffic on those flows. This learning is done in an on-line manner: route discovery\noperates simultaneously with packet delivery.\nOur learning strategy is loosely based on work in swarm intelligence: those systems whose design is inspired\nby models of social insect behaviour. In particular, we adapt the ant-colony optimisation meta-heuristic as a\nlearning strategy for the ad-hoc routing learning problem. In our protocol, each data packet routed by the protocol\ncauses incremental changes in the routing policy of the network.\nWe have found that a continuous model of link quality is very beneficial in congested multi-hop networks.\nWhereas a bimodal link model will interpret any dropped packet as an indication of node mobility and trigger\nroute updates throughout the network, a routing protocol based on a continuous model can respond to dropped\npackets by incrementally adapting it?s routing behaviour. In congested network scenarios simulated in NS-2, the\nperformance of our protocol in terms of packet delivery ratio and routing traffic is found to be superior to that of\nAODV or DSR.
Author: Eoin Curran
Language: English
Topic: Mobile Ad Hoc Networks
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2145140883
