Journal of International Council on Electrical Engineering

ISSN: (Print) 2234-8972 (Online) Journal homepage: http://www.tandfonline.com/loi/tjee20

Multi-agent systems and their applications

Jing Xie & Chen-Ching Liu

To cite this article: Jing Xie & Chen-Ching Liu (2017) Multi-agent systems and their
applications, Journal of International Council on Electrical Engineering, 7:1, 188-197, DOI:
10.1080/22348972.2017.1348890

To link to this article: http://dx.doi.org/10.1080/22348972.2017.1348890

© 2017 The Author(s). Published by Informa

UK Limited, trading as Taylor & Francis
Group

Published online: 14 Jul 2017.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=tjee20

Download by: [185.158.120.234] Date: 14 July 2017, At: 06:05

Journal of International Council on Electrical Engineering, 2017
VOL. 7, NO. 1, 188–197
https://doi.org/10.1080/22348972.2017.1348890

OPEN ACCESS

Multi-agent systems and their applications

Jing Xiea and Chen-Ching Liua,b§
a
School of Electrical Engineering and Computer Science, Washington State University, Pullman, WA, USA; bSchool of Mechanical and Materials
Engineering, University College Dublin, Dublin, Ireland

ABSTRACT ARTICLE HISTORY

The number of distributed energy components and devices continues to increase globally. As a Received 14 January 2017
result, distributed control schemes are desirable for managing and utilizing these devices, together Accepted 26 June 2017
with the large amount of data. In recent years, agent-based technology becomes a powerful tool KEYWORDS
for engineering applications. As a computational paradigm, multi-agent systems (MASs) provide a Multi-agent system (MAS);
good solution for distributed control. In this paper, MASs and applications are discussed. A state-of- intelligent agent;
the-art literature survey is conducted on the system architecture, consensus algorithm, and multi- multi-agent platform;
agent platform, framework, and simulator. In addition, a distributed under-frequency load shedding distributed control; system
scheme is proposed using the MAS. Simulation results for a case study are presented. The future of architecture; consensus
MASs is discussed in the conclusion. algorithm; smart grid; under-
frequency load shedding

1. Introduction the number of devices in a system, distributed methods

are needed to shift the computational burden to local
1.1. Motivation of distributed control
controllers.
The energy system is evolving rapidly. Indeed, from At the same time, integrating renewable energy gen-
smart buildings to smart grids, digital technologies are eration, energy storage, and electric vehicles brings new
producing numerous data streams that offer in-depth challenges [3,4]. A critical problem comes from the decen-
information about the system. For example, the num- tralized ownership of energy system components. For
ber of distributed intelligent electronic devices (IEDs) example, in Pullman’s distribution system (Washington,
and distributed energy resources (DERs) continue to U.S.A.), a 1 MW flow battery is installed by Avista and
increase globally. The total number of installed pha- a 72 kW solar panel is established on the campus of
sor measurement units (PMUs) has increased to more Washington State University (WSU). While the electric
than 2000 in North America [1]. In addition, many energy trading and regulatory environment is evolving,
more remote control switches have been installed in decentralized ownership of energy system components
distribution systems to enhance the resilience of dis- becomes a problem for centralized control methods with
tribution feeders. Furthermore, development of renew- participation from a large number of consumers. Another
able energy contributes to the increase in DERs. By difficulty arises from the random nature of these compo-
2017, the total capacity of utility-scale solar panels nents. For instance, the energy demand, arrival time, and
operating in the U.S. is expected to be over 20 GW departure time of electric vehicles (EVs) are random [5,6].
[2]. Decentralized and distributed control schemes Centralized methods are complex and impractical for
are needed for managing and utilizing these widely modeling and managing these stochastically behaved EVs.
distributed devices.
Two reasons behind the increasing number of devices
1.2. Why multi-agent systems?
are the growing investment driven by policies and lower
hardware prices. Moore’s law has been valid for a long Modeling and computation tasks are becoming much
period of time. Hardware is becoming cheaper and bet- more complex as the size continues to increase. As a result,
ter in performance. This provides the foundation for it is laborious and difficult to handle using centralized
distributed control. Considering the increasing scale of methods. Although motivations to apply multi-agent

CONTACT Jing Xie [email protected]

§
Dr. Chen-Ching Liu is a Visiting Professor at the School of Mechanical and Materials Engineering, University College Dublin, Dublin, Ireland
© 2017 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided the original work is properly cited.
 JOURNAL OF INTERNATIONAL COUNCIL ON ELECTRICAL ENGINEERING  189

systems (MASs) for researchers from various disciplines research on the system architecture, consensus algorithm,
are different, as indicated in [7], the major advantages of and multi-agent platform, framework, and simulator. A
using multi-agent technologies include: (1) individuals MAS-based UFLS scheme is discussed in Section 4 with
take into account the application-specific nature and envi- a study case. The future of MASs is discussed in the
ronment; (2) local interactions between individuals can conclusion.
be modeled and investigated; and (3) difficulties in mod-
eling and computation are organized as sublayers and/or
2. System design – system architecture, agent
components. Therefore, MASs provide a good solution to
type, and consensus algorithm
distributed control as a computational paradigm. In addi-
tion, artificial intelligence (AI) techniques can be utilized. System design is critical as it covers many aspects in the
In [8], an agent is defined as a computer system that is development of MASs, e.g. agent models, coordination,
situated in an environment that is capable of autonomous data collection, and interaction among agents. In this sec-
actions in this environment to meet its design objectives. tion, a literature survey of system architecture, agent type,
In [9], a MAS is defined as a system that comprises two and consensus algorithm is presented.
or more agents, which cooperate with each other while
achieving local goals.
2.1. System architecture and agent type
There are abstract and concrete architectures [8].
1.3. Applications of MASs
Components and the basic engine structure are defined
MASs have been applied to various problems, including in the abstract architecture, which needs to be as generic
market simulation, monitoring, system diagnosis, and as possible. Starting from an abstract architecture, the con-
remedial actions [10–12]. In [13], a substation physical crete architecture is developed by assigning a type to each
security monitoring (SPSM) system is proposed within component and implementing each macro instruction of
the framework of strategic power infrastructure defense the engine.
(SPID) [14,15]. It monitors from remote the physical Intelligent agents are classified into several types with
security of power substations. In [16], an approach is respect to their functionalities and decision-making
developed to prevent interconnected power systems from mechanisms. (1) Purely reactive agents make decisions
catastrophic failures. It uses a MAS-based defense sys- using only the present information without referring to
tem that allows agents to have adaptive decision criteria. historical data. Thus, they utilize direct mapping from sit-
In [17], an integrative and flexible method is proposed uation to action and respond to the environment directly.
that uses agent-based modeling for assessment of market For example, in [21], reactive agent techniques are uti-
designs. Agents are facilitated by Q-learning. Compared lized to build up car-following models based on artificial
with the competitive benchmark, they can exploit market neural networks. (2) Logic-based agents make decisions
flaws to make higher profits. For remedial actions, a dis- through logical deduction. In [22], a method is developed
tributed under-frequency load shedding (UFLS) scheme for handling multiple hypotheses and performing log-
is developed in [18] using the MAS. It is improved and ic-based fault diagnosis. Scenarios from the Italian power
being implemented as a demonstration for the RIAPS system are used for evaluation. (3) Belief-desire-intention
platform [19]. Details of the UFLS scheme are presented (BDI) agents are built using symbolic representations of
in Section 4. the intentions, beliefs, and desires of agents. In [23], the
Progress in AI, hardware, and sensor technologies have stages of autonomy determination for software agents
been achieved by the MAS community, resulting in agent are discussed. The recognition of potential autonomy is
technologies that are applied successfully to real-world provided utilizing the BDI paradigm. (4) Layered archi-
industrial problems. A project [20] was supported by U.S. tectures incorporate several software layers. Each layer
Department of Energy (DOE) to transfer the VOLTTRONTM combines agents that deal with different abstract levels
software platform to Tranformative Wave. Details of of the environment. For example, in [14,15], a SPID sys-
VOLTTRONTM developed by Pacific Northwest National tem is designed in the layered architecture for integrating
Lab (PNNL) are discussed in Section 3.2. In addition, it various types of protection systems and defense schemes
provides Tranformative Wave with technical support to in different levels.
develop products and services that improve the operating
efficiency of buildings and resilience of power grids. This
2.2. Consensus algorithm
project indicates that industry is adopting agent technologies.
The remaining of this paper is organized as follows. A The cooperation of agents is achieved through information
literature survey is presented in Sections 2 and 3, including interaction to reach a consensus. The performance and
190  J. XIE AND C.-C. LIU

functionalities highly rely on the communication layer, where Δ and L are the degree and Laplacian matrices,
especially the connection topology and associated proto- respectively.
cols. The consensus and interactive consistency problems
are two important aspects of the Byzantine agreement 2.2.3. Fault tolerance
problem [24]. The terms of the above three problems are Fault tolerance is focused on both continuous availability
used interchangeably in the literature, although they have and the elimination of recovery time. As an important
different formal definitions. In this study, the consensus performance metric of fault tolerance, downtime refers to
problem is addressed. The well-known and widely used periods of time in which a system is not operational [29].
Paxos and average-consensus algorithms are described in In order to minimize the downtime, specialized software
this subsection. In addition, the fault tolerance is discussed. routines are needed to detect failures of hardware (e.g.
sensors, actuators, storage devices, and communication
2.2.1. Paxos algorithm channels) and switch to backup devices automatically.
The Paxos algorithm was originally proposed by L. This type of self-checking logic should be provided by
Lamport in 1988 [25]. In 2001, it was published in [26] the operating system (OS) and governed by the software
with a user-friendly presentation. As one of the simplest platform designed for distribution applications. In addi-
distributed algorithms, the Paxos algorithm is designed to tion, the capability to remove, disconnect, and repair the
implement a fault-tolerant distributed system. The ‘synod’ problematic devices without disruption to the computer
consensus algorithm [25] serves as the heart of the Paxos system is critical. Furthermore, important data sources
algorithm. Each Paxos agent can be a proposer, accep- should be checked and calibrated periodically, e.g. PMUs.
tor, or learner. If most nodes are available, this algorithm With respect to the requirements on cyber security,
guarantees that agents will converge to one value. It has firewalls and access control should be provided by the OS
been used by many software products, e.g. (1) the BigTable to prevent unauthorized users and access. Encryption is
which is adopted by many products of Google; and (2) the needed to secure the communication among nodes. The
search engine, Bing, of Microsoft. basic mechanisms of encryption should be provided by
the OS. The security level can be enhanced further with
2.2.2. Average-consensus algorithm digital certificates if it is supported by the platform. Once
The average-consensus algorithm has been applied to the cyber intrusions are detected and traced promptly and
many fields for the consensus and cooperation of net- accurately by the OS and platform, mitigation actions are
worked MASs. Its convergence and convergent speed are applied to block unauthorized access. In most applica-
reported in [27] and [28]. As an adaptive distributed algo- tions, it is assumed that non-Byzantine fault is considered.
rithm, it requires only communication among neighbor- Therefore, no cheating agent(s) is (are) considered in the
ing agents. Therefore, the communication burden is low. multi-agent based applications. Both fault tolerance pol-
All available agents will reach an agreement, which is equal icies and mechanisms become simpler.
to the average of their initial values. The simple mathe- In general, the OS and platform should support the
matic model is summarized. A graph, N = (V , B), repre- recovery of nodes due to software or other defects (e.g.
sents the network of n agents. The set of agents is denoted power failure). The recovery includes two parts: (1) auto-
by V = {1, 2, … , n}. A nonnegative n × n adjacency matrix matically restarting the node; and (2) recovering the node
B = [bij ]specifies the interconnection topology of network to a well-known state. It is expected that multiple states
N. If an active communication link exists from agent i to will be recorded and available for recovery. As a result,
agent j, bij is a positive value. Otherwise, bij = 0. applications will be able to achieve fault tolerance and
∑n ( ) decide which state should be selected for recovery. In
ẏ i (t) =
j=1
bij yj (t) − yi (t) (1) addition, self-checking should be performed immediately
once a node restarts, including hardware, software, and
( ) interfaces. If functionalities of the restored node cannot
Y (t) = col y1 (t), y2 (t), … , yn (t) (2) be guaranteed, the node may be required by applications
to remain silent instead of participating in the coordina-
(∑n ∑n ∑n ) tion. Therefore, self-checking is critical and its reliability
Δ = diag
j=1
b1j , … ,
j=1
bij , … ,
j=1
bnj (3) matters.
Fault tolerance becomes more complicated if commu-
nication channels are unreliable. The outage of communi-
cation lines and noisy measurements should be taken into
L = −B + Δ (4)
account. Therefore, it is harder to reach a consensus if the
underlying communication channels are unreliable. For
Ẏ (t) = (−Δ + B)Y (t) = −LY (t) (5) example, the balanced digraph may become unbalanced
 JOURNAL OF INTERNATIONAL COUNCIL ON ELECTRICAL ENGINEERING  191

due to the loss of edge(s), i.e. communication channel(s). distribution and development. In addition, the com-
Note that problematic communication links usually can- patibility with OSs matters. For example, JADE bene-
not be recovered soon. fits greatly from its pure design using JavaTM, which is
cross-platform with respect to the associated Java virtual
machine (JVM). Applications developed using JADE can
3. Multi-agent platforms, frameworks, and
run on multiple operation systems without modification.
simulators
Therefore, it becomes one of the most widely used plat-
Much effort has been made in designing the system archi- forms for research purpose. Furthermore, the supported
tecture, agent model, and interaction between agents. programming languages are likely to influence the choices
Nevertheless, the real implementation of MASs remains in of developers. Programmers usually intend to use tools
an early stage. Since late 90s, a large number of multi-agent that support programming languages with which they are
platforms, frameworks, and simulators have been devel- familiar and are easy to use, regardless of characteristics
oped targeting the gap between concept design and imple- of the problem and model. For instance, agent-oriented
mentation. As a result, it is a critical issue for developers to programming languages are not a preferred option for
select the tool properly. In this section, a comparison study new developers due to its lower level of abstraction.
is performed and tabulated. In particular, VOLTTRONTM
that is targeted for energy system applications is discussed.
3.2. VOLTTRONTM
Almost all platforms are developed to target a wide spec-
3.1. Comparison
trum of applications in different fields. Nevertheless, there
In order to perform a meaningful comparison, the evalu- is a strong desire for specialized platforms that focus on
ation criteria must be selected systematically. In [30], four particular fields. To this end, a flexible and modular soft-
platforms are selected and compared using four criteria: ware platform, VOLTTRONTM [36], is developed and spe-
completeness, applicability, complexity, and reusability. cialized for applications of energy systems, e.g. integration
The process of developing MASs is divided into four of DERs, decentralized control of microgrids, and distrib-
stages. Besides analysis, design, and development stages uted automation systems. The predecessor and prototype
of traditional software construction, deployment is added of VOLTTRONTM [40] was initially presented in 2011 as
as an important stage due to the complexity of MASs. In a framework that brings intelligence to the actuators and
[31], a comparative review of 24 multi-agent platforms, sensors in a smart grid. A year later, it was developed from
frameworks, and simulators is reported using 28 univer- a framework to a platform called VOLTTRONTM [35]. A
sal criteria. It is focused on all available platforms despite demonstration is shown in [41]. It is further applied to
the fact that the development of some platforms has been the integration of electric vehicles and smart grids [42].
terminated. Applications can be developed on top of VOLTTRONTM
Typical multi-agent platforms, frameworks, and sim- to improve energy efficiency, support grid services, mon-
ulators developed by different academic and industrial itor the performance of building systems, etc.
groups (e.g. universities, research institutes, and com- The VOLTTRONTM platform is developed in Python
panies) are compared. The up-to-date results are shown utilizing many open source libraries. In addition, a set
in Table 1. A notable feature of most platforms is the of utilities has been created to speed up development in
open source and non-proprietary nature that allows free Python-based agents. However, applications are not tied

Table 1. Comparison of selected multi-agent platforms, frameworks, and simulators.

Software Type Developer Programming language Operation system(s) Open source
JADETM [32] Framework Telecom Italia JavaTM All that supports JVM ✓
Agent Factory [33] Framework University College Dublin Agent Factory Agent Program- All that supports JVM ✓
ming Language (AFAPL)
OpenFMBTM [34] Framework Duke Energy JavaTM Linux ✓
RIAPS [19] Platform Vanderbilt University Python (and others in future) Linux ✓
VOLTTRONTM [35], [36] Platform Pacific Northwest National All Linux ✓
Laboratory (PNNL)
Janus [37] Platform Laboratoire Systèmes et Trans- SARL, JavaTM Linux, Unix, Windows, Mac OS, ✓
ports (SET) and Grupo de Android
Investigación en Tecnologías
Informáticas Avanzadas
(GITIA)
MASON [38] Simulator George Mason University JavaTM Windows ✓
GAMA [39] Simulator IRD/UPMC International Re- GAML Windows, Linux, Mac OS ✓
search Unit UMMISCO
192  J. XIE AND C.-C. LIU

to a specific programming language, such as Python or 4.1. Methodology

JavaTM. This lives up to developers’ expectation of flex-
The system framework is shown in Figure 1. Buses and
ibility. Agents written in other languages are supported
transmission lines are depicted in the power system layer
once they can communicate over the message bus. The
using thick bars and thin lines, respectively. The mul-
common VOLTTRONTM base can simplify and accelerate
ti-agent based UFLS scheme has two types of agents:
the development of applications. It includes ‘base’ classes
generation agent (GA) and substation agent (SA). If an
and ‘helper’ utilities that support rapid development and
agent has a generation unit, it belongs to the GA type.
deployment of applications in a scalable and cost-effective
Otherwise, it is a SA. The agent framework is shown in
way. The goal is to allow researchers to focus on their algo-
Figure 2. Each RIAPS node includes a number of agents
rithms rather than interaction with the platform.
for each application. Each agent has components for typ-
As an open source and open architecture platform,
ical functionalities. The flowchart showing monitoring,
VOLTTRONTM supports the peer-to-peer network of
estimation, and distributing steps is given in Figure 3. In
distributed agents that bring computation closer to data
the monitoring step, frequency variation is monitored
sources cooperatively. It supports non-Intel central pro-
by agents through two triggers. If abnormal conditions
cessing units (CPUs). In addition, the requirements on
are detected, agents start to communicate with adjacent
CPU, memory, and storage footprint are low. Therefore,
neighbors. Based on the shared information, the amount
applications can be built to manage energy use more effi-
of load shedding is estimated and distributed.
ciently among appliances and devices.
VOLTTRONTM is widely used for implementation of
applications. Besides, it is employed as the base of other 4.2. Extended distributing step
platforms. The BEMOSSTM system [43,44] developed by
It is known that load buses that are closer to generators
Virginia Tech serves as a good example. It adds auto-dis-
are more effective in load shedding. Therefore, they have
covery capability for several devices together with a user
higher priorities to shed the load. In the distributing step,
interface for controlling those devices. All features are
each agent decides how much load should be shed by itself.
built on the core of VOLTTRONTM. Another example is
the Transactional Network Platform (TNP) [45], which ∑Nk
ΔPAk = ΔPLSki (6)
consists of VOLTTRONTM as agent execution software. It i=1

includes agents that perform critical services and specific

functions (e.g. demand response, fault detection, weather ⎧ ΔP k , if ΔPik > 0 and ΔPik < PLik
service, etc.). TNP is proposed to achieve operational, ⎪ i

financial, and energy transactions between connected ⎪ PLik , if ΔPik > 0 and ΔPik ≥ PLik
ΔPLSki =⎨ (7)
individuals. ⎪
⎪ 0, if ΔPik ≤ 0
⎩
4. Distributed UFLS scheme using MAS
In [46], it is indicated that the wide-area special pro- ∑
ΔPik = ΔP − PLmk (8)
tection system (WASPS) is critical for dynamic stability m

and frequency problems. As an important component of

the WASPS, distributed UFLS is a natural problem for { }
m ∈ Ii = u|u ∈ B and au < ai (9)
the application of agent technologies. In this section, an
adaptive UFLS algorithm is described. The purpose of
this UFLS scheme is to mitigate power system blackouts
SA
using the MAS. It was originally proposed in [18]. In this SA GA SA
Agents layer
paper, the distributing step has been extended and a case GA SA SA GA

study is performed. It is being implemented as an appli-

cation of the RIAPS (‘Resilient Information Architecture
Platform for the Smart Grid’) platform. In the future, the
proposed UFLS method will be improved to deal with Power system
layer
wind turbines, solar panels, and batteries, targeting the
increasing penetration and impact of DERs integration
on UFLS schemes [27]. Figure 1. System framework.
 JOURNAL OF INTERNATIONAL COUNCIL ON ELECTRICAL ENGINEERING  193

{ }
ai = min LGi (10) where Nk denotes the number of load buses of the kth
agent. PLik is the amount of active power load available to
be shed at load bus i. ΔPLSki is the active power load to be
⎧� � ⎫ shed by the kth agent at load bus i. ΔP is the total active
⎪ ��𝜃i − 𝜃g �� ⎪ power deficiency, i.e., the total amount of load to be shed
LGi = ⎨ � g ∈ G⎬ (11)
⎪ Sg ⎪ by all agents. It is calculated by agents in the estimation
⎩ ⎭
Other Nodes
Data Inputs/Outputs
Substation Sensors TCP
DFR 1
PMU 1
TCP Data
Acquisition
PMU 2 DNP3 Agent GA/SA
CBM 1 Other Nodes
RTU 1 Control Events
Buffer
Substation Actuators TCP
DFR - Digital Fault Recorder DNP3 Device
CBM - Circuit Breaker Monitor Circuit Breaker 1
Interface
PMU - Phasor Measurement Unit Circuit Breaker 2
RTU - Remote Terminal Unit
RIAPS Node

Figure 2. Agent framework.

Figure 3. Flowchart of GA/SA.

194  J. XIE AND C.-C. LIU

step. All load bus indices of agents are comprised in a finite G8 G9

index set B. The symbols θi and θg are voltage angle of the 37
North Island 38

load bus i and generator g, respectively. Sg is the nominal G10

apparent power of the generator g. G is the finite index set 30 25 26 28 29

that comprises all generator indices.

It is assumed that the amount of reactive power shed, 2 18 17 27 24
ΔQLSki follows the same percentage of the active power G6
load to be shed. 1 3 16 21 35

G1
QLki 39 4 14 15 22
ΔQLSki = ΔPLSki (12)
PLki
5 12 23
19
Where QLSki is the amount of reactive power load available
to be shed at load bus i of the kth agent. 9 7 6 11 13 20 36
Each agent determines the timing and sequence of load 10 G7
buses by the priorities. The timing is determined by 8 32 34 33
31

Tall ∑ G2 G3 G5 G4
tAi k = tfm + top + PLmk (13) South Island
⋅
Pall m

Figure 4. IEEE 39-bus system and contingency scenario.

j Tall
tAi k − tA = k
⋅ ΔPLS (14) Table 2. Load shedding actions.
k Pall j

Reactive Active
where tAi k denotes the timing to shed load at bus i by the kth power power
load load
agent. tfm and top are the delays of monitoring frequency Agent Load bus (MVAR) (MW) Percentage Time (s)
and opening circuit breakers, respectively. Tall indicates PA5 39 135.54 598.53 54.215% 2.74
the time required to shed all load in the system. In [48], PA2 31 4.6 9.2 100% 2.72

it specifies 0.9 s to shed 451 MW load in the IEEE 39-bus

system. Therefore, Tall is set to be 12.17 s. The total load of Table 3. Comparison results.
the test system is denoted as Pall. The priority of load bus i
Multi-agent based
of agent k is one level lower than the load bus j of agent l. Methods distributed Advanced
Reactive power load (MVAR) 140.14 162.07
Active power load (MW) 607.73 643.35
4.3. Case study Frequency decay ends (s) 3.97 8
Frequency (Hz) 59.83 59.8
The IEEE 39-bus system is used and the contingency sce-
nario is shown in Figure 4. Transmission lines are tripped
100.00%
at 0.5, 1.5, and 2.2 s. An advanced load shedding strategy 90.00%
developed in [18] is adopted for the performance compar- Advanced Load Shedding Method
Load Shedding Percentage

80.00%
Multi-Agent Based Method
ison. An amount of 607.73 MW active power imbalance is 70.00%

estimated at 2.54 s. Load shedding actions are presented 60.00%

50.00%
in Table 2. If there is no load shedding scheme, a system 40.00%
collapse takes place, leading to the loss of all loads in the 30.00%
North Island. 20.00%
10.00%
The results of comparison are shown in Table 3. The
0.00%
proposed method sheds 35.62 MW and 21.93 MVAR less 3 4 7 8 18 25 26 27 28 29 31 39

load but stops the frequency decay earlier. See Figures 5 Bus No.

and 6. The frequency does not return to 60 Hz because

Figure 5. Distributed load shedding.
the secondary control is not included in the simulation.
In the simulation, all loads are assumed to be of the
same category. However, in a real power grid, there generators within the same priority category. In the third
are several categories of loads and a tradeoff is needed. step of the UFLS scheme, through communications, all
One simple option for tradeoff is to shed loads closer to agents share the information of loads that can be shed.
 JOURNAL OF INTERNATIONAL COUNCIL ON ELECTRICAL ENGINEERING  195

provides a limited number of interfaces for the interac-

60
tions. Therefore, after a tool is determined, options for
other tools will be reduced.
Frequency (Hz)

59.5 Once the design has been evaluated, implementation

is the final stage except for maintenance. In Section 3, a
59
survey on multi-agent platforms, frameworks, and simu-
Multi-Agent Based Method lators is reported. Again, a critical issue is how to select
Advanced Load Shedding Method
the tool properly for developers. Some rules apply: (1)
58.5
Requirements on agent performance and response time
0 10 20 30 40 50 60 70 80
are highly dependent on the supported programming lan-
Time (S)
guage. The C/C++ implementation of an algorithm should
Figure 6. System responses. be faster than a JAVA implementation. Thus, VOLTTRON
and RIAPS are more suitable for the UFLS scheme com-
In addition, the information of loads includes the type pared with JADETM, Agent Factory, etc.; (2) From a cost
and category. Thus, agents will be able to determine the perspective, an open-source platform is preferred espe-
category of loads which should be involved to reach the cially if the budget is constrained. In addition, flexibility
total amount of load shedding. The load at a lower prior- is provided by the open-source code. The platform can be
ity category will be shed first. Among loads in the same revised by developers for convenience of agent implemen-
category, the load closer to the generator will be shed first. tation. Both VOLTTRON and RIAPS are open-source;
and (3) If agents need to be implemented in hardware
nodes/boards, the supported operation system becomes
4.4. From design to evaluation and implementation an important issue. Most single-board computers, e.g.
Once the MAS is selected and applied to a problem, the first Raspberry Pi and BeagleBone Black, support Linux well.
step starts from system design. Based on the properties and Although some of them also support Windows, there is
characteristics of this problem, a suitable agent architecture an extra cost to purchase the Windows OS and associ-
can be determined together with the consensus protocol. In ated software for development. In addition, the burden
this step, flexibility is a major concern with respect to future on running the OS should be minimized for agents to
system extension and its integration with other systems. reboot quickly from a fault. Both VOLTTRON and RIAPS
In addition, major restrictions on the consensus protocol support Linux well.
arise from communication channels, network topology,
and requirements on performance. For instance, the UFLS 5. Conclusion and discussion
scheme adopts the layered architecture and average-con-
Research on MASs is mainly focused on system architec-
sensus algorithm. The rationale is: (1) The cyber-physical
ture, consensus algorithm, distributed optimization, and
model of modern power systems is layered; (2) As a reme-
software tools for simulation and implementation. It is
dial action scheme (RAS), the MAS-based UFLS scheme
anticipated that agent technologies will continue to evolve
should be easily integrated with other RASs. This flexibility
in the future. In addition, more industries are looking for
can be achieved by utilizing the layered architecture; and (3)
ways to use MASs, indicating that agent technologies are
UFLS requires fast decision-making and quick response.
making good progress in commercial use.
Therefore, the lightweight average-consensus is selected
The software development process of MASs requires
rather than other distributed optimization algorithms.
technical support from the selected platform. It matters
Next, from the software engineering point of view, the
whether documentation and interface are user-friendly.
MAS should be modeled and evaluated before imple-
The development efficiency can be significantly enhanced
mentation. Therefore, in this case, agents of the UFLS
if good software development kit (SDK) is provided. Once
scheme and the power grid are modeled in MATLAB
a MAS is deployed, maintenance becomes a long-term
and DIgSILENT, respectively. Data from the transient
issue concerning the complex environment and geo-
simulation is fed to agents for evaluation and debugging
graphically widely distributed nodes. Specialized and
purpose. Of course, researchers have many options (dif-
professional services on maintaining distributed hardware
ferent programming languages and software tools) in this
devices and software components are critical in future.
step. For example, agents can be modeled in Python and
DIgSILENT can be replaced by comparable tools. If the
physical environment and intelligent agents are modeled Acknowledgment
in different tools/platforms, the interface and interactions The authors would like to thank Dr. Marino Sforna from
between them becomes a critical issue. Typically, each tool TERNA, Rome, Italy, for his helpful suggestions.
196  J. XIE AND C.-C. LIU

The views and opinions of the authors expressed herein do not [5] Wang Y, Nazaripouya H, Chu C-C, et al. Vehicle-to-grid
necessarily state or reflect those of the United States Govern- automatic load sharing with driver preference in micro-
ment or any agency thereof. grids. Proceedings of Innovative Smart Grid Technologies
(ISGT Europe ‘14); 2014 Oct; Istanbul, Turkey.
[6] Wang Y, Shi W, Wang B, Chu C-C, Gadh R. Optimal
Disclosure statement operation of stationary and mobile batteries in distribution
grids. Appl Energy. Mar. 2017;190:1289–2017.
No potential conflict of interest was reported by the authors.
[7] Yu N-P, Liu C-C. Multiagent systems. In Advanced
solutions in power systems: HVDC, FACTS, and artificial
intelligence. Hoboken, NJ: John Wiley & Sons; 2016. p.
Funding
903–930.
This work is supported by the Advanced Research Projects [8] Wooldridge M. An introduction to multiagent systems.
Agency - Energy (ARPA-E), U.S. Department of Energy, New Jersey: Wiley; 2008.
at Washington State University under [grant number DE- [9] Weiss G. Multiagent systems: a modern approach to
AR0000666] through the RIAPS project, ‘Resilient Informa- distributed artificial intelligence. Cambridge: MIT Press;
tion Architecture Platform for the Smart Grid’. This research 1999.
was supported by EU Framework Programme FP7 at Univer- [10] McArthur SDJ, Davidson EM, Catterson VM, et al. Multi-
sity College Dublin (UCD) through the AFTER project, ‘A agent systems for power engineering applications – part
Framework for electrical power sysTems vulnerability identi- I: concepts, approaches, and technical challenges. IEEE
fication, dEfense and Restoration’. Trans Power Syst. 2007 Nov;22(4):1743–1752.
[11] McArthur SDJ, Davidson EM, Catterson VM, et al.
Multi-agent systems for power engineering applications
Notes on contributors – part II: technologies, standards, and tools for building
multi-agent systems. IEEE Trans Power Syst. 2007
Jing Xie received his B.E. from Tongji University, Shanghai, Nov;22(4):1753–1759.
China, in 2010. He was a M.S. student at Tongji University from [12] Catterson VM, Davidson EM, McArthur SDJ. Practical
2010 to 2011. Dr. Xie received his Ph.D. degree at University applications of multi-agent systems in electric power
College Dublin, Ireland, in 2015. He is currently an Assistant systems. Eur Trans Electr Power. 2012;22(2):235–252.
Research Professor with the School of Electrical Engineering [13] Xie J, Liu C-C, Sforna M, et al. On-line physical security
and Computer Science, Washington State University, Pullman, monitoring of power substations. Int Trans Electr Energy
WA, USA. His research interests include physical security of Syst. 2016 Jun;26(6):1148–1170.
substations, distributed control of power systems, and testbed [14] Liu C-C, Jung J, Heydt GT, et al. The strategic power
technologies. infrastructure defense (SPID) system. A conceptual
Chen-Ching Liu (F’94) is Boeing Distinguished Professor design. IEEE Control Syst Mag. 2000 Aug;20(4):40–52.
at Washington State University, Pullman, WA. He is [15] Li H, Rosenwald GW, Jung J, et al. Strategic power
also Visiting Professor at the School of Mechanical and infrastructure defense. Proc IEEE. 2005 May;93(5):918–933.
Materials Engineering, University College Dublin, Ireland. [16] Jung J, Liu C-C, Tanimoto SL, et al. Adaptation in load
At WSU, Professor Liu serves as Director of Energy Systems shedding under vulnerable operating conditions. IEEE
Innovation Center (ESIC). Professor Liu received an IEEE Trans Power Syst. 2002 Nov;17(4):1199–1205.
Third Millennium Medal in 2000 and the Power and Energy [17] Yu N-P, Liu C-C, Price J. Evaluation of market rules using
Society Outstanding Power Engineering Educator Award in a multi-agent system method. IEEE Trans Power Syst.
2004. In 2013, Dr. Liu received a Doctor Honoris Causa from 2010 Feb;25(1):470–479.
Polytechnic University of Bucharest, Romania. Professor Liu [18] Xie J, Liu C-C, Sforna M. Distributed underfrequency
is a Fellow of the IEEE and Member of the Washington State load shedding using a multi-agent system. Proceedings
Academy of Sciences. of IEEE PowerTech (POWERTECH ‘15); 2015 Jul;
Eindhoven, Netherlands.
[19] Emfinger W, Dubey A, Volgyesi P, et al. Demo abstract:
RIAP – a resilient information architecture platform
References for edge computing. Proceedings of 2016 IEEEACM
[1] Nuthalapati S, Phadke AG. Managing the grid: using Symposium on Edge Computing (SEC); 2016 Oct;
synchrophasor technology. IEEE Power Ener Mag. 2015 Washington, DC, USA. p. 119–120.
Sep;13(5):10–12. [20] “VOLTTRONTM – Tranformative Wave,” DOE [Online].
[2] “Solar industry data – solar industry growing at a record Available from: https://energy.gov/eere/buildings/
pace”. Solar Energy Industries Association (SEIA®). volttron [Accessed 2016 Nov 21).
[Accessed 2016 Nov 21]. [21] Panwai S, Dia H. Neural agent car-following models.
[3] Jian L, Zheng Y, Xiao X, et al. Optimal scheduling for IEEE Trans Intell Transp Syst. 2007 Mar;8(1):60–70.
vehicle-to-grid operation with stochastic connection [22] Jung J, Liu C-C, Hong M, et al. Multiple hypotheses and
of plug-in electric vehicles to smart grid. Appl Ener. their credibility in on-line fault diagnosis. IEEE Trans
2015;146:150–161. Power Deliv. 2001 Apr;16(2):225–230.
[4] Chan CC, Jian L, Tu D. Smart charging of electric vehicles [23] Hexmoor H. Stages of autonomy determination. IEEE
– integration of energy and information. IET Electr Syst Trans Syst Man Cybern C Appl Rev. 2001 Nov;31(4):509–
Transp. 2014;4(4):89–96. 517.
 JOURNAL OF INTERNATIONAL COUNCIL ON ELECTRICAL ENGINEERING  197

[24] Pease M, Shostak R, Lamport L. Reaching agreement in [38] Luke S, Cioffi-Revilla C, Panait L, et al. Mason: a
the presence of faults. J ACM. 1980 Apr;27(2):228–234. multiagent simulation environment. Simulation.
[25] Lamport L. The part-time parliament. ACM Trans 2005;81(7):517–527.
Comput Syst. 1998 May;16(2):133–169. [39] Grignard A, Taillandier P, Gaudou B, et al. GAMA 1.6:
[26] Lamport L and others. Paxos made simple. ACM SIGACT advancing the art of complex agent-based modeling and
News. 2001 Dec;32(4):18–25. simulation. Proceedings of 16th International Conference
[27] Olfati-Saber R, Murray RM. Consensus problems in on Principles and Practice of Multi-Agent Systems
networks of agents with switching topology and time- (PRIMA, 2013); 2013 Dec; Dunedin, New Zealand.
delays. IEEE Trans Autom Control. 2004;49(9):1520– p. 117–131.
1533. [40] Akyol B, Haack J, Tews C, et al. An intelligent sensor
[28] Xiao L, Boyd S. Fast linear iterations for distributed framework for the power grid. 5th International
averaging. Syst and Control Lett. 2004;53:65–78. Conference on Energy Sustainability; 2011 Aug;
[29] Laudon K, Laudon JP. Management information systems: Washington, DC, USA. p. 1485–1494.
managing the digital firm. Vol. 7. New Jersey: Pearson [41] Haack J, Akyol B, Carpenter B, et al. Volttron: an agent
Education, Inc., Upper Saddle River; 2015. platform for the smart grid. Proceedings of the 2013
[30] Ricordel P-M, Demazeau Y. From analysis to international conference on Autonomous agents and
deployment: a multi-agent platform survey. First multi-agent systems (AAMS); 2013; Richland, SC.
International Workshop on Engineering Societies in p. 1367–1368.
the Agents World (ESAW); 2000 Aug; Berlin, Germany. [42] Haack J, Akyol B, Tenney N, et al. VOLTTRON: an agent
p. 93–105. platform for integrating electric vehicles and smart grid.
[31] Kravari K, Bassiliades N. A survey of agent platforms. J International Conference on Connected Vehicles and Expo
Artif Soc Soc Simulat. 2015;18(1):11. (ICCVE); 2013 Dec; Las Vegas, Nevada, USA. p. 81–86.
[32] Bellifemine FL, Caire G, Greenwood D. Developing [43] Khamphanchai W, Saha A, Rathinavel K, et al. Conceptual
multi-agent systems with JADE. Vol. 7. Hoboken, NJ: architecture of building energy management open source
John Wiley & Sons; 2007. software (BEMOSS). Proceedings of Innovative Smart
[33] Russell S, Jordan H, O’Hare GMP, et al. Agent factory: a Grid Technologies (ISGT Europe ‘14); 2014 Oct; Istanbul,
framework for prototyping logic-based AOP languages. Turkey. p. 1–6.
Proceedings of 9th German Conference on Multiagent [44] Khamphanchai W, Pipattanasomporn M, Kuzlu M, et al.
System Technologies (MATES ‘11); 2011 Oct; Berlin, An agent-based open source platform for building energy
Germany. p. 125–136. management. Proceedings of Innovative Smart Grid
[34] Mallett E. NAESB developing framework for current Technologies-Asia (ISGT ASIA ‘15); 2015 Nov; Bangkok,
and future grid interoperability. Nat Gas Electr. 2015 Thailand. p. 1–6.
Nov;32(5):6–10. [45] Katipamula S, Lutes RG, Ngo H, et al. Transactional
[35] Akyol B, Haack J, Carpenter B et al. Volttron: an agent network platform: applications. Pacific Northwest National
execution platform for the electric power system. Laboratory (PNNL); 2013 Oct. (Tech. Rep. PNNL-22941).
Proceedings of the Third International Workshop on [46] Nakajima A, Morita M, Hayashi T, et al. Development of
Agent Technologies for Energy Systems; 2012 Jun; an advanced wide-area special protection system. J Int
Valencia, Spain. p. 117–131. Counc Electr Eng. 2013;3(4):334–339.
[36] VOLTTRON 3.0: User Guide. Pacific Northwest National [47] Xue F, Li Y, Song X-F, et al. Impact of large-scale wind
Laboratory (PNNL). 2015 Nov. power integration on the UFLS scheme. J Int Counc
[37] Galland S, Gaud N, Rodriguez S, et al. Janus: another yet Electr Eng. 2014;4(3):251–257.
general-purpose multiagent platform. Proceedings of 7th [48] Terzija VV. Adaptive underfrequency load shedding
Agent-Oriented Software Engineering Technical Forum based on the magnitude of the disturbance estimation.
(TFGASOSE-10); 2010 Dec; Paris, France. IEEE Trans Power Syst. 2006;21(3):1260–1266.