fccjxxw.com
非常超级学习网 学习超级帮手
当前位置:首页 >> >>

Innovation and Creativity


Anders Kofod-Petersen

A Case-Based Approach to Realising Ambient Intelligence among Agents

Thesis for the degree doctor scientiarum

Trondheim, May 2007

Norwegian University of Science and Technology Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science

Innovation and Creativity

NTNU Norwegian University of Science and Technology Thesis for the degree doctor scientiarum Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science ? Anders Kofod-Petersen ISBN 82-471-2128-3 (printed version) ISBN 82-471-2131-3 (electronic version) ISSN 1503-8181 Doctoral theses at NTNU, 2007:97 Printed by NTNU-trykk

To my family near and far.

i

Abstract
This thesis presents a multi-agent based architecture and implementation to realising ambient intelligence. The architecture separates the abilities of an ambient intelligent system into three main areas: the ability to perceive the world which the system inhabits, the ability to reason about the state of the world by being context aware, and the ability to behave intelligently in occurring situations by being context sensitive. The aim of this work is to consider the notion of ambient intelligence from a holistic perspective, where all the aforementioned abilities are accounted for. Furthermore, this work argues that the ability to classify situations by being context aware is closely coupled with being knowledgeable about the world. Thus, achieving context awareness is approached from a knowledge intensive perspective, in particular through the episodic perspective of case-based reasoning. Case-based reasoning is utilised as the method of choice to assess and classify situations, thereby identifying the goal of the system in particular situations. This work argues that as an ambient intelligent system is situated in occurring situations involving human users, the ability to assess and classify situations is closely linked with understanding ongoing human activities. The knowledge model developed as part of this work combines the traditional pervasive computing perspective on context, with the aforementioned episodic perspective, and a socio-technical perspective on activities. The work presented in this thesis builds on both theoretical and empirical work. The underlying ideas were based on the literature and experience gained through the AmbieSense EU project. The main contributions of this work are a coherent conceptual architecture for realising ambient intelligences, a knowledge intensive method for modelling and representing context as an integrated part of a system’s knowledge model, and applying a small foot-print knowledge intensive case-base reasoning agent to assess and classify ongoing situations.

ii

iii

Preface
This thesis is submitted to the Norwegian University of Science and Technology in partial ful?lment of the requirements for the degree doctor scientiarum. This work has been conducted at the Department of Computer and Information Science, NTNU, Trondheim. Part of this work has been conducted as part of the research project AmbieSense, which has been funded by the European Commission within the 6th frame programme.

iv

v

Acknowledgments
The work presented in this thesis is partly funded by the European Commission and the Norwegian University of Science and Technology. First, I would like to thank my supervisor Agnar Aamodt for his advice and support throughout the whole of my work. Also I would like to thank co-supervisors Dag Svan?s and Rob Engels. In addition I would like to thank my many colleagues in the AmbieSense project, for their support, discussions and the exiting work within the AmbieSense project which I was lucky to be able to contribute to. I would like to extend my gratitude to all of my colleagues without whom a lot of this work would have been impossible. In particular I would like to thank the people with whom I have co-written papers: Marius Mikalsen, Sari E. Hakarainen, Carlos Buil Aranda, Odd Erik Gundersen, Sobah Abbas Petersen, and in particular J¨ rg Cassens for the great discussions. Inger Dybdahl S?rby for facilitating the data o collection at St. Olavs Hospital in Trondheim, and Siri Haug Str?mmen for collecting them. Frode S?rmo for always answering my never-ending stream of CREEK questions. Knut Harald Hassel Nielsen for being around when I needed to take my mind of this thesis. Last but not least, I would like to thank my family Mie, Oscar, Cornelia and Jose?ne, for their love, encouragement and support.

Anders Kofod-Petersen Trondheim, February 1, 2007

vi

Contents
I
1

Introduction and Overview
Introduction and Overview 1.1 Motivation and Aim . . . . . . . . . . 1.2 Problem De?nition . . . . . . . . . . 1.3 Research Method . . . . . . . . . . . 1.4 Research Background . . . . . . . . . 1.5 Research Activities and Contributions 1.6 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
3 3 4 5 6 7 12

II
2

Framework and State of the Art
Ambient Intelligence 2.1 From Ubiquitous Computing to Ambient Intelligence 2.2 The AmbieSense Project . . . . . . . . . . . . . . . 2.2.1 The Architecture . . . . . . . . . . . . . . . 2.2.2 The Context Model . . . . . . . . . . . . . . 2.2.3 The Context Middleware . . . . . . . . . . . 2.2.4 The Agency . . . . . . . . . . . . . . . . . . 2.3 Context . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Context in Ambient Intelligence . . . . . . . 2.3.2 Socio-technical Approaches to Context . . . 2.3.3 Context Categories . . . . . . . . . . . . . . 2.4 Situation Awareness . . . . . . . . . . . . . . . . . . 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13
15 15 22 22 25 26 29 29 33 39 44 46 49 51 51 54

3

Case-Based Reasoning 3.1 Foundation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Knowledge Intensive Case-Based Reasoning . . . . . . . . . . . . . .

viii 3.3 3.4 4

Contents Applying Case-Based Reasoning to Ambient Intelligence . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 58 61 61 67 70

Agent Technology 4.1 Agents and Agencies Basics . . . . . . . . . . . . . . . . . . . . . . 4.2 Agencies as an Architecture for Ambient Intelligence . . . . . . . . . 4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

III
5

Research Results
Knowledge Model and Conceptual Architecture 5.1 Knowledge Model . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Context Awareness and Sensitivity . . . . . . . . . . . . 5.1.2 Context Meronomy . . . . . . . . . . . . . . . . . . . . 5.1.3 Socio-technical Approach to Context Modelling . . . . 5.1.4 Domain Dependent Knowledge and Cases as Knowledge 5.2 Conceptual System Architecture . . . . . . . . . . . . . . . . . 5.2.1 Perception . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Awareness . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . Design and Implementation of Architecture 6.1 Context Middleware . . . . . . . . . . . . . . . 6.2 Agency . . . . . . . . . . . . . . . . . . . . . 6.2.1 Choice of Agent Platform . . . . . . . 6.2.2 Context Agent . . . . . . . . . . . . . 6.2.3 CREEK Agent . . . . . . . . . . . . . 6.2.4 Decomposer Agent . . . . . . . . . . . 6.2.5 Application Agents . . . . . . . . . . . 6.3 Running Example of the Hospital Ward Domain . . . . . . . . . . . . . . . . . . . . . . . . . . .

71
73 73 74 75 78 81 84 85 86 88 91 92 93 95 99 102 109 113 114

6

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

IV
7

Evaluation and Conclusion
Tests 7.1 Initial AmbieSense Multi-agent System Test 7.2 Usability Tests . . . . . . . . . . . . . . . 7.3 Hospital Ward Test . . . . . . . . . . . . . 7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

123
125 126 129 130 137

Contents 8 Discussion 8.1 Evaluation of the Knowledge Model . . . . . 8.2 Evaluation of the Architecture . . . . . . . . 8.3 Evaluation of the Implementation . . . . . . . 8.3.1 Case-based Reasoning Sub-system . . 8.3.2 Decomposer and Application Agents

ix 139 140 141 143 143 146

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

9

Conclusions and Future Work 149 9.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 9.2 Directions for Future Work . . . . . . . . . . . . . . . . . . . . . . . 151 153 171

Bibliography Author Index

V

Appendices

177
179 181 181 191 213 225 241 261 269 287

List of Reports Papers Case-Based Situation Assessment in a Mobile Context-Aware System . . . Context: Representation and Reasoning . . . . . . . . . . . . . . . . . . . An Architecture Supporting Implementation of Context-Aware Services . . Multiagent Based Problem-solving in a Mobile Environment . . . . . . . . Using Activity Theory to Model Context Awareness . . . . . . . . . . . . . Using Activity Theory to Model Context Awareness: a Qualitative Case Study . . . . . . . . . . . . . . . . . . . . . . . . Contextualised Ambient Intelligence through Case-Based Reasoning . . . . Challenges in Case-Based Reasoning for Context Awareness in Ambient Intelligent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . .

x

Contents

List of Figures
2.1 2.2 2.3 2.4 2.5 2.6 2.7 3.1 4.1 4.2 4.3 4.4 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 6.1 6.2 6.3 The Path to Ambient Intelligence . . . . . . . . . . . . Ambient Intelligence Situated . . . . . . . . . . . . . The AmbieSense System Architecture and Components Basic Context Meronomy . . . . . . . . . . . . . . . . Activity Theory: The Basic Triangle of Mediation. . . Cultural Historical Activity Theory . . . . . . . . . . . Model of Situation Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 21 24 25 41 43 48 53 63 64 65 68 76 77 79 80 82 83 85 86 87 88 89 92 93 94

Case-Based Reasoning Cycle . . . . . . . . . . . . . . . . . . . . . . Agent Taxonomy . . . . . . . . . . . . . . . . . . A Partial View of an Agent Typology . . . . . . . . Agent Classi?cation . . . . . . . . . . . . . . . . . Mediation Between Information and Physical Space Basic Context Meronomy . . . . . . . . . . . Knowledge Model for the Tourist Domain . . Mapping from CHAT to the Context Model . Knowledge Model containing Activity Theory Observation Schema . . . . . . . . . . . . . The Five Different Layers of Knowledge . . . Conceptual System Overview . . . . . . . . . The Perception Layer . . . . . . . . . . . . . The Awareness Layer . . . . . . . . . . . . . The Sensitivity Layer . . . . . . . . . . . . . Duality of Context Use in the Three Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Structured System Architecture . . . . . . . . . . . . . . . . . . . . . Context Sources and Clients Interacting with the Context Middleware Context Template Example . . . . . . . . . . . . . . . . . . . . . . .

xii 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 7.1 7.2 7.3 7.4 Agency Overview . . . . . . . . . . . . . . . . FIPA Agent Platform Reference Architecture . CREEK Agent State Transitions . . . . . . . . The Domain Independent Layers of Knowledge Subset of the Observed Data . . . . . . . . . . Case Example with Graph Representation . . . Example of a Complete Plan . . . . . . . . . . Decomposer Agent State Transitions . . . . . . Pre-ward Round 1301 Case . . . . . . . . . . . Pre-ward Round 1301 Task Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 98 104 105 106 108 111 112 117 120 126 128 129 134

Overview of Areas Covered by the Tests . . . . . . . Sequence Diagram for the Initial AmbieSense Test . Sequence Diagram for the FIPA Communication Test Ward round 1301 vs. Pre-ward round 1306 . . . . . .

List of Tables
2.1 2.2 2.3 5.1 Distribution of Work Packages . . . . . . . . . . . . . . . . . . . . . Context Aware Applications Survey . . . . . . . . . . . . . . . . . . Explanation of the Abbreviations used in the Survey . . . . . . . . . . Basic Aspects of an Activity and their Relation to a Meronomy of Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Agent Platform Requirements . . . . . . . . . . . . . Agent Platform Survey . . . . . . . . . . . . . . . . . Central Relations in the CREEK Implementation . . . Case Example with Traditional Representation . . . . . Agents and their Capabilities for Pre-ward Round 1301 Component Responsibilities . . . . . . . . . . . . . Captured Aspects . . . . . . . . . . . . . . . . . . . Distribution of Observed Situations for OL9, day 13 . Result of Matching Test (Run 1) . . . . . . . . . . . Result of Matching Test (Run 2) . . . . . . . . . . . Result of Matching Test (Run 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 36 37

78

6.1 6.2 6.3 6.4 6.5 7.1 7.2 7.3 7.4 7.5 7.6 8.1

. 96 . 97 . 107 . 109 . 119 . . . . . . 127 130 131 132 133 136

Absolute Accuracy of Case-Based Reasoning Test . . . . . . . . . . . 145

xiv

List of Tables

Listings
6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 Extract of the Ontology . . . . . . . . . . . . . . . Example of Context in XML . . . . . . . . . . . . INFORM Message from Context Agent . . . . . . SUBSCRIBE Message from the CREEK Agent . . Request Message from the CREEK Agent . . . . . Example of Request Message to Application Agent Pre-ward round 1301 Current Context . . . . . . . Inform Message Containing Pre-ward Round 1301 Request Message Containing Pre-ward Round 1301 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 101 102 103 110 113 115 116 118

xvi

Listings

Part I

Introduction and Overview

Chapter 1

Introduction and Overview
1.1 Motivation and Aim

Since Mark Weiser coined the term ubiquitous computing the ?eld has inspired work in several directions. The original vision dealt primarily with augmenting ordinary household and of?ce items with computer capabilities. Thus, allowing for a whole new approach to computing. The advancements in computer hardware, both with regards to size, price and connectivity, propel us towards the ubiquitous paradigm ?rst proposed by Weiser [1991]. When (computer) technology is disappearing and embedded into everyday objects many hardware problems arise. Problems such as: wiring of many small artefacts, and communication between artefacts and users. Much of the current research is focused on what we can augment and how. Research in the ubiquitous computing area has been focused on new gadgets and not so much on how to use them, and for what. With the ever-increasing access to new gadgets, there has been a push for understanding how to build systems and architectures that could incorporate all of this hardware into the existing world. One of the most important aspects of all these new items is the fact that the amount of information available already surpasses the capabilities of ordinary humans. There are numerous new information channels available, containing both information available from the net, and more local information available from the more or less embedded systems in our surroundings, such as sensors. Traditionally, the area of ubiquitous computing has not been focused on the system and architecture aspects; rather, this has fallen on the area of pervasive computing. A pervasive computer system should aim at being calm, or non-intrusive. To do this, the main approach is for a system to be context aware. Currently, no clear idea of what context encompasses exists. Whether context is to be regarded as data, information or knowledge is also not clear. Even if context is viewed at the knowledge level,

4

Problem De?nition

it is debatable whether it is a particular kind of knowledge, a sub-set of the general knowledge, or if it is indistinguishable from other types of knowledge contained in a system. Clearly, a more thorough treatment of context from a knowledge perspective is important to really talk about systems being context aware. The insight that context awareness is more that just a representational issue, has lead to the developments jointly labelled as ambient intelligence [Ducatel et al., 2001]. It is often argued that the ability to be context aware is the core functionality of an ambient intelligent system. However, the nature of context awareness is blurry at best, and the difference between the ability to be context aware and context sensitive is often disregarded. The core of ambient intelligent systems is their ability to appreciate the environment in which they are situated, infer needs of the users and respond to them intelligently. Thus, a clear separation of these different abilities of such a system is warranted. This is of particular importance when looking at a system’s ability to perceive the environment, be aware of its context and assess situations, and respond intelligently by being sensitive to its context. This separation is traditionally not made in pervasive computing. However, the work presented here argues that this distinction is very important. Systems that wish to exhibit context awareness must be more than simple reactive systems. When the word aware and awareness is used in conjunction with the word context, it implies a consciousness about context, and not only a stimuli-response based reaction to context and contextual changes. When a system exhibits context awareness, it is capable of understanding, or at least assessing, a situation. Once this understanding has been achieved, either by reasoning or through hard-wiring, it is possible to respond in a suitable manner, or being context sensitive. The aim of the work presented here is to consider the problem of ambient intelligent systems from a holistic viewpoint. Primarily, how a system recognises situations and responds to them in meaningful ways. Understanding situations and reacting intelligently to them presupposes a clear understanding of the nature of how context is modelled and represented, and what it means to be context aware and context sensitive. Finally, the aim is to design and implement a knowledge intensive architecture for an ambient intelligent system, which utilises case-based reasoning as the primary means for being context aware.

1.2

Problem De?nition

The aim of the work presented here is focused on investigation of the ability of an ambient intelligent system to assess and classify situations through reasoning about perceivable information. The ?rst and primary requirement for such an endeavour is to clarify the concept

Introduction and Overview

5

of context in light of the ability to reason about situations. How can context be expressed from a knowledge perspective to encompass both perceivable information, as well as knowledge about situations and their aspects? How can this knowledge be represented, and by which method can it be modelled? In conjunction with the clari?cation of context, both context awareness and context sensitivity are important abilities that must be clari?ed and incorporated into an ambient intelligent system. The required abilities of an ambient intelligent system is, among others, the ability to be aware of the presences of persons, perceive their needs and respond to them intelligently. To achieve this goal a coherent, yet ?exible, system architecture is required; one that includes the ability to perceive the environment, be aware of the environment, and be sensitive to the environment. So how can such a system architecture be described and through what means can it be realised? The ability to be aware of the environment and reason about occurring situations are paramount for intelligent systems in general and for ambient intelligent systems in particular. One important question is then how can we assess and classify situations through arti?cial intelligence methods? These methods should be able to reason about the current state of the world by maintaining a model of the causal relationships that exists. Further, the ability to reason about the world is closely linked with being knowledgeable about the world, thus a knowledge intensive perspective on reasoning methods is a potentially feasible approach. Case-based reasoning in general [Kolodner, 1993; Aamodt and Plaza, 1994], and knowledge intensive case-based reasoning [D?az-Agudo and Gonz? lez-Calero, 2000; Aamodt, 2004] in particular, appears to be ? a a promising candidate method for achieving context awareness. Case-based reasoning builds on theories on how humans assess situations and react to them. In addition, knowledge intensive case-based reasoning closely couples episodic memory, in the form of cases, with the more general domain knowledge. As the work presented here is conducted within a case-based reasoning environment and the properties of casebased reasoning seems to lend itself easily to context awareness, a central question is: how can case-based reasoning be used as the a tool to achieve context awareness? To sum up, the problem de?nition is: how can ambient intelligence be realised through applying knowledge intensive case-based reasoning within a multi-agent environment?

1.3

Research Method

The research method adopted here is partly theoretical and partly empirical. The research context gained from prior experience and the literature studies are provided in the theoretical section. Designing the architecture and implementing an instance of it is documented in the empirical section. The combination of these two perspectives

6

Research Background

seems appropriate. The theoretical aspect assists in shaping the holistic view on ambient intelligent systems. The empirical approach demonstrates that the theoretical developments can indeed be realised as an ambient intelligent system. The research presented here is founded on literature and research experience primarily in intelligent systems. A signi?cant part of this research was carried out within the AmbieSense research project.

1.4

Research Background

The AmbieSense project1 was a two and a half year EU-funded project within the 6th framework programme (IST-2001-34244), which focused on the development of an ambient intelligent infrastructure. The consortium consisted of industrial and academic partners that covered large areas of Europe: The private research facility Sintef in Norway; CognIT, a Norwegian company focusing on knowledge management; The Robert Gordon University in Aberdeen, Scotland; Siemens Austria; Yellowmap, a German company providing map services; The Norwegian University of Science and Technology; Oslo Airport; Sevilla Global, the agency for economic development in Sevilla, Spain; Lonely Planet, the publisher of travel guide books; and Reuters, the news agency. The main objective of the AmbieSense project was to construct an ambient intelligent landscape, within which users can receive personalised information. The AmbieSense project had four main areas of interest: i) wireless context tags supplying contextual information as well as content; ii) mobile computers to be used as clients for the users; iii) intelligent agents implementing the context aware personalisation services; and iv) personalised and context oriented information services. Even though the AmbieSense system and architecture was to be universally useable, the test domain was ?xed on the tourism sector [AmbieSense, Annex 1]. The four main scienti?c objectives of the AmbieSense project were: design and construction of the context tags; making information, such as digital maps, travel guides, and news databases available; pro?ling of users; and combining the three points above. The AmbieSense project, its solutions and technology is more thoroughly described in Section 2.2.

1 http://www.ambiesense.net

Introduction and Overview

7

1.5

Research Activities and Contributions

The main scienti?c contributions of this thesis are: ? A knowledge intensive method for modelling and representation of context, which combines an action oriented perspective on situations, with traditions in pervasive computing, and theories of episodic memory from case-based reasoning. ? A coherent architecture for realising ambient intelligence, which makes a necessary separation between the ability to perceive the world, reason about occurring situations by being context aware, and behave rational in occurring situations by being context sensitive. ? The use of case-based reasoning to reason about context, assessing situations and achieving context awareness. Parts of the work presented in this thesis are contained in 12 published peer reviewed workshop, conference and journal papers. During a period around the publication of this thesis these papers are available from my website2 . The following section gives an overview of the papers, their place of publication, as well as abstracts. Eight of the most central papers are available in the Appendix “Papers”. These are the papers: 2, 3, 5, 6, 7, 8, 10 and 12 in the below list. These papers provide additional details to issues presented in the main text. As part of the AmbieSense project, four reports have been published within the AmbieSense consortium, which are listed in the appendix “List of Reports”. Finally, two additional papers [Hakkarainen et al., 2005b,a], which did not emerge directly from this work have been published. Published Papers 1. Petersen, S. A. and Kofod-Petersen, A. (2006b). The non-accidental tourist: Using ambient intelligence for enhancing tourist experiences. In CamarinhaMatos, L., Afsarmanesh, H., and Ollus, M., editors, Proceedings of the 7th IFIP Working Conference on Virtual Enterprises, volume 224 of IFIP International Federation for Information Processing, Helsinki, Finland. Springer Verlag Abstract: An Ambient Intelligent environment is aware of the presence of a person, perceives her needs and responds intelligently. Ambient intelligence can
2 http://www.idi.ntnu.no/?anderpe/uk/research.php3

8

Research Activities and Contributions be used to guide a tourist through the city in such a way that is mutually bene?cial for both the tourist and the city. The composite set of services delivered to a tourist can be personalised. This adaptive behaviour is best supported by Virtual Enterprises that are formed dynamically, at runtime. This paper considers an Ambient Intelligent scenario from the service providers’ perspective and discusses how such scenarios can be realised through Virtual Enterprises. 2. Kofod-Petersen, A. and Aamodt, A. (2006). Contextualised ambient intelligence through case-based reasoning. In Roth-Berghofer, T. R., G¨ ker, M. H., o and G¨ venir, H. A., editors, Proceedings of the Eighth European Conference u on Case-Based Reasoning (ECCBR 2006), volume 4106 of Lecture Notes in ¨ u Computer Science, pages 211–225, Ol¨ deniz, Turkey. Springer Verlag Abstract: Ambient Intelligence is a research area that has gained a lot of attention in recent years. One of the most important issues for ambient intelligent systems is to perceive the environment and assess occurring situations, thus allowing systems to behave intelligently. As the ambient intelligence area has been largely technology driven, the abilities of systems to understand their surroundings have largely been ignored. This work demonstrates the ?rst steps towards an ambient intelligent system, which is able to appreciate the environment and reason about occurring situations. This situation awareness is achieved through knowledge intensive case-based reasoning. 3. Kofod-Petersen, A. (2006). Challenges in CBR for context awareness in ambient intelligent systems. In Schwarz, S., Bogaerts, S., and Schulz, S., editors, CACOA 2006, 1st Workshop on Case-based Reasoning and Context Awareness, volume 221. CEUR Workshop Proceedings Abstract: One of the most important issues in an ambient intelligent environment, indeed in any problem-solving situation, is the ability of a system to appreciate its environment and assess the situation in which problems are to be solved. Recently, case-based reasoning has gained some momentum in the area of context awareness for ambient intelligent systems. When applying casebased reasoning in this type of system some challenges arises, each well known for case-based reasoning in general, but each gaining a new particular angle. The main challenges are: how to acquire the initial cases, coping with the potential for a very large amount of cases, when to execute a case-based reasoning process, and knowing if the reasoning was correct. The work presented here builds on the experiences gained by applying case-based reasoning to situations assessment. It will analyse each of these challenges from the perspective of different domains, and give some suggestions as to how to best approach these challenges.

Introduction and Overview

9

4. Petersen, S. A. and Kofod-Petersen, A. (2006a). Learning in the city: Context for communities and collaborative learning. In Proceedings of the 2nd International Conference on Intelligent Environments (IE06), Athens, Greece. The Institution of Engineering and Technology Abstract: A language learner, new to the city wishes to explore the city, interact with local people and improve her language skills. The learner interacts with several communities who support her in her learning process. This paper explores the context for mobile collaborative learners in an intelligent environment, in particular the social and environmental context. A scenario is used to identify how existing context models support modelling the context of mobile collaborative learners. 5. Kofod-Petersen, A. and Cassens, J. (2006). Using activity theory to model context awareness. In Roth-Berghofer, T. R., Schulz, S., and Leake, D. B., editors, Modeling and Retrieval of Context: Second International Workshop, MRC 2005, Revised Selected Papers, volume 3946 of Lecture Notes in Computer Science, pages 1–17, Edinburgh, UK. Springer Verlag Abstract: One of the corner stones of any intelligent entity is the ability to understand how occurrences in the surrounding world in?uence its own behaviour. Different states, or situations, in its environment should be taken into account when reasoning or acting. When dealing with different situations, context is the key element used to infer possible actions and information needs. The activities of the perceiving agent and other entities are arguable one of the most important features of a situation; this is equally true whether the agent is arti?cial or not. This work proposes the use of Activity Theory to ?rst model context and further on populate the model for assessing situations in a pervasive computing environment. Through the socio-technical perspective given by Activity Theory, the knowledge intensive context model, utilised in our ambient intelligent system, is designed. 6. Cassens, J. and Kofod-Petersen, A. (2006). Using activity theory to model context awareness: a qualitative case study. In Proceedings of the 19th International Florida Arti?cial Intelligence Research Society Conference, pages 619–624, Florida, USA. AAAI Press Abstract: In this paper, we describe an approach to modelling context-aware systems starting on the knowledge level. We make use of ideas from Activity Theory to structure the general context model and to assess empirical data. We further on describe how the data-driven and the model-driven aspects of our approach are combined into a single knowledge model. We outline the design of

10

Research Activities and Contributions an empirical study conducted to gather information about a concrete workplace environment to populate our context model. We further on describe how the collected data can be used to validate our approach. 7. Gundersen, O. E. and Kofod-Petersen, A. (2005). Multiagent based problemsolving in a mobile environment. In Coward, E., editor, Norsk Informatikkonferance 2005, NIK 2005, pages 7–18. Institutt for Informatikk, Universitetet i Bergen Abstract: As users are becoming more mobile and information services are increasingly being personalised, the need for computer systems to adapt to the user is also increasing. One important aspect of this adaptation is the ability to solve problems in a dynamic environment. We propose a multiagent based approach to dynamically planning and problem solving. This approach allows for construction of plans based on available resources in the environment. We demonstrate the capabilities of this system by showing an implemented prototype. 8. Kofod-Petersen, A. and Mikalsen, M. (2005a). An architecture supporting implementation of context-aware services. In Flor? en, P., Lind? n, G., Niklander, e e T., and Raatikainen, K., editors, Workshop on Context Awareness for Proactive Systems (CAPS 2005), Helsinki, Finland, pages 31–42. HIIT Publications Abstract: This work describes an approach to facilitate to use of contextual information to improve the quality of service in a mobile ambient environment. The approach advocated here de?nes two levels of applications: a simple stimuli-response based system using the Context-Middleware, and a more complex and adaptive version that utilises a multi-agent environment to supply information and context to mobile users. The main goal of this work is to show how a developer easily can implement a simple responsive system, and by following simple speci?cation of agent communications, the local ontology and complying with the speci?cations of the Context Middleware, develop adaptive context-aware applications. 9. Kofod-Petersen, A. and Cassens, J. (2005). Activity theory and context awareness. In Schulz, S., Leake, D. B., and Roth-Berghofer, T. R., editors, Proceedings of the IJCAI-05 Workshop on Modeling and Retrieval of Context (MRC 2005), volume 146, pages 1–12. CEUR Workshop Proceedings Abstract: A lot of research has been done in the area of context-aware computing. Even though, the term context seems often not to be well de?ned. We attribute this problem partly to the fact that research often focuses on syntactical and technical issues of contextuality and does not take a knowledge level

Introduction and Overview

11

perspective on context. When including the knowledge level, some sort of analysis is required on what aspects need to be modelled. In this paper, we propose the use of an Activity Theory (AT) based approach on modelling components, and outline how it can be combined with the AmbieSense context modelling framework we have proposed earlier. 10. Kofod-Petersen, A. and Mikalsen, M. (2005b). Context: Representation and reasoning – representing and reasoning about context in a mobile environment. Revue d’Intelligence Arti?cielle, 19(3):479–498 Abstract: Today the computer is changing from a big, grey, and noisy thing on our desk to a small, portable, and ever-networked item most of us are carrying around. This new found mobility imposes a shift in how we view computers and the way we work with them. When interaction can occur anywhere at any time it is imperative that the system adapts to the user in whatever situation the user is in. To facilitate this adaptivity we propose a two tier architecture. A middleware layer implementing a general mechanism for aggregating and maintaining contextual information. The second part offers automatic situation assessment through Case-Based Reasoning. We demonstrate a multi-agent system for supplying context-sensitive services in a mobile environment. 11. Mikalsen, M. and Kofod-Petersen, A. (2004). Representing and reasoning about context in a mobile environment. In Schulz, S. and Roth-Berghofer, T., editors, Modeling and Retrieval of Context 2004 (MRC), volume 114, pages 25–35. CEUR Workshop Proceedings Abstract: This paper describes an approach to representing contextual information through a domain independent solution. It, further more, shows how case-based reasoning can be used to reason about user-situations. The solutions described is implemented within a context-aware system supporting users in an mobile environment. 12. Kofod-Petersen, A. and Aamodt, A. (2003). Case-based situation assessment in a mobile context-aware system. In Kr¨ ger, A. and Malaka, R., editors, Arti?u cial Intelligence in Mobile Systems 2003 (AIMS), pages 41–49. Universit¨ t des a Saarlandes Abstract: This paper describes how to utilise Case Based Reasoning for identifying and user situations assessment in a context-aware mobile system. The Case Based Reasoning mechanism attempts to identify what situation the user is in, and utilises a Multi Agent System, consisting of information suppliers, to provide the user with personalised and context-sensitive information.

12

Thesis Structure

1.6

Thesis Structure

The rest of this thesis is divided into three main parts plus appendices. The parts are structured as follows: Part II, Framework and State of the Art This part initially covers the technological background from ubiquitous computing to ambient intelligence. It then describes the different work on context modelling and representation, ranging from the traditional use in ambient intelligence through socio-technical aspects of activities to situational awareness. Following this, case-based reasoning and its use in ambient intelligence are addressed. Finally, agents and agencies are described. Part III, Research and Results This part is divided into two main chapters: the knowledge model and conceptual architecture, and the design and implementation of the architecture. The former describes how the model starts off from the initial AmbieSense meronomy, and arrives at a highly complex knowledge model containing aspects from traditional ambient intelligence, human activities and case-based reasoning. The implementation chapter covers how the agency and its agents are implemented. Part IV, Evaluation and Conclusion This part is divided into three chapters. The initial chapter describes the tests conducted to verify the credibility of the chosen solution. The second chapter evaluates and discusses the contributions made in this thesis: the proposed knowledge model is evaluated with regards to its cognitive plausibility by analysing its ability to capture relevant knowledge; the conceptual architecture is also evaluated with respect to its cognitive plausibility; and the implementation is evaluated against its ability to implement the conceptual architecture as well as its performance. The ?nal chapter concludes the work presented here by summarising the results and contributions, comparing them to the research questions posed, discussing some of the challenges not yet addressed, and ?nally describes some directions for future work.

Part II

Framework and State of the Art

Chapter 2

Ambient Intelligence
2.1 From Ubiquitous Computing to Ambient Intelligence

Mark Weiser ?rst proposed the ubiquitous computing paradigm through the realisation that: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” [Weiser, 1991, p. 1]. Since the birth of this paradigm the vision has grown in both scale and complexity. Originally, Weiser envisioned a world occupied by a variety of ordinary artefacts augmented with computing capabilities, which would assist humans in their day-to-day activities. Weiser saw this new paradigm as a natural evolution in the way we use computers. Originally we had mainframes, where many people shared one computer; we then moved on to personal computers, where one person used one computer, and we are now entering ubiquitous computing, where many computers serve one person. Since the originally vision was conceived the followers of Weiser’s proposal have increased in numbers; as have the number of views on computers as ubiquitous artefacts. Today we can count at least four different research directions that more express aspects of the same vision. These research directions are: ? Ubiquitous computing ? Pervasive computing ? Context awareness ? Ambient intelligence Even though no clear hierarchy between these areas exists, it is possible to see a trend in increasing capabilities, autonomy, intelligence and complexity of the systems, going from the original ubiquitous computing paradigm to the vision of ambient

16

From Ubiquitous Computing to Ambient Intelligence

Distributed Systems

Ubiquitous Computing Pervasive Computing

Mobile Computing

Human Computer Interaction

Context Sensitive

Arti?cial Intelligence

Context Awareness

Ambient Intelligence

Figure 2.1: The Path to Ambient Intelligence

intelligence. Figure 2.1 depicts the relationship between some of the most important research areas, which contribute towards the ambient intelligence research ?eld. The arrows show how research that originated in each of the included areas have contributed to the development of other ?elds. Weiser, and his colleagues, argued that in contrast to the popular desktop paradigm of modern computers, where one attempts to cram in a full sized real desktop onto a screen signi?cantly smaller, “scrap computers” should be spread out across the real desktop. These scrap computers should be comparable to ordinary artefacts that we might ?nd in an of?ce, except for the fact that they should be augmented with computer capabilities to improve their usefulness. Examples ranged from simple notebooks like the now so ubiquitous yellow stickers, to whiteboards that could communicate with other artefacts. Along with other examples, such as doors opening when somebody wearing a correct badge approaches, rooms that greet people with names, and automatic telephone forwarding, the whole world would at some point be populated with these simple computational units, and the full power of ubiquitous computing would be unleashed. From the very beginning of ubiquitous computing, the ?eld has always dealt with distributed systems of some sort. Some of the initial simple artefacts did indeed have communication capabilities, and the ability to distribute information and activities. The way these augmented artefacts could be used to access a digital world is in itself not remarkable or revolutionary. To a large degree, the original vision was mostly a challenge for how we interact with the digital world. This has naturally intrigued the research area of human-computer interaction. A very important realisation within

Ambient Intelligence

17

human-computer interaction is that, when using multi-modal interfaces, the systems have to be sensitive to the context in which the user works. Even though the original envisioned artefacts were in many ways simple, many of them also displayed certain levels of pro-activity, and even reasoning capabilities. Examples include, the systems mentioned above where computers are to greet people, or even know when a meeting is occurring. Weiser sketches out a solution, where simply having access to people’s calendars would lead to knowing that a meeting is occurring. However, relaying solely on peoples calendars to classify something, as a meeting, seems a bit na¨ve. Assuming that calendars are not complete, which is ? very often a safe assumption; it was never really clear how a system would separate a meeting from an arbitrary gathering of people in the same room. However, it is argued that “no revolution in arti?cial intelligence is needed – just the proper embedding of computers into the everyday world” [Weiser, 1991, p.3], since: “If a computer merely knows what room it is in, it can adapt its behavior in signi?cant ways without requiring even a hint of arti?cial intelligence.” [ibid., p. 2]. We will come back to this argument later in this section. Weiser demonstrates a few examples of his view of pure ubiquitous computing. One particular example is an automatic coffee maker, which at ?rst glance appears to be a simple artefact, yet when scrutinised more closely is more complicated. The coffee maker can sense the movements of a person in bed and deduce that he is about to wake up. Knowing that he is waking up, the machine will ask the magic morning word: “Coffee?”, and only knowing “yes” and “no”, will behave accordingly to the answer. Initially, this seems like a trivial task. However, it might not be trivial to decide whether a person is actually waking up, or simply moving about while sleeping. Knowing that this new ubiquitous computing approach has its limitations, Weiser discusses some of the issues that must be resolved before the vision can be attained. As ubiquitous computing is primarily concerned with hardware issues, problems such as power consumption, wireless networking, and miniaturisation are key concerns; but also privacy and user interfaces are important [Weiser, 1993]. This view on ubiquitous computing, as being focused on interconnectivity of simple computer augmented artefacts and access to computerised systems through everyday artefacts, has been the prevailing trend in the area. This trend is clearly evident when we examine the areas of research that have gained the most attention at the main ubiquitous computing conference [Abowd et al., 2001; Borriello and Holmquist, 2002; Dey et al., 2003; Davies et al., 2004; Beigl et al., 2005]. Areas such as perceiving the location of an object or a person, physical infrastructures, user interfaces, and privacy are widely covered. Examples of work on positioning and tracking systems with respect to ubiquitous computing are many, and the techniques vary. A typical example includes use

18

From Ubiquitous Computing to Ambient Intelligence

of radio frequency and ultrasonic signals, as described by Randell and Muller [2001], to position large objects, such as persons, inside rooms. For positioning of more minute items, the work reported by Hoffmann and Scott [2002] is interesting. Here Networked Surfaces not only provide network connectivity, but also allows for an estimation of what devices are in contact with the surface. The commercial available of-the-shelf RFID-tag systems were used by Fishkin et al. [2004] to infer the location of everyday objects in everyday situations, such as books, magazines and TV remote controls. On a much larger scale Krumm et al. [2003] describes positioning of object via FM radio stations. Analogue to the Networked Surfaces described in [Hoffmann and Scott, 2002], Fukumoto and Shinagawa [2005] describes the use of an unconventional wireless LAN, where ?oor surfaces and the human body is used as the LAN; not only for delivering network connectivity but also for positioning. The small sample of techniques described above is a clear indication of the fact that hardware development has advanced dramatically in terms of miniaturisation, high calculation power and interconnectivity. All necessary hardware to realise Weiser’s vision is readily available [Satyanarayanan, 2002]. However, as aforementioned, it is not only research into hardware that facilitates the advancement of ubiquitous computer. Research in different ?elds within computer science also propels us towards our goal. As an example, Satyanarayanan [2001] discusses how the advancements in distributed systems and mobile computing have led to our current level. Even though the term ubiquitous computing and pervasive computing often are used interchangeable, there is a trend towards different meanings. When dealing with the original idea of augmenting ordinary artefact to allow access to the digital world and the interconnectivity between these, the term ubiquitous computing is often preferred. Whereas when investigating systems, architectures, and integration of systems into human surroundings, there is a tendency to use the term pervasive computing. Distribution of capabilities is also a more important aspect in pervasive computing compared to ubiquitous computing. In pervasive computing the focus has expanded, from small more or less autonomous artefacts to complete systems. Advancements in the ?eld of mobile computing has, in particular contributed to the realisation of pervasive computing. These advancements have primarily been within miniaturisation and availability of hardware. Accompanying the increased use of the term pervasive computing, the visions and scenarios have become more vivid. Even though the basic view on simple, small and tangible computer augmented artefacts has survived, a trend towards pervasive systems displaying more complicated behaviour has appeared. One example is the Aura1 system described in the work by Satyanarayanan [2001], where the system often behaves proactively and assumes responsibility from the user, for an example: “Aura
1 http://www.cs.cmu.edu/?aura/

Ambient Intelligence

19

transfers the state of his [the user’s] work from his desktop to his handheld. . . ” [ibid., p. 3]. Compared to the traditional approach, where the user would request a transfer, it is clear that Aura proactively assumes responsibility for the user’s actions. Aura also demonstrates the ability to reason about its own, and its user’s, behaviour, when it is capable of inferring that sensitive information should not be disseminated, since: “. . . the room’s face detection and recognition capability indicates that there are some unfamiliar faces present.” [ibid., p. 3]. All in all, Aura displays a remarkable array of common-sense reasoning and intelligent behaviour (for a more extensive analysis and overview of challenges see Lueg [2002c]). Aura is not the only example of pervasive systems that are envisioned to exhibit such complicated behaviour, however, it is a good example of the move from investigating simple artefacts, to large complicated pervasive systems. Where traditional software applications “know” by design in which situations they are to function, applications in pervasive computing do not necessarily have this luxury. Due to the very nature of the dynamism in the world that pervasive systems like Aura interact with, they have to dynamically adapt their behaviour in run time. To do this, they must be able to somehow interpret the environment in which they are situated. This ability to appreciate the environment is, within pervasive computing, referred to as being context aware. However, as will be described later, there are some unclear issues with the use of the terms context and context awareness, in particular with regard to whether or not context is regarded as data, information or knowledge, and if reasoning is an important part of being context aware. Both the word context and aware implies certain meanings. According to the Oxford dictionary context is: “the circumstance that form the setting for an event, statement, or idea, and in terms of which it can be fully understood and assessed”, and awareness is about: “having knowledge or perception of a situation or fact”. So, the fact that an artefact is considered context aware, implies that it has knowledge about a situation through the terms that describe a situation. How an artefact’s knowledge or understanding of a situation is de?ned is largely subjective, as the knowledge and perception used is unique to the particular artefact. This distinction between a situation as something personal, and context as the terms, or parameters, available when assessing a situation is what Lueg describes in the following way: “A ’situation’ is an observer-independent and potentially unlimited resource that is inherently open to reinterpretation. ’Context’, to the contrary, as an expression of a certain interpretation of a situation is observer-dependent and therefore no longer open to re-interpretation” [Lueg, 2002b, p. 44]. When we use the word aware we are making a strong statement about the capabilities of the entity described. As being knowledgeable is central to being aware, some kind of reasoning capabilities, or intelligence, is implied. Thus, a computer system

20

From Ubiquitous Computing to Ambient Intelligence

that aims at becoming context aware must regard context from the knowledge level [Newell, 1982]. However, many of the projects described here, which implements context aware artefacts can hardly claim to be aware, and the distinction between what we regard as a standard computer program, which transforms input to output, and context aware applications can be hard to spot. One examples is the Active Badge System [Want et al., 1992] that relies solely on location as its input parameter. It should be clear that this application hardly is aware. If we look at robotics, the famous Braitenberg Vehicles [Braitenberg, 1984], where different vehicles exhibit behaviour, such as fear or aggression, are based on simple hardwired connections between sensors and motors. Analogue to the applications relying on a simple set of parameters these vehicles are commonly not referred to as being aware of their environment; or context aware. Where pervasive computing still, to some degree, supports the original thought by Weiser that arti?cial intelligence is totally unimportant for realising the vision, the realisation that reasoning is an important feature when approaching the next generation of systems has lead to the area of ambient intelligence. This term is the result of the IST Advisory Group in European Union (ISTAG) embracing the concepts in pervasive computing and extending them with an explicit focus on intelligence. ISTAG describes ambient intelligence as human beings surrounded by intelligent interfaces, supported by computing and network technology embedded in everyday objects. More importantly, the environment should be aware of the presence of a person, perceive the needs of this person and respond intelligently to them. Finally, an ambient intelligent system should be able to adapt to the needs of the users in a relaxed and unobtrusive manner [Ducatel et al., 2001]. ISTAG2 has been set up by the Commission in the European Union (EU) to advise on the overall strategy to be followed when carrying out the thematic priorities for research on information and communication technology (ITC) within the frame programs of the EU Commission. The term ambient intelligence and its explicit focus on intelligence and reasoning as important means for realising the next generation of computer systems stands in contrast to the original proposal by Weiser, where he explicitly argued that: “no revolution in arti?cial intelligence is needed” [Weiser, 1991, p. 3]. The choice of focus on intelligence in ambient intelligence is closely connected to the scenarios envisioned, and the realisation that to achieve these visions of a brave new world, the requirement of reasoning to interact meaningful with the world has to be tackled. Along with the Aura example by Satyanarayanan [2001], where a huge amount of reasoning and intelligent behaviour is conveyed, the scenarios described in the ISTAG scenarios [Ducatel et al., 2001] equally well expresses high degrees of pro-activity, autonomous
2 http://cordis.europa.eu/ist/istag.htm

Ambient Intelligence

21

Arti?cial Intelligence
(awareness, intelligent response)

Pervasive Computing
(architectural aspects)

Ubiquitous Computing
(access)

Ambient Intelligence

Figure 2.2: Ambient Intelligence Situated (Adopted from Petersen and KofodPetersen [2006b])

behaviour and reasoning capabilities. One of the scenarios in the ISTAG document describes “Maria the Road Warrior” [ibid., p. 4] who is visiting a foreign country that has implemented an ambient intelligent infrastructure. This infrastructure allows her to, among other things, stroll right through the immigration control at the airport since her personal communication device negotiates all identi?cation checks. Her personal agent has also negotiated a good deal with the local car rental agent, and ?nally her hotel room adapts to her personality as she enters. The description of ambient intelligence suggests some necessary requirements to guarantee its success. There must be some technology that in accessible to the user and serve the user in a proactive, yet unobtrusive fashion. Further, some technology must be aware of the user and his needs and respond intelligently to the context and these needs. In this way ambient intelligence can be seen as a combination of ubiquitous computing, pervasive computing and arti?cial intelligence. Figure 2.2 depicts the relationship between these three paradigms, where: the ubiquitous computing aspects address the notion of accessibility to the technology, where access and connectivity is available through everyday objects, which are present in the user’s environment; pervasive computing supports the architectural aspects required to realise the system; and arti?cial intelligence techniques provide the context awareness abilities required

22

The AmbieSense Project

to establish the user’s needs and to respond in an appropriate way. As stated above, the basic technological challenges in pervasive computing, and in that sense also in ambient intelligent systems, have to a large degree been solved. Most basic devices are readily available. However, most of the scenarios presented in the literature still sound like science ?ction. This is largely due to the fact that these scenarios include a large degree of context awareness, inference and decision-making [Lueg, 2002c]. The ever-important area of context awareness, and how representation and reasoning about context and situations affects ambient intelligence, are still largely an open question. The main challenges are concerned with the exact nature of context, the representation of context, how context is related to situations, and how context awareness, context sensitivity and situation awareness are related.

2.2

The AmbieSense Project

As described in the ?rst part of this thesis, the AmbieSense project was a two and a half year EU-funded project within the 6th framework programme (IST-2001-34244), which focused on the development of an ambient intelligent infrastructure. The main objective of the AmbieSense project was to construct an ambient intelligent landscape, where users can receive personalised information. The AmbieSense project had four main areas of interest: i) wireless context tags supplying contextual information as well as content; ii) mobile computers to be used as clients for the users; iii) intelligent agents implementing the context aware personalisation services; and iv) personalised and context oriented information services. The four main scienti?c objectives of the AmbieSense project were: the design and construction on the context tags; making information, such as digital maps, travel guides, and news databases available; pro?ling of users; and combining the three points above. The AmbieSense project’s activities were divided into 11 different work packages (WP). The work presented in this thesis contributed to most of the different work packages, however the main contributions were within WP 6 concerning the intelligent agents for context-based adaptation. The work carried out in this thesis has contributed to most of the work packages in the AmbieSense project. Table 2.1 shows all of the packages, and how many man months were contributed.

2.2.1

The Architecture

The AmbieSense architecture can be dissected into three main components, as depicted in Figure 2.3. These are the context aware environments consisting primarily of the context tag, which acts as the gateway to sensors and actuators in the envi-

Ambient Intelligence WP WP 1 WP 2 WP 3 WP 4 WP 5 WP 6 WP 7 WP 8 WP 9 WP 10 WP 11 Description Functional and non-functional requirements Modelling and design of the AmbieSense technology Context tags technology and wireless networking User context middleware and persistent storage Information sharing and distribution for personalisation, context sensitive use Intelligent agents for context-based adaptation User interface design for the mobile/travelling user Application and components integration Test and evaluation Dissemination and exploitation Project management and technical management Man months 1 4 0 5 1 10 0 0 0 5 0

23

Table 2.1: Distribution of Work Packages ronment. The AmbieSense mobile computer, which is the devices that the users are equipped with, allowing them access to the digitally augmented environment. Finally, the net-based information services, which are supplied through the context tags to the user’s mobile device. The context tag (left box in Figure 2.3) is the primary means of communication between the user and the ambient intelligent environment. The context tag can store and communicate structured data describing the context of its surroundings. It has; furthermore, the ability to communicate locally stored content. The nature of these tags allow them to be mounted, more or less, anywhere: in buildings, in vehicles, in furniture, and outdoors. The context tag is networked with other computers and networks, through wireless or wired LANs, as well as through Bluetooth. The Bluetooth connection is furthermore used to supply location information about user’s mobile devices in the vicinity. However, it is important to note that the location service built into the context tag is to be regarded as an extra ability, since the primary characteristic of the context tag is not a location device. The primary purpose of the context tag is to supply the contextual data used by the rest of the AmbieSense system. As the nature of mobile users dictate that location is important, in the sense of vicinity, it is obviously an important parameter when the context tags transmit contextual data. Any arbitrary sensor can in theory be connected to the tag, thus contextual data can contain any type of sensed data, as well as information already stored in the context tag, and supplied by the users’ devices in the

24

The AmbieSense Project

Context-aware environment

AmbieSense mobile computer
Ambient and intelligent user interface

Netbased information services
Information services

Context tags

Intelligent agents

Personalised and contextaware information sharing and distribution Central user context service

Context tags administration

User context middleware

Sensors in the environment

Effectors in the environment

Local ?le system

Local user context storage

Central ?le system

Central user context storage

Figure 2.3: The AmbieSense System Architecture and Components

vicinity. The contextual data on the context tag can, if required, be communicated through the context middleware to the user’s mobile devices, as well as to net-based information services. The AmbieSense mobile computer (middle box in Figure 1) is an of-the-shelf available portable computer such as mobile phones, PDAs, or smart phones. These devices can access the AmbieSense infrastructure through wireless LANs, GSM/UMTS, or Bluetooth; or any combination of the above. Aside from these communicative capabilities these devices include parts of the AmbieSense software, in particular the Context Middleware, the agency containing the required agents for personalisation, and the user interface. The Context Middleware provides a generic and user-friendly context management infrastructure, which collects and maintains context. The Context Middleware allows applications to utilise relevant context information, paying little or no attention to the details of context management. The net-based information services (right box in Figure 2.3) is an open systemarchitecture for sharing and distribution of information from different services and content providers. This component serves two main purposes. It functions as a hub for adapting and personalising available information. Content providers can make their information accessible to the users through the intelligent agents gathering the information dictated by the user’s context. It furthermore functions as storage for context; in such a way that contextual information is accessible for future use.

Ambient Intelligence

25

User Context

Task Context

Social Context

Personal Context

Spatio-Temporal Context

Environmental Context

Physiological Context

Mental Context

Figure 2.4: Basic Context Meronomy

2.2.2

The Context Model

To facilitate the development of the AmbieSense system, a context meronomy was used as the basis for representation of context. G¨ ker and Myrhaug [2002] ?rst proo posed this basic structure. The context model is divided into ?ve sub-categories: 1. Environmental context: this part captures the users’ surroundings, such as things, services, light, people, and information accessed by the user. 2. Personal context: this part describes information about the mental and physical state of the user, such as mood, expertise, disabilities and weight. 3. Social context: this describes the social aspects of the user, such as information about friends, relatives and colleagues. 4. Task context: the task context describes what the user is doing; it can describe the user’s goals, tasks, activities, etc. 5. Spatio-temporal context: This type of context is concerned with attributes like: time, location and movement. This meronomy depicted in Figure 2.4 build on tradition in the pervasive computing area, where the problem of context models has been viewed as a problem of enumerating possible parameters, and then categorise them. In particular this meronomy builds on the de?nition given by Dey [2001, p. 2]: Context is any information that can be used to characterise the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.

26

The AmbieSense Project

The AmbieSense meronomy extends the above de?nition to comply more tightly with the assumptions made for the AmbieSense project, applying the following de?nition: Context is the set of suitable environmental states and settings concerning a user, which are relevant for a context sensitive application in the process of adapting the services and information offered to a user. [Mikalsen et al., 2003] The motivation for this particular standardisation of context is the need for a combination of facts that motivated much of the work within the AmbieSense project: Users want useful services that are personalised, context sensitive, and extends throughout a user’s life; computers are used as tools for sharing of knowledge and experiences between human users; and users wish to be able to access these services anywhere at anytime; i.e. being mobile. The meronomy depicted in Figure 2.4 deals with the syntactical relationships between the different concepts regarded as context. In that sense it does not stipulate a knowledge perspective on context.

2.2.3

The Context Middleware

The Context Middleware is one of the most important parts of the AmbieSense system. Its main purpose is to acquire and aggregate contextual data from the sources available and supply this information to the rest of the system. The primary responsibility for the development of the Context Middleware was in the hands of Sintef. Some of the work presented in this thesis contributed to the development and design of the Context Middleware. However, Sintef was in charge of the main implementation work, in particular M. Mikalsen. The speci?cations and implementation details has been thoroughly documented in Mikalsen et al. [2003]. The context middleware builds on the assumption put forth in Dey [2001], stating that there are essentially three categories of features that a context aware application can support: i) presentation of information and services to a user; ii) automatic execution of a service for a user; and; iii) tagging of context to information to support later retrieval. The context information available in the Context Middleware is provided from some context sources. Examples of sources are mobile users (through a user interface), applications, software agents and sensors (such as the context tag). Clients may be context sensitive applications or software agents. The sources and clients interact with the Context Middleware using services relevant to their particular needs, without integrating with each other directly. Several sources deliver contextual information

Ambient Intelligence

27

into the Context Middleware. Input from all of these sources must be merged into one coherent context model. The context merging service allows applications to de?ne what parts of a context that can be merged with similar elements from other contexts. Using context templates and the validation mechanisms one can verify that two context representations have the same structure and can safely be merged. Context aware clients use the subscription mechanism of the Context Middleware to indicate that they are interested in noti?cations when context changes. Changes in context are a result of one or more sources publishing new contexts into the context space. Clients can subscribe to several kinds of changes: i) changes of the entire current context, i.e., it is replaced by an entirely new current context; ii) changes in a sub-context in the overall context structure; and iii) changes in the attribute-value level in the context representation (e.g. be noti?ed when location changes in the spatio-temporal context). In Kofod-Petersen and Mikalsen [2005b] it was showed how context templates could be used to create context patterns that constrain contexts and de?ne contextual information relevant within a particular domain. The context template de?nes context structures with valid attributes and values. The validation mechanism of the Context Middleware ensures that a given context instance always is valid toward the context template for a domain. The Context Middleware supports the following high level features: Context template creation; context creation and validation; context storage and retrieval; context publishing and subscription; and context security. The following sub-sections summarise the high-level features of the context middleware. Context templates were introduced to unify the two con?icting goals of representing context in a standardised way, while maintaining the ?exibility of de?ning context models that are relevant for speci?c domains. The context templates de?ne a pattern, which all contexts added to the middleware must comply with. When using a context template a designer may specify structure of the context instances, as a subset of the context meronomy. It is worth noticing that the use of context templates effectively supports the implementation of context sensitive applications, while bypassing the need of being context aware. Context awareness is not always necessary, or even desirable, for all applications. For many areas context sensitivity is suf?cient. Since context templates can constrain the contexts instantiated, it is quite obvious that the Context Middleware allows for creation of new contexts based on any data received from the potential sources. Furthermore, the context middleware also validates any new contexts against the corresponding templates; thus removing that burden from the application developer. The Context Middleware implements a context space [Myrhaug, 2001]. A context

28

The AmbieSense Project

space consists of a context history, a current context and a context future, populated by context instances. The context space abstraction is essential to capture the difference between transient and persistent context [Lei et al., 2002]. Transient context re?ects the environment at a single point in time, whereas persistent context represents a recurrent pattern of a transient context. The fact that Marius eats a Danish pastry at 10:30 in the morning is transient context, whereas the fact that he eats a Danish every morning at 10:30 is persistent context. The current context is the relevant contextual information at the present time. When it is no longer relevant, it can be stored in the context history for future reference. From the context history clients can retrieve contexts using the Context Middleware’s search routines. This separation of context into past and present will allow developers to react to changes in the current situation, i.e. current context. The storing of historical contexts will allow for reasoning resources to draw on the history of a user, thus facilitating the possibility of adapting to patterns in behaviour. Context history can help applications to predict actions and intentions of users based on their previous (stored) contexts. Once deduced, this can be put in the context future (context cache). The current context is the context with corresponding attribute-value pairs currently relevant. Given the generic context de?nition, the nature of the context meronomy, and the domain independent nature of the Context Middleware, there is a risk of in?nite de?nitions of context. To remedy this, context templates are used to create context patterns that constrain context representation and de?ne contextual information in a domain dependent way. Using the context template, application developers de?ne what context structure (contexts with sub-contexts), permissible attributes and valid values a context can have. Context instantiations that comply with the pattern laid out by the context template are valid context instances. Consequently, the context template de?nes what is context, and ef?ciently avoids in?nite de?nitions of context. By using the built in context validation tool of the Context Middleware, one has the measures to ensure that a given context instance is valid toward a context template. As mentioned above, tagging of context to information for later retrieval has been de?ned as a core task for context aware applications. This is explicitly supported by the Context Middleware, allowing application developers to create links from contexts, attributes and values, to relevant information. As a context is entered into the context space it can be linked to a relevant information entity. When this context is retrieved the information entity can also be retrieved. As context ?ows into a context space from various sources, applications need a mechanism to integrate this information into one common context representation. The context merging service allows applications to de?ne what parts of a context that are subject to change, and can be merged with other context representations. Using

Ambient Intelligence

29

the validation mechanism and a context template, one can assure that two context representations have the same structure and attribute-value pair, and consequently can be merged correctly.

2.2.4

The Agency

The use of intelligent agents in an agency was also stipulated in the AmbieSense description of work [AmbieSense, Annex 1]. This work was lead by CognIT. However, contrary to the Context Middleware, much of the work presented in this thesis did actively contribute to all aspects of the agents and the agency. Thus most of the description of the agency can be found in Section 6.2. One of the prime premises set by the AmbieSense project was the fact that the whole system was to be implemented in Java, using XML as the primary tool for data exchange between modules. Since one of the primary goals of the project was to run the system on as many different devices as possible, while maintaining code portability, the Java platform was chosen. It supports, more or less, all desirable platforms, from large serves using the Enterprise Edition (J2EE), through personal computers (J2SE), PDAs (J2ME), down to smart cards. The same argument holds for the choice of XML. XML has become the de facto standard for exchanging data between applications. The speci?c agent platform chosen was the Jade platform.

2.3

Context

As described in Section 2.1, context awareness is the term most often used to describe an ambient intelligent system’s ability to adapt to the environment, thus facilitating the intelligent response to users’ needs. To approach a useable de?nition of context and describe its use, an account of the different approaches is warranted. The concept of context is a closely related to reasoning and cognition in humans. Even though context might be important for reasoning in other animals, it is commonly agreed that context is of huge importance in human reasoning. Beside the more mechanistic view on reasoning advocated by neuro-science, psychology and philosophy play important roles in understanding human cognition. Humans use an abundance of more or less subtle cues to de?ne the context. A human, as any agent, is situated in the environment [Brooks, 1991], and is even inseparable form the environment [Gibson, 1979]. Humans are also able to perceive their environment through a variety of sensors, such as sight, smell and touch, as well as capable of a large amount of ?exibility in interpretation of these sensors. Thus, humans are capable of reasoning about perceived information, and use this information as context when assessing situations, and ultimately decide how to behave in certain situations. However, as

30

Context

humans are individuals, context and situation assessment is subjective and can be regarded as collective tacit [Polanyi, 1964] knowledge [Br? zillon and Pomerol, 2001]; e that is, knowledge regarding assessment of situations belongs to somebody and is not shared, objective or public. If we regard an ambient intelligent system as an entity situated in the environment, it should also have the ability to appreciate its environment through its sensors. Hence, from the perspective of the machine it should have the capability to infer what situation it is in, based on its contextual inputs. However, compared to humans, computers are equipped with a more limited range of sensors. Thus, a machine can, compared to humans, only have a limited awareness of its context. Yet, it is important to note that, as with humans, a system’s understanding of context and its assessment of a situation is also subjective. This view is somewhat contrary to the view put forth by Erickson [2002] where he states that the nature of context awareness in computer systems is fundamental different from that of humans. Greenberg [2001] supports the subjectivity of situation assessment, by arguing that similar looking contextual situation may differ due to personal experiences and internal state. He also argues that it is (most likely) impossible to enumerate the set of contextual states that exists. Yet, he later argues that an arti?cial system can at best make educated guesses about the real context [ibid., p. 262]. This point is problematic as it, ?rst and foremost, assumes that there is an objective, global and correct de?nition of a given current context, much in the same way as Aristotle argues for Episteme [Aristole, Nicomachean Ethics], which he de?nes as knowledge that is objective, non-dogmatic, and orderly. This view also indirectly contradicts the idea that it is impossible to enumerate the set of contextual states that exists. Secondly, Greenberg states that a system can only make approximations, implicitly stating that humans are correct in their assessment of context. As described in Section 2.1, arti?cial intelligence techniques are the enabling technology for context awareness in ambient intelligent systems. The ?eld of arti?cial intelligence has close relations to the study of reasoning in the real world, most prominently psychology and philosophy. Since arti?cial intelligence and psychology are closely related, and context is an important aspect of human reasoning, context also plays an important role in the understanding and implementation of arti?cial intelligence. Arti?cial intelligence has historically been closely connected to formal logic. Formal logic is concerned with explicit representation of knowledge. This leads to the need to codify all facts that could be of importance. This strict view on objective truth is also known in certain directions within philosophy, where such a concept as knowledge as an objective truth exists. As aforementioned, this view springs from the father of logic, Aristotle, who believed that some subset of knowledge had that characteris-

Ambient Intelligence

31

tic (Episteme). This view stands in stark contrast to the views advocated by people such as Polanyi, who argues that no such objective truth exists and all knowledge is at some point personal and hidden (tacit) [Polanyi, 1964]. Since context is an elusive type of knowledge, where it is hard to quantify what types of knowledge are useful in a certain situation, and possible why, it does not ?t very well with the strict logical view on how to model the world. According to Ekbia and Maguitman [2001] this has led to the fact that context has largely been ignored by the arti?cial intelligence community. The paper by Ekbia and Maguitman is not a recipe on how to incorporate contextual reasoning into logistic systems, rather an attempt to point out the de?ciencies and suggest possible directions arti?cial intelligence could take to include context. The work by Ekbia and Maguitman builds on the work by the American philosopher John Dewey (1859-1952). According to Ekbia and Maguitman, Dewey distinguishes between two main categories of context: spatio and temporal context, coherently know as background context, and selective interest. The spatio context covers all contemporary parameters. The temporal context consists of both intellectual and existential context. The intellectual context is what we would normally label as background knowledge, such as tradition, mental habits, and science. Existential context is combined with the selective interest related to the notion of situation. A situation is in this work is viewed as a confused, obscure, and con?icting thing, where a human reasoner attempts to make sense of this through the use of context. This view, by Dewey, on context leads to the following suggestion by the pragmatic approach [ibid., p. 5]: 1. Context, most often, is not explicitly identi?able. 2. There are no sharp boundaries among contexts. 3. The logical aspects of thinking cannot be isolated from material considerations. 4. Behaviour and context are jointly recognisable. Once these premises have been set, the authors shows that the logical approach to (arti?cial) reasoning has not dealt with context in any consistent way. The underlying argument is that arti?cial intelligence has been using an absolute separation between mind and nature, thus leading to the problems associated with the use of context. Instead, the authors argue a view on the inseparability of mind and nature, which is also based on Dewey’s work. This view is not unique the Dewey. In recent years Gibson [1979] has proposed this in ecological psychology and Brooks [1987, 1991] as situatedness in robotics. Through the discussion of different logical based arti?cial intelligence methods and systems, the authors argue that arti?cial intelligence has not yet parted company with the limitations of logic with regard to context. Furthermore, they stress

32

Context

the point of intelligence being action-oriented; based on the notion of situations described above. Many people within the computer science milieu share this notion of intelligence being action-oriented, thus making context a tool for selecting the correct action: Most notably the work by Strat [1993] where context is applied to select the most suitable ¨ u algorithm for recognition in computer vision, and by Ozt¨ rk and Aamodt [1998] who utilises context to improve the quality and ef?ciency of case-based reasoning. Strat [1993] reports on the work done in computer vision to use contextual information in guiding the selection of algorithms in image understanding. When humans observe a scene they utilises a large amount of information (context) not captured in the particular image. At the same time, all image-understanding algorithms use some assumptions to function. Examples are algorithms that only work on binary images, or not being able to handle occlusions. Strat de?nes three main categories of context: physical, being general information about the visual world independent of the conditions under which the image was taken; photogrammetric, which is the information related the acquisition of the image; and computational, being information about the internal state of the processing. The main idea in this work is to use context to guide the selection of the image-processing algorithms to use on particular images. This is very much in line with the ideas proposed by Ekbia and Maguitman, where intelligence is action-oriented, and context can be use to bring order to diffuse situations. ¨ u Ozt¨ rk and Aamodt [1998] demonstrate the use of context to improve the quality and ef?ciency of case-based reasoning. They also advocate an action-oriented view on reasoning and use of context by arguing that the essential aspects of context are the notion of relevance and focus. To facilitate this improvement to case-based reasoning a context model is constructed. This model builds on the work by Hewitt, where the notion of intrinsic and extrinsic context types is central. According Hewitt3 , intrinsic context is information related to the target item in a reasoning process, and extrinsic is the information not directly related to the target item. This distinction is closely related to the concepts of selective interest and background context as described by Dewey. The authors re?ne the view by focusing on the intertwined relationship between the agent doing the reasoning, and the characteristics of the problem to be solved. This is exactly the approach recognised as being missing in arti?cial intelligence by Ekbia and Maguitman. ¨ u Ozt¨ rk and Aamodt [1998] build a taxonomy of contextual categories based on this merger of the two different worlds of information (internal vs. external). Beside
3 Hewitt’s original work Context Effects in Memory: A review, Cambridge University Psychological University, is inaccessible as it is an unpublished manuscript. However, it has, among others, been referred to in [Baddeley, 1982] and [Tiberghien, 1986]

Ambient Intelligence

33

this categorisation, the authors impose the action, or task, oriented view on knowledge in general, and contextual knowledge in particular. The goal of an agent focuses the attention, and thereby the knowledge needed to execute tasks associated with the goal. The domain used in this paper is medical diagnostics, where a doctor attempts to diagnose a patient by the hypothesise-and-test strategy. The particular method of diagnostics in this case-based reasoning system is related to the strategy used by Strat. Though with the minor modi?cation that Strat used contextual information to select ¨ u the algorithms to use, whereas Ozt¨ rk and Aamodt have, prior to run-time, de?ned the main structure of a diagnostic situation, and only uses context to guide the sub-tasks in this process. Finally, Zibetti et al. [2001] focus on the problem of how agents understand situations based on the information they can perceive. Their work is the only one that does not attempt to build an explicit ontology on contextual information prior to run-time. The idea is to build a (subjective) taxonomy of ever-complex situations solely based on what a particular agent gathers from the environment in general, and the behaviour of other agents in particular. The implementation used to exemplify this approach contains a number of agents “living” in a two-dimensional world, where they try to make sense of the world by assessing the spatial changes to the environment. Obviously the acquisition of knowledge starting with a tabula rasa is a long and tedious task for any entity. To speed up the process the authors prede?nes some categories with which the system is instantiated. All in all, this approach lies in between a complete bottom-up and the more top-down approaches described earlier.

2.3.1

Context in Ambient Intelligence

As described in Section 2.1, the ability to appreciate the environment and thus adapt to this is, within ambient intelligence, referred to as being context aware. One of the major challenges in context awareness is the lack of a common agreement on what context really constitutes. Where the preceding section described some of the work on the nature of context, a lot of work within context awareness goes into de?ning what context and context parameters are, and little work into analysing the situation in which the applications are to function. Dey and Abowd [2000] describe the different de?nitions of context available in the literature. The authors distil the different uses into a very ?exible de?nition: Context is any information that can be used to characterize the situation of an entity. An entity is a person, place or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.

34

Context

Parallel to the analysis by Dey and Abowd, Chen and Kotz [2000] investigate the use of context in several available systems. Some of the investigated work was overlapping, thus they unsurprisingly arrived as quite similar de?nitions. The de?nition by Chen and Kotz is as follows: Context is the set of environmental states and settings that either determines an application’s behavior or in which an application event occurs and is interesting to the user. It is obvious that whether or not the word situation is explicitly used, the two de?nitions are very alike. Furthermore, the information referred to by Dey and Abowd is closely related to, if not equivalent to, the use of environment by Chen and Kotz. The main difference between these two works is the fact that Dey and Abowd extend their de?nition and de?ne very speci?c categories of contextual information based on what they perceive as important. These categories are: location, identity (of the user), activity (of the user), and time. The rationale for the categorisation is practical; these categories are, according to the authors, the ones that are most useful in a mobile context aware setting. Since we strive to build context aware applications, pragmatic choices are important. In this regard, theoretical-pure solutions must often be substituted for feasible ones (e.g. see the example by Zibetti et al. [2001] above), however it can be counter productive to de?ne parameters as always being context. When parameters a priori are being de?ned as always being context it leads us down the objective-truth path of Episteme. Br? zillon and Pomerol [1999], among others, argue that a particular e kind of knowledge can be considered context in one setting and domain knowledge in another, or in their own words: “. . . knowledge that can be quali?ed as ’contextual’ depends on the context!” [ibid., p. 7] To exemplify this view we can look at the following quote: “For example, in a context-aware tour guide, a user carrying a handheld computer approaches some interesting site resulting in information relevant to the site being displayed on the computer” [Dey and Abowd, 2000, p. 4]. In this example location is viewed as belonging to a particular context category. More or less along the same line as any other type of contextual information, perhaps with the exception of importance. From a pragmatic and top-down conceptual view it could be argued that location is part of the context model for the particular system, thus making it context. However, in this example location is of paramount importance. If no location is supplied to the system there is no way it can execute the (possible) function show relevant information(location). In this example it would be more feasible to look at location as a classical input parameter and any other types of available information as context. This view of contextual parameters being de?nable a priori is a common approach in context aware pervasive computing. Much of the work carried out focuses on a few

Ambient Intelligence

35

common parameters, which easily could be regarded as classic input parameters in computer science. As part of the work in this thesis an investigation into the use of the term context has been conducted. Table 2.2 gives a summary of the work in context aware pervasive computing surveyed. When investigating the table it becomes clear that location (of the user) is the preferred parameter de?ned as context. The only two exceptions to this are: the work by Abowd [1999], where the object of interest was a classroom, and therefore the location of the participants could not change; and the work by Yan and Selker [2000] where a context sensitive doorman is installed at of?ce doors. Again, as this is not a mobile system so no concept of location is required. Most of the applications surveyed employ an ad hoc approach to de?ning context. The choice of what parameters that make up the context is based on what is required as input parameters; much in line with the example from Dey above. This approach is founded on the wish to construct speci?c applications that solve speci?c problems; thus, no well-founded model of context, situations, and awareness is demanded. Examples of this ad hoc procedure include Schilit and Theimer [1994], where context encompasses location, identity of nearby entities, and changes to these entities. The stick-e document system described in Brown [1996] and Pascoe [1997] employs a hierarchy model to describe the: “. . . many different types of context that a computer can utilize: location, presence of objects or people, temperature, blood pressure of the user, etc.” [Pascoe, 1997, p. 1], without explicitly describing how to distinguish what is context and when. The de?nitions, along with most others utilised, de?ne context by giving examples, which is not very useful when trying to decide if a particular piece of information not listed can be regarded as context or not. Exactly this problem was one of the main aspects debated in Anind K. Dey’s Ph. D. work. This led to the de?nition described above and to the implementationoriented view on context presented in the Context Toolkit [Dey, 2000]. The Context Toolkit builds on the idea from graphical user interface (GUI) production, where users communicate with an application through typically keyboard and mouse. To facilitate transparency for the application developer it is rare that applications communicates directly with these peripheral devices, they often communicates through widgets, which normalises communication. This idea is transferred into the world of context aware systems. The Context Toolkit de?nes an architecture consisting of the widgets, servers and interpreters. The widgets are responsible for connecting to the physical sensors available. They offer the same bene?ts as GUI widgets, such as: separation of concern, reuse, and easy access to data, as well as a common interface [Dey et al., 1999b, p. 4]. In a pervasive computing environment several types of data will be available concerning an entity. To collect all information about a certain entity from all the different

36

Name x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x

a

b

c

d

e

f

g

h

i

j

k

l

m n

o

p

q

r

s

t

u

v

Reference

x x

x x x x x x x x x x x x x x

Table 2.2: Context Aware Applications Survey
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x

Active Map Adaptive GSM phone and PDA Augmentable Reality Bat Call Forwarding Classroom 2000 Conference Assistant Cooltown Cyberdesk Cyberguide Enhanced Reality Fieldwork Fieldwork Guide Healthcare Healthcare Listen Location-aware Information Delivery Mobisaic Web Browser MyCampus Netman Of?ce Assistant SeTrigger Shopping Assistant Smart Sight Stick-e document Take me with you Teleporting WearNet

[Schilit and Theimer, 1994] [Schmidt et al., 1999a] [Rekimoto et al., 1998] [Harter et al., 1999] [Want et al., 1992] [Abowd, 1999] [Dey et al., 1999a] [Barton and Kindberg, 2001] [Dey, 1998] [Long et al., 1996] [Ryan et al., 1998] [Pascoe, 1997] [Cheverst et al., 2000] [Jahnke et al., 2005] [Mu? oz et al., 2003] n [Zimmermann, 2003] [Marmasse and Schmandt, 2000] [Voelker and Bernshad, 1994] [Sadeh et al., 2005] [Kortuem et al., 1998] [Yan and Selker, 2000] [Brown et al., 1997] [Asthana et al., 1994] [Yang et al., 1999] [Brown, 1996] [Fujinami et al., 2004] [Bennett et al., 1994] [Lukowicz et al., 2002]

Context

Ambient Intelligence a b c d e f g h i j k l m n o p q r s t u v Identity User location User preferences Terminal location Time Activity Schedule Nearby People Nearby Objects Light level Pressure Friends Classmates State User state Season Temperature Others Physical Social Mental Emotional

37 The identity of the user The location of the user The user’s preferences The location of the terminal, which is not necessarily the same as the user The time of day The activity of the user The schedule of the user People physical located nearby the user Objects physical located nearby the user The level of lighting, or Illuminance Pressure applied to the portable device Known friends of the user Known classmates of the user The state of the system The state of the user The season The temperature Other parameters not included above Category Category Category Category

Table 2.3: Explanation of the Abbreviations used in the Survey

widgets, a Context Server is implemented. This is responsible for acquiring all data for a speci?c entity, most often a user. The idea is that the application developer should be relieved from the tedious work of enquiring all the widgets about an entity; he should rather focus on the work to be done regarding the data available. Beside the widget and the server a Context Interpreter exists. This is a module that allows easy implementation of reasoning capabilities on the data available from the widgets and the servers. There is really no good information available on how this reasoning is to be accomplished. In the Context Toolkit it is regarded as a black-box reasoner. The two examples presented [Dey et al., 1999b], concerns a simple problem of mapping a room location into a building location, and a more complex one that identi?es meeting situations based on location, identity, and sound information.

38

Context

All in all, the Context Toolkit can most easily be regarded as a middleware layer, which handles acquisition and (limited) interpretation of contextual data. In this sense it is in the same tradition as other middleware solutions. Along the same line of middleware thinking, Capra et al. [2001] designed a system for aggregation of context. They describe how a marriage of re?ection and metadata is a suggested means to create a middleware that give applications dynamic access to information about their execution context. In their view, middleware is seen as a network middleware. Network middlewares sit on top of a network operating system and provide application developers with higher levels of abstractions, hiding complexities introduced by distribution. This is conceptual close to the Context Toolkit where a distributed network of contextual sensors is made available in a coherent and transparent way. Network middlewares have been designed and worked successfully on stationary computers, but they do not appear to be suitable for the mobile setting for the following reasons: Firstly, the interaction primitives assume high-bandwidth connection between components, as well as constant availability. This is not the case in mobile systems, where unreachability and low bandwidth is the norm rather than the exception. Secondly, completely hiding implementation detail from applications makes little sense. Mobile systems need to detect and adapt to drastic change occurring in the environment, such as battery power. If we have complete transparency, the middleware needs to make decisions on behalf of the application. Applications however, make more ef?cient and better quality decisions based on application speci?c information. Re?ection and metadata are used to build the systems that support context aware applications. Applications pass metadata to the middleware. This metadata constitutes a policy as to how the applications want the middleware to behave as a result of a speci?c context occurrence. As context and applications need changes continuously, one cannot assume that metadata is static, therefore applications use re?ection mechanisms offered by the middleware to inspect their own metadata, and possibly alter it according to changing needs. Where the above mentioned systems have a data perspective on context, Chen [2004] approaches the context problem form a knowledge perspective. He describes the Context Broker Architecture (CoBrA), where an intelligent broker agent maintains a shared model of the context data available. This model also assumes responsibility for aggregating available contextual data. The main difference between this model and the Context Toolkit is that the CoBrA also includes a knowledge model of the contextual information available, and in that sense facilitates more distributed reasoning capabilities for agents wishing to use the CoBrA.

Ambient Intelligence

39

2.3.2

Socio-technical Approaches to Context

When we examine the different views on context, both within the world of arti?cial intelligence and ambient intelligence, some common issues stand out. Initially, context is some knowledge related to the ability of making sense of the surrounding world. Thus, allowing entities to modify their behaviour to suit the situations that they experience. The exact nature of this type of knowledge is hard, if not impossible to pinpoint. This has been claimed in the work ranging from Dewey who argues that context is not explicitly identi?able and has no sharp boundaries (see p. 31), via Br? zillon and Pomerol who states that contextual knowledge depends on the context e (see p. 34), to the work presented in ambient intelligence, by Dey and Abowd, and Chen and Kotz, which states that any information used to characterise a situation is regarded as context (see p. 33). Yet, two points stand out: actions or activities are of supreme importance when understanding situations, thus as context is used to make sense of situations, it should also be closely related to activities; and it is possible to categorise the knowledge that are to be used as context. The integration of intelligent systems into workplace environments marks a shift from mere tool usage to partnership between humans and intelligent artefacts. It furthermore fosters the need for social adept systems [Marsh, 1995]. The ability to understand situations and knowing that different situations require different behaviour is paramount — in particular when entering partnerships with human users and interacting with them socially. As the research into context aware systems have been largely technology driven, not a lot of attention has been given to analysing context from socio-technical perspective [Lueg, 2002a]. However, if our systems are to play a role in a social setting an analysis of the social aspects of situations are needed. Further, one of the most important aspects of any situation, namely the activity, has largely been ignored. As activities are central to occurring situations, an ambient intelligent system must have some means of representing situations and activities, and even being aware of occurring situations. Parallel to the fact that psychology and philosophy have been sources of inspirations for work on context in ambient intelligence, they have also in?uenced work on understanding activities and situations. The area of humancomputer interaction has been one of the more in?uential sources for situation understanding in ambient intelligence. Much of the work in human-computer interaction regarding context and situation understanding has been grounded in cognitive science. Several approaches to investigate activities have been proposed: such as Actor-Network Theory [Latour, 1988], or the Locales Framework [Fitzpatrick, 1998]. Yet, some of the most interesting approaches are Situated Actions [Suchman, 1987] and Activity Theory [Leont’ev, 1978; Vygotsky, 1978]. Common to both of these approaches is the fact that none of them

40

Context

were developed with arti?cial cognition in mind. Situated Actions springs from an anthropological approach to understand human learning and work. Whereas, Activity Theory emerges from psychology, where the understanding of development and learning is central. Situated Actions deal with the relationship between observable behaviour and the hidden processes that make this behaviour meaningful. It is not only of interest in psychology or social studies, but also of particular interest when dealing with machine intelligence. Situated Actions are focused on the continuous and emergent character of human activity [Suchman, 1987]. Opposite the formal studies of cognitive properties of artefacts, this approach to activity is situated, in the sense that activities are viewed as inseparable from interaction between actors in an environment. This view perpetuates the notion that saturates ecological psychology where, in the words of Gibson [1979], animals are inseparable from the environment and each term implies the other. Equally so, a relationship exists between this view of situated actions and that of situatedness robotics, which is argued by researchers such as Brooks [1991]. The argument put forward in Situated Actions is that every course of action is a construct of its material and social circumstances. Thus, the actual planning is made largely in a bottom-up fashion, where an objective and a priori de?nition of situations and plans are inconceivable. Suchman points out ?ve propositions on purposeful action and shared understanding [Suchman, 1987, p. 50]: 1. Plans are representations of situated actions. 2. In the course of situated actions, representation occurs when otherwise transparent activity becomes in some way problematic. 3. The objectivity of the situation of our action is achieved rather than given. 4. A central resource for achieving the objectivity of situations it language, which stands in a generally indexical relationship to the circumstances that it presupposes, produces, and describes. 5. As a consequence of the indexicality of language, mutual intelligibility is achieved on each occasion of interaction with reference to situation particulars, rather than being discharged once and for all by a stable body of shared meanings. The focus by Suchman is on bottom-up construction of plans in situ. She argues that traditional planning ahead is only one of the possible sources of knowledge people can use when ?guring out what to do next. Suchman places a strong emphasis on careful examination of speci?c situations, rather than pursuing theoretical principles. The exact same approach is argued by activity theory, were the focus lies on individual and collective work practise. Activity theory is a descriptive psychological

Ambient Intelligence

41

Artefact

Subject

Object

Figure 2.5: Activity Theory: The Basic Triangle of Mediation. framework that is to help us in understanding the unity of consciousness and activity. One of its strengths is the ability to identify the role of material artefacts in the work process, thus supplying a framework that allows for modelling and representation of activities. Where situated actions have a strong focus on the actions as behaviour, activity theory deals primarily with the role of artefacts as mediators in actions. An activity (Figure 2.5) is composed of a subject, an object, and a mediating artefact or tool. A subject is a person or a group engaged in an activity. An object is held by the subject and motivates activity, giving it a speci?c direction. The foundation is a set of basic principles. These guiding principles, as described by Bannon and B?dker [1991] include: Hierarchical structure of activity Activities (the topmost category) are composed of goal-directed actions. These actions are performed consciously. Actions, in turn, consist of non-conscious operations. Object-orientedness Objective and socially or culturally de?ned properties. Our way of doing work is grounded in a praxis, which is shared by our co-workers and determined by tradition. The way an artefact is used and the division of labour in?uences the design. Hence, artefacts pass on the speci?c praxis that they are designed for. Mediation Tools, language, etc. mediate human activity. The artefacts as such are not the object of our activities, but appear already as socio-cultural entities. Continuous Development Both the tools used and the activity itself is constantly reshaped. Tools re?ect accumulated social knowledge; hence they transport social history back into the activity and to the user.

42

Context

Distinction between internal and external activities Traditional cognitive psychology focuses on what is denoted internal activities in activity theory, but it is emphasized that these mental processes cannot be properly understood when separated from external activities, that is the interaction with the outside world. While these principles describe a general model, more speci?c theories deal with different aspects of human activities. Unifying these theories is the insight that human beings live in an objective reality where the artefacts, social relationships, governing rules etc. always carry social and cultural properties. Taking a closer look on the hierarchical structure of activity, we can ?nd the following levels: Activity This is the topmost level. An individual activity is for example to check into a hotel, or to travel to another city to participate at a conference. Individual activities can be part of collective activities, e.g. when someone organises a workshop with some co-workers. Action Activities consist of a collection of actions. An action is performed consciously, the hotel check-in, for example, consists of actions like presenting the reservation, con?rmation of room types, and handover of keys. Operation Actions themselves consist of collections of non-conscious operations. To stay with the hotel example, writing your name on a sheet of paper or taking the keys are operations. That operations happen non-consciously does not mean that they are not accessible. It is important to note that this hierarchical composition is not ?xed over time. If an action fails, the operations comprising the action can get conceptualised, they become conscious operations and might become actions in the next attempt to reach the overall goal. This is referred to as a breakdown situation. In the same manner, actions can become automated when done many times and thus become operations. In this way, we gain the ability to model a change over time. A basic notion of activity theory is that the subject participating in an activity does so because he wants to achieve a certain goal. The subject’s interest is directed towards the object of an activity, which he tries to use and modify to achieve an anticipated outcome. Tools, creating the basic triangle of Subject, Object, and Mediating Artefact, mediate his interaction with this object. Since we consider social activities, the acting subject is part of a community. The relations between the acting subject and the community as well as between the community and the object are mediated by a set of rules and the division of labour (since the desired outcome is anticipated to be shared by the community, a solitary view on

Ambient Intelligence

43

Artefact

Subject

Object

Rules

Community

Division of Labour

Figure 2.6: Cultural Historical Activity Theory

the relation between one subject and the object would miss important aspects). An expanded model of activity theory, Cultural Historical Activity Theory (CHAT), covers the facets of social and cultural context. The expanded model takes these aspects into account by adding a community component and other mediators, especially rules (an accumulation of knowledge about how to do something) and the division of labour (see Figure 2.6). In order to be able to model that several subjects can share the same object, the community is used to represent that a subject is embedded in a social context. Now a relationship between subject and community and between object and community exists. These relationships are themselves mediated, with rules regarding the subject and the division of labour regarding the object. Activity theory has for some time been attracting interest in the human-computer interaction ?eld. One of the main advocates for the use and study of activity theory in this context, argues the this is exactly what the human-computer interaction ?eld needs, as: “Activity theory offers a set of perspectives on human activity and a set of concepts for describing that activity. This, it seems to me, is exactly what HCI research needs as we struggle to understand and described ’context,’ ’situations,’ ’practice’ ”[Nardi, 1996, p. 8]. Looking at the design of IT-systems in general, we ?nd that activity theory has been applied to many different areas of system development. For example, activity

44

Context

theory was used in health care settings as a tool to support development of information systems [Korpela et al., 2001]. It has also been used in the design of augmented reality systems, as reported by Fjeld et al. [2002], and for a posteriori analysis of computer systems in use [B?dker, 1996]. A comparative survey of ?ve different activity theory based methods for information systems development with pointers to additional examples was conducted by Quek and Shah [2004]. Recently, activity theory has also received some attention within the pervasive computing research community. Li et al. [2004] propose an activity based design tool for context aware applications. The authors’ focus lies not on the use of activity theory in the context model itself, but on supporting the designer of context aware applications with a rapid prototyping tool. An interesting idea is the proposed integration of temporal probabilistic models. Wiberg and Olsson [1999] also make use of activity theory, where their focus on the design of context aware tangible artefacts. Their usage situation is well de?ned a priori and no reasoning about the context is required. Kaenampornpan and O’Neill [2004] report on their work focusing on modelling features of the world according to an activity theoretic model. The authors do not carry out a knowledge level analysis of the activities. However, Kaenampornpan and O’Neill further on develop a notion of history of context in order to elicit a users goals [Kaenampornpan and O’Neill, 2005]. This work also deals with the interesting problem of representing the user’s history in context models.

2.3.3

Context Categories

With the inspiration from situated actions and activity theory, the tools supporting an activity centred view on situations, and thus context is now available. Activity theory can be used to capture and model contextual knowledge regarding the person involved, or subject, the object in question, the community contributing to achieving the objective of the activity, and the mediating artefact, rules and division of labour. These dimensions of a situation can make signi?cant contributions to the structure of a knowledge model for ambient intelligent systems. As was stated earlier, beside the focus on activities, most work on context argues that some sort of categorisation of contextual knowledge is possible, event though the exact use of the knowledge might differ from situation to situation. As described earlier, the selection of what knowledge to be regarded as context has often been ad hoc based, or even arbitrary. Yet, even though the exact knowledge used differ across situations, it is possible to categorise it. The idea of categorisation without committing to speci?c use underlines the idea behind the AmbieSense context meronomy. The AmbieSense project builds around a context meronomy, which is based on the traditional view projected in pervasive computing. The meronomy is

Ambient Intelligence

45

divided into ?ve main sub-categories of context. Figure 2.4 on p. 25 depicts the context meronomy as originally described in G¨ ker and Myrhaug [2002] as part of o the AmbieSense project. The ?ve categories are: Environmental context This part captures the user’s surroundings, such as things, services, light, people, and information accessed by the user. Personal context This part describes the mental and physical information about the user, such as mood, expertise, disabilities and weight. Social context This describes the social aspects of the user, such as information about friends, relatives and colleagues. Task context the task context describe what the user is doing, it can describe the user’s goals, tasks, activities, etc. Spatio-temporal context This type of context is concerned with attributes like: time, location and movement. This meronomy uses the de?nition by Dey and Abowd [2000], as described earlier, and extends it to comply more tightly with the assumptions made for the AmbieSense project, applying the following de?nition: Context is the set of suitable environmental states and settings concerning a user, which are relevant for a context sensitive application in the process of adapting the services and information offered to a user. The motivation for this particular standardisation of context is the need for a combination of facts that motivate much of the work within the AmbieSense project: Users want useful services that are personalised, context sensitive, and extends throughout a user’s life; computers are used as tools for the sharing of knowledge and experiences between human users; and users wish to be able to access these services anywhere at anytime, i.e. being mobile. If we examine the ?ve different categories, we will observe that the parameters traditionally used in pervasive computing can be captured within categories such as environmental context and spatio-temporal context. Whereas, the components emphasised by an activity perspective on context can be encapsulated inside the categories of social context and task context. Finally, the idiosyncrasies of the user, which is important for ambient intelligent systems to achieve their goal of personalised services can be contained within the personal context. It should now be clear that the notion of context awareness implying more than just being sensitive to one (or more) parameters. This has also been discussed in

46

Situation Awareness

work such as Schmidt et al. [1999b] where an argument against the, at the time, de facto view that location and context are more or less synonymous, and Lueg [2002a] that discusses the reasons and issues concerning the design of truly context aware artefacts. One of the main reasons for this focus on context sensitive applications is that the research often is technology driven. It is ”... driven by what is technically feasible rather than by what might be helpful in a situation.” [Lueg, 2002b, p. 1] More recently Barkhuus [2004] has investigated the ”context gap”, which discusses the gap between the sensor-derived technical representation of context and the social human perception of context.

2.4

Situation Awareness

The ability to perceive and adapt to the user’s needs are the most important abilities of an ambient intelligent system. So far, it has been argued that to achieve this, a system must be able to perceive the world, structure the acquired knowledge coherently, and reason about the context to make sense of an occurring situation, and that this ability to make sense of the world traditionally has been referred to as being context aware. When a system is to take in the entire situation, and possibly be an active participant in it, it could be feasible to substitute context awareness with the more powerful concept situation awareness. Situation awareness is a theory that original grew out of the wish to help pilots maintain the knowledge of situations when piloting an airplane, while minimising the cognitive load. The focus has mostly been on combat pilots, where the stress level and amount of information processed is very large. Recently a book by Endsley et al. [2003] has expanded the focus to include a more general view on how to design computer applications with situation awareness in mind. Situation awareness is not considered a technology, it is primarily regarded as an architectural framework for designing systems that are capable of managing complex situations occurring in a real environment, including sentient entities. Situation awareness as a concept is of great importance to realising ambient intelligence. To achieve proactive systems that exhibit intelligent behaviour and can response to situations in a meaningful fashion, the ability to perceive and comprehend the environment is a prerequisite. Revisiting the scenarios described in Section 2.1 where the Aura system infers what behaviour to exhibit [Garlan et al., 2002, p. 2], it is clear that some model of the world must exists, and awareness of situations are paramount. It is equally true that the same capabilities to perceive and be aware of situations are required to facilitate the behaviour demonstrated by Maria’s P-Comm in Ducatel et al. [2001]. Even though situation awareness, as described by Endsley et al. [2003], deals with more conventional computer systems, there is no evident reason as

Ambient Intelligence

47

to why it should not be capable of covering ambient intelligent systems as well. Several de?nitions of situation awareness have been given, most have had a very strong focus on aircrafts and aerial combat. Tolk and Keether [1982] de?ne it as the ability to predict future behaviour of airplanes and surface threats. Whitaker and Klein [1988] look at pilots’ knowledge about the surroundings. Mckinnon [1986] and Harwood et al. [1988], who enumerate the types of knowledge a pilot possess, give more a speci?c de?nitions. Endsley [1988] gives the most general and less pilot speci?c de?nition, where situation awareness is all about what is occurring around an entity and the understanding of the future implications. Situation awareness is formally de?ned as: The perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future. [Endsley, 1988] For an overview of these different de?nitions, and a discussion of how well they ?t when measuring situation awareness in pilots see Fracker [1988]. The above de?nition of situation awareness by Endsley is the most useful when dealing with a general view on what situations are and how they can be understood. The de?nition is further developed in Endsley et al. [2003], where it is applied as a tool for constructing computer applications from a user-centred perspective. Endsley et al. argue that a clear understanding of one’s situation is paramount when constructing technologies that will enhance their abilities rather that hinder them. A discussion of how the framework model proposed by Endsley is based on the information processing theory [Wickens and Hollands, 1999] can be found in Endsley [2000]. The above de?nition of situation awareness can be broken down into three distinct levels [Endsley et al., 2003, p. 14]: : Level 1 Perception of the elements in the environment Level 2 Comprehension of the current situation Level 3 Projection of future status Level 1 deals with the ability to perceive the properties of relevant elements in the environment. The exact nature of these elements will naturally vary depending on the environment in question; thus the (necessary) array of sensors utilised will also vary. This level is only concerned with the raw sensory input. Level 2 is concerned with understanding the data perceived, and how these data are related to relevant goals and objectives of the acting entity. This understanding is based on the integration of many, possible, disjoint Level 1 elements, and a comparison of this information to the entity’s goals.

48
Feedback

Situation Awareness

Situation Awareness State of the Environment Perception of Comprehension Projection Elements in of future of Current Current Situation Situation status
Level 1 Level 2 Level 3

Decision

Performance of Action

Figure 2.7: Model of Situation Awareness (adapted from Endsley [1988])

Level 3 allows an entity to predict the behaviour of the elements recognised as being important in Level 2. When the raw sensory data have been perceived, it is possible to assess the situation based on information available in both short-term and long-term memory. The long-term memory structures used goes by many names; depending on the theories applied. The most obvious examples in this setting are memory organization packets [Schank, 1982], schemata [Rumelhart, 1980], or semantic networks. Important parts of the available knowledge are the: goals, objectives, and preconceptions of the entity assessing the situation. This speci?c and more general knowledge aid the entity in assessing the situation and predict the future states of the environment. When a suitable understanding has been reached the entity can now decide what to do, and perform any actions deemed necessary; which again will in?uence the environment. A simpli?ed model of situation awareness is depicted in Figure 2.7. Situation awareness has only recently emerged in the world of pervasive computing; even though the core concept is an integrated part. As described in Section 2.1 the focus in pervasive computing has been on base technologies. However, with the current level of technology, work on systems that are aware of their environment and situations occurring is picking up. One example of explicit use of situation awareness as a concept can be found in Yau et al. [2002], where the typical ubiquitous computing application is ascribe the characteristics of: context-sensitive, situation-aware, and ad hoc communication. The loose de?nition of situation awareness can be found in the work describing smart classrooms: “Situation-awareness is the capability of a device to determine What’s going on? in it [sic] surroundings” [Yau et al., 2003, p. 1]. This use of situation awareness is similar to the Comprehension of Current Situation shown in Figure 2.7, as Yau et al. de?ne a situation as: “. . . an expression on previous device-action record [sic] over a period of time and/or the variation of a set of contexts relevant to the application software on the device over a period of

Ambient Intelligence

49

time. Situation is used to trigger further device actions.” [Yau et al., 2002, p. 3]. The perception of the environment (level 1) has been classi?ed as context: “Context is considered any detectable attribute of a device, its interaction with external devices, and/or its surrounding environment.” [Yau and Karim, 2001, p. 4], and projection has to a certain degree been left to other modules in their architecture.

2.5

Summary

If we now revisit the objectives of an ambient intelligent system: being aware of the presence of a person, perceive the needs of this person and respond intelligently to these needs in a relaxed and unobtrusive manner, through the spectacles of situation awareness. An ambient intelligent system should contain three components: a component for perceiving the world (level 1); one to reason about the current situation, by being context aware (level 2); and ?nally one to exhibit appropriate behaviour by being context sensitive (level 3). The ?nal component is a combination of the ability to project future states and actually decide to act. Further, as reasoning is the most important ability when being context aware, and context has been described as knowledge, it should be feasible to demonstrate context awareness through a knowledge intensive method such as case-based reasoning.

50

Summary

Chapter 3

Case-Based Reasoning
This chapter is concerned with the method of case-based reasoning. Case-based reasoning approaches reasoning by the use of episodic knowledge, as opposed to the more traditional use of general rules or domain knowledge. Case-based reasoning is based on research into how humans use scripts and schemata to reason about situations and their outcome. This chapter starts out with an overview of case-based reasoning as a method and its foundation in psychology. The following section will describe how knowledge intensive case-based reasoning is achieved through the CREEK method, which expands traditional case-based reasoning by enriching cases with explicit domain knowledge. Finally, the use of case-based reasoning in ambient intelligence will be described.

3.1

Foundation

Case-based reasoning approaches the reasoning process, not by the classic approach of rules or general knowledge, but by using episodic memory. The knowledge base in any case-based reasoning system consists of prior episodes experienced; or cases. The solution to a new problem is found by matching the problem to an existing case and reusing the solution. This method springs from work on understanding reasoning as an explanation process [Schank, 1982]. Our understanding of common occurrences assists us in comprehending stories, in such a way that details omitted or assumed implicitly does not make a story incomprehensible for us. Our general knowledge about situations, the expectations, and the behaviour which should be exhibited are stored in what has been referred to in psychology as scripts [Schank and Abelson, 1977]. Scripts are closely related to the concept of schema [Bartlett, 1932; Rumelhart, 1980]. Scripts are also known as event schemata, which are a cluster of knowledge about speci?c events or

52

Foundation

actions expected to take place in a certain way in a particular setting, or situation. A script can be viewed as procedural knowledge, whereas a schema is declarative knowledge. Both scripts and schemata are highly context dependent, as scripts in particular encode a speci?c behaviour given a speci?c context. Research shows that the ef?ciency of retrieval, or ?nding the correct script, is highly correlated with learning and using speci?c behaviour in the same environment, or context [Baddeley, 1982]. These are the premises from which case-based reasoning takes its approach. Via the memory organization packets (MOP) proposed by Schank [1982], a relationship between scripts and cases emerge. MOPs contain the general knowledge describing the type of experiences, as well as indexes how cases differ from the norm. The main idea of the organisation is the fact that cases that are worth remembering are those that somehow differ from what was expected; or in the words of Kolodner [1993]: “A case is a contextualized piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goals of the reasoner” [p. 13]. Case-based reasoning can, to some degree, be compared to a rule-based system; where the rules are very large. A case’s ?ndings can be viewed as a problems antecedent, and its solution as the consequence. However, they differ in several important ways: i) rules are patters, whereas cases are constants; ii) rules are retrieved based on exact matching, where cases only require partial matching; iii) rules are small, independent, and consistent pieces of domain knowledge, cases can be large chunks of, potential redundant, domain knowledge. It is not only rule-based systems that case-based reasoning resembles. Case-based reasoning also resembles model-based reasoning. Both methods assume knowledge composed into large chunks, and uses these chunks for reasoning. However, as with rule-based systems, there are signi?cant differences: i) model-based systems store causal models of domains, whereas case-based reasoning systems store speci?c descriptions of processes; ii) model-based reasoning requires a well understood domain, where case-based reasoning can function in ill understood domains; iii) model-based systems only provide ways of solution veri?cation, but no guiding of the solution generation, case-based reasoning provides ef?cient solution generation and evaluation base on the best case available. The theoretical foundation, in particular the fact that case-based reasoning gains its expertise through remembering the irregularities, are the main reason why casebased reasoning is very suitable for ill understood and/or open domains. This has lead to the fact that case-based reasoning today is a thriving and growing community. Today, many different versions of case-based reasoning exist. However, all methods use the four main steps of reasoning. The classic case-based reasoning cycle, depicted in Figure 3.1, contains these four steps, which are: 1. Retrieve the most similar case or cases

Case-Based Reasoning

53

Problem

New Case

Re
Learned Case

trie v

e
Retrieved Case New Case

General Knowledge

Con?rmed Solution

Tested/ Repaired Case

Reuse
Solved Case

Previous Cases

Figure 3.1: Case-Based Reasoning Cycle (adapted from Aamodt and Plaza [1994])

2. Reuse the information and knowledge in these cases to solve the problem 3. Revise the proposed solution 4. Retain the parts of the experience likely to be useful for future problem solving A common approach to retrieval in case-based reasoning is to compare cases solely based on their surface features. These features are the ?ndings, or description, of a case, and are often represented as attribute-value pairs. In other approaches the similarity can be computed based on their structural properties. This approach is often more computational expensive that pure syntactical comparison. Yet, it often produces more relevant cases [L? pez de M? ntaras et al., 2006, p. 6]. A few o a approaches uses features derived through the use of a domain model to measure the similarity. Once a suitable case, or set of cases, has been retrieved, the reuse phase can commence. This process can range from the trivial, often used for classi?cation purposes,

Retain

Revise

Suggested Solution

54

Knowledge Intensive Case-Based Reasoning

where the best matching case is returned, unchanged, to far more complicated processes. When there is a large discrepancy between the new problem and the retrieved case’s problem, some sort of adaptation is required. Adaptation is most often achieved through either substitution, where some parts of the retrieved solution is kept; transformation, where the structure of the retrieved solution is altered; or generative adaptation, where the method of deriving the retrieved solution is replayed on the new problem [ibid., p. 15]. Revision of the proposed solution is closely coupled with the adaptation process. In classi?cation systems, where the best matching case is returned without any adaptation, the revise step is a discrete step where the proposed solution is validated on the new problem. However, when a lot of adaptation is required the reuse and revise step can be integrated. Yet, any proposed solution has to be evaluated to ?gure out, not only if it works, but also if it is of such a calibre that it should be retained for future use. The retain phase is where the outcome of the recent problem-solving process is kept for future use. In line with the argument that a case is: “. . . an experience that teaches a lesson. . . ” [Kolodner, 1993, p. 13], cases are traditional kept when they represent a new path in the reasoning process, rather than reinforcing existing pathways. Most systems simply record the ?ndings of the case and the outcome, implicitly stating that the outcome was a success. In more complex scenarios some more ?ne granulated information, on how well different aspects of the solution were addressed, is stored.

3.2

Knowledge Intensive Case-Based Reasoning

When dealing with words such as data, information and knowledge it is important that an agreement upon the meaning of these terms is reached. It is meaningless to discuss methods being knowledge intensive if different de?nitions of knowledge are available. It is commonly agreed that in the context of knowledge systems, the three concepts of data, information and knowledge can to a certain degree be viewed as a hierarchy. Data is traditional viewed as unorganised and unprocessed facts. Data can in theory be anything from raw sensory input to this thesis. When data are being interpreted, organised or assigned meaning they become information. Thus, the data received from a sensor might become information when interpreted by an entity, or this thesis might become information when read by somebody knowledgeable of English. Finally, knowledge is information that has been incorporated into an entity’s reasoning resource. Knowledge is closely related to the widely shared view that learning is the process of integrating information into a body of knowledge. All computer systems operate on data, many work with information, and some with knowledge. As with all computer programs, case-based reasoning comes in many guises, in

Case-Based Reasoning

55

particular with respect to the degree of knowledge involved. The architectures involving case-based reasoning can range from purely database type of systems, where cases represent data used for indexing and retrieval; to full grown knowledge-based systems, where the case base is not only a source of information for the user, but knowledge is actively used by the system to reason. When dealing with knowledge intensive systems, the knowledge base involved can be viewed as a: “. . . qualitative model of the part of the real world that the system is to reason about.” [Aamodt, 2004, p. 5]. According to Davis et al. [1993], the knowledge of a system will be biased in several ways: i) a knowledge representation, or model, is a surrogate, which enables an entity to reason rather than act; ii) it is set of ontological commitments, which determines in what way one should think about the world; iii) it is a fragmentary theory of intelligent reasoning, expressed as the fundamental conception of intelligent reasoning, the inference that is sanctioned, and the inference that is recommended; iv) it is a medium for pragmatically ef?cient computation, which is the computational environment where reasoning is accomplished; v) it is a medium of human expression, or the language which we use to talk about the world. The CREEK method [Aamodt, 1991, 1994b] is a knowledge intensive case-based reasoning architecture; particularly developed to tackle problem-solving and learning in open and weak-theory domains. The main asset of CREEK is the fact that cases are submerged into the general domain knowledge. This model is realised through a multi-relational semantic network, where an object-oriented, frame-based representation is used to capture both cases and domain knowledge. When modelling knowledge in CREEK, a combination of a top-down process for initial knowledge acquisition, and a bottom-up process of continuous learning through retaining cases, is used. The top-down process is used to acquire and develop the conceptual model required to de?ne the domain in question. The model must be of such a nature that the ontological commitments are expressed in a way where a developing team can re?ne and populate the model, and be suitable for acquiring and maintaining cases. To facilitate the knowledge modelling, and reasoning process, CREEK is integrated with a top-level ontology, or knowledge model. This model describes the fundamental concepts used in CREEK, such as: real world entities and relationships, a strategy model, a model for combining case-bases, and a model for combining learning methods [Aamodt, 2004]. The case-based reasoning process in CREEK is divided into the four-step process, containing the retrieve, reuse, revise and retain process. Each of these three steps utilises a general explanation engine [Aamodt, 1994a], consisting of an activateexplain-focus cycle. This cycle initially activates the relevant parts of the semantic network, it then generates and explains derived information within the activated knowl-

56

Applying Case-Based Reasoning to Ambient Intelligence

edge structure, and ?nally, it focuses the selection of a conclusion that conforms to the goal in question. CREEK has been applied to several application areas, including medical image understanding [Grimnes and Aamodt, 1996], decision support in oil well drilling [Skalle et al., 2000; Skalle and Aamodt, 2004], intelligent tutoring systems [S?rmo and Aamodt, 2002], and explanation of gene-gene relationships for understanding diseases at functional genomics level [Kusnierczyk et al., 2004].

3.3

Applying Case-Based Reasoning to Ambient Intelligence

Traditional, case-based reasoning has been applied to monolithic decision support systems. Recently, however, case-based reasoning has also been applied within the ambient intelligent community. Zimmermann [2003] reports on case-based reasoning used to generate recommendations based on the user’s context in a mobile environment. The user context is encapsulated inside cases to facilitate comparison of contexts, generating recommendations based on case similarities, and learning of user behaviour. This work was part of the LISTEN project, which was concerned with audio augmentation of the real world in the context of the Kunstmuseum (art museum) in Bonn. To model the users’ situations a context model was proposed. This model falls in to the category of typical ad hoc context models commonly used in pervasive computing (see Section 2.3.1). The model consists of four dimensions: identity of the user in question, the location of the user, the time of day, and the environment describing the objects surrounding the user. Instances of this context model were used to describe the ?ndings of the cases. As for the solution part of the case, this was a recommendation consisting of an audio ?le being played to the user. When a user move about in the exhibition, the change in location triggers the instantiation of new cases and a matching to existing cases to ?nd a suitable recommendation is executed. The author points out some problems associated with using a case-based reasoning approach to tackle location-based services. Most importantly, the tracking of users in the real environment and in real time, will quite rapidly grow the number of cases in the case base to a staggering number. Zimmermann proposes to initial cluster similar cases, and consecutively generalise cases prior to the reasoning process. However, the most important contribution from this work is the approach to tackle the underlying problem of all context aware pervasive computing applications; namely identifying similarities between contextual parameters, using a general reasoning mechanism. Along the same lines of adapting solutions to users is the work by Ma et al. [2005], where case-based reasoning is used to adapt the behaviour of smart homes to users’ preferences. Multi-user smart homes can, even with a very limited amount of con-

Case-Based Reasoning

57

nected devices, present itself with a very large amount of possible key processes and dependencies between these. Case-based reasoning is being used to identify these interdependencies because of its ability to reason in ill understood and poorly structured domains. The cases are represented as frames, where the ?ndings are: the user, the environment, the time, and the values of the active devices. When new cases are instantiated the similarities between the new and existing cases are calculated. The best matches are calculated using 1- or k-nearest neighbour. As smart homes have no user interface per se, a novel approach has been used to revise a proposed solution. The revision is based on measuring if the user modi?es the values set on the different devices, and depending on any user initiated changes, cases are adapted as required. Kwon and Sadeh [2004] have applied case-based reasoning in a multi-agent environment, to estimate best purchase in comparative shopping. The idea of comparative shopping in this context is that consumers wishing to purchase a product can compare prices, warranties, and other aspects between suppliers, thus ?nding their optimal combination. At the same time, the suppliers can tailor their products to speci?c customers, or customer groups; yet still make a pro?t. All in all, it seems that the goal of the system is to achieve a Pareto optimal solution between seller and buyer. A multiagent architecture is adopted under the assumptions that many selling agents are ready to serve the customer and intelligent agents can autonomously produce proposals by delegation from their human users. The agency used consists of three types of agents: a single buyer agent, several seller agents, and a single negotiation agent. Since it is hardly possible to know the buyer’s exact utility function, the negotiation agent employs case-based reasoning to match the current negotiation to known outcomes. The ?ndings of a case are the product in question, the price, the level of quality, and some contextual information, which are the location, weather and calendar. As with the other work mentioned above, no explicit or grounded context model exists, just an ad hoc de?nition of extra parameters. The solution part of the case is the bidding result. When a buyer wishes to acquire an item, a request is send to the negotiator, which assess the user’s preferences by retrieving the most similar case from its case base. The system was tested in simulation where several experiments were executed. The three tests were: no negotiation, only price negotiation, and price as well as quality negotiation. As expected by the authors, the complex negotiation outperformed the other two, with respect to buyer’s payoff, seller’s payoff, and seller’s rate of bid winning. Whether or not this result is a direct result of using case-based reasoning, or rather a result of just optimising over several parameters is a bit hazy. However, the authors argue that it is directly linked to adjusting proposals to the buyer’s preferences through case-based reasoning.

58

Summary

The MyCampus [Sadeh et al., 2005] system uses case-based reasoning to learn a user’s context sensitive message ?ltering preferences. The cases’ ?ndings are a message received by the user and the user’s preferences regarding this particular message. When a new message arrives it is compared to the existing messages stored in the case base. The new case is compared to the type of message, the sender of the message, the user’s current calendar activity, the user’s current location, and the weather. These parameters could be viewed as an ad hoc context model for this particular application. Cases are matched using Aha’s Nearest Neighborhood Algorithm [Aha et al., 1991], as adapted by Cercone et al. [1999]. An experiment was carried out to validate the feasibility of a case-based approach. This experiment showed that the accuracy of the ?ltering process grew from 50% to over 80% when using the case-based reasoning approach. Recently B? nard et al. [2006] has investigated the use of case-based reasoning as e a mechanism for selecting suitable behaviour in different situations. They propose an agent-based architecture that uses perceived information, or context, as the ?ndings of a case and the proposed action as the solution. Contrary to most work the authors ground their context model in a psychological perspective that resembles the idea of selective interest and background context as argued by Dewey (see Section 2.3). B? nard et al. separates the context into external and internal context, where the former e includes the entities present in the environment, and the latter includes skills, states of the agent, the agent’s strategies and the agent’s history. The existing cases in the case base are pre-classi?ed situations modelled by a domain expert. The authors brie?y describe how the case-based reasoning process approaches the problem of behaviour selection. Initially a new context is compared to the ?ndings in the existing cases and the best matching case is retrieved. Secondly, the retrieved case is adapted to ?t the new context. If an existing case cannot be retrieved the domain expert or the agent itself decides on an appropriate behaviour and retrains it for future use. To demonstrate their approach the authors gives an example where a football-playing agent has to decide when to pass on the ball.

3.4

Summary

On of the main objectives of an ambient intelligent systems, is its ability to reason about ongoing situations by being context aware. Case-based reasoning springs from a wish to understand situations through the concepts of scripts. Thus, it appears to be a promising candidate for context awareness. In addition, the knowledge intensive approach to case-based reasoning ?ts well with the argument that context awareness requires reasoning as well as an knowledge level perspective on context. When incorporating case-based reasoning into an ambient intelligent system, it

Case-Based Reasoning

59

must be encapsulated in such a way that the reasoning process can be executed as a separate component. Investigating the existing literature, it appears that agents are a feasible approach.

60

Summary

Chapter 4

Agent Technology
4.1 Agents and Agencies Basics

For some time now, agents have been an important concept in computer science in general, and in arti?cial intelligence in particular [Russell and Norvig, 1995]. However, agents are a dif?cult concept to de?ne rigidly. Agents cover a range of capabilities that not only software entities can exhibit. Historically, agents have been de?ned ad hoc to ?t the particular set of agents the de?ner used. However, as agent technology has matured, more general and system independent de?nitions and classi?cations have emerged. Goodwin [1994] approaches the problem of a common de?nition of agents by de?ning a common vocabulary and framework. The foundation for discussing agents is, according to Goodwin, that: “an agent in an entity that performs some task. . . ” [ibid., p. 2]. Thus, any property of an agent must be de?ned according to the tasks performed and the environment inhabited by the agent. Adhering to the behavioural psychology approach, he de?nes only observable behaviour as an acceptable measure of success. This view leads to an informal de?nition of agent properties [ibid., p.2]: Successful an agent is successful to the extent that it accomplishes the speci?c task. Capable an agent is capable if it possesses the effectors needed to accomplish the task. Perceptive an agent is perceptive if it can distinguish salient characteristics of the world that would allow it to use its effectors to achieve the task. Relying on observing an agent’s behaviour only will not allow us to consider any internal organisation of an agent. However, deliberating agents, those who maintain

62

Agents and Agencies Basics

an internal model of the world and reasons about it, are typically of particular interest. Aside from the before mentioned properties, they also have the following additional properties [ibid., p. 3]: Predictive an agent is predictive if its model of how the world works is suf?ciently accurate to allow it to correctly predict how it can and cannot achieve a task. Interpretive an agent is interpretive if it can correctly interpret its sensor readings. Rational an agent is rational if it chooses to perform commands that it predicts will achieve its task. Sound an agent is sound if it is predictive, interpretive and rational. Along with these properties, a framework for describing agents must also include an account of the tasks involved and the environment inhabited by the agents. The author develops a formal de?nition of these properties. The analysis and framework present should allow for a useful analysis and comparison of agents. However, for practical purposes in this context, the separation of environment, agent and task, along with the properties described is the most important addition to a general understanding of agents. Following this notion of agents, as autonomous entities inhabiting an environment and solving tasks, is the work by Wooldridge and Jennings [1995]. In this work the aim is to express important issues regarding theoretical and practical aspect of using agents in software development. Wooldridge and Jennings approach agents by separating their properties into two main groups. The ?rst group is the weak notion of agency, which to a large degree corresponds to the general properties from Goodwin. These notions are [ibid., p. 4]: Autonomy agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state; Social ability agents interact with other agents (and possibly humans) via some kind of agent-communication language; Reactivity agents perceive their environment and respond in a timely fashion to changes that occur in it; Pro-activeness agents do not simply act in response to their environment; they are able to exhibit goal-directed behaviour by taking the initiative.

Agent Technology

63

Autonomous Agents

Biological Agents

Robotic Agents

Computational Agents

Software Agents

Arti?cial Life Agents

Task-speci?c Agents

Entertainment Agents

Viruses

Figure 4.1: Agent Taxonomy (adopted from Franklin and Graesser [1996])

These weak notions are very much in agreement with the properties described by Goodwin. The notion of pro-activeness, where goal-directed behaviour is exhibited, is equivalent to: “an agent . . . that performs some task” [Goodwin, 1994, p. 2]. The notion of reactivity, where an agent perceives the environment, is the perceptive property. Wooldridge and Jennings also support the division of agent and environment, as an agent is situated in an environment. The combination of being able to perceive the environment, (successfully) solving tasks, and the ability to distinguish salient tasks described by Goodwin, is what Wooldridge and Jennings to some degree captures as autonomy. The only addition to Goodwin’s properties is the social ability of agents. This is an addition that arrives through the idea of agency, which is not explicitly recognised by Goodwin. All in all, the weak notion of agency is equivalent to the general agent properties, with the addition of the social aspects. Both of these perspectives assume the behavioural psychology approach, where agents are viewed as black-box entities acting in an environment. However, as the internal workings of an agent are of great importance, in particular when constructing them, Wooldridge and Jennings also recognise the need to tackle what Goodwin refers to as the deliberative agent properties. This leads to the strong notion of agency, where mentalistic notions are ascribed to agents [Wooldridge and Jennings, 1995, p. 5]. Most importantly are the notions of belief, desire, and intention [Shoham, 1993]. In addition to these mentalistic notions, several other notions are often ascribed to agents, the most common are: mobility; veracity, agents will be truthful; benevolence, agents will do what they are told; and rationality. The considerations of agents described above give us a frame of reference when discussing agents and how agents distinguish themselves from any arbitrary program.

64

Agents and Agencies Basics

Smart Agents

Collaborative Learning Agents

Cooperate

Learn

Autonomous

Collaborative Agents

Interface Agents

Figure 4.2: A Partial View of an Agent Typology (adopted from Nwana [1996])

Franklin and Graesser [1996] attempts to construct a taxonomy of agents, partly based on the de?nitions above. The taxonomy, depicted in Figure 4.1, starts out from the classical biological classi?cation scheme, where “living creatures” are located at the root and individual species at the leaves [ibid., p. 6]. Where Franklin and Graesser attempt to construct a taxonomy based on a biological analogy, where species are structured. Nwana [1996] chooses another approach and constructs a typology based on agent’s attributes. Nwana argues that mobility of an agent is a parameter for classi?cation. Secondly, following Goodwin, agents can be classi?ed as either deliberative or reactive. Thirdly, Nwana argues that agents should demonstrate three primary attributes: autonomy; cooperation, which is an important reason for having multiple agents; and learning, which is paramount for demonstrating intelligent behaviour. When looking at these three attributes four main types of agents are recognised, they are: smart agents, collaborative learning agents, collaborative agents and interface agents, see Figure 4.2. Fourthly, the role assumed by an agent is an important aspect when classifying them. Finally, any combination of the above attributes constitutes a hybrid agent. The typology constructed based on these attributes is shown in Figure 4.3. Following the notion of agents’ social abilities, the classi?cation by Nwana introduces the notion of collaborative agents and agent systems. According to Wooldridge [2002] the creed of the multi-agent system community is that: “There’s no such thing as a single agent system” [ibid., p. 105]. This thesis holds true regardless of which type of agent one examines in the taxonomy by Franklin and Graesser (Figure: 4.1),

Agent Technology

65

Heterogeneous Agent Systems Mobile Agents Information Agents Reactive Agents

An Agent Typology

Smart Agents Collaborative Agents Interface Agents Hybrid Agents

Figure 4.3: Agent Classi?cation (adapted from Nwana [1996])

or the classi?cation by Nwana. We know from experience that biological agents live in an environment consisting of several agents. This is also true for arti?cial agents, since most, if not all, systems today are interacting with their environments, something which is the very nature of agents. From an engineering point of view, the idea of multi-agent systems is a very attracting idea. Such multi-agent systems can encapsulate specialised task knowledge, exhibit specialised behaviour useful in a heterogeneous environment, and organise themselves in such a way as to avoid processing bottlenecks [Sycara et al., 1996]. Following the creed that no single agent system exists, a common platform where agents can interact smoothly is required. According to Sycara [1998] the characteristics of these platforms, or multi-agent systems, are: each agent has a limited viewpoint on the world, there is no global system control, data are decentralised, and computation is asynchronous. These characteristics leads to multi-agent systems viewed as programmable platforms that provides low-level functionality, such as administration of agents’ life cycle, handling communication between agents and communication with the outside world. Through the use of these platforms it is possible to construct and program agents and collection of agents, focusing on the high-level functionality. One of the most important issues in a multi-agent system is the ability to communicate in a coherent and ef?cient way. When communicating, two important issues must be covered: what is the format of the messages, and how is the content expressed? To approach these issues common languages for agent communication have been developed.

66

Agents and Agencies Basics

Among the initiatives for standardising the content of messages is the Knowledge Interchange Format (KIF)1 [Genesereth and Fikes, 1992], which is based on ?rst-order logic. KIF can, for example, express properties of things in a domain, the relationship between these things, and general properties of the domain. As KIF does not state anything about how messages are de?ned, only about the content, other initiatives attempt to tackle that issue. One of the most important ones is the Knowledge Query and Manipulation Language (KQML) [Finin et al., 1993]. KQML is a message-based language, which de?nes a common standard for the format of messages. Each message consists of a performative and a number of parameters. The performatives describe the type of messages send. This version of KQML contains 41 different performatives. Each of these performatives can contain several parameters. Currently, eight reserved parameters exist [ibid., p. 9]. The four most important are: :content, which is the content of the message; :language, which speci?es what language the content is in; :ontology, which speci?es what terminology is used; and :sender, which speci?es who is sending this message. KQML has had a large impact on the agent community. Several multi-agent systems have been developed that use KQML as the language of communication. One of the most in?uential is probably the RETSINA system developed at the Carnegie Mellon University [Sycara et al., 2003]. RETSINA is a full-?edged multi-agent system infrastructure that supports and facilitates all the services required by agents to communicate and interact. This includes the management services, infrastructure for use of the agent communication language and name service; see Sycara et al. [2003, Figure 1, p. 4] for an overview. Even though KQML has been widely used, several issues where never fully covered, thus leading to some problems when construction KQML-agents. According to Wooldridge [2002, p. 175], the following critiques where the most important (paraphrased): ? The basic KQML performatives were never tightly constrained, leading to incompatibilities between different KQML implementations. ? The transport mechanism were never de?ned precisely, again hampering interoperability. ? The semantics of KQML were never strictly de?ned and the meaning where open to interpretation. ? There were no performatives de?ning commitments, which severely limits cooperation between agents.
1 http://www-ksl.stanford.edu/knowledge-sharing/kif/

Agent Technology ? The choice of performatives were rather ad hoc.

67

These de?ciencies – among others – lead to the development of the Agent Communication Language (ACL) by the Foundation for Intelligent Physical Agents (FIPA)2 . FIPA is a non-pro?t association of companies and organisations devoted to producing speci?cations for generic agent technology. FIPA has developed speci?cations for all categories within agent technology: agent communication, agent transport, agent management, abstract architecture and applications; 25 speci?cations are included in the current speci?cation set. At the core of all agent systems is the agent communications speci?cation. The FIPA ACL speci?cation [FIPA ACL, 2002] is currently the standard par excellence in the agent community. On the surface FIPA ACL looks quite similar to KQML. It de?nes 20 performatives and does not deal with what language is used for the content. One of the most important critiques of KQML was the lack of adequate semantic. This critique has been dealt with by leaning on the speech act theory by Cohen and Levesque [1990]. The semantics are given with regards to the formal language SL. This language allows for expression of actions performed by agents, as well as beliefs, desires and uncertain beliefs of agents; so following Shoham [1993], SL allows for communication between deliberative agents. Another of the critiques put forth against KQML is that performatives are chosen in a seemingly arbitrary way. FIPA ACL attempts to remedy this by moving performatives not ?tting the speech act paradigm to the agent management level. Among others, the Agora [Matskin et al., 2001] infrastructure employs the FIPA ACL speci?cation. However, for communication inside the agency a local speech act type of communication language was developed. This communication is based on the JATLite3 package, which is KQML based. For communication in and out of the agency, both the FIPA ACL and KQML are used, thus allowing for communication between the Agora system and others, such as with the RETSINA systems.

4.2

Agencies as an Architecture for Ambient Intelligence

Plaza [2004] argues that the ability to exhibit context awareness has two levels of requirements: i) the ability to sense the physical environment, and ii) the ability to use common sense knowledge, as :“. . . context-awareness is essentially an interpretation of the world situation, . . . [thus] common sense knowledge is needed.” (author’s emphasis). The current view on common sense, or any symbolic reasoning for the matter, is that two parts are required: i) an ontology, or knowledge model, de?ning
2 http://www.fipa.org 3 http://java.stanford.edu

68

Agencies as an Architecture for Ambient Intelligence

CAPIA

CAPIA

Delivery Services

CAPIA CAPIA

CAPIA

Awareness Services

Information Spaces

Physical Spaces Social Activity

Figure 4.4: Mediation Between Information and Physical Space (adapted from Plaza and Arcos [2001, p. 2])

the objects existing that we wish to talk about, and ii) an inference engine, which is capable of using the knowledge model to conclude new facts and statements about these facts. Lacking these abilities removes the possibility of an application being context aware. However, only missing the latter will allow an application to be, what Yau et al. [2002] de?nes as context sensitive (see Section 2.4, p. 48). Beside these requirements, Plaza argues that context aware applications must have some properties that set them aside from ordinary applications. These are: the ability to be persistent; it needs to be responsive; and the ability to be autonomous. According to the author, one such computation paradigm ?ts these requirements; namely intelligent agents. In line with these views, Arcos and Plaza [2001] present a society of personal information agents that are aware of the physical and social context of their users. The framework consists of a collection of context aware personal information agents (CAPIAs), which are working in an information space (left side of Figure 4.4). Together with two mediating services, the awareness services and the delivery services they supply context aware services to humans inhabiting the physical world. The awareness service is responsible for transferring information emerging in the physical world into the information world. The information in question is the information about the world that the system can perceive through its sensors; which is

Agent Technology

69

comparable to level 1 in situation awareness (see Section 2.4, p. 47). The delivery services, on the other hand, is the service that is responsible for delivering information from the information space to the users located in the physical space; through the use of subscription. The society of agents consists of personal agents, one for each user. Each of these agents are capable of reacting to changes in the context, as sensed trough the awareness services, and reason about the information suitable to its user’s needs. Information is evaluated based on the interestingness, which is a function of the user model; and relevance, which is a function of the user’s context Implementation and test of the architecture was conducted within the COMRIS project [Plaza and Arcos, 2001]. The ontology used includes activities occurring at a conference, the roles of the attendees, the activities of the roles, locations, the schedule of the conference, and an instruction set given to the CAPIAs by their users. The tasks that the system are able to perform are: information gathering, where the CAPIAs attempts to gather information relevant and interesting to its user; appointment proposal, where appointments between users who share common interests are negotiated; proximity alert, which will alert the user when he is near an interesting person or facility; and a context sensitive appointment reminder. Several experiments were performed in simulation. The population of conference attendees were 2.000, able to use 420 different locations. The system was tested in two different modes: one with the context aware service activated, and one without. Furthermore, four different setting for the attendees’ level of interest were used: 100%, 50%, 25%, and 10%. The authors show that as the level of interest decreases, the ability of the context aware system to deliver correct information increases. Opposed to the test case where context awareness was turned off, where the ability was independent of the attendees interest level. All in all, the authors demonstrate that including context awareness is bene?cial to the quality of the services performed. Somewhat along the same line as the idea of context aware agents as information ?lters, described above, is the work by Jayaputera et al. [2003a]. They demonstrate how the eHermes multi-agent system can adapt the content delivered to users, to their context; in this case the capabilities of the users’ devices and the bandwidth available. eHermes is a system that has been developed with ubiquitous computing in mind [Jayaputera et al., 2003a,b]. The main idea is to be able to generate plans and execute them in run-time. Plans are in the form of task-hierarchies, which are referred to as Missions. Missions are generated on the basis of a user request and the pro?le of the user making that request. Unlike the work by Plaza and Arcos, Jayaputera et al. use stereotype user modelling in the tradition of Rich [1989] to build user pro?les. The same problem may be solved in different ways according to which user makes a request. Agents do

70

Summary

not generate the Missions; rather they are generated by a system component called the Mission Generator. Agents are not part of the planning process; they only participate in the problem-solving process. One agent, the Mission Control agent, is responsible for co-ordinating the agents performing the problem solving. These problem-solving agents are assembled on demand from the mission speci?cation, an agent component library, and ontologies. The agent assembly is done by a system component called the Agent Generator, which assemble agents on request from the Mission Control agent. The system is reported to be capable of solving problems in domains where the overall purpose of agents may be explicitly identi?ed and expressed. The work by Kwon and Sadeh [2004], which was described in Section 3.3, p. 57, uses a multi-agent system to estimate the best purchase in the context of comparative shopping. As in the cases of Plaza and Arcos [2001] and Jayaputera et al. [2003a], a user pro?le is one of the most important contextual parameters used to ?lter the results. Where Plaza and Arcos [2001] build the user pro?le by letting the user ?ll in some preferences, and Jayaputera et al. use stereotype modelling, Kwon and Sadeh relied on case-based reasoning to estimated the user’s pro?le based on prior behaviour. Finally, Sycara [1999] shows how the RETSINA system can use context to re?ne the results of an information retrieval algorithm. The RETSINA MAS distinguishes between three different agent types: Interface agents, Task agents and Information agents. Interface agents are the interface between the user and the system. The user speci?es a problem to be solved and the Interface agents translate the problem into a task that is handed to a Task agent. Task agents may decompose tasks into subtasks. The decomposition speci?es a plan for how to solve a problem. One or more Task agents may cooperate to design the solution to the problem. Task agents query Information agents that are in control of different information sources. Task agents assemble the received answers into an answer to the initial problem. The answer to the initial problem is sent back to the requesting Interface agent, which present the solution to the user.

4.3

Summary

Realising the objectives of an ambient intelligent system through the use of agents appears to be a very suited approach. Not only does the use of agents allow us to compartmentalise the main components, agents are the only paradigm that can realise the requirements on ambient intelligent systems. Thus, it seems reasonable that the functionality of perception, context awareness and context sensitivity should be encapsulated in different agents. Further, as a system’s behaviour depends on ongoing situations, as well as the ?exibility in availability of different services, the use of agencies with proper use of agents communication languages supports exactly this.

Part III

Research Results

Chapter 5

Knowledge Model and Conceptual Architecture
This chapter describes the conceptual architecture proposed in this thesis. The chapter is divided into two main parts: a part about the knowledge model developed and used, and a part on the three layered architecture developed for realising an ambient intelligent system. The section dealing with the knowledge model will initially make a separation between a system’s ability to be context aware and context sensitive, and show how this separation will affect the perspective on knowledge in general and context in particular. Following this, the original context meronomy used in the AmbieSense project will be described, how it is expanded to include a representation of tasks, goals and actions, and how it is moved into a multi-relational semantic network. Finally, this model is enhanced into the full ?edge knowledge model by including the sociotechnical perspective of activity theory, general domain knowledge and situations in the sense of episodic knowledge through cases. The second half of this chapter deals with the three-layered system architecture inspired by the work on situation awareness. This part describes the overall architecture as well as how the Context Middleware is used as the perceptions layer; case-based reasoning as the awareness layer; and how agents are used in the sensitivity layer, and how the separation of awareness and sensitivity fosters a dualistic view on context.

5.1

Knowledge Model

The development of a knowledge model useful for building an ambient intelligent application is of utmost importance. This section describes the different sources of

74

Knowledge Model

inspiration, the motivation for approaching modelling from a knowledge level perspective, as well as the socio-technical approach employed. Section 5.1.1 de?nes the difference between the ability to be context aware and context sensitive, as well as why this distinction is important, and how it fosters a knowledge level approach to context modelling. Section 5.1.2 describes the initial meronomy [Sharov, 1995] —which describes an ordering by the part-of relation— used in the AmbieSense project to describe context. It furthermore argues why syntactical relationships are not adequate when aiming for context awareness. Section 5.1.3 argues that a socio-technical approach is bene?cial when moving the basic meronomy into the knowledge level, and that the use of activity theory is a promising method for constructing a knowledge model. The knowledge model describing a domain independent perspective on context is de?ned. Section 5.1.4 describes how domain dependent knowledge is integrated into the knowledge model, how activity theory is used to facilitate knowledge engineering through observations, and how cases play an important role in the knowledge model concerning knowledge about situations.

5.1.1

Context Awareness and Sensitivity

When dealing with the construction of artefacts, whether they are tangible in the form of hardware, or less tangible in the form of software, we often ascribe context awareness to them. As was described in Section 2.1 the term context aware is used to convey a certain idea of the capabilities of the artefacts we construct, and many of the systems that claim to be context aware can hardly be described as being aware, rather they can be perceived as being sensitive to the context. The work presented in this thesis postulates that there is an important and worthwhile distinction between applications being context aware and applications that are context sensitive. This distinction is important when describing the capabilities required by the systems we construct, but even more important when discussing what attributes that are viewed as context, and how context is to be modelled. As already mentioned in Section 2.1, context awareness deals with the problem of understanding the situation in which something occurs. In the case of context awareness, context is viewed as resources available to a reasoner, in particular for making sense of the situation. In this sense, all information is potential context. However, for determining a particular situation only a subset of all the knowledge contained inside a reasoner’s knowledge base is used. For an example, we might know who won the latest UEFA cup; however, this is hardly relevant when determining that my three colleagues are having lunch. Whereas, it is essential when classifying the situation to observe that it is noon and all of my colleagues are eating. This view on context as knowledge that are important in determining a situation leads to the following de?nition of what attributes are considered context for context awareness:

Knowledge Model and Conceptual Architecture Attributes are context if they are essential (for understanding)

75

Context sensitivity, on the other hand, deals with how we can adapt and improve our computer systems in such a way that they respond to situations or supplies tailormade information. Any computer system takes some input and transforms it to some output. For an example, a function might take your location and return all restaurants located near you. So far this is a typical program that can handle the input-parameter location. Since the number of restaurants located near you might be very large, and you really only enjoys Italian restaurants, it is possible to ?lter the result based on your context (the fact that you only enjoy Italian restaurants). The fact that the system can improve the restaurant suggestion based on your context makes it context sensitive. This draws a clear line between ordinary input-parameters and context. The fact that location is essential for the program to execute excludes it from context (in this situation). Yet, the preference for a particular type of restaurant is non-essential, thus making it context (in this situation). This leads to the following de?nition of what attributes can be considered context for context sensitivity: Attributes are context if they are non-essential (for tasks) These two de?nitions draw a line between the different sub-sets of the knowledge base that are to be used when working with context. The second de?nition is by far the simplest to implement, since the input attributes are given when designing a particular application, and the designer de?nes the context attributes. However, the ?rst de?nition is more complicated, as it is not obvious what parts of the world-model are important. The best, or at least most natural solution would be to start the system from scratch and allow it to learn the structures and attributes necessary to understand situations. However, as for an example described by Ekbia and Maguitman [2001], reported in Section 2.3, the acquisition of knowledge starting with a tabula rasa is a long and tedious task for any entity. To speed up the process prede?ning some categories with which the system is instantiated is feasible.

5.1.2

Context Meronomy

Since we are making software that is supposed to assist humans in their activities we need a pragmatic approach to tackle the representation problem. The work presented here imposes the meronomy a priori (see Figure 5.1) that was suggested in the AmbieSense project (see Section 2.2.2). This meronomy will assist developers in structuring their knowledge model to enable the use of context for both aware and sensitive applications. Furthermore, the meronomy imposed at design time should

76

Knowledge Model

User Context

Task Context

Social Context

Personal Context

Spatio-Temporal Context

Environmental Context

Physiological Context

Mental Context

Figure 5.1: Basic Context Meronomy (identical to Figure 2.4) limit the subset of the domain model in such a way that we will not be caught in the frame problem. The original AmbieSense meronomy depicted in Figure 5.1 deals with the syntactical relationships between the different concepts regarded as context. In that sense it does not stipulate a knowledge perspective on context. However, as described in Section 2.1, the ability to be aware demands a knowledge perspective. To approach this knowledge perspective the original meronomy is extended and structured as a multidimensional semantic network, and enriched with domain dependent and independent knowledge. The initial attempt to construct such a model within the tourist domain has been reported in Mikalsen and Kofod-Petersen [2004], and later extended in Kofod-Petersen and Mikalsen [2005b]. The extended model depicted in Figure 5.2, imposes a knowledge perspective on the meronomy. The original meronomy has been integrated with the general concepts of task, goal, action, user state, service provider, service, and entity; as well as the domain speci?c concepts of restaurant, the speci?c services, and the actions that they perform. This enriched model enables the system to infer relationships between concepts by constructing context-dependent paths between them. This is an important ability, in particular when matching two features that are syntactically different, by explaining why they are similar [Aamodt, 1994a; J?re et al., 2002]. For an example, the concept of A la Carte matches Bar in the context of Restaurant. As can be seen in Figure 5.2, the original meronomy is retained through the part-of relationship between the different parts of a user context. In addition, the task-actiongoal triangle has been introduced. This triangle represents the relationships between tasks, which is an abstract construction that achieves a goal and triggers actions; actions that are concrete tasks, which can be executed and result in a goal; and goals represent a desirable state of the user, the system or a combination of both. This triangle is located in the task context. It is further related to a user’s role, which can de?ne any number of tasks that the user is responsible for. The goal is related to the user’s

Knowledge Model and Conceptual Architecture
Context
Isa Isa

77

Tag Context
Isa Has a Isa Has a

User Context

User

Context Tag

Part of

Part of

Part of

Part of

Part of

Task Context
0 *

Social Context
1 *

Personal Context

Spatio?Temporal Context

Environmental Context

Task
Triggers

* Achieves

1

Role

Physiological Context
1

Mental Context

Location

Has a

Service Provider
1

Entity

Goal
* Results in 1 * *

Action

User State
Performs Isa Has a

Service

Isa

Isa

Restaurant
Isa Isa Isa

Isa

Isa Performs

Isa

Serves Food Serves Drinks
Performs

Food Service
Has a

A la Carte
Has a

Cafe

Bar

Drink Service

Figure 5.2: Knowledge Model for the Tourist Domain state by the fact that a user’s state can have one or more goals. Finally, the actions are regarded as services offered by service providers. Within the environmental context both entities, such as the user, the context tags, and service providers are located. It is important to notice that the user’s actions are not explicitly modelled here. The reason for this is that the ambient intelligent system is attempting to assist the user by executing actions that are not done by the user, and that it is assumed that the user is aware of the situation and knowledgeable about the actions that the user has to execute. This de?nition of a knowledge model for context could insinuate that context is regarded as a special kind of knowledge, where the meronomy and knowledge model is disconnected from the system’s general knowledge model. It is important to note that the work presented in this thesis does not regard context as a special kind of knowledge. In that sense this work adheres to the view advocated by Br? zillon and e Pomerol, as reported in Section 2.3 on p. 34, who argues that knowledge that are to be regarded as context is in the eye of the beholder: “. . . knowledge that can be quali?ed as ’contextual’ depends on the context!” [Br? zillon and Pomerol, 1999, p.7]. e As described in Section 2.3, the notion of activities are at the core of using context to gain an understanding of situations. An understanding of situations is paramount to ?guring out how to behave in an appropriate manner. Thus, the notion of activities is very important when describing ambient intelligent systems. The follow section describes how a socio-technical perspective on context modelling will bene?t an understanding of situations, and how activity theoretical aspects can be incorporated into the existing context meronomy.

78 CHAT aspect Subject Object Community Mediating Artefact Mediating Rules Mediating Division of Labour

Knowledge Model Category Personal Context Task Context Spatio-Temporal Context Environmental Context Task Context Social Context

Table 5.1: Basic Aspects of an Activity and their Relation to a Meronomy of Context

5.1.3

Socio-technical Approach to Context Modelling

Section 2.3.2 described how an understanding of activities from a socio-technical perspective could facilitate the shift from mere tool usage to partnership between humans and intelligent artefacts. It further showed how activity theory, and the expansion cultural historical activity theory, is a promising method for capturing the knowledge required for making systems socially adept [Marsh, 1995]. The work presented here employs the expanded model of activity theory, cultural historical activity theory, which covers the fact that human work is done in a social and cultural context (see Section 2.3.2). This model is useful when capturing the knowledge level view of contextual knowledge that should be incorporated into an ambient intelligent system. This contextual knowledge should include knowledge about the acting subjects, the objects towards which activities are directed and the community as well as knowledge about the mediating components, like rules or tools. To meet this end, the mapping from the basic structure of an activity into the meronomy of contextual knowledge is depicted in Table 5.1. It is important to note that a strict one to one mapping does not exist or is desirable at all. The view on contextual knowledge is contextualised itself in the sense that different interpretations exist, and what is to be considered contextual information in one setting, is part of the general knowledge model in another one. Likewise, the same piece of knowledge can be part of different categories based on the task at hand. The same holds for the activity theory based analysis: the same thing can be an object or a mediating artefact depending on the perspectives and task settings. The mapping, shown in Figure 5.3, should guide the development process, and allow for a focus on knowledge level aspects instead of being lost in the modelling of details, without being able to see the relationship between different aspects on a socio-technical level. Some of the work described here has been presented in Kofod-Petersen and Cassens [2005] and later expanded in Kofod-Petersen and Cassens [2006].

Knowledge Model and Conceptual Architecture
Environmental Context Artefact

79

Personal Context

Subject

Object

Task Context

Outcome

Rules Task Context
Rules Time

Community Spatio?Temporal Context
Persons

Division of Labour Social Context
Roles

Figure 5.3: Mapping from CHAT to the Context Model As an example, consider a software development setting where a team is programming a piece of software for a client. The members of the team are all subjects in the development process. They form a community together with representatives of the client and other stakeholders. Each member of the team and personnel from other divisions of the software company work together in a division of labour. The object at hand is the un?nished prototype, which has to be transformed into something that can be handed over to the client. The task is governed by a set of rules, some explicit like coding standards, some implicit like what is often referred to as a working culture. The programmers use a set of mediating artefacts, like methods for analysis and design, programming tools and documentation. When designing

更多相关文章:

非常超级学习网 fccjxxw.com

copyright ©right 2010-2021。
非常超级学习网内容来自网络,如有侵犯请联系客服。zhit325@126.com|网站地图