A Multi-agent Based Model of Human Recognition Process

for Analysis of the Advertisement ’s Effectiveness

Michiaki IWAZUME�

Yuichiro KATO�

Akihito KANAI§

� ������ ��� � ��� ���������� � �������������! #" $&% '�(�)+*�,!*�-�.0/21435*�6�(7.�879�%:1�8�*% ;�*�<=*?>�@�-�.�AB/�C�D!E0*�F�*�GH IKJ4L4M�J&N�O�P�Q�R Q S�Q�T?J7UVIKTW�X�O�J7Y J�Z�M [?\�]4^�_0`�_�a=_�b�c0deb�c0d2f�gih2j4k�l�]7c ^�k�f�mK]4^4`�]nd2o�b�c�p7p7q7q&r0_�s�_�t

§ u0v�w�v�x�y�z7{| }�~ ����z4�B~ ��}�����z4�=z4~�| z4x�z7�#y�{| }�x�{}��V������������| ��z7�7v� �������K�4�4���n���4���������B���0������� iwazume@brain.riken.go.jp ‡yu-kato@valdes.titech.ac.jp, §kanai@umako.c.u-tokyo.ac.jp

AbstractIn this paper, we propose a multi -agent model for analyzingrecognition process of advertising information. We selectedbrand communication in enterprises as our research targetand analyzed customer ’s recognition process of TVcommercials from the viewpoints of audio-visual rhetoricand cognitive science. And we designed multi -agent modelconcerned with customer ’s recognition process. The multi -agent model makes clear the interaction between elementsof customer ’s knowledge structure and the rhetoricalcharacteristics of audio-visuals. It is found that the model iseffective in the evaluation of advertisement.

1 Introduction

As spreading personal medium (e.g. the www, a cellularphone etc.), information environment between informationprovider and receiver has being more complicated andheterogeneous. It has been pointed out that that designingsystems taking characteristics of man-machine interactionand computer mediated human communication intoconsideration is important.However, conventional studies on media communication(for example groupware, CSCW (Computer SupportedCooperative Work), CMC (Computer MediatedCommunication) etc.) did not discuss the effects ofrhetorical structure on information recognizing process.¡�¢&£ ¤�¥:¦#§�¨�§�©�ª+«4¬!©�§7ª�­4§�­�¦+©®¨�¯±°�² £�¥ ³�¨2´7©�¢�£µ¯=­7¶7©²�·�­4ªK¨�¢�¨² ¸7¹�¥ ¢7´ª�©º­�´4¢�¥ £�¥ ­4¢&§7ª�­7º©�¦»¦K­7· ¼½4¾�¿�ÀÂÁ�Ã:Ä+à Å7Æ Ç È�É�Ê4ËÂÌ=Í�Î�Ç Ê4ÈÐÏ

Advertising Information Processing

2.1 Role of Advertisements on Brand KnowledgeFrom the perspective of brand management, advertising builds apreferred brand knowledge structure within the consumerperceptions [8], or modifies an existing structure. Systemizedbrand values reduce information processing load at the time ofdecision making of the brand, which underscores the need forstrategic brand equity design to be tightly linked to the consumergoal/plan knowledge structure associated with the productcategory [8]. Naturally, advertising plays a crucial role in the

construction and management of the advertiser’s intended brandequity as a consumer knowledge structure.

However, goal/plan knowledge structure is affected at the aspectsof memory and remembering not only by functional and rationalgoal [1; 6], but also through consumer consumption mode. Thisconsumption mode is the process where the consumer siftsthrough information and alternatives, which is influenced by thepurpose and context of the product, or by the consumerexpectations and outlook mode Therefore, the ads understandingprocess to construct the brand knowledge structure with theconsumer goal/plan knowledge structure, needs a higher-ordercognitive process which includes understanding of affectivesymbolic benefit, not just the process which reasonably conveysthe functional benefit.

From the perspective of advertising studies, not a fewresearches have noted that emotions are assumed toinfluence on thought processes [4]; to create a positiveattitude toward the ad and translate this into attitudestoward the brand [5]; or to work by transforming the user ’sexperience. Despite of this, there is currently no cohesivesystem that satisfactorily explains the role of emotions inthe perception and understanding of advertisinginformation; and consequently, there is littl e confidence inmethod used to assess emotional responses to ads.Vakratsas and Ambler (1999) classified over 250advertising studies concerned with ad effects into 7categories, based on the primary dimensions in consumerbehavior, such as cognition and affect. Since prior studiesof the information processing paradigm have focusedprimarily on the consumer cognitive actions, examininginformation acquisition, integration, and memory with anemphasis on rational aspects emotional response hastended to be viewed as a relatively peripheral process [7].

2.2 Cognitive Analysis of AdvertisementRecognizing ProcessHowever, cognition and emotion form a complex andinseparable relationship within higher-order humancognitive behavior [3]. Higher-order image processingexists in emotions. In the central route of the elaboration

li kelihood model, emotions play a substantial role inunderstanding product features.

From this perspective, the ads understanding process asbrand knowledge forming need to be considered as ahigher-order cognitive process which includes not onlyreasonable understanding of functional benefit, butunderstanding of benefit based on user and usage imageryand brand personality

2.3 Cognitive Appraisal TheoriesThis theory stresses the role of cognition in the generationof emotion, treating emotion as a product of cognitivebehavior, such as understanding, reasoning, anticipation,memory, evaluation, and motivation proposed thefollowing layers of understanding involving the processthat occurs while reading a novel.

Understanding as Problem Solving (PS). This kind of understanding of human actions is madepossible by a common understanding of a goal/planknowledge structure. Not only is this essential inunderstanding the characters in the advertisement, it iscentral to understanding the perception of productproperties, in which consumers calculate that a productproviding a given function is capable of providing a knownbenefit.

Understanding as Affective Reasoning (AR).This kind of understanding is invoked by using a sharedawareness of emotional states. It is applied to dramatispersonae in advertising narratives, films, and novels. If acharacter is walking with an expression of self-confidence,the perceiver/reader implicitly perceives the character tohave accomplished something important and satisfying asfollowing process. This is a rational understanding basedon correlation between action outcomes (achieved, failed)and the mental states of persons.

Understanding as Emotion Evocation (EmE).If the targeted viewer also happens to enjoy the advertisingcontent, the interpersonal emotions to the characters in theadvertisement which is different from AR are also invoked.In addition to the interpersonal emotions, the viewer alsoevokes the aesthetic emotions of the overall advertisingnarrative, or the manner in which the product is presented.This understanding is important for perceivers tocomprehend non-product aspect of brand, such as brandpersonality.

Understanding as Evaluation Evocation (EvE). Within this process, we also posit (interpersonal)evaluation, a type of cognition that differs frominterpersonal emotion. This provides a clear distinctionbetween the evocation of emotion and evaluation and a

correct understanding of the advertising content, whichtakes place in the realm of information processing. ‘Empathy’ is considered to be an affective responseinvoked when EmE is coupled with EvE. When empathyor sympathy occurs for a specific brand, we speak of ‘acceptance’ and ‘confidence’ .

Understanding as Desire Evocation (DE).Tokosumi [9] describes the following transition with regard o a shift in viewers’ reasoning process while watching a TV drama:observation phase --> understanding phase --> desire evocationphase. As Observation phase, the viewers typically perform metacognitions (i. e. , about TV dramas generally) reasoning based onthe titles and background music. As the story unfolds, the viewers’ desires emerge in the form of the reasoning ofexplanation creation in desire evocation phase. Considering thegoal of advertising communication which is consistent withchanging feelings to the direction of purchasing products [2], thegoal of advertisers is to achieve DE toward the brand.

3 Multi-Agents Based Approach

We regarded these simple processes as agents anddeveloped a multi -agent model of human recognitionprocess of TV commercials. How does each agent workand influence on the processes? How does each agentcollaborate and organize the brand learning process? Weconsider these matters through the implementation of themulti -agent system.

We selected brand communication in enterprises as ourresearch target and studied customer ’s recognition processof TV commercials form the viewpoints of audio-visualrhetoric and cognitive science taking into account offollowing points.

Relation between the recognition process (e.g. storyunderstanding process) and the rhetorical structure ofaudio-visuals

Affective and cognitive effects on human recognition byaudio-visuals that involve story depend and storyindependent elements.

A methodology for making audio-visuals based onconsideration of the relation between the intention ofinformation sender, rhetorical characteristics of the audio-visuals and the information receiver ’s recognitionprocesses.

As the results, the recognition process is regarded as aninteraction between story processes, character processes(two process as line) and brand processes (one process aspoint). It has been cleared that there are some interactiontypes. It has also been cleared that setting up viewpoint instory processing, affective and cognitive effects brought byaudio-visual contents are very important factors.

Fig.1 Viewer's Sensory Ontology

However, to develop information model for designingaudio-visual medium concretely, it is necessary to makeclear the structures of audio-visual, brand knowledge andgoal-plans, model the interaction between the elements ofthe these structures, and understand their mechanisms.Therefore, we tried to develop the multi -agent model foranalyzing customer ’s ads recognition. We design themodel that consists of sensory agents, short-term and long-term memory agents and make clear the inner structures ofagents and collaboration mechanisms.We develop a multi -agent model to simulate understandingprocess based on the Cognitive Appraisal Theories. Themodel consists of f ive modules (See Fig1). We alsoimplemented and evaluated a prototype system based onthe model by using two protocol samples.

3.1 Ontological Analysis of TV CommercialThe elements in the audio-visual scenes of TVcommercials are classified to threefold, (a) object in thescene (e.g. product, person, character etc.), (b) space-timeevent in the scenes (e.g. story, action etc.), (c) rhetoricalmood in the scenes (e.g. visual tone, narration etc.)

To investigate the relation between the recognitionprocesses and the elements that appear in the scenes of TVcommercials, we did an ontological analysis of TVcommercial(see Fig.1). In concrete, we developed theontology of the recognized objects in the TV commercialsby explicitly describing the concepts and their relationsform protocol data.

Ontology is a term in philosophy and defined as “ thebranch of metaphysical enquiry concerned with the studyof existence itself ” . In AI community, an ontology isdefined as “an explicit specification of conceptualizatio n”which is intended to use as base for knowledgerepresentation. Ontology is represented as systems of

symbols in computers, i.e., symbols and their relationsrepresent concepts and their relations respectively.Ontology commits only declarative nature of systems ofconcepts because its purpose is to provide content forknowledge representation.

To computational implementation of the human reasoningand understanding process, it is important that we study theontological structures concerned and involved with theabove processes. We can understand the human intelli gentactivities by making clear the ontological structure.

3.2 Multi-Age nt ModelFig.3 shows the overview of our multi -agent model. Themodel consists of sensory agents, monitoring agents, aworking memory, high-order recognition agents and long-term memory agents.

There are there typical agent model, considering agentmodel (e.g. BDI model), reactive agent model (e.g.subsumption architecture) and hybrid model of the twoformer models. Our model is the hybrid model, thesubsumption model is adopted for sensor agents, theconsidering agent model is adopted for short-term andlong-term memory agents.

Ñ#Ò7ÓKÔÕ�Ö2×ÙØ=Ú�Ò7Ó#Û0ÔSensory agents write information about recognized objectsto the working memory. In actual human cognition process,there are interaction sensor and long-term memory, inpresent our model, the agents treat only object whichappear in protocol data. The sensory agent family consistsof the background, character, behavior, article and rhetoricagents (see Fig. 2).

ÜÞÝ7ß#àâáeÝ�ãà�ß�ä±å=ä�æ7ß#á0ç

Fig.2 Inner Structure of A Sensory Agent

In a monitoring Agent Family, there is fourfold agent:meta-monitoring agent, story processing agent, characterprocessing agent, rhetoric processing agent.Meta-monitoring agentThe meta-monitoring agent selectively responds to theelement which changes viewer ’s viewpoint in the TVcommercial (e.g. audio-visual effects, scene changes ) anddirects sensory agents and monitoring agents to theremarkable object of the next scene .Story processing agentStory processing agent understand consecutive scene asstory in the TV commercial. If this agent is only active, thebrand knowledge process isn ’ t done well .Character processing agentCharacter processing agent monitors character objects andtheir behaviors.Rhetoric processing agentRhetoric processing agent responds to the rhetorical moodof the audio-visual selectively.Story processing agentThe story processing agent, the character processing agentand the rhetorical processing agent also are related to thenon-product related property of the brand knowledgestructure. The objects and their states that these agentsremark are written into the brand knowledge on brandknowledge processingBrand knowledge processing On the other hand, The Brand knowledge processing agentresponds the information related to the product relatedproperty, it writes and changes the brand knowledge.

èêé�ë7ìKí�îðï�í4ñ�í�ò#ó!í7ôõ�ë�ö#éâ÷»é�õ7öùø=ë�í7ö#÷0úHigher order recognition agents understand advertisinginformation based cognitive appraisal theories. There aresix types of agent: (1) MC agent, (2) AR agent, (3) EvEagent, (4) PS agent, (5) EmE agent, (6)DE agent.

ûýü�þÿ���������� ü�þ �Recently, in cognitive science, working memory isremarked and regarded as the basis of high-order cognitivefunction such as language understanding, consideration etc.

It seem that working memory is related to (1) selectinginformation from environment, (2) maintaining andoperating selected information, (3) generating the results ofunderstanding information (generating answers), (4)outputting the actions based on the answers, (5) evaluatingthe result of actions.

In our multi-agent model, working memory architecture isadopted for multi-agent collaboration.

Sensory agents write the information about TV ads objectsinto the working memory. The information in the workingmemory are monitored, read, changed by meta-monitoringagent, high-order recognition agents, long-term memoryagents.

������������������������ ����� ���"!$#

Long term memory agents are twofold:(1) goal-plan agentand (2) long-term memory agentIn the process of deducting answers such as makingdecision concerned with products, customer ’s brandknowledge is operated as the system of goal-planknowledge represented in if-then rule. The goal-plan agentmaintains and manipulates the customer ’s goal plans.On the other hand, long-term memory agent maintains thecustomer ’s long term memory, for example, knowledgeconcerned with product category, brand, product, character,TV commercial etc. The agent retrieves the relevantinformation form long-term memory responding to thecontent of working memory and requests from other agents.

Fig.3 Multi-Agent Model

Fig4. Scene Analysis

In generally, it is diff icult to describe all of common senseknowledge, so we focus on the knowledge concerned withbrand knowledge management.

4. Evaluation and Discussion

In this section, we analyze a viewer ’s recognition processon each scene(see Fig.4) using our multi -agent model.

Sene1In the first scene, three characters appear and the brandname is narrated at the same time. The representation ofthe scene is focused on portraying character ’s appearanceactively than informing the brand.

The customer ’s understanding ads is concentrated on EmE( “ lovely ” , “active ” and “cheerful ” ) to the characters andtheir actions. The fact seems to enhance the motivation tofollowing brand knowledge process.

Sene2In the second scene, the characteristics of this product areexplained thorough character ’s action. The main messageis “The warm face pack in 60 seconds makes your skinslick. ”In general advertising, such messages are often representeddirectly. For example, characters send messages to viewers,or the process of permeating into skin is described withfigures. However, in this ads, the process of skin ’s beingwarm and slick is described thorough the character ’sbehaviors. Specially, the look of warming skin representedthrough the character ’s action in thermograph. It seem thatthis representation enable to communicate the productproperty more effective than the representation with onlyfunctional figures. And the narration “ the warm face packin 60 seconds ” is announced as if it supplement thedescription.The audience is not only interested in but also evaluatedthe description as the product benefit.

Most of the directive explanations for the functionalproperties of the products are recognized superficially admeta cognition. Even if the product benefits are understoodthorough inference of the product property, the benefits areoften regarded uncertain.

On the other hand, in this case, the audiences understand asaffective reasoning through the scenes that the charactersare active and satisfied with the product concerned. Thisunderstanding contributes to arouse customer ’s expectationto the product benefit. In short, AR works effectively tounderstanding the product property and benefit through thecharacter ’s behaviors.

Sene3

Third scene describes the characters are enjoying shoppingat free market changing the scene that is not related to theproduct.However, as result of analysis of viewer ’s understandingprocess, AR and EmE to this scene are translated tofriendly feeling to the brand. Understanding brandpersonality works by removing the blocking factors ofcustomer ’s use. It is typical case that understanding brandpersonality activates through character ’s behaviors.

Sene4In fourth scene, the product name is announced showingthe package design of the product.The viewer understood the red and roundish design asEmE ( ” It ’ s lovely ” ).

The viewer ’s motion to characters and their behaviors formscene1 to last scene4 is translated to the emotion to brandpersonality. The viewer ’s background knowledge related

to esthetic activates through the brand name ”Tsuru-tsuruface estetic (silky face esthetic) ” (see Fig. 5).

5.Conclution

In this paper, we propose a multi -agent model foranalyzing recognition process of advertising information.We selected brand communication in enterprises as ourresearch target and analyzed customer ’s recognitionprocess of TV commercials form the viewpoints of audio-visual rhetoric and cognitive science taking into account offollowing points with our model. As result of analysis, it isfound that the multi -agent model make clear the interactionbetween elements of customer ’s knowledge structure and Itis possible us to treat the sender ’s purpose, the rhetoricalcharacteristics of audio-visual and viewer ’s cognitiveprocess with explicit and computer-executable form.

Future works are as follows:Developing a model considered an interaction betweenvarious mediaExperimental evaluation with large-scale protocol dataDesign the design supporting system for audio-visualcontents

References

[1]Bersalou, Lawrence W. , “Ad Hoc Categories, “ Memory & Cognition, 11 (3), 211-227. 1983.[2]Dutka, Solomon, “DAGMAR, “ trans. Toshio Yamaki,Diamond Publishing (In Japanese), 63-88. 1995.[3]Fischbach, Gerald D. , “Mind and Brain, “ ScientificAmerican, 267 (September), 48-57. 1992[4]Gardner, Meryl P. , “Mood States and ConsumerBehavior: A Critical Review, “ Journal of ConsumerResearch, 14 (December), 281-300. 1985[5]Gorn, Gerald J. , “The Effect of Music in Advertising on Choice Behavior: A Conditioning Approach, “ Journalof Marketing, 1 (Winter), 94-101. 1982[6]Izumi, Kiyoshi , “Restructuring Advertising Awarenessthrough Goals, “ Yoshida Hideo Memorial FoundationResearch paper (In Japanese). 1996.[7]Kato, Yuichiro , “Brand Quality Control: A New Strategic Brand Building and Management, “ Proceedings of SUGI-J ’99, 339-342. 1999.[8]Keller, Kevin L. , “Strategic Brand Management:Building, Measuring, and Managing Brand Equity, “ Prentice Hall , 87-129, 217-266. 1997.[9]Tokosumi, Akifumi (1997), “Media Understanding andKansei, “ In Approaches to Kansei Information Processing,ed. Saburou Tsuji , Science Publishing (In Japanese), 110-113.[10]Vakratsas, Demetrios and Tim, Ambler (1999), “How Advertising Works: What Do We Really Know?” Journalof Marketing, 63 (January), 26-43.