1

Convocatoria de ayudas de Proyectos de Investigación

MEMORIA TÉCNICA PARA PROYECTOS DE LA CONVOCATORIA DE I+D TIPO A ó B

RESUMEN DE LA PROPUESTA (Debe rellenarse también en inglés) INVESTIGADOR PRINCIPAL: FRANCISCO JOSE PERALES LOPEZ

TITULO DEL PROYECTO: Interfaces Tangibles Avanzadas en Domotica Asistencial (ITADA).

RESUMEN (debe ser breve y preciso, exponiendo sólo los aspectos más relevantes y los objetivos propuestos):

El proyecto presentado pretende avanzar un paso más en la definición de nuevos paradigmas de interacción multimodal entre el ordenador y el usuario final. Hoy en día es un hecho la comunicación electrónica entre personas a diferentes niveles y en diferentes entornos (Chat, servicios GMS, E-Healt, Escenarios Inmersivos Virtuales, etc.). En experiencias anteriores se han desarrollado los sistemas VBI (Visual Based Interfaces) por ser la vista nuestra fuente principal de información e interacción con el entorno. Por otra parte, se han explorado e implementado sistemas hapticos convencionales, es idea de este proyecto aumentar el grado de multimodalidad del interface tangible de manera que se incluya un estudio y aplicación de sentidos como el oído, el gusto y el olfato y su combinación para el diseño de interfaces multimodales tangibles. Por otra parte, los agentes inteligentes distribuidos permitirán modelar adecuadamente el comportamiento del sistema y la interacción con el usuario. Dado que la aplicación final del proyecto es la implementación de un sistema domotico de asistencia en ancianos o discapacitados en su propio hogar, también un aspecto importante es la implementación de métodos fiables de verificación de la identidad de los ancianos y cuidadores, es por ello que se incluye un modulo de multibiometria en el desarrollo del proyecto propuesto. Finalmente, la implementación de los métodos de asistencia virtual en un prototipo de robot físico real que permita ayudar en determinadas tareas al anciano se presenta como un reto muy interesante y una demostración precisa de la viabilidad de los interfaces naturales aplicados a la robótica móvil. Como aplicaciones fundamentales del desarrollo teórico anterior, nos planteamos el desarrollo de un avatar tangible en un ambiente domotico de asistencia diaria al anciano y con funcionalidades de e-Healt de carácter crónico. Por experiencia de proyectos anteriores (TIN2004-07926, TIC2001-0931, TIC1998-0302-C02) los sistemas VBI permiten la captura del movimiento y la interpretación de las acciones del usuario mediante agentes inteligentes, por ello la petición de este proyecto se basa el experiencia ya adquirida en VBI y hapticos para avanzar en sistemas mas portables y ubicuos en el paradigma final del interface natural. La interacción persona-hogar permitirá crear sinergias muy importantes en múltiples campos. Nos centramos en el desarrollo de herramientas TTS-ARS, Sistemas sinestesicos, y la combinación de ambos en modelos de agentes inteligentes que manejen esta multimodalidad. Como requisito habitual, las herramientas y software utilizado deben adaptarse a los estándares actuales y buscando siempre la portabilidad y compatibilidad máxima posible al mismo tiempo que los sistemas sean de bajo coste o bien con dispositivos estándar evitando complejos sistemas de realidad virtual sino son estrictamente necesarios para el discapacitado. Por experiencias anteriores la aplicación de una metodología para la especificación de requerimientos, análisis y diseño de los procesos se considera muy necesaria en proyectos grandes donde se manejan muchos recursos físicos y lógicos.

2

PROJECT TITLE: Advance Tangible Interfaces in Assistive Domotics

SUMMARY: The project we present hopes to advance another step in the definition of new multimodal interaction paradigms between the computer and the final user. Nowadays, electronic communication among people at different levels and different environments (chat, GMS services, virtual immersive scenes) is a fact. In previous experiences VBI (Visual Based Interface) systems have been developed, since they are the main font of information and interaction with the environment. On the other hand, conventional haptic systems have been explored and implemented. This project pretends to enlarge the interface’s multimodal degree of the tangible interface to include the study and application of senses such as hearing, taste, smell and their combination in a new design. Likewise, distributed intelligent agents will allow the proper modeling of the system’s behavior and its interaction with the user. The project’s final application is the implementation of a domotic home assistance system for the elderly and disabled. Consequently, an important aspect to be considered is the implementation of reliable verification methods for the identification of the elderly or disabled and their caregivers. To this effect, a multi biometric module is included in the proposed project. Finally, the implementation of the virtual assistance methods in a robot that helps an elderly in certain chores is an interesting challenge that will demonstrate the feasibility of natural interfaces applied to mobile robotics. As a basic application of the previous theoretical results, we propose the development of a tangible avatar in a domotic environment for the elderly’s daily assistance and with tele-assistance functionalities in chronic cases. Our experience in previous projects (TIN2004-07926, TIC2001-0931, TIC1998-0302-C02) include VBI systems that allow movement capture and the interpretation of the user’s actions using intelligent agents. Therefore, this project’s petition, is based on acquired experience in VBI and haptics, in order to advance to more portable and ubiquious systems in the final paradigms of the natural interface. The man-home interaction will allow the creation of very important synergies in a wide range of fields. We will concentrate in the development of TTS-ARS tools, synestetic systems and their combination in intelligent models that handle this multimodality. The tools and software used must adjust to actual standards and seek the highest level of portability and compatibility. At the same time, systems must be low cost or use standard devices avoiding complex virtual reality systems unless strictly needed by the elderly or disabled. Due to prior experience, the application of a methodology for requirement specification, analysis and design is very necessary in large projects that manage many physical and logical resources.

3

2. INTRODUCCIÓN (máximo cinco páginas)

Deben tratarse aquí: la finalidad del proyecto; los antecedentes y estado actual de los conocimientos científico-técnicos, incluyendo la bibliografía más relevante; los grupos nacionales o internacionales que trabajan en la misma materia específica del proyecto, o en materias afines.

According to the expectation of life, people’s social and vital needs are currently on the rise and the TIC area are a key item in present society in order to guarantee this quality of life to all citizens of any level or social class. ALthough it is certain that, the data processing systems have advanced in its benefits and its associated services, the paradigm of interaction between the person and the computer has been maintained stable by means of the classical GUI (Graphical User Interface) and with the mouse and conventional keyboard as physical devices. Although advances in natural communication systems exist, by means of voice and also by recognition of gestures, they are not of common or habitual use in the majority of common settings. Generalization of the utilization of data processing systems therefore requires a more natural and accessible communication and interaction between the end users and the computers. Because of this, research in IPO or Person-Computer Interaction appears as a natural consequence of the previous needs and develops thanks to the apparent improvement of the data processing equipment and the apparition of more sophisticated tools and applications. Human’s natural interaction is accomplished by means of the human body and its senses, therefore it is necessary to enlarge the classical paradigm of GUI so that the computer may be capable of identifying our gestures, phrases and expressions with the idea of a convergence toward a natural communication between computer and user. In this field, we encounter multiple areas of possible investigation. Obviously, at the moment of undertaking the problem it is necessary to mention with large detail the areas considered. Our prior experience in research projects has permitted us to explore in depth the VBI (Vision Based Interfaces) systems and the haptic devices. At the same time, we have developed systems of facial animation with a degree of acceptable realism. Finally multiagent systems have been explored, using commercial low priced robots and a basic prototype domotic system. The main objective of this new project, using the previous systems and the models and algorithms designed, is the introduction of multimodality with new senses (voice, touch, smell, flavor, etc…). On the other hand, biometrics allows identity verification by means of a non invasive system, that use systems that can be similar to the techniques used in the design of VBI. The objective is the study of multibiometric systems that improve the system’s reliabillity. Finally, the management and interpretation of this information, of different nature, is carried out through distributed intelligent agents. Therefore it is necessary to study in depth the agent’s models and architecture, their intercommunication, their coordination and which methods and tools are available for its implementation and testing. As a final aspect, we intend to apply the previous advances in a robot that aids the disabled with basic tasks in a home environment. The following sections present in depth what we intend to developed in each field of the proposal. Finally a general diagram of the system is given together with links to previous projects and concurrent petitions of the group and other similar research centers.

Tangible Interfaces. VBI Systems, ARS/TTS, Haptics, Smell and Taste

Actually, electronic communication includes multiple levels of complexity, fulfilled by means of a simple "Chat" or "GSM" services or more sophisticated Collaborative Virtual Reality systems. What is evident is that the systems capacities have evolved whereas the user-computer interaction remains constant and is based on Windows and Icons (GUI) classical paradigms and the management of the keyboard and mouse. A solution proposes the development of systems that allow the user to communicate with the machine in the same way as it does with humans. This is where VBI systems appear, the ARS/TTS (Automatic Recognition Speech/Text to Speech) and finally haptics and other devices of sensory perception. As it is commented in the text [ShP06], new technologies offer us some fantastic possibilities at the moment of handling and processing information by means of computers. But it is the interdisciplinar design science, called person-computer interaction (also HCI, Human Computer Interaction) the one that has begun combining the data acquisition methods with the

4

conceptual frameworks of experimental psychology and the powerful data processing tools to approach technological advances to the way of acting of humans. Interfaces have evolved in a parallel manner to the data processing systems and their requirements, and at the same time the necessary tools have been used in each case. In the following figure by Palanque, the evolution of the systems, proposed solutions and the tendency in a next future can be seen.

As we can observe, some key applications presented require of ubiquitous and tangible interfaces so that the potential user can interact with the environment that surrounds him in a natural manner. Chapter 9 of [ShP06] shows in detail the opportunities of these interfaces, the technologies used and the possible obstacles to solve. As key characteristics of the tangible interfaces and/or precepts we emphasize the signal multimodality and the naturalness in the human-computer communication process. Being more precise, tangible Interfaces [Car01] are those that associate determined physical objects (physical representations) to certain digital information, using these objects at the same time as representations and as information control of what they represent (actually of its computational substrate). The term, minted by Ishii [Ish97], refers to a new paradigm of interaction that modifies (or eliminates) the traditional functional separation between inputs and outputs of the data processing system’s interface.

The integration without separating representation and control differs substantively from the main current based on user’s graphic interfaces (GUI), the most common in the HCI (Interaction Person-Computer in English) discipline. The GUI establishes a fundamental distinction among the input peripherals (mouse, keyboard or other), associated to the system’s control and the output devices (screen), and associated to the system’s viewing and its changes of state. The tangible interfaces explore the conceptual space that al is opened to eliminate this separation.

Tangible interfaces (ITU) give physical form to digital information, using physical devices that serve the data processing system at the same time as representations and controls [Hol04]. The traditional interface follows the MVC (Model-View Controller) model, separating the digital or view representation, given by the screen, from the control, given through the keyboard and themouse. Instead, the tangible interface follows the MCRpd (Model-control-representation (physical and digital)) model, that blurs the view-control separation and in its place separates the view in two types of elements: the cited real objects that allow direct manipulation (physical representation), and devices like screens, workbench, loudspeakers, etc. that

5

allows us to visualize or to listen digital information without no controllable real object (digital representation). The ITU provide a great variety of associations between physical objects and digital information. The physical objects often include embedded detectors or sensors; Duch as the following [Ull00]: - Pre-existing objects as chips, models, demos, etc. - Mechanical or electronic Elements, for example, real components or their models - Components originating from prior experience, like those used in control and air navigation. These objects can be associated with information elements such as the following: - Static multimedia elements such as digital images and 3D models - Dynamic multimedia elements like video or animated 3D models - Digital attributes such as color, texture and other - Computacional operations and applications - Simple data structures (trees, lists,) of any type of information - Complex data structures (data combinations, operations, attributes) - People, places and object representation The ITU tend to select and to interpret the systems of physical objects according to one of the following categories: - Based on spatial relations, such as object’s form and position, distances and relative orientation among them, movements, turns, etc. - Based on abstract relations between objects, such as sequence, proximity, color, etc. That allow property mapping on the digital side by the definition of adequate "physical-digital languages". - With a constructive base, such as games with modular pieces, LEGO type, from which assemblies are carried out, associated to robots, flow simulations, shadows, etc. - With a mixed base, since the previous classification is not exclusive. It is not the objective of this project to present an exhaustive description of all the methods applied to the design, analysis and interaction in the area of user interfaces. The bibliography is extensive and the advances can be consulted for each field. In the following sections, various aspects of the fundamental concepts are commented related to the final objective of obtaining a natural interface that allows us to work with the computer as if were another real user. Thus, we will comment those points related to the speech and its comprehension, the biometric systems, distributed intelligent agents and finally the applications of the project. In the speech and generation of conversational agents area considerable progress has been carried out. The work of [PeB03] presents the problems and the details involved in the creation of Conversational Agents (Embodied Conversational Agents). They have a human aspect and they can be "embodied" in a user’s interface, being capable of interacting with it. This is, they should be capable of perceiving and understanding what the user is saying, but they should also respond in non-verbal and verbal form. These agents have been created to interact with the users within a person to person conversation. The synthesis of voice, in order to reproduce the conversations that accompany to facial gestures, is controlled by TTS Festival. Besides, TTS Festival gives the necessary information for the synchronization of the facial expressions with the voice (such as, list of phonemes and their duration). Some systems are capable of identifying individual moods and to act consequently. Such is the case of [LNP01] system, where a prior summary of the methods used for the automatic classification of the different types of voice depending on the emotional state of the speaker is done. The collection of data used for the analysis comes from taped dialogues in a commercial application developed by SpeechWorks. These dialogues were obtained from telephone conversations between users and an agent-machine, developed for a telephone application by SpeechWorks. . The popularization of the mobile phone system generates an increment in the mobile interfaces systems creation, as is the case of [BPR03], where a generic platform for the implementation of mobile and distributed interfaces capable to reproduce dialogues is proposed. The motivation is the increase of situations in which a human user should interact with machines, such as logistics, inventory management

6

distribution centers. This interaction is carried out by means of Human-Machine interfaces (Human-Computer Interfaces). In many cases the interfaces must be mobile, for example, pencils, jewels, etc. The authors list the implementation of a software solution with voice technologies (TTS, ASR, and management of dialogues) in mobile environments and its integration in architectures of other suppliers: dates browsers, data presentation servers and information management systems. Finally, if the final objective is the creation of a naturally interaction environment, the voice should be integrated with synthetic avatars. The system proposed by [DoM04] is an example. In this article, Baldi®’s implementation is explained, that is, a three-dimensional agent gifted with synthetic and natural voice. This agent guides students through a variety of exercises designed to learn the vocabulary and grammar of a language, to improve the articulation, and to develop phonological and linguistic capacities. In this application, the development of the voice’s synthesis was carried out along the whole life cycle of the application, carrying out improvements from the results obtained of the different tests. It tested to be very effective in people with auditory disability, autistic people, and in the learning of a second language. The use of these systems extends to all people with certain functional limitations. The SYNFACE system [BKK04] is a clear example of it. It is a telephone application designed for people with auditory dissability; it shows speaker’s lip movement at the other side of the telephone, in synchronization with the voice. The SYNFACE system is composed by a voice-recognizer, that recognizes the incoming voice, and a synthetic head that speaks. The recognizer’s output is used to control the articulated movements of the synthetic head. This prototype is available in three languages: English, Dutch and Swedish. Both, the voice recognizer and the controller of articulations and facial animation were developed in C + +. The previous example sufficiently enlightens the need of research in Conversational Agents, fundamental in the Tangible Interface’s area. Because of this, it is included as a key point in the integration of the information generated by the VBI and by the systems ARS/TTS based systems. Even though the proposal is very extensive and ambitious we cannot forget other senses that habitually have been consigned to a second plane, but that in determined activities are fundamental (tasters, maitres or cooks, etc.). Without being the main objective of the proposal, a desire to know the present systems, to contrast them and to validate them, exists. In the future, a possible simple application that improves the tangible interface designed can be studied.

Multi-biometry. Aplicacions

In general, the biometric systems allow us to access the computer systems by means of some personal body identification, in such a way that the identification is univocal. Security and access control acquire more and more importance, in an already massively computerized society, where the stored data is also confidential in many occasions. An option selected more and more frequently when providing a restricted access to identify those users that access a system or service is biometry or measurement of characteristics that only apply to the human being. This allows us to establish a one-to-one relationship between the user stored in the system and the real person, without the necessity of other elements subject to loss or undue acquisitions as cards or passwords. In spite of it, the biometric systems that use only one characteristic or feature are far from perfect and can present errors, although always in a small percentage. These errors can be exibited at the time of accepting an imposter as a system’s user (what is known as false acceptance) or at the time of rejecting an existing user (false rejection), or when registering a user in the system. In addition to this, in many occasions we deal with biometric data with a high level of noises which can produce “fraud”. Many of these inconveniences can be palliated combining multiple biometric features. This usually improves the performance of the system. The intention of this project is to create a library, independent of the device and the type of biometry, that represents a flexibleand complete tool for the creation of multimodal biometric applications and that can be integrated into the domotic system. This library will allow the developer to add new types of biometries, to manage their devices for scanning purposes, to select what types of biometric features define each user and which algorithm is used to combine them and to manage users within a biometric data base.

7

In this way, the same library can be used to develope different biometric applications, allowing us to adjust application’s behavior to the characteristics of the population susceptible to use the system and to the necessities of the security and access control system that we intend to implement. In addition, the fact that all applications were developed on the same interface will allow as well a simple and agile communication between different securities systems developed within the same organization. On the other hand, the bookstore implements a new system that allows, from an algorithm that combines two biometries, improvement in the user’s biometric features stored in the data base. Also changes in the different accesses to the system are determined, instead of remaining static from the moment of the user’s registration as it happens with classic biometric systems. A generic biometric system is basically made up by four differentiated units: Sensor unit: featue capture by means of a scanner or other input device, as unprocessed biometric data. Feature extraction unit: processes the biometric data in order to extract a set of features that represent the captured biometric feature in a compact and controlled way. Comparison unit: compares the set of extracted features with the templates stored in the data base and generates a similarity value. Decision unit: uses the similarity values to determine or validate the identity. As a result of the action of these units, the biometric systems handle a series of entities. The different classes that compose the library represent these usual entities that can be found in any biometric application and at the same time implement the different related units: - Biometric image: image from which some determined biometric features belonging to an individual are extracted. This image can be acquired by a scanner or other input devices, it can also be imported from a file. - Biometric feature: class that controls and characterizes a determined individual biometric features and carries out the acquisition and comparison process. This entity centralies the functionalities that allow the acquisition of biometric images from the biometric device used (by means of the sensor unit), the feature’s extraction and storage process, from the acquired biometric images biométricas (feature extraction unit) and the comparison of the biometric features belonging to different individuals (comparison unit). - Biometric user: system’s user or potential user with restricted access. A set of features that represent the user are extracted using the previously mentioned entities. They will allow us to differentiate the user from all other users that use the system and from all possible intruders that desire to pass for the user in order to access the system. - Biometric database: those biometric users defined by the same set of biometric features are stored jointly to represent the identity of those individuals with system’s access. If we say that all users of a same database are defined by the same set of biometric features, we mean that the use of different biometric features makes no sense for each user, since its comparison is impossible (for example: we will not store the iris and figerprints of some users and the retina and facial image for others). At this level it should also be mentioned that as well as the different user’s features, the algorithm that combines these features will be common to all the users. These two application characteristics are very important since its choice can produce systems with different security. A basic plan of this methodology of action (with UML notation) can be seen in the following figure:

8

Figure: Application Biometric Structure This methodology assures one of the main objectives of the library, that is, to make the user’s management independent from the biometric databases, as well as the general application operation from the devices and biometric features used. The importance of the proposed system is in the multimodal biometric supported systems. These types of systems can be consulted in [RNJ06], since an exhaustive enumeration escapes the scope of this project. I simply want to stress the idea of multibiometric system design. On the other hand, there is a need to design different decision algorithms that will improve the user's validation process.

9

Distributed Multiagent Systems. Methodology and Tools

In general, Artificial Intelligence (AI) is more convenient for problems where a solution, an explanation or planning is needed. On the other hand AI [Lan89] is more adequate in those cases in which there not really anything to solve is, but something to perceive, and before which one must act. The elements or tools that presently exist to create Intelligent Systems, the majority belonging to the AI field, conform what is known as Technology based on Agents. Since the 90’s, there has been a fast development of these technologies based on agents [Mae94] [Mül96] [Woo97] [Les95]. The Agents are autonomous (or semi-autonomous) systems based on Software [Bra97] and/or Hardware that carry out complex tasks in dynamic environments. We understand autonomy as the capacity to decide in function of an internal representation of the "world", without having to resort to any central instance. The agents communicate with the environment (Perception and Action) and can perform changes in it, by executing actions.

In order to be able to solve this type of complex problems, an agent (or a multi-agent system) should offer the following basic aptitudes: • Reaction Capacity, in the presence of events and changes in the environment. • Planning and Decision Making Capacity, in order to perform tasks in agreement with one or more objectives. • Efficiency in decisión making and in the execution of the actions. • Interaction and Communication (with other agents) Capacity. • Capacity to adapt to new environments. An autonomous agent is a program or device that contains some type of sensor-actuador system. The agent operates in a virtual environment that perceives through its sensors and uses its actuadores to modify the state of the environment or the state of other agents. The agents are called autonomous because they are or should be self-sufficient. The agents are intelligent because the tasks that they carry out are classified as follows: "intelligence is necessary to solve this problem". Furthermore, many of these agents learn and evolve, what ratifies their description of "intelligent". Another preocupation in the design of synthetic agents is that they must be credible, that is, their behavior must seem natural, they must express emotions or they must use natural language to communicate [Bat94] [Loy97].

Finally, the concept of cooperative work is also ratifying the importance of this type of agents, distributed and autonomous. More and more applications are designed using different independent modules that cooperate. Such as the Multi Agent platforms, with collective or cooperative emerging behaviours. A practical implementation related to the proposal consists in the utilization of a low price mobile robot that integrates the VBI and ARS/TTS developments and the haptics in a real home aid system. Prior experiences with the PIONEER II and the AIBO of Sony robots have been carried out, but the idea can be generalized by using more versatile systems that adapt to the user’s disability.

Domotic assistance and e-Health.

One of the current emerging technological concepts is that of Intelligent Environment, which includes all of the devices and applications whose objective is to facilitate interaction with the user in a more natural way. The theoretical concepts and algorithms designed in the previous sections should be summarized in a real application that allows us to improve society’s living standard. In particular, the European society suffers a natural aging process of its population, which increases the welfare needs of hospitals and health centers that must deal with patients with chronic problems and with the elderly. If the trip of these users to the health centers is avoided we can improve the quality in other patient’s treatment also improving the economic aspect of each individual’s treatment. Because of this, a basic task of the project is the application of the tangible interfaces designed in the home environment. The supervision of the elderly and/or sick will be guaranteed by the intelligent agent that will be in permanent contact with the health centers and the respective doctors. The following figure shows a proposed system with the main classes and information.

10

The experience of the “Graphics by Computer, Vision and Artificial Intelligence Group” of the Math and Computer Science Department of the UIB in previous projects (TIN2004-07926, TIC2001-0931 and IST-2001-32202), and the execution of the preceding tasks, will also allow the development of the system’s natural answers in graphic form by means of Distributed Intelligent Agents.

Project Environment & Previous work.

The objective of this section is to situate the origin of the investigation that will be carried out, as well as its tasks. It is not an exhaustive introduction but allows us to obtain a precise idea of the amplitude, interdisciplinary nature and complexity of the final problem or proposed project and at same time obtain an idea of the wide range of applications in the actual society. In particular we are interested in the results developed for final user. Given the experience of the group in similar projects, and as an organizing idea, we intend to situate the proposal’s environment and to justify the completed project in the global context of the research areas of the “Graphics by Computer, Vision and Artificial Intelligence Group” of the Math and Computer Science Department.

The proposed system is just another step in the development of advanced multimodal interaction systems. It is based on some partial results obtained in a previous project TIN2004-07926 where VBI (Vision Based Interfaces) systems were designed and implemented. Some haptic RN aspects were also explored and the definition of a butler’s basic control prototype in a domotic environment was given. Another fundamental part of the previous project was the graphical component for a trustworthy representation of the virtual actor or avatar.

In this new proposal, the VBI concepts are assumed to be known and will be used in new designs; known graphical aspects will also be used. New ideas in the field obtained from other projects or from projects done in collaboration with other universities will be imported. This will allow us to clearly center the proposal in the new aspects that we consider: the tangible interface’s multimodality and its management by means of distributed intelligent agents in an asistencial domotic environment.

11

The proponed project is another step in the research group consolidation, in accordance with the politics of the ministry and the guidelines of the national plan. The research proposals have been divided in specialized areas promoting the generation new young researchers without the loss in the continuity of the work carried out by the initial group. This proposal does not pretend to generate greater funds but to structure the objectives achieved. Because of this, collaborations with other projects of similar centres by yielding EDPS with the idea of complementing the proposals are presented. In a global context we intend to synthesize the proposals already granted and finished by the group, the present proposal, and the short-term petitions and the middle to long term actions, both nationally and internationally. To finalize, in recent years (2000-2006) an explosion of research tasks in the lines of the proposal’s objectives have been produced and they are explicitly mentioned in the MEC’s priority lines. The advances have been spectacular and nowadays multiple applications related to human movement by computer, intelligent systems based on agents, domotic homes, speech synthesis and recognition systems, advanced inmersive viewing systems, advanced user’s interface exist. Different updated collections can be looked up they incorporate an extensive bibliography that has been carefully brought up to date with the most recent papers and "surveys" in the area. Even though this is very important, we would like to point out some aspects not so technological but equally important: the social component or dimension of the defined applications that will allow us to improve the quality of life of people, especially of the elders and disabled. The technological advances should be oriented to facilitate accessibility and equal opportunities to all of the potential users. These objectives are insured by the design of tangible interfaces that adapt to the person’s abilities. The TIC's will make posible for older people and dependent people to live independently more time and with more security, and at the same time, will facilitate a greater and more effective integration in the professional and social life, being thus preferential instrumental technologies. 3. Originality & Degree of Innovation. As it already has been exposed in the previous sections of the proponed project, we consider the project as another step in the integration of the data processing technologies that generate services at different levels by introducing the concept of environmental intelligence (Ambiental Intelligence, Aml). AmI allows us to integrate intelligence in the daily environment that surrounds us (home, street, city transportation, public places, hospitals,) which will make our life easier. It allows us to forsee the future of the information society and to see knowledge as a convergence of computation and the ubiquitous communication, and the easy to use interfaces, in a digital environment that will be conscious of its presence and of the general context of the situation, being been able to adapt and to respond to the needs, customs and emotions of the end user. The main and innovative objectives of the new project consist in integrating the theoretical VBI concepts and the mutlimodal person-computer interaction systems with systems based on distributed intelligent agents. Also, an environment for a real application with a strong social component that affects the quality of life of people with physical limitations will be defined.

Our proposal’s main objectives are: 1. Development of a tangible Interface that combines basic VBI with more advanced TTS/ARS, haptic systems and other senses. 2. Improvement of the distributed intelligent agent models in order to control multimodal environments with user’s natural information. 3. Improvement of the biometric systems and particularly exploration of fusion systems in biometry. 4. Adaptation of the designed systems to mobile robotic elements for home aid. 5. Specification of all of these aspects for commercial products in domotic TVassisting environments.

Regarding the software tools, those libraries needed for the development of open environments will be used, although concrete aspects may be implemented for each platform. The idea is to use the available standards in each field of the proposal and preferably open code standards. Likewise, design tools and analysis oriented to objects mentioned in detail in the specific subtask will be used. The use of tools and procedures based on software engineering has proved efficient and necessary in projects where the complexity is high and a rigorous control of the specification requirements should be carried out together with the quality control of the software developed.

12

4. Contacts with other universities, research centrers and companies The “Graphics by Computer, Vision and Artificial Intelligence Group” (GGVeIA), and others member of the Math and Computer Science Department of the UIB, are in permanent contact and collaborate activelly with other national and internacional groups that work in the areas presented in this project proposal. Although the subject matter and the areas covered are very extensive, well known experts in all the areas will collaborate very closely with the main applicant group. At any rate, following the MEC’s proposed philosophy and promoting the creation of new groups or new areas of investigation due to the specialization of the previous projects, the VBI systems have been separated from the present proposal in order to make their own investigation area in the project. In other words, a new line of investigation in VBI has been created by a young investigator although knowledge of previous projects will be used. In a similar way, those aspects related to visual realism of the possible avatars or virtual humanoids as well as the generation of realistic scenes and of virtual reality have been separated in parallel proposal, giving another young investigator the possibility to ask for a project in the field of geographical navigation with RV and the use of VBI's.

In an analogous way, the experience of the researchers of the “Grupo de Informática Gráfica Avanzada” (GIGA), belonging to the Computer Science Department of the University of Zaragoza, in the field of modeling and deformation of persons or humanoids, is key in order to complete graphical area of the proponed project. Common interests make possible the incorporation of members of our group (GGVeIA) in the application of the TANGIBLE project that they (GIGA) request. At the same time this effective and real collaboration offers the possibility to test in a realistic manner the topics presented in both projects since both institutions dispose of comminication infrastructures, domotic and virtual eality systems. At the same time, an intense collaboration CVC (“Centre de Visió per Ordinador” of the UAB)-UIB is maintained in all the aspects related to vision by computer, exchanging experiences in all fields. Even a framework agreement between UIB and CVC for cooperation at all levels exists. Periodic visits between investigators from both institutions are carried out. They have even collaborated in the request for joint European projects.

With respect to the UPC, we obviously know the extensive research carried out by some of the most consolidated groups in area of computer graphics in Spain. We know of all of their projects, which are considered as a reference Spain and in Europe. We mantain a fluid contact with the Professor Pere Brunet and his team, of the Languges and Systems Department, his help and advice withVirtual Reality topics is of great importance. Also a close relationship with the Electronics and Computer Science Departement of the USC is maintained, particularly in the field of development of virtual reality systems for navigation in virtual cities environments. The natural device control and the integration of VBI in its systems is an area of common research. Relationship is also maintained with the Dr. Julián Flores by the exchange of professors and doctorands. Internationally, the every other year the meeting of the AMDO (articulated motion and deformable objects) Conference in Majorca has allowed us to consolidate our work relation with the main investigators in fields related to the project’s proposal. Particularly, very close relationships are maintained with the following investigators: Dr. Ronan Boulic, MediaLab, EPFL, Switszerland. He is a specialist in the field of prioritized cinematic inverse methods and collaborations have been published in international magazines [BVU05]. Even some of them have been awarded with an international prize to the better paper of the NoE ENACTIVE network of excellence. Also, the project’s main investigator has been invited to be a new member of the NoE network which shows an international recognition of his research.

Dr. A. Hilton, Center for Vision Speech and Signal Processing, School of Electronics and Physical Sciences of the University of Surrey has begun a relationship with the group in the human movement capture without scoreboards field. He has visited the UIB recently and has carried out a series of tutoriales and conferences. He has attended the last AMDO06 meeting with a high degree of satisfaction and we plan to expand our collaboration with him. Dr. B. Fisher, Member of Institute of Perception, Action and Behaviour, has been involved in the AMDO06 conference as general chairman, he has supervised the group’s research and he knows of its advances. He

13

was an EU evaluator for the HUMODAN project, giving a favorable report of the work developed by the UIB in that European project. Currently we plan to cooperate with him in the improvement of the VBI based systems in exterior environments and Dr. F. J. Perales will complete an investigation stay in his center from March to June of 2007. From a business point of view, it pleases us to mention that EPTRON INC., a national leader in the field of applications related to vision by computer, has already collaborated with us in the development of a humanoid animation system for multimedia and videoconference applications. We consider its participation as a business cazable of contributing with technical information and as a medium in possible resultant applications. From previous experience, we maintain their contribution in this new project. A very important point is the consolidation of the TAGrv S.L. business (founded in 2003), with a strong technological base and with newly graduates and UIB students, it participates fully in the development of proposed applications and their commercialization. The business has already participated in various research projects with the GGVeIA group, but it has also participated in projects from the Govern Balear and in the national UNIEMPREDIA competition where it was selected as the only Majorcan business of I+D projects in the virtual reality environment. On the other hand, the GGVeIA group is connected with all the nacional groups of the area by means of its active participation in the TIC2001-4570-E thematic network. Thematic network in Structuring, processing and generation of Digital Video in Multimedia Applications (2001-2004), which recently has been renewed for two years more. With all other national groups, periodic contacts are maintained, either for seminars or specific visits, well by means of the Spanish Congress of Computer Graphics (CEIG) and, at present, by means of the National Symposium of Form Recognition and Image Analysis (SNRFAI). At the same time we assist to international events in these areas (EG2003, IEEE, SIGGRAPH, etc.). The group also promotes international collaboration by means of the periodic organization of the AMDO (Articulated Motion and Deformable Objetcs) conference, http: //dmi.uib.es/~ugiv/AMDO2006/. This past July its fourth edition was carried out, it is considered to be consolidated conference and leader in its area at European level. The conference obtained a store of 85 by the Spanish ministry aid committee, the cut for aid was of 54 points and the evaluators commented on the very high quality of the conference.

Of the previous sections, it can be deduced that the group’s activity as for its collaboration with other national and internacional centers is active and expansive, promoting consortiums and the proposal of European projects such the one presented in the EU VI framework agreement, AALIVE, which was not approved but that obtained an evaluation very closet o the cut grade and that will be taken up again and presented to the next EU framework agreement in the AAL field. Since this is an interdisciplinary project with differentiated modules, a bibliography by specific sections is included. We exclude, except important cases, references previous to 1995.

14

Relevant Bibliography:

Papers Developed by Own researchers:

[MVM05] C. Manresa-Yee, J. Varona, R. Mas, F.J. Perales. “Hand Tracking and Gesture Recognition for Human-Computer Interaction”. Electronic Letters on Computer Vision and Image Analysis 5(3):96-104, ISSN: 1577-5097, 2005. [PVM05] F.J. Perales, J. Varona, M. Miro, G. Fiol, P. Fuster, E. Cerezo, S. Baldassarri, V.Remiro, F.J. Seron, A. Pina, I. Azkue. “El proyecto INEVAI 3D: Agentes Autónomos 3D, Escenarios virtuales e Interfaces Inteligentes para aplicaciones de Domótica y de Realidad Virtual”, in Actas del I Simposio sobre Computación Ubicua e Inteligencia Ambiental (UCAmI'2005), ISBN: 84-9732-442-0, Granada. September 2005. [VBP05] J. Varona, J.M. Buades, F.J. Perales. “Hands and face tracking for VR applications”. Computers & Graphics 29(2):179–187, ISSN: 0097-8493, 2005. [BVH05] R. Boulic, J. Varona, B. Herbelin, L. Unzueta, A. Suescun, A. Jaume-i-Capó, F. Perales, D. Thalmann. “Vision-Based Comparative Study of IK Techniques for Arm Movements”, in First International Enactive Workshop, Pisa, Italy. March 2005. [BVU05] R. Boulic, J. Varona, L. Unzueta, M. Peinado, A. Suescun, F.J. Perales. "Real-Time IK Body Movement Recovery from Partial Vision Input", in Proceedings of the 2nd International Conference on Enactive Interfaces (ENACTIVE05), Genova, Italy. November 2005. 1st ENACTIVE Best Collaborative Paper Award. [AMB04] A. Aguiló, P. Martinez, J.M. Buades, F.J. Perales, M. Gonzalez. "Human motion analysis and synthesis using graphical biomechanics models applied to disable swimming people", in the 3rd International Workshop on Virtual Rehabilitation, Lausanne, Switzerland. 16-17 September 2004. [VBP04] Javier Varona, José M. Buades, Francisco J. Perales. "Hands and face tracking for VR applications ", CEIG 04, Sevilla 2004 pp 201-218, ISBN 84-688-6998-8

[JVP06] A. Jaume-i-Capó, J. Varona, F.J. Perales. “Interactive applications driven by human gestures”, in SIACG 2006 Ibero-American Symposium on Computer Graphics, Santiago de Compostela, Spain. July 2006. [JVG06] A. Jaume-i-Capó, J. Varona, M. González-Hidalgo, R. Mas, F.J. Perales. “Automatic Human Body Modeling for Vision-Based Motion Capture”, in Short Communication Proceedings of the 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG'06), Plzen, Czech Republic. February 2006. [MAP06] C. Muñoz, D. Arellano, F. J. Perales, G. Fontanet. "Perceptual and Intelligent Domotic Systems for Disabled People", in IASTED International Conference on Visualization, Imaging and Image Processing(VIIP 2006), Palma de Mallorca, Spain. August 2006. [MVP06] C. Manresa-Yee, J. Varona, F.J. Perales. “Towards Hands-Free Interfaces Based on Real-Time Robust Facial Gesture Recognition”. Lecture Notes in Computer Science 4069: 504-513, ISSN: 0302-9743, 2006. [MVR06] C. Manresa-Yee, J. Varona, T. Ribot, F.J. Perales. “Non-verbal communication by means of head tracking”, in SIACG 2006 Ibero-American Symposium on Computer Graphics, Santiago de Compostela, Spain. July 2006. [MVP06b] C. Manresa-Yee, J. Varona, F.J. Perales. “Face-Based Perceptual Interface for Computer-Human interaction”, in Short Communication Proceedings of the 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG'06), Plzen, Czech Republic. February 2006. [BPV06] J.M. Buades, F.J. Perales, J. Varona. “Matching for Perceptual User Interface”, in Poster Proceedings of the 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG'06), Plzen, Czech Republic. February 2006. [BPG06] J.M. Buades, F.J. Perales, M. González-Hidalgo, J. Varona. “Upper Body Tracking for Interactive Applications”. Lecture Notes in Computer Science 4069: 514-523, ISSN: 0302-9743, 2006. [RVG06] I. Rius, J. Varona, J. González, J.J. Villanueva. “Action Spaces for Efficient Bayesian Tracking of Human Motion”, in the 18th International Conference on Pattern Recognition (ICPR06), Hong Kong, China. August 2006. [RVU06] R. Boulic, J. Varona, L. Unzueta, M. Peinado, A. Suescun, F.J. Perales. “Evaluation of on-line analytic and numeric inverse kinematics approaches driven by partial vision input”. Virtual Reality 10(1): 48-61, ISSN: 1359-4338, 2006.

15

[BPV04] José M. Buades, Francisco J. Perales, and Javier Varona , "Real time segmentation and tracking of f ace and hands in VR Applications", in International Workshop on Articulated Motion and Deformable Objects 2004, Palma de Mallorca, Spain. September 2004 [BPG04b] José M. Buades, Francisco J. Perales, and M. Gonzalez , "Human Body Analysis with Biomechanics Criteria" , International Workshop on Articulated Motion and Deformable Objects 2004, Palma de Mallorca, September 2004 [BPG04] J.M. Buades, F.J. Perales, M. Gonzalez, A.Aguiló and P.Martinez, "Human Body Segmentation and Matching using Biomechanics 3D models" , in 8th International Conference on Information Visualisation, London, England. 14-16 July 2004. [MVM04] Cristina Manresa, Javier Varona, Ramon Mas, Francisco J. Perales, "Real -Time Hand Tracking and Gesture Recognition for Human-Computer Interaction" , in Electronic Letters on Computer Vision and Image Analysis 0(0):1-7, 2004. ELCVIA. [BGP03b] Buades Rubio, J.M., Gonzalez Hidalgo, Manuel, Perales Lopez, Francisco; "A New Method for Detection and Initial Pose Estimation Based on Mumford-Shah Segmentation Functional"; LNCS 2652, Springer-Verlag; 2652, 117-125 (2003). [BGP03] Buades Rubio, J.M., Gonzalez Hidalgo, Manuel, Perales Lopez, Francisco; "Face & Hands segmentation in color Images na Initial Matching"; ICViW03, Las palmas de Gran Canaria, pp.43-48 (2003). [MMP03] M. Mascaró Portells, A. Mir, F. J. Perales López. " Shape deformation Models Using Non-uniform Objects in Multimedia Applications" , in First Iberian Conferenece, IbPRIA 2003, Puerto de Andratx, Mallorca, Spain, June 2003 Proceedings. LNCS. [PMM03] F. J. Perales López, R. Mas, M. Mascaró, P. Palmer, A. Igelmo, A. Ramírez. " A Colour Tracking Procedure for Low-Cost Face Desktop Applications" , in First Iberian Conferenece, IbPRIA 2003, Puerto de Andratx, Mallorca, Spain, June 2003 Proceedings. LNCS. [BPI03] M. Bez , F.J. Perales López, A. Igelmo. " Análisis Estadístico Multivariante de Medidas de la Cara para Aplicaciones de Animación Facial" , in Ceig 2003, XIII Congreso Español de Informática Gráfica. Universidade da Coruña. July 2003. [AbP03] M.J. Abásolo, F.J. Perales López. "Wavelet Analysis for a New Multiresolution Model for Large-Scale Textured Terrains ", in WSCG'2003 - 11th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision '2003. Czech Republic. [BuP02] J. M. Buades Rubio, F. J. Perales López. "Voxel Matching Reconstruction in Real Image Sequences of Human Avatars", in 1st Ibero-American Symposium in Computer Graphics. Guimaraes, Portugal. 1-5 July 2002. [MMP02] M. Mascaró Portells, A. Mir, F. J. Perales López. "P3DMA: A Physical 3D Deformable Modelling and Animation System" , in Second Internacional Workshop, AMDO 2002, Palma de Mallorca, Spain. November 2002 Proceedings. LNCS

[MMP00] M. Mascaro, A. Mir, F. Perales. “Elastic Deformations Using Finite Element Methods in CG Applications”. pp. 145-158. H.-H. Nagel - F. J. Perales Ed., Proceeding Workshop on Aritculated Motion and Deformable Objects, CTS AMDO2000, LNCS 1899, Springer-Verlag.. [PMG00] P. Palmer, A. Mir, M. González. “Stability and Complexity Study of Animated Elastically Deformable Objects”. pp. 59-71. H.-H. Nagel - F. J. Perales Ed., Proceeding Workshop on Aritculated Motion and Deformable Objects, CTS AMDO2000, LNCS 1899, Springer-Verlag.

[BIP00] J. M. Buades, A. Igelmo, F. J. Perales. “Modelos antropométricos a partir de secuencias de imágenes”. CEIG 2000 X Congreso Español de Informática Gráfica. pp. 395-396, 2000. [AbP99] M. J. Abásolo, F. J. Perales, “Generating Synthetic Images Integrated With Real Images in Open Inventor”. International Conference on Information Visualisation. IEEE, 1999 [YRP98] C. Yañiz, J. Rocha, F. Perales. “3D Part Recognition Method for Human Motion Analysis”. CAPTECH’98 Modelling and Motion Capture Techniques for Virtual Environments. pp 41-55. [MMP97] M. Mascaró, A. Mir, F. Perales. “Modelado y Animación de Objetos Deformables basado en la Teoría de la Elasticidad.” Congreso Español de Informática Gráfica.1997, pp.95-117,ISBN 84-8498-429-X. [PeT94] F. Perales, J. Torres. “A System for Human Motion Matching Between Synthetic and Real Images based on a Biomechanic graphical model”. Proceedings of the workshop on motion of non-rigid and articulated objects, IEEE, pp. 83-88, November, 1994, Autin, TEXAS.

16

Similar Projects & Systems: [AgC99] J. K. Aggarwal ; Q. Cai. “Human Motion Analysis: A review”. Computer Vision and Image Understanding. Vol. 73, N. 3, March, pp. 428-440, 1999. [AzP96] A. Azarbayejani, A. Pentland. “Real-time self-calibrating stereo person tracking using 3-D shape estimation from blob features”. Proc. ICPRV'96, Vienna, Austria. August 1996.

[BTha99] C. Babski, D. Thalmann, “A Seamless Shape for H-ANIM compliant Bodies”, Proc. VRML 99, ACM Press, pp 21-28, 1999. [CoH96] I. Cox, S, Hingorani. “An efficicient Implementation of Reids Multiple hypotesis Tracking algorithm and its evaluation for the purpose of visual tracking”. IEEE Trans on PAMI, Vol. 18, N.2, pp. 138-150, 1996.

[GaD96] M. D. Gravila, L. S. Davis. “3D model-based tracking of human in action: a multi-view approach”. pp. 73-80, 1996, IEEE. [Gav99] D. M. Davis. “The Visual Analysis of Human Movement: A survey”. Computer Vision and Image Understanding, Vol. 73, N. 1, January, pp. 82-98, 1999. [Gda96] D. M. Gravila, L. S. Davis. “3-D model-based tracking of humans in action: a multi-view approach”, Computer Vision Laboratory, Proc. CVPR, pp. 73-80, IEEE, 1996.

[GSB 01] M. Garau, M. Slater, S. Bee, M. A. Sasse. “The impact of eye gaze on communication using humanoid avatars”. Proceedings of the SIG-CHI Conference on Human Factors in Computing Systems, 2001, pp. 309-316.

[HHD00] I. Haritaoglu, D. Harwood, L. S. Davis. “W4: real-time surveillance of people and their activities”. IEEE Transactions on Pattern Analysis and Machine Intelligence, V: 22 Issue: 8, 2000. pp. 809-830.

[KaM96] I. A. Kakadiaris, D. Metaxas. “Model-Based Estimation of 3D human motion with Oclussion Based on Active-Viewpoint Selection”, pp. 81-87, 1996, IEEE.

[MEN00] A. Menache. “Understanding Motion Capture for Computer Animation”. Morgan Kaufman, Academic Press, 2000.

[MTA99] N. M. Thalmann, D. Thalmann, B. Arnaldi. “Computer Animation and Simulation”. Springer-Verlag, Computer Science, Proceedings, 2000.

[NYA00] T. Nunomaki, S. Yonemoto, D. Arita, R. Taniguchi, “Multipart NoN-Rigid Object Tracking Based on Time Model-Space Gradients”, AMDO 2000, First International Workshop. Palma de Mallorca, September 2000. pp. 72-82.

[ORC 02] J. Ostermann, J. Rurainsky, R. Civanlar. “Real-time streaming for the animation of talking faces in multiuser environments”. Proceedings of the IEEE International Symposium on Circuits and Systems, 2002.

[WAD97] C. Wren, A. Azarbayejani, T. Darrell, A. Pentland. “Pfinder: Real-Time Tracking of the Human Body”. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 19, no 7, pp. 780-785, 1997. [WPe99] C. Wren, A. Pentland. “Understanding Purposeful Human Motion”, ICCV 1999.

Domotic References: [Ess01] I. A. Essa. “Ubiquitous sensing for smart and aware environments”. IEEE Personal Comm. 2001. [FST02] D. Focken, R. Stiefelhagen. “Towards vision-based 3-D people tracking in a smart room”. Proceedings Fourth IEEE International Conference on Multimodal Interfaces. 2002. [Sat01] M. Satyanarayanan. “Pervasive Computing: Vision and Challenges”. IEEE Personal Comm. 2001.

[Bir98] S. Birchfield. “Elliptical head tracking using intensity gradients and color histograms”. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'98), 1998, pp. 232-237. [BMa98] C. Bregler, L. Malik. “Tracking people with twists and exponential maps”. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1998. pp. 8-15.

17

[CRM03] D. Comaniciu, V. Ramesh, P. Meer. “Kernel-based object tracking”. IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 25 Issue: 5 , May 2003; pp. 564 -577. [Hil99] A. Hilton. “Towards model-based capture of a persons shape, appearance and motion”. Proceedings IEEE International Workshop on Modelling People, 1999. pp. 37 -44. [KFO02] N. Kaempchen, U. Franke, R. Ott. IEEE Intelligent Vehicle Symposium, Volume: 2 , 2002; pp. 459-464. [MGr01] T. B. Moeslund, E. Granum. “A Survey of Computer Vision-Based Human Motion Capture”. Computer Vision and Image Understanding: CVIU, Volume: 81 Issue: 3, 2001. pp. 231-268. [MRC02] L. P. Morency, A. Rahimi, N. Checka, T. Darrell. “Fast stereo-based head tracking for interactive environments”. Proceedings of the IEEE Int. Conference on Automatic Face and Gesture Recognition (FG’02), 2002; pp. 375-380. [OBB00] J. O'Brien, B. Bodenheimer, G. J. Brostow, J. K. Hodgins. “Automatic Joint Parameter Estimation from Magnetic Motion Capture Data”. Proceedings of Graphics Interface, 2000.

[SBF00] H. Sidenbladh, M. J. Black, D. J. Fleet. “Stochastic Tracking of 3D Human Figures Using 2D Image Motion”. Proceedings European Conference Computer Vision, Volume:2 (2), 2000. pp. 702-718. [BPG03] J. M. B. Rubio, F. J. Perales, M. González. “A New Method for Detection and Initial Pose Estimation Based on Mumford-Shah Segmentation Functional”. LNCS 2652, pp. 117-125, June, 2003. Domotic & HCI references:

[Ahl 02] J. Ahlberg. “An active model for facial feature tracking”. Eurasip Journal on Applied Signal Processing, Vol. 6, 2002, pp. 566-571. [Bar01] M. Bartlett. “Face image analysis by unsupervised learning”. Boston: Kluwer Academic Publishers, 2001. [Bra98] G. R. Bradski. “Computer video face tracking for use in a perceptual user interface”. Intel Technology Journal, Q2’98, 1998.

[BBL 01] M. Bartlett; B. Braathen; G. LittleWort-Ford; J. Hershey; I. Fasel; T. Marks; E. Smith; T. Sejnowski; J. R. Movellan. “Automatic analysis of spontaneous facial behavior: A final project report”. (Tech. Rep. No. 2001.08). San Diego, CA: University of California, San Diego, MPLab. [Car01] Carroll J.M. Human-Computer Interaction in the New Millennium. (ed). Addison Wesley Professional (ACM Press). ISBN-10: 0-201-70447-1; ISBN-13: 978-0-201-70447-1, 2001.

[CCC 01] J. C. Chou, Y. Chang, Y. Chen. “Facial feature point tracking ad expression analysis for virtual teleconferencing systems”. Proceedings of the International Conference on Multimedia and Expo, 2001, pp. 25-28.

[CPG 01] M. D. Cordea, E. M. Petriu, N. D. Georganas, D. C. Petriu, T. E. Whalen. “3D head pose recovery for interactive virtual reality avatars”. Proceedings of the IEEE Instrumentation and Measurement Technology Conference, 2001.

[ECC 02] N. Eveno, A. Caplier, P.Y. Coulon. “Key points based segmentation of lips”. IEEE International Conference on Multimedia and Expo, 2002.

[ECC01] N. Eveno, A. Caplier, P. Y. Coulon. “A new color transformation for lips segmentation”. Workshop on Multimedia Signal Processing, 2001.

[FeC01] R. Feris, R. Cesar. “Detection and tracking of facial landmarks using Gabor wavelet networks”. In Lecture Notes in Computer Science, Rio de Janeiro, Brasil, March 2001. Springer-Verlag Press.

[FuV04] Fundacion Vodafone, Telemedicina, Fundacion Vodafone-España, 2005. [FuV05] Fundacion Vodafone, Dependencia y Diversidad, Fundacion Vodafone-España, 2005. [Hol04] Holmquist L.E., Schmidt A., Ullmer B. Tangible interfaces in perspective. Personal and Ubiquitous Computing, 2004 - Springer [Ish97] Ishii, H., and Ullmer, B. “Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms.” In Proceedings of CHI’97, pp. 234-241.

[KiP01] S. A. King, R. E. Parent. “A parametric tongue model for animated speech”. Journal of Visualization and Computer Animation, 12(3), 2001, pp. 107-115.

18

[Mor01] S. Morishima. “Face analysis and synthesis for duplication expression and impression”. IEEE Signal Processing Magazine. May-01, pp. 26-34.

[MaP01] M. Malciu, F. Prêteux. “MPEG-4 compliant tracking of facial features in video sequences”. Proceedings of the International Conference on Augmented Virtual Environments and 3-D Imaging, 2001, pp. 108-111.

[ORC 01] J. Ostermann, J. Rurainsky, R. Civanlar. “RTP payload format for phoneme/facial animation parameters (PFAP)”. RTF: draft-ietf-avt-rtp-pfap-00.txt, 2001.

[SLL 01] K. L. Sum, W. H. Lau, S. Leung, A. W. Liew, K. W. Tse. “A new optimization procedure for extracting the point-based lip contour using active shape model”. Proceedings of the Int. Conf. Acoustics Speech and Signal Processing. 2001.

[SR 01] S. Spors, R. Rabestein. “A real-time face tracker for color video”. Proceedings of the International Conference on Acoustics Speech and Signal Processing. 2001.

[VAD 01] S. Valente, A. C. Andrés del Valle, J. L. Dugelay. “Analysis and reproduction of facial expressions for realistic communicating clones”. Journal of VLSI and Signal Processing, 29, 2001, pp. 41-49.

[PaB 02] M. Pardàs, A. Bonafonte. “Facial animation parameters extraction and expression recognition using Hidden Markov Models”. Eurasip Signal Processing: Image Communication, 17(9), 2002, pp. 675-688.

[PaF 02] I. S. Pandzic, R. Forchheimer. “MPEG-4 – The standard, implementations, applications”. John Wiley & Sons, Inc. 2002.

[CRM 03] D. Comaniciu, V. Ramesh, P. Meer. “Kernel-based object tracking”. IEEE Transactions on Pattern Analysis and Machine Intelligence. v. 25, pp. 564-577, May 2003. [Sim03] M. Simunek. “Visualization of talking human head”. Electronic version retrieved July, the 15th, 2003, from: http://www.cg.tuwien.ac.at/studentwork/CESCG/CESCG-2001/MSimunek/

[ShP06] B. Shneiderman, C. Plaisant, Diseño de Interfaces de Usuario, 4 edicion, Pearson. Addison Wesley 2005.

[TuK05] Turk, M., Kölsch, M.: Perceptual Interfaces. In Medioni, G., Kang S.B.(eds): Emerging Topics in Computer Vision, Prentice Hall, 2005. [TuR00] Turk, M., Robertson, G.: Perceptual User Interfaces. Communications of the ACM 43: 32–34, 2000.

[Ull00] Ullmer B., Ishii, H. Emerging Frameworks for Tangible User Interfaces.” In IBM Systems Journal, v39, n3-4, 2000, pp. 915-931.

[VOH 01] S. Varakliotis, J. Ostermann, V. Hardman. “Coding of animated 3-D wireframe models for Internet streaming applications”. Proceedings of the IEEE International Conference on Multimedia and Expo. 2001.

[Wei05] S. Weinstein, The Mutimedia Internet, Spriger Verlag, 2005.

[ZWL 01] P. Zhenyun, H. Wei L. Luhong, X. Guangyou, Z. Hongjian. “Detecting facial features on image sequences using cross-verification mechanism”. Proceedings of the Second IEEE Pacific-Rim Conference on Multimedia. 2001.

Biometric Bibliography:

[BrF95] R. Brunelli and D. Falavigna, “Person identification using multiple cues,” IEEE Transactions on PAMI, vol. 12, no. 10, pp. 955–966, Oct 1995. [BBD97] E.S. Bigun, J. Bigun, B. Duc, and S. Fischer, “Expert conciliation for multimodal person authentication systems using Bayesian statistics,” in Proceedings AVBPA’97, Crans-Montana, Switzerland, March 1997, pp. 291–300. [Dau03] Daugman, John. “Combining Multiple Biometrics”. Cambridge University, 2003 [HJP99] L. Hong, A. K. Jain, and S. Pankanti, “Can multibiometrics improve performance?,” in Proceedings AutoID’ 99, Summit(NJ), USA, Oct 1999, pp. 59–64. [JaR02] Jain, A. K. and Ross, A. “Learning User-specific Parameters in a Multibiometric System”. Proceedings International Conference on Image Processing (ICIP), 2002, [JBP99] A. K. Jain, R. Bolle, and S. Pankanti, Eds., Biometrics: Personal Identification in Networked Society, KluwerAcademic Publishers, 1999.

19

[JHP97] A. K. Jain, L. Hong, S. Pankanti, and R. Bolle, “An identity authentication system using fingerprints,” Proceedings of the IEEE, vol. 85, no. 9, pp. 1365– 1388, 1997. [JRP99] A. K. Jain, A. Ross, and S. Pankanti, “A prototype hand geometry-based verification system,” in Proceedings AVBPA’99, Washington, D.C., USA, March 1999, pp. 166–171. [KWS01] L. I. Kuncheva, C. J. Whitaker, C. A. Shipp, and R. P. W. Duin, “Is independence good for combining classifiers?,” in Proceedings ICPR, Barcelona, Spain, 2001, vol. 2, pp. 168–171. [Man01] Mansfield, T. et al. "Biometric Product Testing Final Report". UK Biometrics Working Group, 2001, [OGo02] L. O’Gorman, “Seven issues with human authentication technologies,” in Proceedings AutoID , Tarrytown, New York, Mar 2002, pp. 185–186. [PrJ02] Prabhakar, S. and Jain, A. K. “Decision-level Fusion in Biometric Verification”.Pattern Recognition v35 n4, 2002, [PRW02] „Multi Modal Technology makes Biometrics work“. PRWeb Press Release, Aurora Defense LLC, 2002, [RJA01] A. Ross, A. K. Jain, and J. Qian, “Information fusion in biometrics,” in Proceedings AVBPA’01, Halmstad, Sweden, Jun 2001, pp. 354–359.

[RNJ06] A. Ross, K. Nandakumar, A. Jain. Handbook of Multibiometrics, Springer Verlag, 2006. [TuP91] M. Turk and A. Pentland, “Eigenfaces for recognition,”Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991. [VwC99] P. Verlinde and G. Cholet, “Comparing decision fusion paradigms using k-NN based classifiers, decision trees and logistic regression in a multi-modal identity verification application,” in Proceedings AVBPA’99, Washington D.C., USA, March 1999, pp. 188–193

References about Agents (IA):

[Bad97] Badler, N.I. and Reich, B.D. and Webber, B.L. (1997); Towards Personalities for Animated Agents with Reactive and Planning Behaviours; Creating Personalities for Synthetic Actors; Trappl R.

[Bat94] Bates J. :The role of Emotion in Believable Agents. Communications of the ACM Vol. 37, Number 7 (July 1994) 122-125.

[Bee90] Beer, R. D. (1990); Intelligent as Adaptive Behaviour; Academic Press

[Bra97] Bradshaw J.M. Editor : Software Agents. AAAI Press/ The MIT Press, Menlo Park, CA (1997).

[Fau94] Faussett, L. (1994); Fundamentals of Neural Networks, Architectures, Algorithms, and Applications; Prentice Hall International.

[Gol89] Goldberg, D.E. (1989); Genetic Algorithms in Search, Optimisation, and Machine Learning. Addison Wesley.

[Gut97] Gutowitz H.: Artificial-Life Simulators and their Applications. http://alife.santafe.edu/alife/topics/simulators/dret/dret.html.

[Hay97] Hayes-Roth, B. and Van Gent, R. and Huber, D. (1997); Acting in Character; Creating Personalities for Synthetic Actors; Trappl R. and Petta P. Eds., Springer-Verlag.

[IsG04] P.Isasi, I.M. Galvan. Redes de Neuronas Artificiales, Pearson, Prentice Hall, 2005..

[Les95] Lesser V.R.: Multiagent Systems: An Emerging Subdiscipline of AI. ACM Computing Surveys Vol. 27 Number 3, 340-342 (September 1995).

[Loy97] Loyall A.B. “Some Requirements and Approaches for natural Language in a Believable Agent”. Creating Personalities for Synthetic Actors (1997), Trappl R., Petta P. (Eds)

[Mae94] Maes P.: Modeling Adaptive Autonomous Agents. Artificial Life Vol. 1, Number 1/2 (1994) 135-162.

[Mae95] Maes P. (1995); Artificial life meets Entertainment: Lifelike Autonomous Agents. Communications of the ACM] VOL 38, Number 11 (pp. 108-114).

20

[Mas05] A. Mas Agentes Software y Sistemas Multi-Agente, Pearson, Prentice-Hall, 2005.

[Mer98] Merelo, J.J. Mundos artificiales. Herramientas computacionales para la síntesis de vida artificial. Curso de verano de Vida Artificial de la Universidad de Castilla la Mancha en Albacete (1998).

[MT94] Magnenat Thalmann N., Thalmann D. Eds. “Artificial Life and Virtual Reality”. John Wiley \& Sons, Ltd., (1994)

[MT96] Magnenat Thalmann, N. and Thalmann, D. (1996); Computer Animation. ACM Computing Surveys, VOL 28, Number 1, March 1996 (pp. 161-163).

[Mül96] Müller J.P.: The design of Intelligent Agents. Lecture Notes in Artificial Intelligence, 1177, Springer Press, (1996).

[NPC96] Noser, H., Pandzic, I.S., Capin, T.K., Magnenat Thalmann, N. and Thalmann, D. (1996); Playing games through the Virtual Life Network; Proceedings Artificial Life V, Nara, Japan.

[PCS00] Pina A., E. Cerezo E., Serón F.J., “Computer Animation: From Avatars to Unrestricted Autonomous Actors (A survey on replication and modelling mechanisms)”. Computers & Graphics, Vol. 24, No. 2, section "Surveys", pp. 297-311 (2000)

[PSG02] A. Pina, F.J. Seron, D. Gutierrez, “The ALVW system: an interface for smart behavior-based 3D computer Animation”, 2nd. International Symposium on Smart Graphics, ACM Press, Smart Graphics 2002, pp. 17-20, 11-13 de junio de 2002, Hawthorne, New York, USA

[Tha96] Thalmann, D. (1996); A new generation of Synthetic Actors: the Real-Time and Interactive Perceptive Actors; Proceedings Pacific Graphics 96; Taipeh, Taiwan.

[TT94] Tu ,X., Terzopoulos, D. (1994); Artificial Fishes :Physics, Locomotion, Perception, Behavior, Computer Graphics Proceedings (SIGGRAPH’94)

[Wat96] Watson, M. (1996); AI Agents in Virtual Reality Worlds; John Wiley & Sons

[Woo97] Wooldridge M.: Agent-based Software Engineering. IEEE Proc.-Soft. Eng. Vol. 144, Number 1 (February 1997) 26-37.

Bibliography about TTS/ARS, Speech and Applications

[Bes04] J. Beskow, Inger Karlsson, Jo Kewley, Giampiero Salvi: SYNFACE - A Talking Head Telephone for the Hearing-Impaired. ICCHP 2004: 1178-1185] [BPR05] M. Bagein, O. Pietquin, C. Ris and G. Wilfart. An Architecture for Voice-Enabled Interfaces over LocalWireless Networks. Facult´e Polytechnique de Mons - TCTS Lab., 2005 [LAO05] Maider Lehr, Andoni Arruti, Amalia Ortiz, David Oyarzun, and Michael Obach. Speech driven facial animation using HMMs in Basque, 2005. [LNR01]C.M. Lee, S. Narayanan, and R. Pieraccini, “Recognition of negative emotions from the speech signal,” in Proc. Automatic Speech Recognition and Understanding, Dec 2001] [Mas04] Dominic W. Massaro. Symbiotic Value of an Embodied Agent in Language Learning. Proceedings of the 37th Hawaii International Conference on System Sciences – 2004] [Ola00] P. OLASZI. Analysis of Written and Spoken Form of Hungarian Numbers for TTS Applications. INTERNATIONAL JOURNAL OF SPEECH TECHNOLOGY 3, 177–186, 2000] [PeB03] C. Pelachaud, M. Bilvi: Computational Model of Believable Conversational Agents. Communication in Multiagent Systems 2003: 300-317].

3. OBJETIVOS DEL PROYECTO (máximo dos páginas)

21

3.1 Describir brevemente las razones por las cuales se considera pertinente plantear esta investigación y, en su caso, la hipótesis de partida en la que se sustentan los objetivos del proyecto (máximo 20 líneas)

The following reasons:

1. Developement of a tangible interface that combines VBI basics with advanced TTS/ARS and if necessary with haptic systems and other senses (smell and taste). 2. Improvement of the models based on distributed intelligent agents for multimodal environment control with natural information of the user. Formalization of the concept of agent and compatibility with technologies and computational platforms 3. Improvement in the biometric systems and particularly exploration of fusion systems in biometrics. Definition of a multibiometric system that must be hybrid with multisampling and multimodality. 4. Adaptation of the designed systems to mobil robotic elements for home aid (articulated arm and mobil robot). 5. Specification of all of these aspects in commercial products used in domotic environments and in routine medical e-Health and if possible of low cost.

are the reasons that will allow us to obtain the previously presented objectives: 1. Development of a tangible interface. 2. Use of Intelligent Humanoids Simulation techniques (3D Agents). 3. Definition of new low cost tangible Interfaces and their application to automated environments. 4. Establishment of some concrete prototypes for automated and medical e-Health.

3.2. Indicar los antecedentes y resultados previos, del equipo solicitante o de otros, que avalan la validez de la hipótesis de partida

As it has already been commented througout this document, the group that applies and investigators of other institutions and universities that participate in the request of the project, have a solid experience in national and Europeans projects that has already been shown with recent international publications that vouch for the validity of the starting hypothesis. Also, information from other foreign and national groups that confirm the importance of carrying out research in the advanced user’s interface field is available. The increase of publications and completed tasks is astonishing. The preparation for European projects proposals requests is also maintained. The applicant’s team has been coordinator of a petition for a European project in the last call of the VI EU framework agreement. The proposal of name ALIVE (Ambient Assitive for Living Virtual Environments) received a high appraisal from the examiners although due to the competitiveness and the high number of projects presented (over 165 in AAL) it finally was not selected. The coordinator of the project was our own research team of the UIB. The project’s consortium is maintained and plans to improve the proposal in the recommended aspects and will be presented in the next call in March of 2007 for the VII EU framework agreement. All the previous projects, either national or internacional, have been positively valued by all of the evaluation commissions. More than 50 papers of on going research tasks have been published and 5 international books have been edited (LNCS, Springer-Verlag). On the other hand, a prize for the better collaborative paper of the Noe ENACTIVE excellence network has been received recently the prize has been received, which indicates the quality level of the research that’s been carried out.

3.3. Enumerar brevemente y describir con claridad, precisión y de manera realista (es decir, acorde con la duración prevista del proyecto) los objetivos concretos que se persiguen, los cuales deben adecuarse a las líneas temáticas prioritarias del Programa Nacional al que se adscribe el proyecto (ver Anexo de la convocatoria).

La novedad y relevancia de los objetivos (así como la precisión en la definición de los mismos) se mencionan explícitamente en los criterios de evaluación de las solicitudes (ver apartado 11º.1 de la Convocatoria)

The objectives proposed are:

22

1. Development of tangible interfaces. 2. Use of Intelligent Humanoids Simulation techniques (3D Agents). 3. Definition of new low cost tangible Interfaces and their application to automated environments. 4. Establishment of some concrete prototypes for automated and medical e-Health.

These objectives are related to:

The National I+D plan, as priority areas in the TIN (Computer Science Technologies), particularly priority is forseen in the area of Agents and Artificial Intelligence. The area of advanced interfaces is also considered fundamental: 3) Agent based Systems.

3.1) Agent architectures and models. Languages and advanced Interfaces. 3.2) Multiagent Architectures. Organization models. 3.3) "Softcomputing". Adaptive and evolutive agents. 3.4) Automatic Learning. 3.5) Information Agents. 3.6) Human Language Engineering.

5) Advanced InterfacesVirtual Reality 5.1) Virtual reality and aumentada. 5.2) Artificial Vision and Advanced Image Treatment. 5.3) Multimodal Interfaces 5.4) sensorial Interactive Advanced Systems

Even the National Program in Technological Services for the Information Society, in section 1.3 and 1.4 mention the tools for accessibility and aid, and applications in areas of teleducacion e-Learning and telemedicina e-Health. All of the areas, that is, the 100% of the sections and topics proposed are prioritized in the programs of the Spanish Ministry. The proposed application combines adequately each one of them and the proposed applications, that validate them, are also priority in the health area.

3.4. En el caso de Proyectos Coordinados (máximo dos páginas):

- el coordinador deberá indicar: los objetivos globales del proyecto coordinado, la necesidad de dicha coordinación y el valor añadido que se espera alcanzar con la misma los objetivos específicos de cada subproyecto la interacción entre los distintos objetivos, actividades y subproyectos los mecanismos de coordinación previstos para la eficaz ejecución del proyecto (NO PROCEDE)

23

4. METODOLOGÍA Y PLAN DE TRABAJO (en el caso de proyectos coordinados deberá rellenarse para cada uno de los grupos participantes)

Se debe detallar y justificar con precisión la metodología y el plan de trabajo que se propone y debe exponerse la planificación temporal de las actividades, incluyendo cronograma (se adjunta un posible modelo a título meramente orientativo).

El plan de trabajo debe desglosarse en actividades o tareas, fijando los hitos que se prevé alcanzar en cada una de ellas. En los proyectos que empleen el Hespérides o se desarrollen en la zona antártica, deberán también incluir el plan de campaña en su correspondiente impreso normalizado. En cada una de las tareas debe indicarse el centro ejecutor y las personas involucradas en la misma (Ver la relación de personal investigador incluido en el formulario de solicitud). En el caso de proyectos coordinados, se deberán indicar las actividades que realizará cada uno de los subproyectos Si solicita ayuda para personal contratado o en formación justifique claramente su necesidad y las tareas que vaya a desarrollar. La adecuación de la metodología, diseño de la investigación y plan de trabajo en relación con los objetivos del proyecto se mencionan explícitamente en los criterios de evaluación de las solicitudes (ver apartado 11º.1 de la convocatoria).

The project will be divided into tasks; each will be carried out by non disjoint groups of researchers and coordinated by the main investigator. Collaboration in tasks implies interdisciplinary groups of researchers from different centers (both National and Foreign). The tasks are enumerated subsequently:

T0: Web page creation, CD' s, DVD' s (divulgative). Global coordination of the project. T1: Development of Tangible Interface VBI + ARS/TTS + Haptic. Other senses (taste and smell). T2: Multibiometric System. T3: Distributed Intelligent Agent. T4: Analysis and Design Module in UML. T5: Modular Integration. Specific Commercial prototypes.

24

Descripción de las Tareas

Task 0: Web page creation, CD' s, DVD' s (divulgative). Global coordination of the project.

Length: from month 1 to month 36. Description: The global tasks regarding all activities and all personel coordination are grouped. The diffusion of results by advanced multimedia media is also included. Previous tasks: Results: Monitoring of the Project. WEB pages, CD´s and DVD´s. Person Responsible: Pere Palmer

Subtask 0.1: Basic web page creation. Global coordination.

Length: from month 1 to month 2.

Description: The objective is the creation of a WebSite that will be used as a reference by all participants of the project. Partial results of each subtask (milestones, publications, conference presentations, useful data for other working parties, etc.) will be included periodically.

Previous tasks: None

Results: Project Website

Personnel: F. J. Perales, M. González, R.Mast, A. Igelmo, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), M. Rosecler, D. Arellano, hired technitian.

Complementary personnel: Possible PFC Students

Subtask 0.2: Improvement and Maintenance of the web pages. Length: from month 3 to month 36. Description: Due to the interactive nature of the Project, the creation of its WebSite cannot be reduced

to a few pages with the basic information that will serve as a reference to all participants of the project. The WebSite will contain more advanced pages that will allow the visualization in an interactive manner of result animations. Because of it, the need to include applets in JAVA is considered and the necessary Plug-ins to visualize VRML environments.

Previous tasks: 0.1 Results: Advanced Website of the Project. Personnel: F. J. Perales, P. Palmer, J.M. Buades, A. Clar (CTIUIB), hired technitian. Complementary personnel: Possible PFC Students

Subtask 0.3: Diffusion of results.

Length: from month 30 to month 36.

Description: Although it has already been outlined in the previous subtask, the objective of this section is the broadcast and diffusion of results in all aspects. The task seeks a level of exchangeof data and results between research centers, and also its divulgation in high level national and international

25

magazines. Periodically, a Internacional Workshop that guarantees the disclosure of the results and advances of the project will be carried out in the UIB (AMDO2008, 2010).

Previous tasks: 0.1, 0.2 and all those that generate results

Results: Website’s final version, national and international publications. Conferences organized. Personnel: F. J. Perales, M. González, R. Mas, A. Igelmo, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), M. Rosecler, hired technitian.

Complementary personnel: Possible PFC Students

26

Task 1: Development of Tangible Interface VBI + ARS/TTS + Haptic. Other senses (taste and smell)

Length: from month 1 to month 24. Description: This section includes all tasks related to the definition of the tangible interface that combines the prior deveoped VBI know-how with the new input forms based on ARS and advanced Haptics. At the same time solutions that study other senses such as taste and smell will be explored. Previous tasks: None. Results: Prototype for the domotic environment Tangible Interface. Person Responsable: F. J. Perales.

Subtask 1.1: System’s Architecture, communication’s infrastructure design.

Length: from month 1 to month 12.

Description: The objective of this subtask is to document the tangible interface’s requirements concretely thinking in the final domotic and medical e-Health application. A system capable of recognizing the face and zones of the face is the starting point, a repertoire of basic gesturesthat allows us to enrich the present number of available actions will be incorporated. One of the fundamental qualities of the functionality of the system is its adaptability to the user. Evaluation and learning tasksfor all symbols and recognizable gestures will be designed, guaranteeing the best adaptation to the patient limitations. As it has been commented in the introduction the VBI are already well known and they constitute a specific area of research. Because of it, our investigation is centered in the exploration of new combinations of inputs, especially voice and haptics. Since we are nor voice specialists we have carried out a thourough study of comercial ARS/TTS systems arriving to the conclusion that the Loquendo’s Software is one of the best in all of its aspects besides having a quite extensive and functional SDK. The integration of visual signals will be carried out with commands delivered and/or recognized by Loquendo. The use of free software has not been ruled out for future versions, but will not be used for the time being.

Previous tasks: None.

Results: Definition of the tangible interface architecture.Basic interface prototype.

Personnel: F. J. Perales, A. Clar (CTIUIB), R. Mas, A. Igelmo, G. Fontanet, hired technitian. Complementary personnel: A. Cross (ULBRA), M. Rosecler (Feevale).

Subtask 1.2: Interface enlargement by means of the inclusion of Hapticos and other Senses Length: from month 6 to month 24. Description: In this section, the functionality of the previous system including the use of some haptic device for the interaction with the home environment that surrounds the person is extended. It has been conceived especially for people with reduced hand mobility and who can carry out complex or heavy tasks by means of a lever’s displacement. The experience in previous projects and the knowledge on prioritized inverse cinematic by some members of the group, guarantee the integration of the haptic in the final tangible interface. The haptic system’s actuador will be manipulated by means of voice, gestures or simple hand movement. The options proposed in the proposal are the manipulation of an articulated arm with advanced haand capacity of “grasping“ that will allow the user a natural interaction with the objects of its environment. On the other hand, the improvement of the algorithms implemented in the PIONEER Robot and Sony’s AIBO is forseen, in order to develop useful activities in the domotic environment (surveillance, detection of anomalies, capture of far away objects, etc.)

27

Finally, the testing of the actual smell and taste capture systems by means of synthetic devices is foreseen. Industrial applications in the area of cosmetics, vine growers or cooks are evident. Our idea for the project is to improve the realism of the patient’s daily activities, expanding the pleasant sensations and minimizing the negative ones. In the domotic environment, basic smell systems can be installed to complete the daily actions of house maintenance. Previous tasks: 1.1. Results: Prototype of Tangible Interface Advanced.

Personnel: F. J. Perales, A. Igelmo, P. Palmer, J.M. Buades, A. Clar (CTIUIB), R. Mas, M. Rosecler, hired technitian. Complementary personnel:

T2: Multibiometric System

Length: from month 1 to month 18 Description: The objective of this task is to define a multibiometric system to validate user’s identity in the domotic system. The habitual unimodal techniques are known and we wish to implement a modular system that fits all variants of multibiometry. Previous tasks: Task 1 Results: Intelligent Agent Systems Person Responsible: A. Igelmo

Subtask 2.1: Study of present models in biometric systems. Its adaptation to biometric data fusion.

Length: from month 1 to month 12.

Description: In this task the present models for design and implementation of biometric systems are studied. Particularly, our interest lies in the specifications on how to build a basic biometric system that can latter be extended to a multibiometric system. Basicly, the different levels of biometric fusion are explored. We consider the following levels: sensor level, "feature" or characteristic level, rank level. Several final projects for degree fulfilment have been already developed using fingerprint recognition systems and some systems based on the iris recognition have been tested. Integration of both procedures has not been posible due to access problems to the SDK software of the iris system of LG. In this project, we foresee to develop fusion criteria more centered in the characteristics fusion level and rank level than in the device level. Normalization, characteristic selection and their transformation will be studied at a "feature" level, and methods based on PCA, ICA and MDS will be used, also Kohonen maps and neural networks will be explored. At rank level, when the biometric system works in identity identification mode, different methods: Higest Rank Mode, Borda Count Method, Logistic Regresion Model can be studied. Finally we will implement a decisión system that guarantees an adequate reliability level and although multiple options exist, the initial approximation selected is Bayesian theory (Naive Bayes combination rules). In a second phase we will explore solutions based on BKS (Behavior Knowledge Space).

Predecessors: Task 1

Results: Models of considered biometric systems.

Personnel R. Mas, A. Igelmo, M. Miró, M. Gonzalez, J.M. Buades (UIB), A. Clar (CCUIB), hired technitian.

Complementary personnel:

Subtask 2.2: Definition and implementation of the domotic multibiometric system’s prototype.

Length: from month 6 to month 12.

28

Description: In this task, previous aspects of basic investigation will be specified for the biometric system and will allow the implementation of the system’s prototype in the domotic environment as another agent of the global distributed multiagent system that constitutes thedomotic control application.

Previous tasks: Task 2.1

Results: Multimodal biometric system’s prototype.

Personnel: R. Mas, A. Igelmo, M. Miró, M. Gonzalez, J.M. Buades (UIB), A. Clar (CCUIB), hired technitian.

Complementary personnel:

29

Task 3: Distributed Intelligent Agent.

Length: from month 6 to month 30. Description: Present design algorithms and distributed multiagent systems implementation will be improved. The study of open environment technologies, coordination models and agents coordination space requires special emphasis. On the other hand, the definition is in need of decision taking and of the definition of contributions and cooperations. Previous tasks: Task 1 and 2 Results: Distributed Intelligent Agent Prototype Person Responsable : G. Fiol

Subtask 3.1: Implementation of the distributed intelligent agent advanced algorithms.

Length: from month 6 to month 12. Description: Living beings carry out a perception-analysis-action chain:

a.- Perception: In order to have an action of a living being in the envionment that surrounds it, a perception of the environment is essential to make adequate decisions, that complies to its survival objectives. Because of this, one must endow the inhabitants of a sensory system. b.- Analysis: The inhabitant should have a mechanism that allows it to evaluate possible behaviors in base of the stimuli perceived (through the sensors) and to decide which is the most adequate behavior. Besides the received stimuli, the objective that the inhabitant is trying to reach with the present behavior can influence the decisión making. c.- Action: All external actions that materialize the different possible behaviors of the inhabitant that we are trying to model should be provided. In this subtask, we intend to endow the agents of a learning capacity, that is, of a cognitive model that complements the reaction capacity (conduct model). This learning capacity can be presented from two points of view:

• Behavior evaluation and analysis based on the environment’s conceptual scheme and prior experiences (memory).

• Memorization of behavior situations (correct and/or incorrect) and their detection on next occasions.

In the project granted previously TIN2004-07926 a simple versión of a virtual butler has already been implemented as a basic intelligent agent that controls the home and its sensors. This agent basically has a reactive and centralized behavior. Therefore, it is fundamental to extend the agent’s functionality to distributed environments, such as the domotic home, where each agent is responsible for a series of chores and with an agent in control that organizes the system’s global operation. In a first phase, open environments based on XML, JAVA, RMI and CORBA will be explored taking results of tasks 1 and 2 into account and thinking that natural language can be incorporated as a basic agent communication tool. The coordination between agents will be accomplished by means of intermediary agents, with mediation services. Adequate description languages for the final domotic application should be determined, concretly WSDL, OWL-S, Service Profile, Service Model, Service Grounding.

Previous tasks: Task 1 and 2

Results: Advanced distributed intelligent agent prototype.

Personnel: G. Fiol, F. J. Perales, M. González, J.M. Buades, A. Clar (CTIUIB), hired technitian.

Complementary personnel:

30

Subtask 3.2: Integration of the distributed Intelligent Agent and the Tangible Interfaces.

Length: from month 12 to month 30.

Description: This is a necessary module for the integration of the tasks 1 and 2 with the agent’s own design. Even though they are certainly related and each one can condition the other. The development of intelligent agent can be carried out in parallel with data entry and conventional real cases although in its final phase it should allow the recognition of the same functionalities with an advanced interface.

Previous tasks: T1, T2

Results: Tangible interface and disributed intelligent agent integrated system.

Personnel: F. Perales, M. González, M. Mascaro O, J.M. Buades, A. Clar (CTIUIB), hired technitian.

Complementary personnel: M. Rosecler (Feevale), A. Cross (ULBRA), P. Noll of Mattos (ULBRA.)

31

Task 4: Analysis and Design Module in UML.

Lenght: from month 1 to month 30 Description: Se agrupan las tareas globales de análisis y diseño de las aplicaciones del proyecto mediante una metodología basada en UML. The global tasks of analysis and design of the project’s applications are grouped by means of a methodology based on UML. Previous tasks: None Resultados: UML specfications for Project’s modules. Person Responsable: G. Fontanet

Subtask 4.1 UML Análisis and Design

Lenght: from month 1 to month 30.

Description: Parting from the results of the TIN2004-07926 project, the objective of this task to plan and to control changes, adaptations and the additions proposed in other tasks. As a consequence of TIN2004-07926 requirements requests was done using use cases technique. Towards the end of the project a migration toward UML and its methods began. The proposal that is presented continues using a methodology based on UML besides carrying out all the new tasks with this method. It has been a very positive experience that has permitted teamwork and that due to the magnitude of the project is completely justified. The selection of UML is supported by the fact that it has becomed standard to work in analysis and design of applications oriented to objects. It was born from the fusion of Jim Rumbaugh, Grady Booch and Ivar Jacobson methods. The result, the Modeled Unified Method, is a set of techniques thought to represent complete systems employing concepts of orientation to objects, keeping in mind the scalability of the complex systems. This will also provide a library of classes for the project, that will contain, conveniently revised the library resultant of the previous project plus more the ones obtained as a result of the new project. The libraries corresponding to the new user’s interface will be included in it. It will also use some type of CASE tool.

Preivous tasks: All tasks

Results: Project’s application análisis and design in UML.

Personnel: M. González, R. Mas, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), T. Rodriguez (EPTRON), M. Santos (TAGRV S.L.), hired technitian.

Complementary Personnel:

32

Task 5: Modular Integration. Specific Commercial prototypes.

Length: from month 24 to month 36 Description: The implementation of a specific system for a domotic home with low price and compatible systems is desired. The applications will be to aid the disabled and of health control on simple chronic pathologies. Previous tasks: All tasks Results: Specific software for domotic welfare. Person Responsable: Francisco J. Perales

Subtask 5.1: Modular Integration

Lenght: from month 24 to month 36.

Description: The objective of this task is the final integration. The integration strategy must be thoroughly studied to facilitate the obtaining of a prototype and its subsequent installation in the premises of potential end users. A special effort must be made since the final result of the project depends on it. All subproyects are integrated in this one generating the version that will be presented to the end users. Control, development and monitoring of the project are included in this section by means of the methodologies oriented to objects mentioned above. Periodic meetings with the businesses involved will be maintained.

Previous tasks: All tasks

Results: Specific application

Personnel: F. J. Perales, M. González, R. Mas, A. Igelmo, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), M. Rosecler, T. Rodriguez (EPTRON), M. Santos (TAGRV), B. Payeras, P. DeMiguel (C. Rotger).

Comlementary Personnel:

Subtask 5.2: Domotic system for basic aid.

Length: from month 24 to month 36.

Description: Since we have all the resources of the previous tasks at our disposal, we intend to implement a low price system and if possible portable to virtual or augmented reality. Because of it portable devices with stereoscopic capacity are considered. We do not have much experience in this section because all systems are high priced and not portable. Therefore, first of all, a detailed study of solutions proponed by other groups with advanced research in the field will be carried out. If it is possible our functionalities will be integrated in some already extensively tested system. In any case we will consider introducing new, still not treated, aspects such as visual capture systems integration or not invasive detection in portable systems (digital cmaras, etc...).

Previous tasks: All

Results: Specific application of portable RV complying with the specifications of the global project.

Personnel: F. J. Perales, M. González, A. Mir, R. Mas, A. Igelmo, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), M. Rosecler, T. Rodriguez (EPTRON), M. Santos (TAGRV).

Complementary Personnel: M . Bez, A. Cruz

33

Subtask 5.3: e-Health Domotic System.

Lengrh: from month 24 to month 36.

Description: As it has already been commented in section 4, some simple specific multimodal interface examples oriented to people with some slight disability will be implemented. Anyway, a standard domotic system with image, video, audio and sign capture capacity of home devices will also be designed. The main objective is to integrate the intelligent avartar with the devices and the end user. To complete this section we count on specialized personnel from clinica Rotger of Palma de Mallorca, at technical level and in the medical field. The key functions according to pathology are being studied in order to carry out an adequate medical protocol for aid by means of the intelligent agent.

Previous tasks: All

Results: Specific application for 3D domotic Agents complying with project’s global specifications.

Personnel: F. J. Perales, M. González, R. Mas, A. Igelmo, G. Fontanet, P. Palmer, J.M. Buades, A. Clar (CTIUIB), M. Rosecler, T. Rodriguez (EPTRON), B.Payeras, P. DeMiguel (C. Rotger).

Complementary Personnel:

3

4

4.1

MO

DEL

O D

E C

RO

NO

GR

AM

A (O

RIE

NTA

TIVO

) E

n es

te c

rono

gram

a de

ben

figur

ar la

tota

lidad

del

per

sona

l inv

estig

ador

incl

uido

en

el fo

rmul

ario

de

solic

itud

y, e

n su

cas

o, e

l per

sona

l con

trat

ado

o en

form

ació

n qu

e se

sol

icite

con

car

go a

l pro

yect

o. D

eb

e s

ub

raya

rse

e

l n

om

bre

de

la

pe

rso

na

re

sp

on

sa

ble

, e

n c

ad

a t

are

a.

Act

ivid

ades

/Tar

eas

C

entr

o E

jecu

tor

Per

sona

re

spon

sabl

e y

otra

s in

volu

crad

as

Prim

er a

ño (

*)

Seg

undo

año

(*)

T

erce

r añ

o (*

)

T0

: W

eb p

age

crea

tion,

CD

' s, D

VD

' s (

divu

lgat

ive)

. G

loba

l co

ordi

natio

n of

the

proj

ect.

. U

IB

P. P

alm

er (

UIB

) XX

XXXX

XXXX

XX

XXXX

XXXX

XXXX

XX

XXXX

XXXX

XX

T1:

Dev

elop

men

t of

Tan

gibl

e In

terf

ace

VB

I +

AR

S/T

TS

+

Hap

tic.

Oth

er s

ense

s (t

aste

and

sm

ell)

UIB

F

. J. P

eral

es (

UIB

),

R. M

as (

UIB

)

XXXX

XXXX

XXXX

X

XXXX

XXXX

XXX

T2:

Mul

tibio

met

ric S

yste

m.

UIB

A

. Ige

lmo

(UIB

) XX

XXXX

XXXX

XX

XXX

XXX

T3:

Dis

trib

uted

Inte

llige

nt A

gent

. U

IB

G. F

iol (

UIB

)

X

XXXX

X X

XXXX

XXXX

XXX

XXX

XXX

T4:

Ana

lysi

s an

d D

esig

n M

odul

e in

UM

L.

UIB

G

. Fon

tane

t (U

IB)

XXXX

XXXX

XXXX

X

XXXX

XXXX

XXX

XXX

XXXX

T5:

Mod

ular

Inte

grat

ion.

Spe

cific

Com

mer

cial

pro

toty

pes

UIB

, EP

TR

ON

, T

AG

RV

, C

línic

a R

otge

r

F. J

. Per

ales

(U

IB)

XXXX

XXXX

XXXX

35

5. BENEFICIOS DEL PROYECTO, DIFUSIÓN Y EXPLOTACIÓN EN SU CASO DE LOS RESULTADOS (máximo una página) Deben destacarse, entre otros, los siguientes extremos: Contribuciones científico-técnicas esperables del proyecto, beneficios esperables para el avance del conocimiento y la tecnología y, en su caso, resultados esperables con posibilidad de transferencia ya sea a corto, medio o largo plazo. Adecuación del proyecto a las prioridades de la convocatoria y, en su caso, del Programa Nacional o Acción Estratégica correspondiente. Plan de difusión y, en su caso, de explotación, de los resultados del proyecto, el cual se valorará en el proceso de evaluación de la propuesta (ver apartado 11º.1 de la convocatoria) y en el de seguimiento del proyecto.

From the information described in the memory of the project it can be deduced, that the final objective is the generation of a domotic system with real capabilities of aid to the elderly or disabled in their own home. There are multiple benefits, of different kinds. The economic benefits are evident, since many problems that nowadays are settled in the hospital, can be settled at a distance. From a social point of view the benefits are even greater, since the user’s independence increases and therefore their quality of life. In any case, and as can be seen from previous experience (TIC2001-0931, TIN2004-07926), research outputs at a publication level is guaranteed. Specifically, papers have been published in periodic publications or magazines and Conference proceedings of the field, such as: Lectures Notes in Computer Science & Artificial Intelligence (LNCS & LNAI) CEIG (Congreso Español de Informática Gráfica) SNRFAI (Sinposium Nacional de Reconocimiento de Formas e Inteligencia Artificial) Image Processing and Machine Vision (Fair and Workshop) Germany IEEE Transaction on Visualization & Computer Graphics ACM Transactions on Graphics With respect to post gtaduate research, the following Ph.D thesis are being implemented: P. Noll do Matos: Intelligent Facial Animation Systems (ULBRA) Director: F. Perales A: Jaume: Human motion analysis system by means of prioritized inverse cinematics Director: F. Perales, X.Varona C. Manresa: A facial control system by computer Director: F. Perales, X.Varona J. Rossi: Coldiesis: Color and Sound as interaction paradigm Director: F. Perales

Al same time, multiple final projects (PFC) are being developed; they are the basis for future Ph.D. thesis inside the future projected framework. In the section of European projects, the group participates actively in the generation of proposals. As it has previously been commented, the group has headed the AALIVE Project proposal as global coordinator and fundamental partner. In the last call, it was not possible to obtain the minimum scoring to pass the cut, but comments were positive and the consortium is maintained. We foresee to present a proposal to the following Call of the VII framework agreement with the opportune improvements.

36

6. HISTORIAL DEL EQUIPO SOLICITANTE EN EL TEMA PROPUESTO (En caso de Proyecto Coordinado, los apartados 6. y 6.1. deberán rellenarse para cada uno de los equipos participantes) (máximo dos páginas) Indicar las actividades previas del equipo y los logros alcanzados en el tema propuesto:

Si el proyecto es continuación de otro previamente financiado, deben indicarse con claridad los objetivos ya logrados y los resultados alcanzados. Si el proyecto aborda una nueva temática, deben indicarse los antecedentes y contribuciones previas del equipo, con el fin de justificar su capacidad para llevar a cabo el nuevo proyecto.

Este apartado, junto con el 3, tiene como finalidad determinar la adecuación y capacidad del equipo en el tema (y en consecuencia, la viabilidad de la actividad propuesta).

The group defined for the execution of the project is formed by investigators that have worked in the fundamental branches that shape the proposal. The main nucleus is formed by the Group of Graphics and Vision by Computer and Artificial Intelligence of the Math and Computer Science Department of the UIB (consolidated group of investigation). Although given the interdisciplinarity and complexity of the Project, we have resorted to specialists of other centers and companies with experience in the field.

The evolution of the subject matter in previous projects (TIC96-0528, TIC98-C0302, TIC2001-0931, HUMODAN IST2001-32202, TIN2004-07926) and the growth in number and specialties of the components of the group has permitted, in this call, to center the lines presented and to create new proposals for young investigators. It is obvious that the project’s main core is the application related to the tangible interfaces, but with a strong artificial intelligence component that controls all of the processes. The UIB’s original Graphics and Computer Vision group has fused with the Artificial Intelligence group of the same department in order to include experts in inductive learning and systems based on distributed intelligent agents.

The TIC96-0528 project was the embryo of this line of work, where the basis was established and subsequently consolidated by means of the TIC98-C0302 and TIC2001-0931projects. All of this has already been commented during the memory of results obtained up till now. In the TEL96-0544 and TIC98-C0302 projects, the cited investigators main work was the study of deformable objects based on physical models. The experience in the modeling of complex objects is very necessary to confront the area of advanced facial modelizacion, where specific knowledge on deformable and multilevel models is required. Besides, we also count with the contribution of external members of the University of Feevale (Brazil) and of the ULBRA University (Brazil), who have worked in advanced facial models and, particularly, in facial aging. This is an example of international contribution in research projects. The last project granted, TIN2004-07926, represents the first step in multimodal interface systems generation, but it has certain limitations and it is oriented towards the humanoid’s visual realism and not toward the application’s real behavior and environment. The participation in European projects is shown in first place with the PAVR (Platform for Animation and Virtual Reality) project, which concentrates the better research centers of the continent. The CHARM (TO COMPREHENSIVE HUMAN MOTION ANIMATION RESOURCE MODEL) Project of the CEE ESPRIT-BRA CHARM, is another example of the intensive participation in European projects related to the area of computer animation. The group’s consolidation has permitted the accomplishment of the HUMODAN-2001-32202 project, formed by a consortium of 3 research centers (CEIT, Fraunhofer, UIB) and three companies (KURSAAL, SYNKRONIX and SYSTEMA). Research was carried out on the analysis and animation of human movement in virtual environments. This project has permitted the design of a virtual reality system with user’s direct interaction in a distributed environment (Spain / Germany) by means of the AVANGO software. As a result of this Project, the Ministry considered it adequate to finance part of it with the Special Action TIC2002-10743-E. The HUMODAN project finalized in 2005, with the positive appraisal of the revisors and a good number of publications.

37

Al same time, the group has been active in European Community project calls and has participated in the last one with proposals carried out by foreign coordinators (Germany and England). In particular, we want to mention the uPress (Prof. Dr. Peter Schulthers, Distributed Systems Departmt, Ulm University) proposal and the ADAM (coordinator collaboration@intergrams.com) proposal. They were not finally accepted but had a good assesment. All this allows us to sustain that the group is considered, at a European scale, as a good partner or associate in proposal requests. We have also decided to assume the role of coordinator and, as already has been commented, we have presented the AALIVE proposal, that did not prosper but that had a good review. At a national level, the continuous contribution among different research teams in the field of the proposal, and more concretely oriented to the vision by computer area, has been summarized in the active participation in the thematic network TIC2001-4570-E, Thematic Network in the Structuring, processing and generation of Digital Video in Multimedia Applications (2001-2003), where multiple meetings, exchange of data, algorithms and ideas have been carried out. This network finalized in December of 2003 holding an Internacional Workshop in Las Palmas de Gran Canaria where the joint work carried out was presented. Recently a new thematic Network has been proposed, it has been granted by the Ministry and it guarantees the continuity of the previous one promoting the synergy among groups. In the Software Engineering area, we count with new investigators with experience in the OO and software reusability field. They have participated in European projects, such as COMPLEMENT, of evaluation of methodologies and in the TIC96-0513 project on environment simulation for client-server architecture and the modeling of its behavior. An important aspect is the direct contact with related companies. This point is especially taken care of by the group, since the final objective is to obtain a commercial system of the developments of the proposed project. Because of this, contacts with EPTRON are maintained and the TAGrv Company has been expanded. This is a newly created company of extensive technological basis located in ParcBit (Balearic technological Park). This it composed by young computer science and communication engineers, which directly promotes the transfer of technology to the emerging businesses.

As it can be appreciated, the group has evolved and acquired experience and new investigators in the Artificial Intelligence area. It therefore counts with a good background in national and/or European development of projects due to the continuous effort of its components and the acceptable quality of its results. To conclude, it can be stated that the defined multidisciplinary group guarantees a good basis to confront the different problems that can appear in this project. This group has evolved in time, through the projects already granted and the favorable evaluations obtained in previous calls. The team’s work methodology, already known perfectly, is correctly applied, obtaining acceptable results in the majority of projects. As a summary, it can be stated that the group counts with experts in Numerical Calculation, Biomechanics, Vision by Computer, Image Processing and Analysis, Design and Programming Oriented to Objects (DOO and POO), Computer Animation, Artificial Intelligence, all of it complemented by some important installations (hardware and software) in Graphical Computer Science, Vision by computer, and RV/RA systems, besides of the support of the Math and Computer Science Depatment of the UIB of the own University.

3

9

6.1

FIN

AN

CIA

CIÓ

N P

ÚB

LIC

A Y

PR

IVA

DA

(PR

OYE

CTO

S Y

CO

NTR

ATO

S D

E I+

D) D

E LO

S M

IEM

BR

OS

DEL

EQ

UIP

O IN

VEST

IGA

DO

R (*

) D

ebe

indi

cars

e ún

icam

ente

lo fi

nanc

iado

en

los

últim

os c

inco

año

s (2

000-

2004

), y

a se

a de

ám

bito

aut

onóm

ico,

nac

iona

l o in

tern

acio

nal.

Deb

en in

clui

rse

las

solic

itude

s pe

ndie

ntes

de

reso

luci

ón.

Sub

venc

ión

conc

edid

a o

solic

itada

Títu

lo d

el p

roye

cto

o co

ntra

to

Rel

ació

n co

n la

sol

icitu

d

que

ahor

a se

pr

esen

ta (

1)

In

vest

igad

or

resp

onsa

ble

EU

RO

E

ntid

ad fi

nanc

iado

ra y

re

fere

ncia

del

pro

yect

o

Per

iodo

de

vige

ncia

o

fech

a de

la s

olic

itud

(2

)

CIC

YT

TIN

2004

-079

26. I

nteg

raci

ón d

e es

cena

rios

virt

uale

s co

n ag

ente

s in

telig

ente

s 3D

1

F

. J. P

era

les

18

9.4

00

C

ICY

T

C

12

/20

04

1

2/2

00

7

CIC

YT

TIC

2001

-093

1. A

nális

is y

Sín

tesi

s de

l Mov

imie

nto

Hum

ano

Med

iant

e T

écni

cas

de V

isió

n y

Ani

mac

ión

por

Ord

enad

or

1

F. J. P

era

les

13

4.2

66

C

ICY

T

C

12

/20

01

1

2/2

00

4

TIC

2002

-107

43-E

. Acc

ión

Esp

ecia

l Sis

tem

a de

Rea

lidad

V

irtua

l y A

umen

tada

1

F. J. P

era

les

13

2.1

50

C

ICY

T

C

12

/20

02

1

2/2

00

5

HU

MO

DA

N IS

T-2

001-

3220

2 (A

n au

tom

atic

hum

an m

odel

an

imat

ion

envi

ronm

ent f

or A

ugm

ente

d R

ealit

y in

tera

ctio

n)

1

F. J. P

era

les

50

0.0

00

C

om

isió

n E

uro

pe

a

C

9/2

00

2-

5/2

00

5

(*)

Véa

se a

part

ado

5º.5

de

la C

onvo

cato

ria

E

scrí

base

0, 1

, 2 o

3 s

egún

la s

igui

ente

cla

ve:

0 =

Es

el m

ism

o pr

oyec

to

1 =

est

á m

uy r

elac

iona

do, 2

= e

stá

algo

rel

acio

nado

, 3 =

sin

rel

ació

n, (

2)

Esc

ríba

se u

na C

o u

na S

seg

ún s

e tr

ate

de u

na c

once

sión

o d

e un

a so

licitu

d.

40

7. CAPACIDAD FORMATIVA DEL PROYECTO Y DEL EQUIPO SOLICITANTE (En caso de Proyecto Coordinado deberá rellenarse para cada uno de los equipos participantes) Este apartado sólo debe rellenarse si se ha respondido afirmativamente a la pregunta correspondiente en el cuestionario de solicitud. Debe justificarse que el equipo solicitante está en condiciones de recibir becarios (del Programa de Formación de Investigadores) asociados a este proyecto y debe argumentarse la capacidad formativa del equipo. En caso de Proyecto Coordinado, debe rellenarse por cada subproyecto que solicite becarios de FPI.

Due to the activities of previous research, the training level of the group is high, generating a reasonable number of doctoral thesis. Currently it has 6 doctorands, and in brief several thesis will be defended. The cooperation with universities of Latin-America, has promoted the arrival of students from universities in Argentina, Venezuela and Brazil. The work environment is adequate, so that graduate students can carry out in a competitive way their first research work. Also, by means of the Ramon and Cajal program we have welcomed doctors (Dr. X. Varona, UIB) that are collaborating actively in the research of the group. They have already begun to ask for their own projects as IP' s. All of this shows the formative capacity of the applicant group.