|
Prima perception Recognition and Integration for Observing and Modeling Activity
|
tarix | 01.07.2018 | ölçüsü | 9,33 Mb. | | #52468 |
|
PRIMA Perception Recognition and Integration for Observing and Modeling Activity James L. Crowley, Prof. I.N.P. Grenoble Augustin Lux, Prof. I.N.P. Grenoble Patrick Reignier, MdC. Univ. Joseph Fourier Dominique Vaufreydaz, MdC UPMF
The PRIMA Group Leaders
The PRIMA Group Members
The PRIMA Group, May 2006 Permanents : - James L. Crowley, Prof. I.N.P. Grenoble
- Augustin Lux, Prof. I.N.P. Grenoble
- Patrick Reignier, MdC. U.J.F.
- Dominique Vaufreydaz, MdC. UPMF.
Assistante : Contractual Engineers - Alba Ferrer, IE INRIA
- Mathieu Langet, IE INPG
The PRIMA Group, May 2006 Doctoral Students : - Stan Borkowski (Bourse EGIDE)
- Chunwiphat, Suphot (Bourse Thailand)
- Thi-Thanh-Hai Tran (Bourse EGIDE)
- Matthieu Anne (Bourse CIFRE - France Telecom)
- Olivier Bertrand (Bourse ENS Cachan)
- Nicolas Gourier (Bourse INRIA)
- Julien Letessier (Bourse INRIA)
- Sonia Zaidenberg (Bourse CNRS - BDI)
- Oliver Brdiczka (Bourse INRIA)
- Remi Emonet (Bourse MENSR)
1) Presentation of Scientific Project - Objectives
- Research Problems and Results
- Bilan 2003 - 2006
- Evolutions for 2007-2010
Objective of Project PRIMA Develop the scientific and technological foundations for context aware, interactive environments Interactive Environment: An environment capable of perceiving, acting, communicating, and interacting with users.
Experimental Platforme : FAME Augmented Meeting Environment 8 Cameras 7 Steerable 1 fixed, wide angle 8 Microphones (acoustic Sensors) 6 Biprocessors (3 Ghz) 3 Video Interaction Devices (Camera-projector pairs)
Augmented Meeting Environment
Research Problems
Research Problems - Context-aware interactive environments
- New forms of man-machine interaction (using perception)
- Real Time, View Invariant, Computer Vision
- Autonomic Architectures for Multi-Modal Perception
Software Architecture for Observing Activity Sensors and actuators: Interface to the physical world. Perception and action: Perceives entities, Assigns entities to roles. Situation: Filter events, Describes relevant actors and props for services. (User) Services: Implicit or explicit. Event driven.
Situation Graph Situation: An configuration of entities playing roles Configuration: Set of Relations (Predicates) over entities. Entity: Actors or Objects Roles: Abstract descriptions of Persons or objects A situation graph describes a state space of situations and the actions of the system for each situation
Situation and Context Basic Concepts: Property: Any value observed by a process Composite entity: A composition of entities Relation: A predicate defined over entities Actor: An entity that can act. Role: Interpretation assigned to an entity or actor Situation: A configuration of roles and relations.
Situation and Context Role: Interpretation assigned to an entity or actor Relation: A predicate over entities and actors Situation: An configuration of roles and relations. A situation graph describes the state space of situations and the actions of the system for each situation Approach: Compile a federation of processes to observe the roles (actors and entities) and relations that define situations.
Acquiring Situation Models Objective: - Automatic acquisition of situation models.
Approach: - Start with simple sterotypical model for scenario
- Develop using Supervised Incremental Learning
Recognition: - Detect Roles with Linear Classifiers
- Recognize Situation using probablisitic model
Video Acquisition System V2.0
Audio-Visual Acquisition System
Research Problems - Context-aware interactive environments
- New forms of man-machine interaction (using perception)
- Real Time, View Invariant, Computer Vision
- Autonomic Architectures for Multi-Modal Perception
Steerable Camera Projector Pair
Portable Display Surface
Rectification by Homography For each rectified pixel (x,y), project to original pixel and compute interpolated intensity
Real Time Rectification for the PDS
Luminance-based button widget
Striplet – the occlusion detector
Striplet – the occlusion detector
Striplet – the occlusion detector
Striplet-based SPOD
Projected Calculator
Research Problems - Context-aware interactive environments
- New forms of man-machine interaction (using perception)
- Real Time, View Invariant Computer Vision
- Autonomic Architectures for Multi-Modal Perception
Real Time, View Invariant Computer Vision Results - Scale and orientation normalised Receptive Fields computed at video rate. (BrandDetect system, IST CAVIAR)
- Real time indexing and recognition (Thesis F. Pelisson)
- Robust Visual Features for Face Detection
- Direct Computation of Time to Crash
- Natural Interest "Ridges"
Scale and Orientation Normalised Gaussian RF's
Natural Interest Points (Scale Invariant "Salient" image features)
Natural Ridge Detection [Tran04] Compute Derivatives at different Scales. For each point (x,y,scale) - Compute second derivatives: fxx,fyy,fxy
- Compute eigenvalues and eigenvectors of Hessian matrix
- Detect local extremum in the direction corresponding to the largest eigenvalue.
- Assemble Ridge points,
Real Time, View Invariant Computer Vision Current activity - Robust Visual Features for Face Detection
- Direct Computation of Time to Crash
- Natural Interest "Ridges" for perceptual organisation.
Research Problems - Context-aware interactive environments
- New forms of man-machine interaction (using perception)
- Real Time, View Invariant, Computer Vision
- Autonomic Architectures for Multi-Modal Perception
Supervised Perceptual Process Supervisor Provides: Execution Scheduler • Command Interpreter
Detection and Tracking of Entities Entities: Correlated sets of blobs - Blob Detectors: Backgrnd difference, motion,color, receptive fields histograms
- Entity Grouper: Assigns roles to blobs as body, hands, face or eyes
Autonomic Properties provided by process supervisor Auto-regulatory: The process controller can adapt parameters to maintain a desired process state. Auto-descriptive: The process controller provides descriptions of the capabilities and the current state of the process. Auto-critical: Process estimates confidence for all properties and events. Self Monitoring: Maintaining a description of process state and quality of service
Self-monitoring Perceptual Process Process monitors likelihood of output When an performance degrades, process adapts processing (modules, parameters, and data)
Autonomic Parameter Regulation Parameter regulation provides robust adaptation to Changes in operating conditions.
Research Contracts (2003-2006) National and Industrial: - ROBEA HR+ : Human-Robot Interaction (with LAAS and ICP)
- ROBEA ParkNav: Perception and action dynamic environments
- RNTL ContAct: Context Aware Perception (with XRCE)
- Contract HARP (Context aware Services - France Telecom)
IST - FP VI: IST - FP V: - Project IST - CAVIAR: Context Aware Vision for Surveillance
- Project IST - FAME: Multi-modal perception for services
- Project IST - DETECT : Publicity Detection in Broadcast Video
- Project FET - DC GLOSS : Global Smart Spaces
- Thematic Network: FGNet (« Face and Gesture »)
- Thematic Network: ECVision - Cognitive Vision
Collaborations INRIA Projects - EMOTION (INRIA RA): Vision for Autonomous Robots; ParkNav, ROBEA (CNRS), Theses of C. Braillon and A. Negre
- ORION (Sophia): Cognitive Vision (ECVision), Modeling Human Activity
Academic: - IIHM, Laboratoire CLIPS: Human-Computer Interaction, Smart Spaces; Mapping Project, IST Projects GLOSS, FAME, Thesis: J. Letissier
Univ. of Karlsruhe (Multimodal interaction): IST FAME and CHIL. Industry - France Telecom: (Lannion and Meylan) Project HARP, Thesis of M. Anne.
- Xerox Research Centre Europe: Project RNTL/Proact Cont'Act
- IBM Research (Prague,New York): Situation Modeling, Autonomic Software Archictures, Projet CHIL
Knowledge Dissemination
Conferences and Workshops Organised
APP Registered Software
Start-up: Blue Eye Video
Blue Eye Video Activity Sensor (PETS 2002 Data)
Blue Eye Video Activity Sensor (Distributed Sensor Networks)
Evolutions for 2006-2010 Context-aware interactive environments - Adaptation and Development of Activity Models
New forms of man-machine interaction Real Time, View Invariant, Computer Vision - Embedded View-invariant Visual Perception
Autonomic Architectures for Multi-Modal Perception - Learning for Monitoring and Regulation
- Dynamic Service Composition
Automatic Adaptation and Development of Models for Human Activity Adaptation: consistent behaviour across environments
Affective interaction Interactive objects that recognize interest and affect and that learn to perceive and evoke emotions in humans.
Embedded View-invariant Visual Perception Embedded Real Time View Invariant Vision in phones and PDA’s (Work with ST MicroSystems)
Distributed Autonomic Systems for Multi-modal Perception
PRIMA Perception Recognition and Integration for Observing and Modeling Activity James L. Crowley, Prof. I.N.P. Grenoble Augustin Lux, Prof. I.N.P. Grenoble Patrick Reignier, MdC. Univ. Joseph Fourier Dominique Vaufreydaz, MdC UPMF
Dostları ilə paylaş: |
|
|