Iso/iec jtc 1/sc 29 N

Scope

This part of ISO/IEC 23000 specifies:

• 
  Scene description elements for representing AR content
  
• 
  Mechanisms to connect to local and remote sensors and actuators
  
• 
  Mechanisms to integrated compressed medias (image, audio, video, graphics)
  
• 
  Mechanisms to connect to remote resources such as maps and compressed medias
  

Normative references

The following documents, in whole or in part, are normatively referenced in this document and are indispensable for its application. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/IEC 14496-1, Information technology — Coding of audio-visual objects — Part 1: Systems

ISO/IEC 14496-3:2009, Information technology — Coding of audio-visual objects — Part 3: Audio

ISO/IEC 14496-11:201X, Information technology — Coding of audio-visual objects — Part 11: Scene description and application engine

ISO/IEC 14496-16:2011, Information technology — Coding of audio-visual objects — Part 16: Animation Framework eXtension (AFX)

ISO/IEC 23005-5:2011, Information technology — Media context and control — Part 5: Data formats for interaction devices

ISO/IEC 14772-1:1997, Information technology — Computer graphics and image processing — The Virtual Reality Modeling Language — Part 1: Functional specification and UTF-8 encoding

ISO/IEC 10646-1:2012, Information technology — Universal Multiple-Octet Coded Character Set (UCS) — Part 1: Architecture and Basic Multilingual Plane

ISO/IEC 8859-1:1998, Information technology — 8-bit single-byte coded graphic character sets — Part 1: Latin alphabet No.1

Terms, definitions, and abbreviated terms

Terms and definitions

For the purposes of this document, the terms and definitions are defined in ISO/IEC 23000-1:2012 Multimedia Application Format, Part 14 Augmented Reality Reference Model.

Abbreviated terms

For the purposes of this International Standard, the following abbreviated terms apply.

AR Augmented Reality

URI Uniform Resource Identifier

URL Uniform Resource Locator

URN Uniform Resource Name

ARAF Components

ARAF principle and context

Augmented Reality (AR) applications refer to a view of a real-world environment whose elements are augmented by content, such as graphics or sound, in a computer driven process. Figure 1 illustrates two real and virtual cameras and the composition of a real image and graphics objects.

Figure 1 — Simplified illustration of the AR principle

The Augmented Reality Application Format (ARAF) is an extension of a subset of the MPEG-4 part 11 Scene Description and Application Engine standard, combined with other relevant MPEG standards (MPEG-4, MPEG-V), designed to enable the consumption of 2D/3D multimedia content as depicted in Figure 2.

An ARAF, available as a file or stream, is interpreted by a device, called ARAF device. The nodes of the ARAF scene point to different sources of multimedia content such as 2D/3D image, 2D/3D audio, 2D/3D video, 2D/3D graphics and sensor/sensory information sources/sinks that are either remote or/and local.

Figure 2 — The ARAF context

ARAF Scene Description

To describe the multimedia scene ARAF is based on ISO/IEC 14496-11 (MPEG-4 Part 11 BIFS) which at its turn is based on ISO/IEC 14772-1:1997 (VRML97). About two hundreds nodes are standardized in MPEG-4 BIFS and VRML, allowing various kinds of scenes to be constructed. ARAF is referring to a subset of MPEG-4 BIFS nodes and external prototypes as defined in ISO/IEC 14496-11 (MPEG-4 Part 11 BIFS) annex E (Predefined Prototype Nodes) for scene description as presented below.

Category	Sub-category	Node, Prototypes / Elements name in MPEG-4 BIFS / XMT
Elementary media	Audio	AudioSource
		Sound
		Sound2D
	Image and video	ImageTexture
		MovieTexture
	Textual information	FontStyle
		Text
	Graphics	Appearance
		Color
		LineProperties
		LinearGradient
		Material
		Material2D
		Rectangle
		Shape
		SBVCAnimationV2
		SBBone
		SBSegment
		SBSite
		SBSkinnedModel
		MorphShape
		Coordinate
		TextureCoordinate
		Normal
		IndexedFaceSet
		IndexedLineSet
Programming information		Script
User interactivity		InputSensor
		OutputActuator
		SphereSensor
		TimeSensor
		TouchSensor
		MediaSensor
		PlaneSensor
Scene related information (spatial and temporal relationships)		Background
		Background2D
		CameraCalibration
		Group
		Inline
		Layer2D
		Layer3D
		Layout
		NavigationInfo
		OrderedGroup
		LoImg
		ReImgProxy
		ReImgServer
		ReImgComp
		Switch
		Transform
		Transform2D
		Viewpoint
		Viewport
		Form
Dynamic and animated scene		OrientationInterpolator
		ScalarInterpolator
		CoordinateInterpolator
		ColorInterpolator
		PositionInterpolator
		Valuator
Communication and compression		BitWrapper
		MediaControl
	Maps	Map
		MapOverlay
		MapMarker
		MapPlayer
Terminal		TermCap

All the above listed elements are specified in MPEG-4 Part 11. However, to facilitate the implementation of ARAF content, the current document contains their XML syntax as well as the semantics and functionality.

MPEG-4 Part 11 describes a scene with a hierarchical structure that can be represented as a graph. Nodes of the graph build up various types of objects, such as audio video, image, graphic, text, etc. Furthermore, to ensure the flexibility, a new, user-defined type of node derived from a parent one can also be defined on demand by using the Proto method.
In general, nodes expose a set of parameters, through which aspects of their appearance and behavior can be controlled. By setting these values, scene designers have a tool to force a scene-reconstruction at clients’ terminals to adhere to their intention in a predefined manner. In more complicated scenario, the structure of BIFS nodes is not necessarily static; nodes can be added or removed from the scene graph arbitrarily.
Certain types of nodes called sensors, such as TimeSensor, TouchSensor, can interact with users and generate appropriate triggers, which are transmitted to others nodes by routing mechanism, causing changes in state of these receiving nodes. They are bases for the dynamic behavior of a multimedia content supported by MPEG-4.
The maximum flexibility in the programmable feature of MPEG-4 scene is carried out with the Script node. By routing mechanism to Event In valueIn attribute of Script node, the associated function (defined in its URL attribute) with the same name Event In valueIn () will be triggered. The behavior of this function is user-defined, i.e. scene-designer can freely process some computations, and then sets the values for every Event Out valueOut attribute, which consecutively affect the states of other nodes linked to them.
Direct manipulation of nodes’ states is also possible in MPEG-4 Part 11: the Field field attribute can refer to any node in the scene; through this link, all attributes of the contacted node will be exposed to direct setting and modifying operators within the Script node. The syntax of the language used to implement the function of Script node is ECMAScript [ISO/IEC DIS 16262 Information technology - ECMAScript: A general purpose, cross-platform programming language].
ARAF supports the definition and reusability of complex objects by using the MPEG-4 PROTO mechanism. The PROTO statement creates its own nodes by defining a configurable object prototype; it can integrate any other node from the scene graph.
ARAF makes extensive use of EXTERNPROTO mechanism which are nodes whose syntax is identified by URNs as given in ISO/IEC 14496-11 (MPEG-4 Part 11 BIFS) annex E (Predefined Prototype Nodes), while their names are only informative and for convenience can be changed by the content creator in the declaration step.
The following table indicates the MPEG-4 Part 11 nodes that are included in ARAF. For each node, it is specified the version of the standard when it was published.