Sunday, March 18, 2018

HVEI'18: A Framework for Adaptive Delivery of Omnidirectional Video

A Framework for Adaptive Delivery of Omnidirectional Video

Christian Timmerer (Alpen-Adria-Universität Klagenfurt / Bitmovin) and Ali C. Begen (Ozyegin University / Networked Media)

Abstract: Omnidirectional or 360-degree videos are considered as a next step towards a truly immersive media experience. Such videos allow the user to change her/his viewing direction while consuming the video. The download-and-play paradigm (including DVD and Blu-ray) is replaced by streaming, and the content is hosted solely within the cloud. This paper addresses the need for a scientific framework enabling the adaptive delivery of omnidirectional video within heterogeneous environments. We consider the state-of-the-art techniques for adaptive streaming over HTTP and extend them towards omnidirectional/360-degree videos. In particular, we review the encoding and adaptive streaming options, and present preliminary results reported in the literature. Finally, we provide an overview about the ongoing standardization efforts and highlight the major open issues.


Wednesday, March 7, 2018

PostDoc Assistant (tenure track) at Alpen-Adria-Universität Klagenfurt, Faculty of Technical Sciences

Alpen-Adria-Universität Klagenfurt announces the following open position in compliance with § 107 para. 1 Universities Act 2002:

PostDoc Assistant (tenure track)

at the Faculty of Technical Sciences. This is a full-time position (initial employment limited to 6 years) with the option of concluding a qualification agreement (promotion to Assistant Professor). Upon fulfilling the qualification agreement, the position progresses from Assistant to Associate Professor (permanent employment). Starting date is earliest possible.

This opening is aimed exclusively at women and is part of a package of measures to increase the proportion of women in professorships and tenure track positions at the Faculty of Technical Sciences. The successful applicant is expected to be assigned to one of the working groups of the 9 departments of the Faculty of Technical Sciences in order to ensure synergies in research and teaching. The departments cover the following fields:
  • Computer Science
  • Didactics of Computer Science
  • Didactics of Mathematics
  • Information Technology
  • Mathematics
  • Statistics 
Further information about the faculty, its departments and their working groups is available at

Tasks and Responsibilities
Participation in the department's research and teaching tasks, including
  • independent research and further development of the candidate's scientific qualification to the level required for Associate Professorship,
  • graduate and undergraduate teaching, examination activities, and supervision of students, 
  • preparation of grant applications and management of research projects, 
  • publications and active participation in international conferences, 
  • establishing international scientific contacts, and 
  • participation in administration, in university committees, and in public relations activities. 
Required Qualifications
  • PhD in one of the above enumerated research fields 
  • Outstanding research achievements and scientific publications 
  • Potential for future scientific work 
  • Teaching experience (at university level) and didactic competence 
  • Excellent English language skills 
Candidates must meet the required qualifications by April 4th 2018 at the latest.

Additional Desired Qualifications
  • Pertinent international experience or practical experience 
  • Embedding in the international research community 
  • Experience in grant applications and project management 
  • Scientific compatibility with at least one of the faculty's research groups 
  • Experience with and interest in interdisciplinary projects 
  • Communication and presentation skills 
  • Leadership, organisational competence, and ability to cooperate in a team 
  • Experience in administration of university departments and in committee work 
  • German language skills 
German language skills are not a formal prerequisite, but proficiency at level B2 is expected with- in two years.

People with disabilities or chronic diseases, who fulfill the requirements, are particularly encour- aged to apply.

Salary and Application

Minimum gross salary for this position is € 51,955.40 per annum (§ 27 Uni-KV B1 lit b), € 61,441.80 after promotion to Assistant Professor (§ 27 Uni-KV A2) and € 66,619.00 after promotion to Associate Professor.

General information for applicants is available on

We welcome applications in English with the usual documents including three references (address- es of persons who can be contacted by the Alpen-Adria-Universität for information) by April 4th 2018. Applications must be submitted online via (please indicate reference code 711/17). In the cover letter, the research area should be mentioned.

For further information, please contact Assoc. Prof. Dr. Angelika Wiegele, e-mail: frauen-

Travel and accommodation costs incurred during the application process will not be refunded.

Tuesday, February 13, 2018

MPEG news: a report from the 121st meeting, Gwangju, Korea

The original blog post can be found at the Bitmovin Techblog and has been updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.

The MPEG press release comprises the following topics:
  • Compact Descriptors for Video Analysis (CDVA) reaches Committee Draft level
  • MPEG-G standards reach Committee Draft for metadata and APIs
  • MPEG issues Calls for Visual Test Material for Immersive Applications
  • Internet of Media Things (IoMT) reaches Committee Draft level
  • MPEG finalizes its Media Orchestration (MORE) standard
At the end I will also briefly summarize what else happened with respect to DASH, CMAF, OMAF as well as discuss future aspects of MPEG.

Compact Descriptors for Video Analysis (CDVA) reaches Committee Draft level

The Committee Draft (CD) for CDVA has been approved at the 121st MPEG meeting, which is the first formal step of the ISO/IEC approval process for a new standard. This will become a new part of MPEG-7 to support video search and retrieval applications (ISO/IEC 15938-15).

Managing and organizing the quickly increasing volume of video content is a challenge for many industry sectors, such as media and entertainment or surveillance. One example task is scalable instance search, i.e., finding content containing a specific object instance or location in a very large video database. This requires video descriptors which can be efficiently extracted, stored, and matched. Standardization enables extracting interoperable descriptors on different devices and using software from different providers, so that only the compact descriptors instead of the much larger source videos can be exchanged for matching or querying. The CDVA standard specifies descriptors that fulfil these needs and includes (i) the components of the CDVA descriptor, (ii) its bitstream representation and (iii) the extraction process. The final standard is expected to be finished in early 2019.

CDVA introduces a new descriptor based on features which are output from a Deep Neural Network (DNN). CDVA is robust against viewpoint changes and moderate transformations of the video (e.g., re-encoding, overlays), it supports partial matching and temporal localization of the matching content. The CDVA descriptor has a typical size of 2–4 KBytes per second of video. For typical test cases, it has been demonstrated to reach a correct matching rate of 88% (at 1% false matching rate).

Research aspects: There are probably endless research aspects in the visual descriptor space ranging from validation of the achieved to results so far to further improving informative aspects with the goal to increase correct matching rate (and consequently decreasing the false matching rating). In general, however, the question is whether there's a need for descriptors in the era of bandwidth-storage-computing over-provisioning and the raising usage of artificial intelligence techniques such as machine learning and deep learning.

MPEG-G standards reach Committee Draft for metadata and APIs

In my previous report I introduced the MPEG-G standard for compression and transport technologies of genomic data. At the 121st MPEG meeting, metadata and APIs reached CD level. The former - metadata - provides relevant information associated to genomic data and the latter - APIs - allow for building interoperable applications capable of manipulating MPEG-G files. Additional standardization plans for MPEG-G include the CDs for reference software (ISO/IEC 23092-4) and conformance (ISO/IEC 23092-4), which are planned to be issued at the next 122nd MPEG meeting with the objective of producing Draft International Standards (DIS) at the end of 2018.

Research aspects: Metadata typically enables certain functionality which can be tested and evaluated against requirements. APIs allow to build applications and services on top of the underlying functions, which could be a driver for research projects to make use of such APIs.

MPEG issues Calls for Visual Test Material for Immersive Applications

I have reported about the Omnidirectional Media Format (OMAF) in my previous report. At the 121st MPEG meeting, MPEG was working on extending OMAF functionalities to allow the modification of viewing positions, e.g., in case of head movements when using a head-mounted display, or for use with other forms of interactive navigation. Unlike OMAF which only provides 3 degrees of freedom (3DoF) for the user to view the content from a perspective looking outwards from the original camera position, the anticipated extension will also support motion parallax within some limited range which is referred to as 3DoF+. In the future with further enhanced technologies, a full 6 degrees of freedom (6DoF) will be achieved with changes of viewing position over a much larger range. To develop technology in these domains, MPEG has issued two Calls for Test Material in the areas of 3DoF+ and 6DoF, asking owners of image and video material to provide such content for use in developing and testing candidate technologies for standardization. Details about these calls can be found at

Research aspects: The good thing about test material is that it allows for reproducibility, which is an important aspect within the research community. Thus, it is more than appreciated that MPEG issues such a call and let's hope that this material will become publicly available. Typically this kind of visual test material targets coding but it would be also interesting to have such test content for storage and delivery.

Internet of Media Things (IoMT) reaches Committee Draft level

The goal of IoMT is is to facilitate the large-scale deployment of distributed media systems with interoperable audio/visual data and metadata exchange. This standard specifies APIs providing media things (i.e., cameras/displays and microphones/loudspeakers, possibly capable of significant processing power) with the capability of being discovered, setting-up ad-hoc communication protocols, exposing usage conditions, and providing media and metadata as well as services processing them. IoMT APIs encompass a large variety of devices, not just connected cameras and displays but also sophisticated devices such as smart glasses, image/speech analyzers and gesture recognizers. IoMT enables the expression of the economic value of resources (media and metadata) and of associated processing in terms of digital tokens leveraged by the use of blockchain technologies.

Research aspects: The main focus of IoMT is APIs which provides easy and flexible access to the underlying device' functionality and, thus, are an important factor to enable research within this interesting domain. For example, using these APIs to enable communicates among these various media things could bring up new forms of interaction with these technologies.

MPEG finalizes its Media Orchestration (MORE) standard

MPEG "Media Orchestration" (MORE) standard reached Final Draft International Standard (FDIS), the final stage of development before being published by ISO/IEC. The scope of the Media Orchestration standard is as follows:
  • It supports the automated combination of multiple media sources (i.e., cameras, microphones) into a coherent multimedia experience.
  • It supports rendering multimedia experiences on multiple devices simultaneously, again giving a consistent and coherent experience.
  • It contains tools for orchestration in time (synchronization) and space.
MPEG expects that the Media Orchestration standard to be especially useful in immersive media settings. This applies notably in social virtual reality (VR) applications, where people share a VR experience and are able to communicate about it. Media Orchestration is expected to allow synchronizing the media experience for all users, and to give them a spatially consistent experience as it is important for a social VR user to be able to understand when other users are looking at them.

Research aspects: This standard enables the social multimedia experience proposed in literature. Interestingly, the W3C is working on something similar referred to as timing object and it would be interesting to see whether these approaches have some commonalities.

What else happened at the MPEG meeting?

DASH is fully in maintenance mode and we are still waiting for the 3rd edition which is supposed to be a consolidation of existing corrigenda and amendments. Currently only minor extensions are proposed and conformance/reference software is being updated. Similar things can be said for CMAF where we have one amendment and one corrigendum under development. Additionally, MPEG is working on CMAF conformance. OMAF has reached FDIS at the last meeting and MPEG is working on reference software and conformance also. It is expected that in the future we will see additional standards and/or technical reports defining/describing how to use CMAF and OMAF in DASH.

Regarding the future video codec, the call for proposals is out since the last meeting as announced in my previous report and responses are due for the next meeting. Thus, it is expected that the 122nd MPEG meeting will be the place to be in terms of MPEG’s future video codec. Speaking about the future, shortly after the 121st MPEG, Leonardo Chiariglione published a blog post entitled “a crisis, the causes and a solution”, which is related to HEVC licensing, Alliance for Open Media (AOM), and possible future options. The blog post certainly caused some reactions within the video community at large and I think this was also intended. Let’s hope it will galvanice the video industry -- not to push the button -- but to start addressing and resolving the issues. As pointed out in one of my other blog posts about what to care about in 2018, the upcoming MPEG meeting in April 2018 is certainly a place to be. Additionally, it highlights some conferences related to various aspects also discussed in MPEG which I'd like to republish here:
  • QoMEX -- Int'l Conf. on Quality of Multimedia Experience -- will be hosted in Sardinia, Italy from May 29-31, which is THE conference to be for QoE of multimedia applications and services. Submission deadline is January 15/22, 2018.
  • MMSys -- Multimedia Systems Conf. -- and specifically Packet Video, which will be on June 12 in Amsterdam, The Netherlands. Packet Video is THE adaptive streaming scientific event 2018. Submission deadline is March 1, 2018.
  • Additionally, you might be interested in ICME (July 23-27, 2018, San Diego, USA), ICIP (October 7-10, 2018, Athens, Greece; specifically in the context of video coding), and PCS (June 24-27, 2018, San Francisco, CA, USA; also in the context of video coding).
  • The DASH-IF academic track hosts special events at MMSys (Excellence in DASH Award) and ICME (DASH Grand Challenge).
  • MIPR -- 1st Int'l Conf. on Multimedia Information Processing and Retrieval -- will be in Miami, Florida, USA from April 10-12, 2018. It has a broad range of topics including networking for multimedia systems as well as systems and infrastructures.

Sunday, January 14, 2018

Delivering Traditional and Omnidirectional Media

This tutorial will be given at the following events:


Universal media access as proposed in the late 90s is now closer to reality. Users can generate, distribute and consume almost any media content, anywhere, anytime and with/on any device. A major technical breakthrough was the adaptive streaming over HTTP resulting in the standardization of MPEG-DASH, which is now successfully deployed in most platforms. The next challenge in adaptive media streaming is virtual reality applications and, specifically, omnidirectional (360°) media streaming.
This tutorial first presents a detailed overview of adaptive streaming of both traditional and omnidirectional media, and focuses on the basic principles and paradigms for adaptive streaming. New ways to deliver such media are explored and industry practices are presented. The tutorial then continues with an introduction to the fundamentals of communications over 5G and looks into mobile multimedia applications that are newly enabled or dramatically enhanced by 5G.
A dedicated section in the tutorial covers the much-debated issues related to quality of experience. Additionally, the tutorial provides insights into the standards, open research problems and various efforts that are underway in the streaming industry.

Learning Objectives

Upon attending this tutorial, the participants will have an overview and understanding of the following topics:
  • Principles of HTTP adaptive streaming for the Web/HTML5
  • Principles of omnidirectional (360) media delivery
  • Content generation, distribution and consumption workflows
  • Standards and emerging technologies, new delivery schemes in the adaptive streaming space
  • Measuring, quantifying and improving quality of experience
  • Fundamental technologies of 5G
  • Features and services enabled or enhanced by 5G
  • Current and future research on delivering traditional and omnidirectional media

Table of Contents

Part I: Streaming (Presented by Dr. Begen and Dr. Timmerer)
  • Survey of well-established streaming solutions (DASH, CMAF and Apple HLS)
  • HTML5 video and media extensions
  • Multi-bitrate encoding, and encapsulation and encryption workflows
  • Common issues in scaling and improving quality, multi-screen/hybrid delivery
  • Acquisition, projection, coding and packaging of 360 video
  • Delivery, decoding and rendering methods
  • The developing MPEG-OMAF and MPEG-I standards
Part II: Communications over 5G (Presented by Dr. Ma and Dr. Begen)
  • 5G fundamentals: radio access and core network
  • Multimedia signal processing and communications
  • Emerging mobile multimedia use cases
  • Detailed analysis for selected use cases
  • Improving QoE


Ali C. Begen recently joined the computer science department at Ozyegin University. Previously, he was a research and development engineer at Cisco, where he has architected, designed and developed algorithms, protocols, products and solutions in the service provider and enterprise video domains. Currently, in addition to teaching and research, he provides consulting services to industrial, legal, and academic institutions through Networked Media, a company he co-founded. Begen holds a Ph.D. degree in electrical and computer engineering from Georgia Tech. He received a number of scholarly and industry awards, and he has editorial positions in prestigious magazines and journals in the field. He is a senior member of the IEEE and a senior member of the ACM. In January 2016, he was elected as a distinguished lecturer by the IEEE Communications Society. Further information on his projects, publications, talks, and teaching, standards and professional activities can be found

Liangping Ma is with InterDigital, Inc., San Diego, CA. He is an IEEE Communication Society Distinguished Lecturer focusing on 5G technologies and standards, video communication and cognitive radios. He is an InterDigital delegate to the 3GPP New Radio standards. His current research interests include various aspects about ultra-reliable and low-latency communication, such as channel coding, multiple access and resource allocation. Previously, he led the research on Quality of Experience (QoE) driven system optimization for video streaming and interactive video communication. Prior to joining InterDigital in 2009, he was with San Diego Research Center and Argon ST (acquired by Boeing), where he led research on cognitive radios and wireless sensor networks and served as the principal investigators of two projects supported by the Department of Defense and the National Science Foundation, respectively. He is the co-inventor of more than 40 patents and the author/co-author of more than 50 journal and conference papers. He has been the Chair of the San Diego Chapter of the IEEE Communication Society since 2014. He received his PhD from University of Delaware in 2004 and his B.S. from Wuhan University, China, in 1998.

Christian Timmerer received his M.Sc. (Dipl.-Ing.) in January 2003 and his Ph.D. (Dr.techn.) in June 2006 (for research on the adaptation of scalable multimedia content in streaming and constrained environments) both from the Alpen-Adria-Universität (AAU) Klagenfurt. He joined the AAU in 1999 (as a system administrator) and is currently an Associate Professor at the Institute of Information Technology (ITEC) within the Multimedia Communication Group. His research interests include immersive multimedia communications, streaming, adaptation, quality of experience, and sensory experience. He was the general chair of WIAMIS 2008, QoMEX 2013 and MMSys 2016, and has participated in several EC-funded projects, notably DANAE, ENTHRONE, P2P-Next, ALICANTE, SocialSensor, COST IC1003 QUALINET and ICoSOLE. He also participated in ISO/MPEG work for several years, notably in the area of MPEG-21, MPEG-M, MPEG-V, and MPEG-DASH where he also served as a standard editor. In 2012, he co-founded Bitmovin to provide professional services around MPEG-DASH where he currently holds the position of the Chief Innovation Officer (CIO).

Saturday, January 6, 2018

What to care about in multimedia communication in 2018?

In the past days/weeks you may have witness a high number of forecasts/predictions for 2018, like this one here. I'm not so good at predictions and we all learned these days to be careful about speculation. Thus, my focus here is on things to care about in 2018.

MPEG and VCEG are working towards a new video coding standard (naming, number scheme yet to be defined) and the call for proposals is out. Responses will be evaluated by the 122nd MPEG meeting in April 2018 (San Diego, CA, USA) and a new standard is expected to be available in late 2020. The main focus of the CfP is (i) 360-degree omnidirectional video, (ii) high-dynamic range (HDR), (iii) wide colour gamut (WCG), and (iv) conventional standard-dynamic-range camera content. The goal is -- simple, as usual -- compress digital video content, i.e., twice as much as you did before with the same video quality, e.g., as HEVC, or get higher quality with the same number of bits (or a combination thereof). Initial, preliminary results indicate this goal is feasible and everyone is looking forward to the MPEG meeting in April; certainly a place to be.

In addition to what MPEG/VCEG is doing, the Alliance for Open Media gained significant attention with its AV1 codec, first demos are available, and recently also Apple joined AOM. AV1 is not longer controlled by a single company, and, thus, it is becoming a real alternative in the video coding landscape, specifically for the streaming market. The nice thing, it's open source and royalty-free! In other words, one should not neglect AV1 and I think we will see many, hopefully good news in 2018.

On this topic, you might be interested in reading this and this.

What about DASH in 2018? We will see a 3rd edition of MPEG-DASH, the DASH-IF will further work on interoperability points, and I expect further convergence of DASH and HLS towards CMAF. However, I also expect minor changes for the main, common use cases utilizing the core technology of HTTP adaptive streaming. Changes, if any, will be transparent to most of us. On the other hand, immersive media and user engagement will become more and more important as more services are delivered over the top leading to more content becoming available to end users, thus, increasing competition among providers, vendors, etc. As a consequence, (a) content, (b) quality, and (c) costs will be important aspects, whereby (a+c) are "easy to sell" but (b) is still difficult to quantify (and "sell") with many open issues to solve in the (near) future.

Before drifting off to forecasts and predictions, I'd like to conclude with a list of scientific events in 2018 which are worthwhile to attend:
  • QoMEX -- Int'l Conf. on Quality of Multimedia Experience -- will be hosted in Sardinia, Italy from May 29-31, which is THE conference to be for QoE of multimedia applications and services. Submission deadline is January 15/22, 2018.
  • MMSys -- Multimedia Systems Conf. -- and specifically Packet Video, which will be on June 12 in Amsterdam, The Netherlands. Packet Video is THE adaptive streaming scientific event 2018. Submission deadline is March 1, 2018.
  • Additionally, you might be interested in ICME (July 23-27, 2018, San Diego, USA; I'm part of a tutorial there;), ICIP (October 7-10, 2018, Athens, Greece; specifically in the context of video coding), and PCS (June 24-27, 2018, San Francisco, CA, USA; also in the context of video coding).
  • The DASH-IF academic track hosts special events at MMSys (Excellence in DASH Award) and ICME (DASH Grand Challenge).
  • MIPR -- 1st Int'l Conf. on Multimedia Information Processing and Retrieval -- will be in Miami, Florida, USA from April 10-12, 2018. It has a broad range of topics including networking for multimedia systems as well as systems and infrastructures.

Tuesday, December 19, 2017

University Assistant (f/m)

Alpen-Adria-Universität Klagenfurt announces the following job vacancy (in accordance with § 107 Abs. 1 Universitätsgesetz 2002):

University Assistant (f/m)
(fixed-term employment for the period of 4 years, 40 hours/week (Uni-KV: B1)

at the Faculty for Technical Sciences, Dept. of Information Technology. The monthly minimum salary for this position as stated in the collective agreement and according to the classification scheme is €2.731 (pre-tax, 14 x per year), but may be higher due to previous employment periods eligible for inclusion and other earnings and remunerations. 

[Important note: this English version is only informative and the official description of the job vacancy is available here, only in German though]

Your duties:
  • Collaboration at the department within the research group “Multimedia Communication” in terms of research and teaching
  • Independent scientific research with the goal to obtain the conferral of a doctorate
  • Participation at the students’ counselling
  • Collaboration at administrative tasks within the department and university committees
  • Collaboration at public relations activities within the department and faculty

The research group “Multimedia Communication” conducts its research within the fields of communication, adaptation, quality of experience (QoE) and use of multimedia data, as well as in the area of future content-aware networks. The goal is to publish in international, high quality professional journals and conference proceedings and to cooperate with various commercial partners. With regard to teaching, additional fields such as computer organization, computer networks, operating systems and parallel systems are covered by our research group.

Your profile:
  • Master or diploma degree of Technical Science in the field of Informatics, Computer Engineering, Electrical Engineering, Information and Communication Engineering or Telematics completed at a domestic or foreign university (with good final degrees)
  • Excellent knowledge and experience in at least one of the following fields: computer organization, operating systems, computer networks, multimedia systems, distributed systems 

Desirable qualifications are:
  • Fluency in German and English, both in written and oral form
  • Excellent programming skills
  • Relevant international and practical work experience
  • Social and communicative competences and ability to work in a team
  • Experience with university teaching and research activities 

All relevant documents for the application (including copies of all school certificates and performance records) have to be submitted via the online application form of Alpen-Adria-Universität Klagenfurt no later than January 10, 2018, mentioning reference number 639/17.

The goal of the position is to equip graduates of a master or diploma programme with the necessary technical and scientific training to complete a doctorate or PhD program in Technical Sciences. Applications of scientists already holding such a degree can therefore not be taken into further consideration.

Alpen-Adria-Universität Klagenfurt lays special emphasis on increasing the number of women in senior and in academic positions and therefore strongly invites qualified women to apply for this position. In case of equal qualifications, female applicants will receive preferential consideration.

Furthermore, persons with disabilities or chronic illnesses who meet the required qualification criteria are also explicitly invited to apply for the position. 

General information for applicants:

Additional information regarding the research group “Multimedia Communication” can be found online ( or by phone +43-463-2700-3612 (Univ.-Prof. DI Dr. Hermann Hellwagner).

Alpen-Adria-Universität Klagenfurt cannot refund any travel or accommodation expenses that arise in connection with the admission procedure.

Thursday, December 14, 2017

MPEG news: a report from the 120th meeting, Macau, China

MPEG Meeting Plenary
The original blog post can be found at the Bitmovin Techblog and has been updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.

The MPEG press release comprises the following topics:
  • Point Cloud Compression – MPEG evaluates responses to call for proposal and kicks off its technical work 
  • The omnidirectional media format (OMAF) has reached its final milestone 
  • MPEG-G standards reach Committee Draft for compression and transport technologies of genomic data 
  • Beyond HEVC – The MPEG & VCEG call to set the next standard in video compression 
  • MPEG adds better support for mobile environment to MMT 
  • New standard completed for Internet Video Coding 
  • Evidence of new video transcoding technology using side streams 

Point Cloud Compression

At its 120th meeting, MPEG analysed the technologies submitted by nine industry leaders as responses to the Call for Proposals (CfP) for Point Cloud Compression (PCC). These technologies address the lossless or lossy coding of 3D point clouds with associated attributes such as colour and material properties. Point clouds are referred to as unordered sets of points in a 3D space and typically captured using various setups of multiple cameras, depth sensors, LiDAR scanners, etc., but can also be generated synthetically and are in use in several industries. They have recently emerged as representations of the real world enabling immersive forms of interaction, navigation, and communication. Point clouds are typically represented by extremely large amounts of data providing a significant barrier for mass market applications. Thus, MPEG has issued a Call for Proposal seeking technologies that allow reduction of point cloud data for its intended applications. After a formal objective and subjective evaluation campaign, MPEG selected three technologies as starting points for the test models for static, animated, and dynamically acquired point clouds. A key conclusion of the evaluation was that state-of-the-art point cloud compression can be significantly improved by leveraging decades of 2D video coding tools and combining 2D and 3D compression technologies. Such an approach provides synergies with existing hardware and software infrastructures for rapid deployment of new immersive experiences.
Although the initial selection of technologies for point cloud compression has been concluded at the 120th MPEG meeting, it could be also seen as a kick-off for its scientific evaluation and various further developments including the optimization thereof. It is expected that various scientific conference will focus on point cloud compression and may open calls for grand challenges like for example at IEEE ICME 2018.

Omnidirectional Media Format (OMAF)

The understanding of the virtual reality (VR) potential is growing but market fragmentation caused by the lack of interoperable formats for the storage and delivery of such content stifles VR’s market potential. MPEG’s recently started project referred to as Omnidirectional Media Format (OMAF) has reached Final Draft of International Standard (FDIS) at its 120th meeting. It includes
  • equirectangular projection and cubemap projection as projection formats; 
  • signalling of metadata required for interoperable rendering of 360-degree monoscopic and stereoscopic audio-visual data; and 
  • provides a selection of audio-visual codecs for this application. 
It also includes technologies to arrange video pixel data in numerous ways to improve compression efficiency and reduce the size of video, a major bottleneck for VR applications and services, The standard also includes technologies for the delivery of OMAF content with MPEG-DASH and MMT.
MPEG has defined a format comprising a minimal set of tools to enable interoperability among implementers of the standard. Various aspects are deliberately excluded from the normative parts to foster innovation leading to novel products and services. This enables us -- researcher and practitioners -- to experiment with these new formats in various ways and focus on informative aspects where typically competition can be found. For example, efficient means for encoding and packaging of omnidirectional/360-degree media content and its adaptive streaming including support for (ultra-)low latency will become a big issue in the near future.

MPEG-G: Compression and Transport Technologies of Genomic Data

The availability of high throughput DNA sequencing technologies opens new perspectives in the treatment of several diseases making possible the introduction of new global approaches in public health known as “precision medicine”. While routine DNA sequencing in the doctor’s office is still not current practice, medical centers have begun to use sequencing to identify cancer and other diseases and to find effective treatments. As DNA sequencing technologies produce extremely large amounts of data and related information, the ICT costs of storage, transmission, and processing are also very high. The MPEG-G standard addresses and solves the problem of efficient and economical handling of genomic data by providing new
  • compression technologies (ISO/IEC 23092-2) and 
  • transport technologies (ISO/IEC 23092-1), 
which reached Committee Draft level at its 120th meeting.
Additionally, the Committee Drafts for
  • metadata and APIs (ISO/IEC 23092-3) and 
  • reference software (ISO/IEC 23092-4) 
are scheduled for the next MPEG meeting and the goal is to publish Draft International Standards (DIS) at the end of 2018.
This new type of (media) content, which requires compression and transport technologies, is emerging within the multimedia community at large and, thus, input is welcome.

Beyond HEVC – The MPEG & VCEG Call to set the Next Standard in Video Compression

The 120th MPEG meeting marked the first major step toward the next generation of video coding standard in the form of a joint Call for Proposals (CfP) with ITU-T SG16’s VCEG. After two years of collaborative informal exploration studies and a gathering of evidence that successfully concluded at the 118th MPEG meeting, MPEG and ITU-T SG16 agreed to issue the CfP for future video coding technology with compression capabilities that significantly exceed those of the HEVC standard and its current extensions. They also formalized an agreement on formation of a joint collaborative team called the “Joint Video Experts Team” (JVET) to work on development of the new planned standard, pending the outcome of the CfP that will be evaluated at the 122nd MPEG meeting in April 2018. To evaluate the proposed compression technologies, formal subjective tests will be performed using video material submitted by proponents in February 2018. The CfP includes the testing of technology for 360° omnidirectional video coding and the coding of content with high-dynamic range and wide colour gamut in addition to conventional standard-dynamic-range camera content. Anticipating a strong response to the call, a “test model” draft design is expected be selected in 2018, with development of a potential new standard in late 2020.
The major goal of a new video coding standard is to be better than its successor (HEVC). Typically this "better" is quantified by 50% which means, that it should be possible encode the video at the same quality with half of the bitrate or a significantly higher quality with the same bitrate including. However, at this time the “Joint Video Experts Team” (JVET) from MPEG and ITU-T SG16 faces competition from the Alliance for Open Media, which is working on AV1. In any case, we are looking forward to an exciting time frame from now until this new codec is ratified and how it will perform compared to AV1. Multimedia systems and applications will also benefit from new codecs which will gain traction as soon as first implementations of this new codec becomes available (note that AV1 is available as open source already and continuously further developed).

MPEG adds Better Support for Mobile Environment to MPEG Media Transport (MMT)

MPEG has approved the Final Draft Amendment (FDAM) to MPEG Media Transport (MMT; ISO/IEC 23008-1:2017), which is referred to as “MMT enhancements for mobile environments”. In order to reflect industry needs on MMT, which has been well adopted by broadcast standards such as ATSC 3.0 and Super Hi-Vision, it addresses several important issues on the efficient use of MMT in mobile environments. For example, it adds distributed resource identification message to facilitate multipath delivery and transition request message to change the delivery path of an active session. This amendment also introduces the concept of a MMT-aware network entity (MANE), which might be placed between the original server and the client, and provides a detailed description about how to use it for both improving efficiency and reducing delay of delivery. Additionally, this amendment provides a method to use WebSockets to setup and control an MMT session/presentation.

New Standard Completed for Internet Video Coding

A new standard for video coding suitable for the internet as well as other video applications, was completed at the 120th MPEG meeting. The Internet Video Coding (IVC) standard was developed with the intention of providing the industry with an “Option 1” video coding standard. In ISO/IEC language, this refers to a standard for which patent holders have declared a willingness to grant licenses free of charge to an unrestricted number of applicants for all necessary patents on a worldwide, non-discriminatory basis and under other reasonable terms and conditions, to enable others to make, use, and sell implementations of the standard. At the time of completion of the IVC standard, the specification contained no identified necessary patent rights except those available under Option 1 licensing terms. During the development of IVC, MPEG removed from the draft standard any necessary patent rights that it was informed were not available under such Option 1 terms, and MPEG is optimistic of the outlook for the new standard. MPEG encourages interested parties to provide information about any other similar cases. The IVC standard has roughly similar compression capability as the earlier AVC standard, which has become the most widely deployed video coding technology in the world. Tests have been conducted to verify IVC’s strong technical capability, and the new standard has also been shown to have relatively modest implementation complexity requirements.

Evidence of new Video Transcoding Technology using Side Streams

Following a “Call for Evidence” (CfE) issued by MPEG in July 2017, evidence was evaluated at the 120th MPEG meeting to investigate whether video transcoding technology has been developed for transcoding assisted by side data streams that is capable of significantly reducing the computational complexity without reducing compression efficiency. The evaluations of the four responses received included comparisons of the technology against adaptive bit-rate streaming using simulcast as well as against traditional transcoding using full video re-encoding. The responses span the compression efficiency space between simulcast and full transcoding, with trade-offs between the bit rate required for distribution within the network and the bit rate required for delivery to the user. All four responses provided a substantial computational complexity reduction compared to transcoding using full re-encoding. MPEG plans to further investigate transcoding technology and is soliciting expressions of interest from industry on the need for standardization of such assisted transcoding using side data streams.

MPEG currently works on two related topics which are referred to as network-distributed video coding (NDVC) and network-based media processing (NBMP). Both activities involve the network, which is more and more evolving to highly distributed compute and delivery platform as opposed to a bit pipe, which is supposed to deliver data as fast as possible from A to B. This phenomena could be also interesting when looking at developments around 5G, which is actually much more than just radio access technology. These activities are certainly worth to monitor as it basically contributes in order to make networked media resources accessible or even programmable. In this context, I would like to refer the interested reader to the December'17 theme of the IEEE Computer Society Computing Now, which is about Advancing Multimedia Content Distribution.
Publicly available documents from the 120th MPEG meeting can be found here (scroll down to the end of the page). The next MPEG meeting will be held in Gwangju, Korea, January 22-26, 2018. Feel free to contact Christian Timmerer for any questions or comments.
Some of the activities reported above are considered within the Call for Papers at 23rd Packet Video Workshop (PV 2018) co-located with ACM MMSys 2018 in Amsterdam, The Netherlands. Topics of interest include (but are not limited to):
  • Adaptive media streaming, and content storage, distribution and delivery 
  • Network-distributed video coding and network-based media processing 
  • Next-generation/future video coding, point cloud compression 
  • Audiovisual communication, surveillance and healthcare systems 
  • Wireless, mobile, IoT, and embedded systems for multimedia applications 
  • Future media internetworking: information-centric networking and 5G 
  • Immersive media: virtual reality (VR), augmented reality (AR), 360° video and multi-sensory systems, and its streaming 
  • Machine learning in media coding and streaming systems 
  • Standardization: DASH, MMT, CMAF, OMAF, MiAF, WebRTC, MSE, EME, WebVR, Hybrid Media, WAVE, etc.
  • Applications: social media, game streaming, personal broadcast, healthcare, industry 4.0, education, transportation, etc. 
Important dates
  • Submission deadline: March 1, 2018 
  • Acceptance notification: April 9, 2018 
  • Camera-ready deadline: April 19, 2018