bootstrap template

MR2AMC 2018: Multimodal Representation, Retrieval, and Analysis of Multimedia Content in Social Media

In Conjunction with 20th IEEE International Symposium on Multimedia (ISM 2018), Taichung, Taiwan, December 12, 2018 (tentative)

If you want to transfer your paper from ISM main conference to our workshop then contact MR2AMC 2018 organizers.

MR2AMC


MR2AMC is a series of workshops on Multimodal Representation, Retrieval, and Analysis of Multimedia Content organized by the MIDAS lab in IIIT-Delhi. MIDAS stands for Multimodal Digital Media Analysis Lab and it consists a group of researchers who study, analyze, and build different multimedia systems for society leveraging multimodal information. The first iteration of the workshop held in conjunction with the IEEE MIPR conference. The second iteration of the workshop will be held in conjunction with the 20th IEEE International Conference on Multimedia in Taichung, Taiwan. This year's workshop theme is social media. Thus, MR2AMC aims to provide an international forum for researchers in the field of multimedia data processing, analysis, search, mining, and management leveraging multimodal information in social media. This workshop will provide a forum to researchers and practitioners from both academia and industry for original research contributions and practical system design, implementation, and applications of multimodal multimedia information processing, mining, representation, management, and retrieval. MR2AMC 2018 invites research papers in the area of multimodal multimedia content analysis, search and retrieval, semantic computing, and affective computing. Accepted papers of MR2AMC 2018 will be published as part of the workshop proceedings in the IEEE Digital Library. Extended versions of the accepted workshop papers will be invited for publication in Springer Cognitive Computation and IEEE Computational Intelligence Magazine.

Call for Papers


Multimodal Representation, Retrieval, and Analysis of Multimedia Content (MR2AMC) in Social Media is the IEEE International Symposium on Multimedia workshop series on multimedia research. MR2AMC aims to provide an international forum for researchers in the field of multimedia data processing, analysis, search, mining, and management leveraging multimodal information. This workshop will provide a forum to researchers and practitioners from both academia and industry for original research contributions and practical system design, implementation, and applications of multimodal multimedia information processing, mining, representation, management, and retrieval. The broader context of the workshop comprehends Web mining, AI, Semantic Web, multimedia information retrieval, event understanding, and natural language processing. For more information, write to mr2amc.group@gmail.com

Rationale

The presence of social media platforms creates an abundance of multimedia content on the web due to advancements in digital devices and affordable network infrastructures. It has enabled anyone with an Internet connection to easily create and share their ideas, opinions, updates, and preferences through multimedia content with millions of other people around the world. Thus, it necessitates novel techniques for an efficient processing, analysis, mining, and management of multimedia data to provide different multimedia-related services. Such techniques should also able to search and retrieve information from within multimedia content. Since much signi cant contextual information such as spatial, temporal, and crowd-sourced information is also available in addition to the multimedia content, it is very important to leverage multimodal information because different representations represent different knowledge structures. However, decoding such knowledge structures into useful knowledge from a huge amount of multimedia content is very complex due to several reasons. Till date, the most of the semantic analysis, sentiment analysis, multimedia representation, multimedia information search and retrieval, opinion mining, and event understanding engines work in the unimodal setting. There is very limited work done which use multimodal information for these tasks. In this light, this workshop will focus on the use of multimodal information to analyze, represent, mine and manage multimedia content to support several semantic and sentiment based multimedia analytics problems. It will also focus on interesting multimedia systems that build upon semantic and sentiment information derived from multimedia data.

Accepted papers of MR2AMC 2018 will be published as part of the workshop proceedings in the IEEE Digital Library. Extended version of the accepted workshop papers will be invited for publication in Springer Cognitive Computation and IEEE Computational Intelligence Magazine (whichever matches closely with papers).

Topics of Interest

The primary goal of the proposed workshop is to investigate whether multimedia content when fused with other modalities (e.g., contextual, crowd-source, and relationship information) can enhance the performance of unimodal (e.g., when only multimedia content) multimedia systems. The broader context of the workshop comprehends Multimedia Information Processing (e.g., Natural Language Processing, Image Processing, Speech Processing, and Video Processing), Multimedia Embedding (e.g., Word Embedding and Image Embedding), Web Mining, Machine Learning, Deep Neural Networks, and AI. Topics of interest include but are not limited to:

  • Multimodal Multimedia Search, Retrieval and Recommendation
  • Multimodal Personalized Multimedia Retrieval and Recommendation
  • Multimodal Event Detection, Recommendation, and Understanding
  • Multimodal Multimedia based FAQ and QA Systems
  • Multimodal based Diverse Multimedia Search, Retrieval and Recommendation
  • Multimodal Multimedia Content Analysis
  • Multimodal Semantic and Sentiment based Multimedia Analysis
  • Multimodal Semantic and Sentiment based Multimedia Annotation
  • Multimodal Semantic-based Multimedia Retrieval and Recommendation
  • Multimodal Sentiment-based Multimedia Retrieval and Recommendation
  • Multimodal Filtering, Time-Sensitive and Real-time Search of Multimedia
  • Multimodal Multimedia Annotation Methodologies
  • Multimodal Sentiment-based Multimedia Retrieval and Annotation
  • Multimodal Context-based Multimedia Retrieval and Annotation
  • Multimodal Location-based Multimedia Retrieval and Annotation
  • Multimodal Relationship-based Multimedia Retrieval and Annotation
  • Multimodal Mobile-based Retrieval and Annotation of Big Multimedia
  • Multimodal Multimedia Data Modeling and Visualization
  • Multimodal Feature Extraction and Learning for Multimedia Data Representation
  • Multimodal Multimedia Data Embedding
  • Multimodal Medical Multimedia Information Retrieval
  • Multimodal Subjectivity Detection Extraction from Multimedia
  • Multimodal High-Level Semantic Features from Multimedia
  • Multimodal Information Fusion
  • Multimodal Affect Recognition
  • Multimodal Deep Learning in Multimedia and Multimodal Fusion
  • Multimodal Spatio-Temporal Multimedia Data Mining
  • Multimodal Multimedia based Massive Open Online Courses (MOOC)
  • Multimodal/Multisensor Integration and Analysis
  • Multimodal Affective and Perceptual Multimedia
  • Multimedia based Education

Multimodal Sentiment Analysis Challenge Guidelines


Introduction

Computational analysis of human multimodal language is an emerging research area in Natural Language Processing (NLP). It expands the horizons of NLP to study language used in face to face communication and in online multimedia. This form of language contains modalities of language (in terms of spoken text), visual (in terms of gestures and facial expressions) and acoustic (in terms of changes in the voice tone). At its core, this research area is focused on modeling the three modalities and their complex interactions. This Challenge on Multimodal Language aims to facilitate the growth of this new research direction in the community. The challenge is focused on multimodal sentiment analysis and emotion recognition on various datasets that are widely used.
Communicating using multimodal language (verbal and nonverbal) shares a significant portion of our communication including face-to-face communication, video chatting, and social multimedia opinion sharing. Hence, it's computational analysis is centric to NLP research. The challenges of modeling human multimodal language can be split into two major categories: 1) studying each modality individually and modeling each in a manner that can be linked to other modalities (also known as intramodal dynamics) 2) linking the modalities by modeling the interactions between them (also known as intermodal dynamics). Common forms of these interactions include complementary or correlated information across modes. Intrinsic to each modality, modeling human multimodal language is complex due to factors such as idiosyncrasy in communicative styles, non-trivial alignment between modalities and unreliable or contradictory information across modalities. Therefore computational analysis becomes a challenging research area. Revise the installation documentation.

Datasets

Each of the datasets described below has corresponding train and test split. All the pre-processed datasets with associated features can be download from this link - https://github.com/soujanyaporia/multimodal-fusion

Comparision Among Datasets

Properties Datasets
MOSEI CMU-MOSI IEMOCAP
No of Videos 23,500 2,199 1,000
Sentiment Annotations Yes Yes No
Emotion Annotations Yes No Yes
Modalities {l,a,v} {l,a,v} {l,a,v}
Labels {positive, negative, neutral} {positive, negative} {anger, sad, happy, neutral, excited,frustrated}

Where l, a, and v stands for language, acoustic and visual modals respectively.

Challenge Description

This challenge provides a platform to build systems that can automatically determine the sentiment and emotion intensity of an utterance in a multimodal setting. Each submitted model is expected to propose a novel fusion method to fuse multiple modalities. We provide strong baselines and invite novel, intuitive and robust models for submission. The problem can be formulated in the following two ways:

Schedule

30-09-2018: Evaluation Deadline
07-10-2018: Paper Submission Deadline
15-10-2018: Notification
31-10-2018: Camera-ready Submission
10-12-2018 - Workshop (tentative date)

Resources

The following resources are given as part of the challenge:

Baselines

For ease of reproducibility and comparison, we provide strong baselines for this challenge.
Baselines: https://github.com/soujanyaporia/multimodal-fusion

System Submission

Participants are required to email the Github link of their implementation and results to mr2amc.group@gmail.com. If the model uses neural network based approaches please report average performance of the system for the following random seeds - {1, 2, 3, 4, 5}.

Evaluation Metric

Participants should report accuracy and weighted F-score. Participants are required to email the Github link of their implementation. If the model uses neural network based approaches please report average performance of the system for the following random seeds - {1, 2, 3, 4, 5}.

Reference

You may refer to the following papers for reference:
Context-Dependent Sentiment Analysis in User-Generated Videos (ACL 2017).
Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis (ICDM 2017).

Acknowledgment

We are thankful to Gangeshwar Krishnamurthy from Institute of High Performance Computing, Singapore.for helping us in preparing this challenge.

Important Dates


October 18, 2018 - Paper Submission Deadline
October 24, 2018 - Notification of Paper Acceptance to Authors
October 31, 2018 - Camera-ready Submission and Author Registration
December 12, 2018 - Workshop (tentative date)

Submission Instructions


The MR2AMC workshop 2018 follows the submission guidelines of the 20th IEEE International Symposium on Multimedia(ISM 2018). Papers reporting original and unpublished research results pertaining to the topics in the CFP are solicited. There are two different submission categories: REGULAR and SHORT whose lengths are expected to be 8 and 4 pages long (using the IEEE two-column template instructions), respectively. Submissions should include the title, author(s), affiliation(s), e-mail address(es), tel/fax numbers, abstract, and postal address(es) on the first page. The online submission site is: Submission Link (click here). If web submission is not possible, please contact the program co-chairs for alternate arrangements. Papers will be selected based on their originality, timeliness, significance, relevance, and clarity of presentation. Paper submission implies the intent of at least one of the authors to register and present the paper, if accepted. For the detailed information on submission templates and instructions, check the ISM's submission instruction page (click here).

Program


TBA

Organizers


We are very much thankful to Dr. Roger Zimmermann for their kind support and being agreed to become honorary co-chairs for the workshop. Their guidence will help workshop co-chairs to organize a very successful workshop.

Rajiv Ratn Shah
MIDAS, IIIT-Delhi, India

Co-chair

Debanjan Mahata
Bloomberg L.P., USA

Co-chair

Yifang Yin
NUS, Singapore

Co-chair

Soujanya Poria
NTU, Singapore

Co-chair

A V Subramanyam
IIIT-Delhi

Co-chair

Roger Zimmermann
NUS, Singapore

Honorary Co-chair

Program Committee

Amir Zadeh, Carnegie Mellon University, USA
Luming Zhang, Hefei University of Technology, China
Zhenguang Liu, Zhejiang University, China
Vivek Singh, Rutgers University, USA
Pradeep K. Atrey, University at Albany, USA
Mukesh Saini, Indian Institute of Technology Ropar, India
A V Subramanyam, Indraprastha Institute of Information Technology Delhi, India
Debashis Sen, Indian Institute of Technology Kharagpur, India
Animesh Prasad, National University of Singapore, Singapore
Muthu Kumar Chandrasekaran, National University of Singapore, Singapore
Omprakash Kaiwartya, Northumbria University, UK
Mukesh Prasad, University of Technology Sydney, Sydney
Kishaloy Halder, National University of Singapore, Singapore
Lahari Poddar, National University of Singapore, Singapore
Vivek Kumar Singh, Banaras Hindu University, India
Erik Cambria, Nanyang Technological University, Singapore
Yogesh Singh Rawat, University of Central Florida, USA
Hisham Al-Mubaid, University of Houston-Clear Lake, USA
Alexandra Balahur, University of Alicante, Spain
Catherine Baudin, eBay Research Labs, USA
Sergio Decherchi, Italian Institute of Technology, Italy
Rafael Del Hoyo, Aragon Institute of Technology, Spain
Paolo Gastaldo, University of Genoa, Italy
Tariq Durrani, University of Strathclyde, UK
Lila Ghemri, Texas Southern University, USA
Marco Grassi, Marche Polytechnic University, Italy
Amir Hussain, University of Stirling, UK
Raymond Lau, City University of Hong Kong, Hong Kong
Saif Mohammad, National Research Council, Canada
Samaneh Moghaddam, Simon Fraser University, Canada
Tao Chen, Johns Hopkins University, USA

Supporters


We are very much thankful to the following sponsors and supporters of our MR2AMC workshop 2018.

Contacts


If you have any queries regarding the MR2AMC workshop 2018, drop an email to mr2amc.group@gmail.com. You may also contact any of the following workshop co-chairs to expedite the resonse.

Dr. Rajiv Ratn Shah, Assistant Professor, IIIT-Delhi, India, rajivratn AT iiitd.ac.in
Dr. Debanjan Mahata, Senior Machine Learning Engineer, Bloomberg L.P., USA, dmahata AT bloomberg.net
Dr. Yifang Yin, Research Fellow, NUS, Singapore, yifang AT comp.nus.edu.sg
Dr. Soujanya Poria, Senior Research Scientist, NTU, Singapore, sporia AT ntu.edu.sg

© 2018. Multimodal Representation, Retrieval, and Analysis of Multimedia Content (MR2AMC) 2018, All Rights Reserved.