Intelligent Analysis of Big Social Media Data for Crisis Tracking

Description

Often, a well-organized, concerted response can greatly reduce the damage inflicted by large-scale disasters. An important factor affecting our ability to efficiently and promptly respond to crises is the availability of quality information to first-responders about the ground realities of the affected areas. During times of crises, conventional sources of communication are often disrupted, and hence, the availability and reliability of information are called into question. However, we can leverage the vast amounts of data from social media sources such as Twitter to provide relevant information on resources and aid required in the affected areas. The project is aimed at shifting the focus of the applications of big social media data from simple tracking and statistics to a principled set of methods for intelligent analysis of data to provide situational and effective insights for crisis tracking, response planning, humanitarian aid, and disaster relief. As the collective data collection capability has improved significantly, the pressing need for intelligent analysis arises to overcome problems associated with big social media data: information overload, misinformation, covert adversaries, false positives, and differential information discovery. Extending the successes of these software systems for the tracking of blog, tweets and information gathering, this proposed project aims to expand data analysis that can intelligently handle information overload and misinformation, and intelligently differentiate novel information among big social media data from information of known sources. This project has three major research objectives for the intelligent analysis of big social media data:
This project has three major research objectives for the intelligent analysis of big social media data:

From exploratory analysis to deep analysis. This objective focuses on finding actionable patterns within big data. This includes the development of conversation-level information tracking and analysis capabilities through ConversationTracker as well as bot detection, request for help prediction, user to crisis region, and sentiment prediction models.
Misinformation. This objective focuses primarily on misinformation as a function of automated content pushed through bot, cyborg, and human propaganda networks. This includes the development of a robust bot detection module tuned for specific misinformation detection tasks.
Differential information discovery - Developing theories for efficient and effective tracking, and evaluation methods. This objective includes the development of a cross-system platform to leverage the rich Humanitarian Data Exchange sources. These sources can then be used to improve modeling and understanding of emergent crisis situations.

TweetTracker

Our latest system TweetTracker focuses on tweets collected from Twitter to support event monitoring and analysis. More information about this project can be found here.

Conference Tutorials

ICDM 2017 Tutorial on Mining Misinformation in Social Media: Understanding Its Rampant Spread, Harm, and Intervention, Nobember 18-21, 2017, New Orleans, LA.
AAAI 2017 Tutorial on Social Data Bias in Machine Learning: Impact, Evaluation, and Correction, February 4-5, 2017, San Francisco, CA.
SBP2016 Tutorial on Misinformation on Social Media: Diffusion, Detection, and Intervention, June 28, 2016, Washington DC. Blog on KDnuggets
ASONAM2015 Tutorial on Bot Detection in Social Media: Networks, Behavior, and Evaluation, August 25, 2015, Paris, France.
WWW2015 Tutorial on LIKE and recommendation in social media, May 18, 2015, Florence, Italy.

Invited Talks

Isaac Jones. Tutorial on TweetTracker for NATO's Assessment of the Information Environment Limited Objective Experiment, Naples, Italy. December 4- 7, 2017.
Huan Liu. ``Big Social Media Data and Some Associated Challenges", Invited Talk in Big Data Analytics, International Conference on Data Science, Shanghai, China. December 17 - 19, 2017.
Huan Liu. ``Ethics and Professionalism in the age of Social Data", Panel, IEEE International Conference on Data Mining (ICDM), New Orlean, Louisianna. November 21, 2017.
Tahora H. Nazer, Suhang Wang, and Justin Sampson. "Deep Learning, Machine Learning, and Artificial Intelligence in Applications." Invited talk at Phoenix Mobile and Emerging Tech Festival, September 26, 2017.
Huan Liu. ``From AI to AI: Acquiring Intelligence in Big Data - A Social Media Study", Taian, AI and Technology Track, AI and Law Conference, Taian, Shandong, China. September 1, 2017.
Huan Liu. ``Big Data + Deep Learning = A Universal Solution?", Keynote, 6th International Workshop on Big Data, Streams and Heterogeneous Souce Mining: Algorithms, Systems, Programming Models and Applications (KDD BigMine 17), Halifax, Nova Scotia, Canada. August 14, 2017.
Huan Liu. ``From AI to AI: Acquiring Social Media Intelliegence via `Big' Data", Keynote, International Conference on Social Computing, Behavioral-Cultral Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS2017), Washington DC, July 7, 2017.
Justin Sampson. "TweetTracker: Social Media Discourse Discovery, Analysis, Understanding, and Lessons Learned", NATO Strategic Communications Centre of Excellence, Riga, Latvia, March 21-24, 2017.
Huan Liu. "Tactics, Techniques and Procedures in the Social Media Environment", Panel, Trends in Social Media and Their Further Development Seminar, Riga, Latvia. March 20, 2017.
Huan Liu. "When AI Meets Social Media Intelligence - The Need for New Thinking and New Methods", South Big Data Hub, Data Science Roundtable on Anti-Social Computing: Bots, Lies, and the New Information Environment, Institute for Software Research, CMU. March 9, 2017.
Fred Morstatter. "An Unbiased Sample or a Spammer’s Paradise? Understanding the Implications of Sampling Strategies in Social Media", at NSF Data Science Workshop hosted by University of Washington. Seattle, Washington, USA. August 2016.
Liang Wu. "User Preference Prediction with Social Media Networks", Invited Talk, Etsy, Brooklyn, NY, December 5, 2016.
Fred Morstatter. "Discovering and Mitigating Bias in Social Media", at Technical University of Munich / Bavarian School of Public Policy. Munich, Germany. August 2016.
Fred Morstatter. "Detecting and Mitigating Bias in Social Media", at Johns Hopkins University Applied Physics Laboratory. Laurel, Maryland, USA. June 2016.
Huan Liu. "The Good, the Bad and the Ugly: Uncovering Novel Opportunities of Data Science", CUHK, June 16, 2016.
Huan Liu. "The Good, the Bad and the Ugly: Uncovering Novel Opportunities of Data Science", CAS Institute of Software, June 13, 2016.
Suhas Ranganath. "Faciliating Information Seeking in Social Media", Xerox Research Center India, May 8, 2016.
Huan Liu. "Big Data and Deep Learning: Are Them Made for Each Other?", NSF Panel, SIAM International Conference on Data Mining (SDM2016), Miami, FL. May 6, 2016.
Huan Liu. "The Good, the Bad and the Ugly: Uncovering Novel Opportunities of Data Science", University of Technology Sydney, AAi, Australia, April 19, 2016.
Huan Liu. "The Good, the Bad and the Ugly: Uncovering Novel Opportunities of Data Science", IACS, Stony Brook University, NY, April 7, 2016.
Liang Wu. "Relational Learning with Social Status Analysis", ASU Graduate Research Symposium, Tempe, AZ, March 15 2016.
Huan Liu. "Mining Social Media Data", Hohai University, Jan 21, 2016; Nanjing University of Posts and Telecommunications, Jan 22, 2016.
Huan Liu. "Toward Trustworthy Information Detection in Social Media", Annual Symposium on Information Assurance Research and Education, Tempe, AZ, November 13 2015.
Tahora H. Nazer, Fred Morstatter, Liang Wu, and Huan Liu. Bot Detection in Social Media, Symposium on Information Assurance Research and Education, Information Assurance Center, Arizona State University, 2015.
Huan Liu. "Mining Social Media Data: Some Lessons Learned", CS Graduate Colloquium, Brigham Young University, Nov 5, 2015.
Jiliang Tang. "Discovering Negative Links on Social Networking Sites", Yahoo!-DAIS Seminar, UIUC, Oct 30, 2015.
Huan Liu. "Machine Learning for Computational Social Science", CCS2015 Satellite Workshop of Computational Social Science: Contagion, Collective Behaviour, and Networks", Tempe, AZ. October 1, 2015.
Huan Liu. "Mining Social Media Data for Behavior Analytics", Plenary Presentation, Conference on Complex Systems (CCS2015), Tempe, AZ. September 29, 2015.
Huan Liu. "Discovering Negative Links on Social Networking Sites", Keynote, AusDM2015 and ICDS2015, Sydney, Australia. August 8-9, 2015.
Huan Liu, "Employing Machine Learning to Help Verifying Research Hypotheses", Invited Talk, Sister-Conference Presentation in Machine Learning Track, IJCAI2015. Buenos Aries, Argentina. July 30, 2015.
Huan Liu. "Data Mining and Advanced Topics", CAS DragonStar Summer Lectures, Guilin, China. July 9-14, 2015.

Publications

Books

Huiji Gao and Huan Liu. "Mining Human Mobility in Location-Based Social Networks", Morgan & Claypool Publisher, April, 2015.
Reza Zafarani, Mohammad Ali Abbasi, and Huan Liu. "Social Media Mining: An Introduction", ISBN-13: 978-1107018853, Cambridge University Press, 2014.
Shamanth Kumar, Fred Morstatter, and Huan Liu. "Twitter Data Analytics", ISBN-13: 978-1461493716, Springer, 2014.

Book Chapters

Kai Shu, H. Russell Bernard, and Huan Liu. Studying Fake News via Network Analysis: Detection and Mitigation. To appear in Lecture Notes in Computer Science (LNCS), Springer Press, 2018.
Liang Wu, Fred Morstatter, Xia Hu, and Huan Liu. "Mining Misinformation in Social Media", Big Data in Complex and Social Networks, CRC Press, 2016.
Liang Wu and Huan Liu, "Detecting Crowdturfing in Social Media", in Encyclopedia of Social Network Analysis and Mining, Forthcoming.

Journal Articles

Suhang Wang, Jiliang Tang, Yilin Wang and Huan Liu, xploring Hierarchical Structures for Recommender Systems. IEEE TKDE, 2018
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. ``Fake News Detection on Social Media: A Data Mining Perspective", SIGKDD Explorations, 19(1):22-36, June, 2017.
H. Nazer, Tahora, Guoliang Xue, Yusheng Ji, and Huan Liu. "Intelligent Disaster Response via Social Media Analysis A Survey." ACM SIGKDD Explorations Newsletter 19, no. 1 (2017): 46-59.
Suhas Ranganath, Suhang Wang, Xia Hu, Jiliang Tang, and Huan Liu. "Facilitating Time Critical Information Seeking in Social Media", IEEE Trans on Knowledge and Data Engineering (TKDE), Forthcoming.
Suhas Ranganath, Xia Hu, Jiliang Tang, Suhang Wang and Huan Liu. "Understanding and Identifying Rhetorical Questions in Social Media", ACM Transactions on Intelligent Systems and Technology (TIST), Forthcoming.
Fred Morstatter and Huan Liu. "Discovering, Assessing, and Mitigating Data Bias in Social Media", Elsevier Journal of Online Social Networks and Media. Volume 1, Pages: 1-13. June 2017.
Fred Morstatter, Liang Wu, Uraz Yavanoglu, Stephen S. Corman, and Huan Liu. "Identifying Framing Bias in Online News", ACM Transactions on Social Computing (TSC), Forthcoming.
Kai Shu, Suhang Wang, Jiliang Tang, Reza Zafarani and Huan Liu. ``User Identity Linkage across Online Social Netwroks: A Review", ACM SIGKDD Explorations. Volume 8, Number 2, December 2016. Pages: 5-17.
Reza Zafarani and Huan Liu. "Evaluation without Ground Truth in Social Media Research", Communications of the ACM, 2015.
Huiji Gao, Jiliang Tang and Huan Liu."Solving the Cold Start Problem in Location Recommendation with Geo-Social Correlations", Data mining and Knowledge Discovery, Springer. 29(2): 299-323, 2015.

Conference Papers

Liang Wu, Jundong Li, Fred Morstatter, Huan Liu, Toward Relational Learning with Misinformation. SIAM International Conference on Data Mining (SDM 2018), May 3-5, 2018. San Diego, California, USA.
Liang Wu, Huan Liu, Tracing Fake-News Footprints: Characterizing Social Media Messages by How They Propagate. The 11th ACM International Conference on Web Search and Data Mining (WSDM 2018), Feb 6-8, 2018. Los Angeles, California, USA.
Jundong Li, Jiliang Tang, Yilin Wang, Yali Wan, Yi Chang and Huan Liu. "Understanding and Predicting Delay in Reciprocal Relations" In Proceedings of the 2018 Web Conference (WWW 2018)
Jundong Li, Chen Chen, Hanghang Tong, Huan Liu. "Multi-Layered Network Embedding", In Proceedings of the 18th SIAM International Conference on Data Mining (SDM 2018)
Jundong Li, Liang Wu, Harsh Dani, Huan Liu, "Unsupervised Personalized Feature Selection", In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI 2018)
Jundong Li, Kewei Cheng, Liang Wu, Huan Liu, "Streaming Link Prediction on Dynamic Attributed Networks", In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM 2018)
Xuying Meng, Suhang Wang, Huan Liu, Yujun Zhang. Exploiting Emotion on Reviews for Recommender Systems In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)
Xuying Meng, Suhang Wang, Kai Shu, Jundong Li, Bo Chen, Huan Liu, Yujun Zhang. Personalized Privacy-Preserving Social Recommendation In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)
Suhas Ranganath, Ghazaleh Beigi, and Huan Liu. ``Leveraging Implicit Contribution Amounts to Facilitate Microfinancing Requests", the 11th ACM International Conference on Web Search and Data Mining (WSDM2018), Los Angeles, CA. Feb 5-9, 2018
Kai Shu, Amy Sliva, Justin Sampson, and Huan Liu. Understanding Cyber Attack Behaviors with Sentiment Information on Social Media. In International Conference on Social Computing, Behavior-Culture Modeling, and Prediction (SBP 2018).
Kai Shu, Suhang Wang, Jiliang Tang, Yilin Wang, and Huan Liu. CrossFire: Cross Media Joint Friend and Item Recommendations. In Proceedings of 11th ACM International Conference on Web Search and Data Mining (WSDM 2018).
Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang and Huan Liu. "Attributed Network Embedding for Learning in a Dynamic Environment", In Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM 2017)
Harsh Dani, Jundong Li and Huan Liu. "Sentiment Informed Cyberbullying Detection in Social Media", In Proceedings of the 2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2017)
Liang Wu, Xia Hu, Fred Morstatter, and Huan Liu. "Adaptive Spammer Detection with Sparse Group Modeling", The 11th International AAAI Conference on Web and Social Media (ICWSM2017), May 15-18, 2017. Montreal, Canada.
Liang Wu, Xia Hu, and Huan Liu."Early Identification of Personalized Trending Topics in Microblogging", Poster paper, the 11th International AAAI Conference on Web and Social Media (ICWSM2017), May 15-18, 2017. Montreal, Canada.
Liang Wu, Xia Hu, Fred Morstatter, and Huan Liu. "Detecting Camouflaged Content Polluters", Poster paper, the 11th International AAAI Conference on Web and Social Media (ICWSM2017), May 15-18, 2017. Montreal, Canada.
Liang Wu, Jundong Li, Xia Hu, and Huan Liu. "Gleaning Wisdom from the Past: Early Detection of Emerging Rumors in Social Media", SIAM International Conference on Data Mining (SDM2017), April 27-29, 2017. Houston, Texas.
Suhang Wang, Charu Aggarwal, Jiliang Tang, Huan Liu. ttributed Signed Network Embedding. In: Proceedings of 26th ACM International Conference on Information and Knowledge Management (CIKM-17)
Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, Huan Liu. What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation. In: Proceedings of the 26th World Wide Web Conference (WWW-17)
Suhang Wang, Charu Aggarwal, Huan Liu. Using a Random Forest to Inspire a Neural Network and Improving on It. In: Proceedings of the Seventeenth SIAM International Conference on Data Mining (SDM-17)
Suhang Wang, Charu Aggarwal, Huan Liu. Using a Random Forest to Inspire a Neural Network and Improving on It. In: Proceedings of the Seventeenth SIAM International Conference on Data Mining (SDM-17)
Suhang Wang, Jiliang Tang, Charu Aggarwal, Yi Chang, Huan Liu. Signed Network Embedding in Social Media . In: Proceedings of the Seventeenth SIAM International Conference on Data Mining (SDM-17)
Suhang Wang, Yilin Wang, Jiliang Tang, Charu Aggarwal, Suhas Ranganath, Huan Liu. Exploiting Hierarchical Structures for Unsupervised Feature Selection . In: Proceedings of the Seventeenth SIAM International Conference on Data Mining (SDM-17)
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. Fake News Detection on Social Media: A Data Mining Perspective. In SIGKDD Explorations, 2017.
Justin Sampson, Fred Morstatter, Liang Wu and Huan Liu. "Leveraging the Implicit Structure within Social Media for Emergent Rumor Detection", short paper, ACM International Conference of Information and Knowledge Management (CIKM2016), October 24-28, 2016. Indianapolis, Indiana.
Fred Morstatter, Liang Wu, Tahora H. Nazer, Kathleen M. Carley, and Huan Liu. "A New Approach to Bot Detection: Striking the Balance Between Precision and Recall", IEEE/ACM International Conference on Advances in Social Network Analysis and Mining (ASONAM2016), August 18-21, San Francisco, CA.
Fred Morstatter and Huan Liu. "A Novel Measure for Coherence in Statistical Topic Models", Association of Computational Linguistics (ACL), August 7 - 12, 2016. Berlin, Germany.
Suhas Ranganath, Xia Hu, Jiliang Tang, Suhang Wang and Huan Liu. "Understanding and Identifying Rhetorical Questions in Social Media", poster paper, The 10th International AAAI Conference on Weblogs and Social Media (ICWSM2016), May 17 - 20, 2016. Cologne, Germany.
Fred Morstatter, Harsh Dani, Justin Sampson, and Huan Liu. "Can One Tamper with the Sample API? - Toward Neutralizing Bias from Spam and Bot Content". Best WWW2016 Poster Award. The 25th International World Wide Web Conference (WWW16). April 11 - 15, 2016. Montreal, Canada.
Liang Wu, Xia Hu, and Huan Liu. "Relational Learning with Social Status Analysis", ACM International Conference on Web Search and Data Mining (WSDM2016), February 22-25, 2016. San Francisco, CA.
Suhas Ranganath, Xia Hu, Jiliang Tang, and Huan Liu. "Understanding and Identifying Advocates of Political Campaigns on Social Media". ACM International Conference on Web Search and Data Mining (WSDM2016), February 22-25, 2016. San Francisco, CA.
Suhas Ranganath, Fred Morstatter, Xia Hu, Jiliang Tang, Suhang Wang, and Huan Liu. "Predicting Online Protest Participation of Social Media Users", The Thirtieth AAAI Conference (AAAI 2016), February 12-17, 2016. Phoenix, Arizona.
User Identity Linkage across Online Social Networks: A Review. Kai Shu, Suhang Wang, Jiliang Tang, Reza Zafarani, and Huan Liu. In SIGKDD Explorations, 2016.
Harsh Dani, Fred Morstatter, Xia Hu, Zhen Yang, and Huan Liu. "Social Answer: A System for Finding Appropriate Sites for Questions in Social Media". IEEE International Conference on Data Mining (ICDM2015). November 14 -17, 2015. Atlantic City, NJ.
Suhas Ranganth, Suhang Wang, Xia Hu, Jiliang Tang, and Huan Liu."Finding Time-Critical Replies for Information Seeking in Social Media". In Proceedings of IEEE International Conference on Data Mining (ICDM2015), November 14 - 17, 2015. Atlantic City, NJ.
Justin Sampson, Fred Morstatter, Reza Zafarani, and Huan Liu. "Real-time Crisis Event Mapping using Sparse Language Distributions". IEEE International Conference on Data Mining (ICDM2015). November 14 - 17, 2015. Atlantic City, NJ.
Fred Morstatter, Jürgen Pfeffer, Katja Mayer and Huan Liu. "Text, Topics, and Turkers: Evaluation of Statistical Topics", The 26th ACM Conference on Hypertext and Social Media (Hypertext2015). September 1 - 4, 2015. Cyprus.
Justin Sampson, Fred Morstatter, Ross Maciejewski, and Huan Liu. "Surpassing the Limit: Keyword Clustering to Improve Twitter Sample Coverage", The 26th ACM Conference on Hypertext and Social Media (Hypertext2015). September 1 - 4, 2015. Cyprus.
Reza Zafarani, and Huan Liu. "10 Bits of Surprise: Detecting Malicious Users with Minimum Information". ACM International Conference on Information and Knowledge Management, 2015.
Jundong Li, Xia Hu, Jiliang Tang, and Huan Liu. "Unsupervised Streaming Feature Selection in Social Media". ACM International Conference on Information and Knowledge Management, 2015.
Suhang Wang, Jiliang Tang, and Huan Liu. "Toward Dual Roles of Users in Recommender Systems". ACM International Conference on Information and Knowledge Management, 2015.

Workshop Papers

Kai Shu, Suhang Wang, and Huan Liu. Understanding User Profiles on Social Media for Fake News Detection. In 1st IEEE International Workshop on Fake MultiMedia (FakeMM 2018).

Demo Papers

Nazer, Tahora H., Fred Morstatter, Harsh Dani, and Huan Liu. "Finding requests in social media for disaster relief." In Advances in Social Networks Analysis and Mining (ASONAM), 2016 IEEE/ACM International Conference on, pp. 1410-1413. IEEE, 2016.

Awards

Ira A. Fulton Schools of Engineering Faculty Exemplar Award, May 13, 2016.
Recipient of the "Best Researcher (Senior Faculty) Award (2015)", School of Computing, Informatics, and Decision Systems Engineering, ASU, May, 2016.
Fred Morstatter, Harsh Dani, Justin Sampson, and Huan Liu, Recipients of the WWW2016 Best Poster, Montreal, Canada. April 11 - 15, 2016.
Huan Liu received the "Best Researcher Award", Senior Faculty, (CIDSE) 2015.
Jiliang Tang received the "ACM SIGKDD2015 Doctoral Dissertation Award Runner Up" , 2015.
Jiliang Tang received the "A Fulton Schools of Engineering Dean's Dissertation Award" , (CIDSE) 2015.
Jiliang Tang received the "Outstanding Computer Science PhD Student in CIDSE Award" , (CIDSE) 2015.
Xia Hu received the "Outstanding Graduate Student Award in Fulton Schools of Engineering", (CIDSE) 2015.

Project Members (current and former)

Huan Liu
Fred Morstatter, PhD (graduated)
Justin Sampson
Christophe Faucon, PhD (graduated)
Suhas Ranganath, PhD (graduated)
Kai Shu

Liang Wu
Isaac Jones
Shamanth Kumar, PhD, graduated
Xia Hu, PhD, graduated
Huiji Gao, PhD, graduated
Reza Zafarani, PhD, graduated
Harsh Dani, MS, graduated
Prabhat Rancherla, MCS
Kunal Bansal, MCS
Ashutosh Bhadke, MCS
Shobhit Sharma, MCS

Acknowledgments

This project is sponsored by ONR grant N00014-16-1-2257

Created on March 13, 2017
Contact: Huan Liu.

Last Updated: April 11, 2018