BlogTrackers: Analyzing Social Media for Cultural Modeling

Description

This research project is a preliminary joint effort to integrate domain knowledge of a special cultural interest with novel search capabilities for assessing political risks. In particular, this project attempts to address three critical questions in order to stay in the frontier of stability operations in a global, forward-looking view: 1. the importance of the blogosphere to military or stability operations; 2. the significance of the blog research to these operations; and 3. the identification and development of tools of blog tracking and analysis to understand the communities of interest and help advance global security and prosperity. We first provide background information on blogs - Web-based media with increasing popularity among youths all over the world, and on Indonesia – a strategically significant country that serves as a case study for the proposal. Secondly, we illustrate the pressing need for studying the blogosphere as a new dimension for identifying pre-conflict indicators and potential threats. Thirdly, we articulate the demand for novel search capabilities that can integrate domain knowledge in search and scale up with the inordinate number of disconnected blogs of special interests in the digital jungle. Finally, we propose to design, develop, deploy, and evaluate BlogTrackers, a proof-of-concept system that distills and manifests the proposed research tasks and derived results.

TweetTracker

Our latest system TweetTracker focuses on tweets collected from Twitter to support event monitoring and analysis. More information about this project can be found here.

News

• Media coverage of TweetTracker project can be found here
• Shamanth Kumar was awarded the Presidential Volunteer Service Award in 2012 for volunteering at Humanity Road Inc.
• First Crisis Response Game Held on ASU Campus, August 2011, In The Loop, CIDSE.
• Research on Blog Mining Breaks New Ground, SCI Monitor, Fall 2008 Pages 10-11.

Invited Talks

• "Some Computational Challenges in Mining Social Media", Keynote, The 2013 IEEE/ACM (ASONAM2013) International Conference on Advances in Social Network Analysis and Mining, August 25-28. Niagra Falls, Canada.
• "Some New Data-Mining Challenges with Social Media Data", Keynote, the 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD2013), Gold Coast, Australia. April 16, 2013.
• "Understanding Behavior in a Networked World via Social Media Data", Plenary Talk, International Conference on Computing, Networking and Communications (ICNC'13), San Diego, CA. January 30, 2013.
• "Computing with Social Media Data", Invited Talk, the Center for Social Dynamics & Complexity", Arizona State University, January 24, 2013.
• "Some Computing Challenges in Understanding Social Media Data", the 4th China National Conference on Social Computing, Keynote, November 15-16, 2012. Beijing, China
• "Toward Mobile Cloud Computing: Data Analysis with Location-Based Social Networks", Mobile Cloud Computing Panel, the 26th IEEE Annual Computer Communications Workshop (CCW), November 9, 2012. Sedona, AZ.
• "Mining Social Media - A Brief Introduction", Invited Tutorial, INFORMS, Annual Meeting, October 16, 2012. Phoenix, AZ.
• "Mining Social Media - A Brief Introduction", Decision Systems Seminar, School of Computing, Informatics, and Decision Systems Engineering, October 11, 2012.
• "Some `Big Data' Challenges in Mining Social Media Data", Computer and Information Science, IUPUI, September 7, 2012.
• "Some Challenges in Understanding Social Media", Shanghai Jiaotong University, Shanghai, May 28, 2012.
• "Some Challenges in Mining Social Media Data", Chinese Academy of Sciences, Beijing, May 25, 2012.
• "Some Challenges in Understanding Social Media", Renmin University of China, Beijing, May 24, 2012.
• "Beyond Crowdsourcing for HADR," presented by Huan Liu, Shamanth Kumar, and Huiji Gao at UCCS HA/DR Tech Conference May 2011

Publications

• Dissertations
• Sai Moturu. Ph.D. dissertation on "Quantifying the Trustworthiness of User-Generated Social Media Content". School of Computing and Informatics, Arizona State University, Summer 2009.
• Nitin Agarwal. Ph.D. dissertation on "Social Computing in Blogosphere". School of Computing and Informatics, Arizona State University, Summer 2009.
• Xufei Wang. Ph.D. dissertation on "Connecting Users with Similar Interests for Group Understanding"


• Magazine Articles
• Fred Morstatter, Huan Liu. "Opening Doors to Sharing Social Media Data," IEEE Intelligent Systems, vol. 27, no. 1, pp. 47-51, Jan/Feb 2012.
• Huiji Gao, Geoffrey Barbier, and Rebecca Goolsby. "Harnessing Crowdsourcing Power of Social Media for Disaster Relief," IEEE Intelligent Systems(CPSS) 26(3): 10-14 2011


• Books and Book Chapters
• Huiji Gao and Huan Liu, "Data Analysis on Location-Based Social Netwoks" in Mobile Social Networking: An Innovative Approach, Editor: Alvin Chin and Daqing Zhang. Springer, Forthcoming
• Nitin Agarwal, Shamanth Kumar, Huiji Gao, Reza Zafarani, and Huan Liu, "Analyzing Behavior Of The Influentials Across Social Media," in Behavior Computing: Modeling, Analysis, Mining and Decision, Editors: Longbing Cao, Philip S. Yu, Springer 2012
• Xia Hu and Huan Liu. "Text Analytics in Social Media" in Mining Text Data, Editor: Charu C. Aggawal and Chengxiang Zhai , Springer. pp 385 - 414. March, 2012
• Xia Hu and Huan Liu. "Text Analytics in Social Media," in Mining Text Data, Editor: Charu C. Aggawal and Chengxiang Zhail , Springer. pp 385 - 414. March, 2012
• Lei Tang and Huan Liu, "Understanding Group Structures and Properties in Social Media", in Link Mining, Models, Algorithms and Applications, Editors: Philip S. Yu, Jiawei Han, and Christos Faloutsos, Springer, 2010
• Lei Tang and Huan Liu, "Graph Mining Applications to Social Network Analysis". In Managing and Mining Graph Data, Editors: Charu Aggarwal and Haixun Wang. Springer, 2010.
• Lei Tang and and Huan Liu, "Community Detection and Mining in Social Media", Morgan & Claypool Publishers, 2010
• Nitin Agarwal and Huan Liu."Modeling and Data Mining in Blogosphere". Synthesis Lectures on Data Mining and Knowledge Discovery No. 1, Morgan and Claypool Publishers, Robert Grossman (Editor), August 2009. Morgan and Claypool website; Amazon website.
• Nitin Agarwal and Huan Liu. "Trust in Blogosphere", in Encyclopedia of Database Systems, Part 20(2009), pp.3187-3191.
• Huan Liu, John J. Salerno, and Michael J. Young, editors, "Social Computing, Behavioral Modeling, and Prediction", 2008, Springer.
• Nitin Agarwal, Huan Liu, and Jianping Zhang. "A Study of Friendship Networks and Blogosphere", Handbook of Research on Text and Web Mining, Technologies. Editors: Min Song and Yi-fang Brook Wu. Idea Group Inc.
• Nitin Agarwal, Huan Liu, John J. Salerno, and Philip S. Yu. "Searching for 'Familiar Strangers' on Blogosphere", Next Generation of Data Mining, Editors: Hillol Kargupta, Jiawei Han, Philip S. Yu, Rajeev Motwani, and Vipin Kumar. Chapman & Hall/CRC Press.
• M. Woodward, H. Goodal, L.Cady, S. Corman, K. McDonald and C. Forbes, "The Iranian Letter to President Bush: Analysis and Recommendations". Weapons of Mass Persuasion. Strategic Communication to Combat Violent Extremism. Editors: S. Corman, A. Trethewey and H.Goodal. New York: Peter Lang, 2008.


• Journal Articles
• Jiliang Tang, Xufei Wang, Huiji Gao, Xia Hu, and Huan Liu. "Enriching short text representation in microblog for clustering," Frontiers of Computer Science in China, vol. 6, no. 1, pp 88-101. 2012
• Geoffrey Barbier, Reza Zafarani, Huiji Gao, Gabriel Fung, and Huan Liu. "Maximizing Benefits from Crowdsourced Data," Computational and Mathematical Organization Theory (2011)
• Merlyna Lim. "Islam and Pop-Politics in Indonesian Blogosphere", Journal of Media and Religion (forthcoming)
• Lei Tang and Huan Liu,"Toward Predicting Collective Behavior via Social Dimension Extraction".IEEE Intelligent Systems, vol. 25, no. 4, pp.19-25, July-Aug. 2010
• Nitin Agarwal, Magdiel Galan, Huan Liu, and Shankar Subramanya. "WisColl: Collective Wisdom based Blog Clustering". Journal of Information Science: Special Issue on Collective Intelligence (INS-CI). Elsevier(2010), 180(1), pp. 39-61
• Nitin Agarwal and Huan Liu. "Blogosphere: Research Issues, Tools, and Applications". SIGKDD Explorations, 10(1): pp. 18-31, July 2008.
• Mark Woodward, "Burma’s Generals and Cyclone Nargis: Incompetence, Callous Indifference or Both?" COMPOS Journal: Analysis, Commentary and News from the World of Strategic Communications, pp. 1-18, May 2008
• Mark Woodward, "PKS Against the Rest. The Justice and Prosperity Party and the 2007 Jakarta Election", Nanayang Technological University, S. Rajaratnam School of International Studies Commentary no. 55, April 2008
• Mark Woodward, "Indonesia's Religious Political Parties: Democratic Consolidation and Security in Post-New Order Indonesia", Asian Security, 4:1 (2008). pp. 41-60
• Mark Woodward, "Time to Stop Fooling Ourselves about Salafis" COMPOS Journal: Analysis, Commentary and News from the World of Strategic Communications, 2008


• Conference Papers
• Xia Hu, Jiliang Tang, Huan Liu, and Zhang Yanchao. "Social Spammer Detection in Microblogging," the Proceedings of IJCAI 2013, August 3-9, 2013. Beijing, China.
• Fred Morstatter, Jürgen Pfeffer, Huan Liu and Kathleen Carley, "Is the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose," The Seventh International AAAI Conference on Weblogs and Social Media (ICWSM 2013), Boston, Massachusetts, USA.
• Xia Hu, Jiliang Tang, Huiji Gao and Huan Liu. "Unsupervised Sentiment Analysis with Emotional Signals," the 22nd International World Wide Web Conference (WWW2013). May 13 - 17, 2013. Rio de Janeiro, Brazil.
• Xia Hu, Jiliang Tang, Huiji Gao, and Huan Liu. "ActNeT: Active Learning for Networked Texts in Microblogging," the 13th SIAM International Conference on Data Mining (SDM 2013). May 2-4, 2013. Austin, Texas
• Shamanth Kumar and Fred Morstatter and Reza Zafarani and Huan Liu, "Whom Should I Follow? Identifying Relevant Users During Crises," Proceedings of the 24th ACM conference on Hypertext and social media(HT 2013). May 1-3, Paris, France
• Xia Hu, Lei Tang, Jiliang Tang, and Huan Liu. "Exploiting Social Relations for Sentiment Analysis in Microblogging," the Sixth ACM International Conference on Web Search and Data Mining (WSDM2013). Best Paper Shortlist. February 4-8, 2013. Rome, Italy
• Ullas Nambiar, Tanveer Faruquie, Shamanth Kumar, Fred Morstatter, and Huan Liu."Faceted Browsing over Social Media," (Short Paper) International Conference on Big Data Analytics(BDA-2012), December 24-26, 2012. New Delhi, India
• Huiji Gao, Jiliang Tang, and Huan Liu. "gSCorr: Modeling Geo-Social Correlations for New Check-ins on Location-Based Social Networks," the 21st ACM International Conference on Information and Knowledge Management (CIKM 2012). October 29-November 2, 2012. Hawaii, USA
• Shamanth Kumar, Fred Morstatter, Grant Marshall, Huan Liu, and Ullas Nambiar."Navigating Information Facets on Twitter(NIF-T)," (Demonstration Paper) 18th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'12). August 12-16, 2012. Beijing, China.
• Huiji Gao, Jiliang Tang, and Huan Liu, "Exploring Social-Historical Ties on Location-Based Social Networks," The Sixth International AAAI Conference on Weblogs and Social Media (ICWSM 2012), Dublin, Ireland
• Xia Hu and Huan Liu, "Social Status and Role Analysis of Palin's Email Network," (Poster) International Conference on World Wide Web (WWW2012). April 16-20, 2012. Lyon, France
• Mohammad Ali Abassi, Shamanth Kumar, Jose Augusto Andrade Filho, and Huan Liu. "Lessons Learned in Using Social Media for Disaster Relief - ASU Crisis Response Game"(Poster). 2012 International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction (SBP12). April 3-5, 2012. College Park, MD.
• Xufei Wang and Jiliang Tang and Huan Liu. "Document Clustering via Matrix Representation," ICDM 2011 IEEE International Conference on Data Mining. December 11-14, 2011. Vancouver, Canada.
• Xufei Wang, Huan Liu, and Wei Fan. "Connecting Users with Similar Interests via Tag Network Inference," 20th ACM Conference on Information and Knowledge Management. (CIKM 2011). October 24-28, 2011. Glasgow, UK.
• Xia Hu, Lei Tang, and Huan Liu. "Enhancing Accessibility of Microblogging Messages Using Semantic Knowledge," (Poster) 20th ACM Conference on Information and Knowledge Management. (CIKM 2011). October 24-28, 2011. Glasgow, UK.
• Pritam Gundecha, Geoffrey Barbier, and Huan Liu. "Exploiting Vulnerability to Secure User Privacy on a Social Networking Site". 17th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2011). August 21-24, 2011. San Diego, CA.
• Shamanth Kumar, Reza Zafarani, and Huan Liu. "Understanding User Migration Patterns in Social Media," Twenty-Fifth AAAI Conference on Artificial Intelligence, August 7-11, 2011. San Fransisco, CA.
• Shamanth Kumar, Geoffrey Barbier, Mohammad Ali Abbasi, and Huan Liu. "TweetTracker: An Analysis Tool for Humanitarian and Disaster Relief, " (Demonstration Paper) 5th International AAAI Conference on Weblogs and Social Media (ICWSM-11), July 17-21, 2011. Barcelona, Spain.
• Nitin Agarwal, Merlyna Lim, and Rolf T. Wigand. "Collective Action Theory Meets the Blogosphere: A New Methodology". 3rd International Conference on Networked Digital Technologies (NDT 2011). July 11-13, 2011. Macau, China.
• Nitin Agarwal, Merlyna Lim, and Rolf T. Wigand. "Finding Her Master’s Voice: The Power of Collective Action Among Female Muslim Bloggers". 19th European Conference on Information Systems(ECIS2011). June 9-11, 2011. Helsinki, Finland.
• Huiji Gao, Xufei Wang, Geoffrey Barbier, and Huan Liu. "Promoting Coordination for Disaster Relief - From Crowdsourcing to Coordination" (Poster Paper), 4th International Conference on Social Computing, Behavioral Modeling, and Prediction (SBP11), March 29-31, 2011. College Park, Maryland.
• Merlyna Lim. “Flipping Coin: Contesting Power through Cyberactivism” Paper presented at the annual meeting of the International Studies Association Annual Conference "Global Governance: Political Authority in Transition”, March 16, 2011
• Huiji Gao, Xufei Wang, Geoffrey Barbier, and Huan Liu."Making Social Media Work for Humanitarian Assistance and Disaster Relief" (Poster Paper). HSCB Focus 2011. February 8-10, 2011. Chantilly, Virginia.
• Xufei Wang, Lei Tang, Huiji Gao, and Huan Liu. “Discovering Overlapping Groups in Social Media”. 10th IEEE International Conference on Data Mining (ICDM'10), December 2010, Sydney, Australia
• Shamanth Kumar, Nitin Agarwal, and Huan Liu. "Towards Building a Social Computing Tool for Social Scientists", 3rd International Conference on Human Computing (HumanCom-10), August 11-13, 2010, Cebu, Philippines
• Shamanth Kumar, Reza Zafarani, Mohammad Ali Abbasi, Geoffrey Barbier, and Huan Liu, "Convergence of Influential Bloggers for Topic Discovery in the Blogosphere" (Poster Paper), 3rd International Conference on Social Computing, Behavioral Modeling, and Prediction (SBP10), March 2010, Bethesda, Maryland
• Reza Zafarani, William D. Cole, and Huan Liu,"Sentiment Propagation in Social Networks: A Case Study on LiveJournal" (Poster Paper), 3rd International Conference on Social Computing, Behavioral Modeling, and Prediction (SBP10), March 2010, Bethesda, Maryland
• Shamanth Kumar, Nitin Agarwal, Merlyna Lim, Huan Liu. "Mapping Socio-Cultural Dynamics in Indonesian Blogosphere", 3rd International Conference on Computational and Cultural Dynamics (ICCCD'09), December 7-8, 2009, College Park, Maryland
• Sai Moturu, Jian Yang, Huan Liu. "Quantifying Utility and Trustworthiness for Advice Shared on Online Social Media", Symposium on Social Intelligence and Networking, IEEE International Conference on Social Computing (SocialCom'09),, Aug 29-31, 2009. Vancouver, Canada.
• Sai Moturu, Huan Liu. "Evaluating the Trustworthiness of Wikipedia Articles through Quality and Credibility" (Poster), 5th International Symposium on Wikis and Open Collaboration (WikiSym 2009),, Oct 25-27, 2009. Orlando, Florida.
• Nitin Agarwal, Huan Liu, Sudheendra Murthy, Arunabha Sen, Xufei Wang. "A Social Identity Approach to Identify Familiar Strangers in a Social Network", 3rd International AAAI Conference on Weblogs and Social Media (ICWSM09), May 17-20, 2009. San Jose, California.
• Nitin Agarwal, Shamanth Kumar, Huan Liu, Mark Woodward. "BlogTrackers: A Tool for Sociologists to Track and Analyze Blogosphere" (Demonstration Paper), 3rd International AAAI Conference on Weblogs and Social Media (ICWSM09), May 17-20, 2009. San Jose, California.
• Nitin Agarwal, Huan Liu, John J. Salerno, and Sanjay Sundarajan. "Understanding Group Interaction in Blogosphere: A Case Study", 2nd International Conference on Computational Cultural Dynamics (ICCCD08), September 15-16, 2008. Washington D.C.
• S. T. Moturu, Huan Liu, and W. Johnson. "Trust Evaluation in Health Information on the World Wide Web", IEEE Engineering in Medicine and Biology Conference (EMBC '08), August 20 - 24, 2008. Vancouver, Canada.
• Nitin Agarwal, Magdiel Galan, Huan Liu, and Shankar Subramanya. "Clustering Blogs with Collective Wisdom", 8th International Conference on Web Engineering (ICWE08), July 14-18, 2008. Yorktown Heights, New York.
• Nitin Agarwal "A Study of Communities and Influence in Blogosphere", 2nd SIGMOD PhD Innovative Database and Research Doctorate Consortium (IDAR08), June 13 2008. Vancouver, Canada.
• Nitin Agarwal, Huan Liu, Lei Tang, and Philip S. Yu. "Identifying Influential Bloggers in a Community", 1st International Conference on Web Search and Data Mining (WSDM08), pp 207-218, February 11-12, 2008. Stanford, California.
• Nitin Agarwal, Huan Liu, John J. Salerno, and Philip S. Yu. "Searching for `Familiar Strangers' on Blogosphere: Problems and Challenges", NSF Symposium on Next-Generation Data Mining and Cyber-enabled Discovery and Innovation(NGDM07). October 10-12, Baltimore, MD.


• Workshop Papers
• Huiji Gao, Jiliang Tang, and Huan Liu, "Mobile Location Prediction in Spatio-Temporal Context", Nokia Mobile Data Challenge Workshop, 2012
• Xufei Wang, Shamanth Kumar, and Huan Liu, "A Study of Tagging Behavior across Social Media," SIGIR 2011 Workshop on Social Web Search and Mining (SWSM 2011). Beijing, China.
• Jiliang Tang, Xufei Wang, and Huan Liu. "Social Media Data Integration for Community Detection," International Workshop on Mining Ubiquitous and Social Environments (MSM-MUSE 2011).
• Lei Tang, Xufei Wang, Huan Liu, and Lei Wang, "A Multi-Resolution Approach to Learning with Overlapping Communities", KDD Workshop on Social Media Analytics (SOMA), July 2010
• Lei Tang, Huiji Gao, and Huan Liu. Network Quantification Despite Biased Labels. KDD Workshop on Mining and Learning with Graphs, July 2010


• Technical Reports
• Huiji Gao, Xufei Wang, Jiliang Tang, and Huan Liu. "Network Denoising in Social Media", Technical Report, TR-11-002, School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, 2011
• Nitin Agarwal, Huan Liu, John J. Salerno, Shankara Subramanya, and Philip S. Yu. "Familiar Strangers: Connecting Dots on Blogosphere", Technical Report, TR-08-005, School of Computing Informatics, Arizona State University, Tempe, AZ 85287, 2008.
• Nitin Agarwal, Magdiel Galan, Huan Liu, Shankara Subramanya. "Clustering with Collective Wisdom - A Comparative Study", Technical Report, TR-08-004, School of Computing Informatics, Arizona State University, Tempe, AZ 85287, 2008.
• Nitin Agarwal, Huan Liu, John J. Salerno, Philip S. Yu. "Searching for `Familiar Strangers' on Blogosphere: Problems and Challenges", Technical Report, TR-07-008, CSE, School of Computing Informatics, Arizona State University, Tempe, AZ 85287, 2007.
• Nitin Agarwal, Huan Liu, Lei. Tang. "Identifying the Influentials in Blogosphere", Technical Report, TR-07-004, CSE, School of Computing Informatics, Arizona State University, Tempe, AZ 85287, 2007.


• Tutorials
• Nitin Agarwal and Huan Liu. A Tutorial on "Blogosphere: Research Issues, Applications, and Tools", 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD08), August 24-27, 2008. Las Vegas, NV.
• Others
• ACM TIST, Special Issue on "AI in Social Computing and Cultural Modeling", October 2010
• A Special Issue of IEEE Internet Computing on "Social Computing in Blogosphere" Mar-Apr 2010, SpringerLink.
• M. Woodward, "On Heresy and Religious Freedom: The Ahmadiyah Movement in Islam and the Front for Defense of Islam in Indonesia", to appear.

Demonstration

• The first video shows the news stories about the death of former Indonesian President Abdurrahman Wahid or Gus Dur as he was popularly known in Indonesia, from the viewpoint of Indonesian blogs
• The second video presents blog articles describing the controversy around the resignation of Indonesian Finance Minister Sri Mulyani to join the World Bank.

PDFTracker

PDFTracker helps a user to parse, store, and analyze PDF (Portable Document Format) documents. PDF is a very flexible and convenient format that is widely used for storing documents. PDF Tracker makes searching and analyzing PDF documents easier.

Download the Demo version of PDFTracker here
User manual for the tool is available here
PDF document corpus used in the demo version of the tool can be downloaded here

Project Members (current and former)

• Huan Liu
• Merlyna Lim
• Shamanth Kumar
• Fred Morstatter
• Huiji Gao
• Xufei Wang (Graduated)
• William Cole (Graduated)
• Patrick McInerney (Graduated)
• Lei Tang (Graduated)
• Nitin Agarwal (Graduated)
• Alan Zheng Zhao (Graduated)
• Sai Moturu (Graduated)
• Shankara Subramanya (Graduated)
• David Webb
• Mark Woodward

Acknowledgments

Created on August 1, 2008.
Contact: Huan Liu.

Last Updated: April 19, 2013