First responders “suit up” with social media and AI
The most critical information that first-response agencies need during large events, disasters and emergencies is a clear understanding of the current situation — what is happening where, where their team is, and where their resources are. This is part of what is known as situational awareness. First responders typically rely on reports from personnel and media accounts for the information, but this is often sparse coverage that doesn’t provide a complete picture.
Yet every day during these occurrences, citizens are communicating to friends and posting information about the events and situations — providing a potential source of valuable intelligence, with more than 500 million posts daily on Twitter alone. People also post millions of images of events, including damage from storms and other natural disasters on Instagram.
We created an AI-powered program called SMART to overcome the difficulty of finding the tweets and Instagram posts that increase first responders’ understanding of the current situation. SMART — which stands for Social Media Analytics and Reporting Toolkit — analyzes public, geotagged tweets and related Instagram posts in real time and classifies them into topics, creating an interactive visual analysis and decision-making environment (visual analytics). This allows us to create “word clouds” on maps, which are updated every few seconds to show what people are posting.
We also let the first responders themselves enter keywords and topics they are interested in to filter out messages that are not on point. We do this based on words and tags that a responder specifies so we can generate information that is more relevant to each event/situation. Through our interactive web page, with the map and associated message and topic tables, emergency responders get constantly updated information at command centers to help them find people in need and identify areas of concern — turning massive, previously indigestible amounts of data into understandable, at-a-glance information in real time.
We’re continually receiving feedback from first responders to improve the analysis and visualizations and to build new features. We recently added machine learning/AI to help eliminate tweets that have the trigger keyword used in a different context. For example, during forest fires, we want to include tweets that contain the keyword “fire” — but only when it refers to a fire, not a tweet like, “They should fire that coach!”
Our approach to solving this problem is to have the user help mark the tweet as germane or not. We then interactively retrain the machine learning algorithm so it highlights tweets similar to the relevant ones and ignores the immaterial ones. This user-guided, interactive machine learning/AI system represents a new area of research, combining advanced machine learning with human expertise to make the results more powerful. Just marking 20 tweets as relevant or irrelevant dramatically improves performance — responders have indicated that it makes the system much more accurate and their jobs much easier.
Our system has been validated and tested during many large-scale events, from Purdue football games to hurricane rescues. The easy-to-understand visualizations, interactive information updates and user guidance enable first responders to trust the results — exemplifying “explainable AI,” in which the rationale for machine-learning-derived solutions can be easily understood and accepted. We’re also constantly working to create new features based on community feedback, and plan to support multiple languages.
Through a partnership with Davista Technologies, a Purdue-based startup, we have developed a commercial version that should be available to first-responder agencies across the country by early 2021. This edition will process a much larger volume of tweets/Instagram posts in the cloud, enabling us to accommodate hundreds of simultaneous users across the country.
The goal is to harness the power of social media and artificial intelligence and get this technology to as many first responders as we can to help them save lives.
David S. Ebert is the Silicon Valley Professor of Electrical and Computer Engineering and Director of VACCINE at the Purdue University College of Engineering.