LUCA Artificial Intelligence in the Industrial Sector: the success story of Repsol Although technological advances have been left behind in recent years, the industrial sector presents opportunities for developpment in the field of Big Data and Artificial Intelligence, and also in...
Richard Benjamins Towards a new Big Data ecosystem to mitigate Climate Change Big Data can be used to help fight Climate Change [1],[2],[3]. Several projects have analysed huge amounts of satellite, weather and climate data to come up with ways to...
Richard Benjamins Lessons learned from The Cambridge Analytica / Facebook scandal It has been now some time that the Cambridge Analytica / Facebook scandal was first revealed on March 17, 2018 by The Guardian and the New York Times. Much has been...
LUCA Traffic spikes before football games? The effect of the World Cup in Germany Post written by Telefónica NEXT Once every four years, The FIFA World Cup brings fans together in a month of surprising events. Every game is different, but each is filled...
LUCA Success Story: pioneering project for economic and social development In a previous article we talked about how important it is to undertake a cultural transformation, that is, of individuals, whilst a company or sector undertakes a digital transformation...
LUCA Improving intent to purchase with mobile advertising: Success Story of Milpa Real The Kantar Millward study demonstrates the effectiveness of the Mobile Advertising platform Data Rewards. The objective of the Brand Lift study is to measure the effectiveness of an advertising campaign,...
LUCA Improving intent to purchase with mobile advertising: Success Story of Milpa Real The Kantar Millward study demonstrates the effectiveness of the Mobile Advertising platform Data Rewards. The objective of the Brand Lift study is to measure the effectiveness of an advertising campaign,...
LUCA How Germany moves Our colleagues in Telefonica NEXT have created, using mobility data, data analysis and big data, an interactive map that lets you visualize nationwide traffic flows. Cities, transport companies and...
Big Data and Elections: We shine a light on Trump and ClintonRichard Benjamins 9 November, 2016 Twitter is widely used as a tool to understand and predict phenomena in the real world. Today on our blog, we have been using Twitter to understand the US Presidential Elections of November 8th 2016. There are no conclusive research results on whether it is possible to predict the outcomes of elections using tweets but we decided to investigate. Figure 1: Shine a light on Trump or Clinton In 2010, an article was published as part of the Fourth International AAAI Conference on Weblogs and Social Media entitled “Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment- concludes that it indeed is possible“. This research used tweets from the German Federal elections in Sept 2009 and concluded that the mere number of messages mentioning a party can reflect the election result. Furthermore, joint mentions of two parties are in line with real world political ties and coalitions. The research also went on to show that an analysis of the tweets’ political sentiment demonstrates close correspondence to the parties’ and politicians’ political positions indicating that the content of Twitter messages “plausibly reflects the offline political landscape.” However, another article published one year later in the same Artificial Intelligence conference -“Limits of Electoral Predictions Using Twitter” stated that “unfortunately, they found no correlation between the analysis results and the electoral outcomes, contradicting previous reports,”, basing their investigation on tweets of the US 2010 Congressional elections. A search in Google will show much more research on the feasibility of using Twitter and other social media networks for election predictions. Additionally, a Google Trends query to check which candidate generates more search activity provides some insight on the matter: Figure 2: Google Trends candidates’ search activity Whatever the conclusion is, there is no doubt that Twitter reflects to some extent what is going on in a country just before important elections. During the two days running up to the elections on the 8th of November 2016, we established a real-time feed from Twitter filtering relevant hashtags, handles names and keywords. As technology, we have used Sinfonier, an ElevenPaths cybersecurity product to detect cyberthreats based on real-time information processing. In this case, Sinfonier automates the capture of tweets in real-time (see Figure 3). Sinfonier encapsulates real-time capabilities of Apache Storm in a very elegant way, so we can build a “digest Twitter-to-MongoDB” within a few minutes and with almost zero code writing. Figure 3: Sinfonier Topology (data extraction pipeline) designed to capture real-time data from Twitter. We then also used real-time capabilities using Elastic Search and Kibana. Apart from visualizing the tweets in real-time in a dashboard (see Figure 4), we wanted to try out something more fun so we also got some Philips Hue lamps involved. Figure 4: Real-time tweets on Clinton (blue) vs. Trump (red). Trump tweets almost double those of Clinton. Trump is slightly more active in replying and both re-tweet equally all tweets they are mentioned in. Actually, in the Big Data era, where taking data-driven decisions is the ultimate goal, in a variety of situations it will be unfeasible to use traditional dashboards to convey the status of real-time KPIs. Imagine the case of factories, call center open floors and large retail centers. It could be very cool to have “realtime transparent dashboards”, so that lights (visual perception) adapt to the (big) data produced in the real world in a fast, intelligent and pervasive way. The applications are limitless! In such cases, using dynamic lights could be a good alternative to convey the main insights of dashboards. For instance, in a call center, the light intensity and color could change according to the number and type of calls (complaints, information, products, etc.) from customers. By connecting lamps to the Internet, we enter the world of the Internet of Things. Hue lamps can be instructed to react to phenomena on the Internet using the “If This Then That” framework. Using IFTTT, you can use the lamps to turn the light on when it is raining in Amsterdam, or when your plane lands safely in another city around the world. So how have we connected the lamps to the tweets related to the US elections? On this occasion, we haven’t performed a complex analysis nor a prediction of the winner as for this purpose, we would need to ask ourselves first if Twitter is the best data stream to use. Our first step was to design two query sets, in the first query, we count all the tweets related to common words used to refer to Hillary Clinton (e.g. Hillary, Clinton, HillaryClinton) or Donald Trump (e.g. Donald, Trump, DonaldTrump). The results of which are displayed in Figure 5: Figure 5: Twitter shows much more activityrelated with Donald Trump but tweet count include both positive andnegative references so direct interpretation could be misleading. The second query (Figure 6) is much more specific as we compare tweets containing their handles, @RealDonaldTrump vs @HillaryClinton: Figure 6. The second query reveals a much more balanced pattern with no clear winner. Additionally, we set rules for the lights as follows: Lamp 1 is connected to a twitter feed on query 1 (Figure 5), blinking the color of the candidate who had more tweets in the last 2 seconds and showing a stable color of the candidate who had more tweets in the last hour. Lamp 2 has the same behaviour of Lamp 1 but this time it is connected to the second query stream (figure 6). On the day of the elections, we used the lamps in our office to engage with Election Day. We observed that the lamps were mostly red, reflecting the fact that Trump has many more mentions than Clinton. However, we saw that many tweets mentioning Trump are actually against him, showing negative sentimient whilst Hillary Clinton attracted less negative tweets. Video: Pedro Antonio de Alarcon explains the technology used (video in Spanish). Now, the votes have been cast and the United States of America has decided – but in the midst of global frenzied reaction to Donald Trump’s election, our lamps keep on flickering. Can Big Data and IOT prevent motorcycle crashes?Chatbots? New? You haven’t met ELIZA
LUCA Artificial Intelligence in the Industrial Sector: the success story of Repsol Although technological advances have been left behind in recent years, the industrial sector presents opportunities for developpment in the field of Big Data and Artificial Intelligence, and also in...
Richard Benjamins Towards a new Big Data ecosystem to mitigate Climate Change Big Data can be used to help fight Climate Change [1],[2],[3]. Several projects have analysed huge amounts of satellite, weather and climate data to come up with ways to...
Olivia Brookhouse The Big Data behind Black Friday It is often the products that seem to have “come out of nowhere” that suddenly experience rocket sales and have everyone talking. Whether its health fads such as...
Olivia Brookhouse How to learn Python: the most important skill for employers The application of Artificial Intelligence to optimize Big Data and provide Data Insights is becoming a necessity to all business processes, therefore programmers are in hot demand. Designing effective...
LUCA SMASSA Success Story: Intelligent Parking Services Today we bring another success story from the public sector: the example of SMASSA, the Sociedad Municipal de Aparcamientos (Municipal Parking Company) whose mission is to solve the parking...
Olivia Brookhouse Big Data, the fuel for Digital Banks In 1981, Bill Gates said “640KB will be enough for anybody”. Surpisingly, it was hard even then, to imagine where we would be today; in a sea of data....