Sentiment Analysis application created with Python and Dash, hosted at socialsentiment.net

Related tags

Data Visualization plotly python3 dashboards plotly-dash
Overview

Social Sentiment Dash Application

Live-streaming sentiment analysis application created with Python and Dash, hosted at SocialSentiment.net.

application example

Dash Tutorials

This application was created in conjunction with the Dash tutorial series.

Repo Contents:

  • dash_mess.py - This is currently the main front-end application code. Contains the dash application layouts, logic for graphs, interfaces with the database...etc. Name is descriptive of the overall state of code :) ...this code is setup to run on a flask instance. If you want to clone this and run it locally, you will be using the dev_server.py
  • dev_server.py - If you wish to run this application locally, on the dev server, run via this instead.
  • twitter_stream.py - This should run in the background of your application. This is what streams tweets from Twitter, storing them into the sqlite database, which is what the dash_mess.py file interfaces with.
  • config.py - Meant for many configurations, but right now it just contains stop words. Words we don't intend to ever count in the "trending"
  • cache.py - For caching purposes in effort to get things to run faster.
  • db-truncate.py - A script to truncate the infinitely-growing sqlite database. You will get about 3.5 millionish tweets per day, depending on how fast you can process. You can keep these, but, as the database grows, search times will dramatically suffer.

Quick start

  • Clone repo
  • install requirements.txt using pip install -r requirements.txt
  • Fill in your Twitter App credentials to twitter_stream.py. Go to apps.twitter.com to set that up if you need to.
  • Run twitter_stream.py to build database
  • If you're using this locally, you can run the application with the dev_server.py script. If you want to deploy this to a webserver, see my deploying Dash application tutorial
  • You might need the latest version of sqlite.
sudo add-apt-repository ppa:jonathonf/backports
sudo apt-get update && sudo apt-get install sqlite3
  • Consider running the db-truncate.py from time to time (or via a cronjob), to keep the database reasonably sized. In its current state, the database really doesn't need to store more than 2-3 days of data most likely.

Tips for Running on Server

  • You can use Gunicorn to run the server
gunicorn dash_mess:server -b 0.0.0.0:80 -w 4

Todo

Want to help contribute???

  • Code is ugly. Low hanging fruit is just making the code not so ugly. Up to this point, I've just been in "make it work" mode.
  • App is ugly. I am not a designer. This app is prettttttyyyyyy gross. Think you have a better design? Halp.
  • Click-able related terms and trending terms would be nice. I tried, but failed at this. It'd be cool to see a related term, and be able to just click on it, and this becomes the new searched term, for example.
  • The interactive search is cool, but also does a search in the database per-character. It would be nice if it didn't search per key-press. Not sure I want a search button, I like the streamlined interactivity, but maybe wait 0.2 seconds or something without any new keypresses to perform the search? Something like that might help with speeds. I really do not know the best option here, I just know this isn't idea.
  • Other manipulations or ideas for interactivity? Feel free to show them in a PR.

Credits

The speed of the application, especially with a database with 10's of millions of records is thanks entirely to Daniel Kukiela who helped us to convert from regular sqlite to using fts, helping with the queries, new database structure, caching, and more.

Comments
  • Hosted example site failing

    Hosted example site failing

    FYI: The SocialSentiment.net link, which I'm assuming is the example site hosted on a live server, seems to be failing. A Error loading dependencies error is being raised.

    Browser: Version 64.0.3282.119 (Official Build) (32-bit)

    opened by jaredtconnor 2
  • Keep getting the error message: no such table: sentiment_fts

    Keep getting the error message: no such table: sentiment_fts

    pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT sentiment.* FROM sentiment_fts fts LEFT JOIN sentiment ON fts.rowid = sentiment.id WHERE fts.sentiment_fts MATCH ? ORDER BY fts.rowid DESC LIMIT 10': no such table: sentiment_fts

    Any Solution?

    OS: MacOS high sierra

    opened by shahud1 1
  • Tweet Language

    Tweet Language

    Aren't Vader/TextBlob not accurate for foreign languages?

    I believe twitter allows you filter out to just english language tweets so would that be something to pursue?

    https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/basic-stream-parameters

    opened by venkashank 0
  • Missing requirement and spelling-error

    Missing requirement and spelling-error

    The master-source as of 17.09.2019 is missing a requirement and has a grammar-error.

    Requirements.txt is missing "textblob", to install it over pip simply type pip install textblob or click here get informations how to install it or simply open requirements.txt and simply add textblob into a new line.

    Under twitter_stream.py, in Line 32 is a " missing to close out the variable asecret. That isnt a big deal thought, youre supposed to enter your asecret manually in the source-code in there so you most likely will notice that yourself.

    Im new on github, and idk how to create pull-requests. :(

    opened by Syrrokz 0
  • Sentiment analysis for tweets written in other language

    Sentiment analysis for tweets written in other language

    Hi! First of all, thanks for your amazing work!

    I would like to share my work based on this great repository, I made sentiment analysis dashboards from Portuguese language, check it out! You are able to change the language.

    • https://github.com/Alro10/twitter-sentiment-live

    Cheers.

    opened by Alro10 0
  • cannot import name 'Event' from 'dash.dependencies'

    cannot import name 'Event' from 'dash.dependencies'

    Looks like the Event system got removed in the latest version of Dash. Is there a reference to know what versions of packages were installed at the time this tutorial was being created?

    opened by krispykres 6
  • sqlite3.ProgrammingError: Recursive use of cursors not allowed.

    sqlite3.ProgrammingError: Recursive use of cursors not allowed.

    While searching for new term this app throws the following error.

      File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/dash/dash.py", line 967, in dispatch
    return self.callback_map[target_id]['callback'](*args)
  File "/home/n3011/miniconda3/envs/twitter/lib/python3.6/site-packages/dash/dash.py", line 907, in add_context
    output_value = func(*args, **kwargs)
  File "/home/n3011/work/twitter/socialsentiment/dash_mess.py", line 228, in update_pie_chart
    sentiment_pie_dict = cache.get('sentiment_shares', sentiment_term)
  File "/home/n3011/work/twitter/socialsentiment/cache.py", line 41, in get
    result = self.cursor.execute("SELECT value FROM {} WHERE key = ?".format(pool), (key,)).fetchone()
sqlite3.ProgrammingError: Recursive use of cursors not allowed.
    opened by saikia1 11
  • In twitter_stream.py

    In twitter_stream.py

    401 401 401 401 forrtl: error (200): program aborting due to control-C event Image PC Routine Line Source libifcoremd.dll 000007FED0BE94C4 Unknown Unknown Unknown kernel32.dll 0000000077774B93 Unknown Unknown Unknown kernel32.dll 00000000777359ED Unknown Unknown Unknown ntdll.dll 000000007786B371 Unknown Unknown

    opened by sidmanish 0
Owner
Harrison
Harrison
GitHub
Yata is a fast, simple and easy Data Visulaization tool, running on python dash

Yata is a fast, simple and easy Data Visulaization tool, running on python dash. The main goal of Yata is to provide a easy way for persons with little programming knowledge to visualize their data easily.

Cybercreek 3 Jun 28, 2021
Getting started with Python, Dash and Plot.ly for the Data Dashboards team

data_dashboards Getting started with Python, Dash and Plot.ly for the Data Dashboards team Getting started MacOS users: # Install the pyenv version ma

Department for Levelling Up, Housing and Communities 1 Nov 8, 2021
GD-UltraHack - A Mod Menu for Geometry Dash. Specifically a MegahackV5 clone in Python. Only for Windows

GD UltraHack: The Mod Menu that Nobody asked for. This is a mod menu for the gam

zeo 1 Jan 5, 2022
Generating interfaces(CLI, Qt GUI, Dash web app) from a Python function.

oneFace is a Python library for automatically generating multiple interfaces(CLI, GUI, WebGUI) from a callable Python object. oneFace is an easy way t

NaNg 31 Oct 21, 2022
Visualization Website by using Dash and Heroku

Visualization Website by using Dash and Heroku You can visit the website https://payroll-expense-analysis.herokuapp.com/ In this project, I am interes

YF Liu 1 Jan 14, 2022
A curated list of awesome Dash (plotly) resources

Awesome Dash A curated list of awesome Dash (plotly) resources Dash is a productive Python framework for building web applications. Written on top of

Luke Singham 1.7k Dec 26, 2022
A guide for using Bootstrap 5 classes in Dash Bootstrap Components V1

dash-bootstrap-cheatsheet This handy interactive cheatsheet makes it easy to use the Bootstrap 5 classes with your Dash app made with the latest versi

null 10 Dec 22, 2022
Custom Plotly Dash components based on Mantine React Components library

Dash Mantine Components Dash Mantine Components is a Dash component library based on Mantine React Components Library. It makes it easier to create go

Snehil Vijay 239 Jan 8, 2023
A dashboard built using Plotly-Dash for interactive visualization of Dex-connected individuals across the country.

Dashboard For The DexConnect Platform of Dexterity Global Working prototype submission for internship at Dexterity Global Group. Dashboard for real ti

Yashasvi Misra 2 Jun 15, 2021
A shimmer pre-load component for Plotly Dash

dash-loading-shimmer A shimmer pre-load component for Plotly Dash Installation Get it with pip: pip install dash-loading-extras Or maybe you prefer Pi

Lucas Durand 4 Oct 12, 2022
Regress.me is an easy to use data visualization tool powered by Dash/Plotly.

Regress.me Regress.me is an easy to use data visualization tool powered by Dash/Plotly. Regress.me.-.Google.Chrome.2022-05-10.15-58-59.mp4 Get Started

Amar 14 Aug 14, 2022
Sky attention heatmap of submissions to astrometry.net

astroheat Installation Requires Python 3.6+, Tested with Python 3.9.5 Install library dependencies pip install -r requirements.txt The program require

null 4 Jun 20, 2022
Wikipedia WordCloud App generate Wikipedia word cloud art created using python's streamlit, matplotlib, wikipedia and wordcloud packages

Wikipedia WordCloud App Wikipedia WordCloud App generate Wikipedia word cloud art created using python's streamlit, matplotlib, wikipedia and wordclou

Siva Prakash 5 Jan 2, 2022
A Python library created to assist programmers with complex mathematical functions

libmaths was created not only as a learning experience for me, but as a way to make mathematical models in seconds for Python users using mat

Simple 73 Oct 2, 2022
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

Sophia Huang 12 Oct 20, 2022
Automatically Visualize any dataset, any size with a single line of code. Created by Ram Seshadri. Collaborators Welcome. Permission Granted upon Request.

AutoViz Automatically Visualize any dataset, any size with a single line of code. AutoViz performs automatic visualization of any dataset with one lin

AutoViz and Auto_ViML 1k Jan 2, 2023
Automatically Visualize any dataset, any size with a single line of code. Created by Ram Seshadri. Collaborators Welcome. Permission Granted upon Request.

AutoViz Automatically Visualize any dataset, any size with a single line of code. AutoViz performs automatic visualization of any dataset with one lin

AutoViz and Auto_ViML 299 Feb 13, 2021
These data visualizations were created as homework for my CS40 class. I hope you enjoy!

Data Visualizations These data visualizations were created as homework for my CS40 class. I hope you enjoy! Nobel Laureates by their Country of Birth

null 9 Sep 2, 2022
Python package for hypergraph analysis and visualization.

The HyperNetX library provides classes and methods for the analysis and visualization of complex network data. HyperNetX uses data structures designed to represent set systems containing nested data and/or multi-way relationships. The library generalizes traditional graph metrics to hypergraphs.

Pacific Northwest National Laboratory 304 Dec 27, 2022