Welcome to the DataDetectives Dashboard, a powerful tool offering visualizations of energy statistics across Europe. This comprehensive dashboard not only presents energy usage data but also integrates and compares it with population and temperature metrics. By doing so, it provides a holistic view, enabling users to explore the correlation between energy consumption and various factors.
There is a known issue where data for Iceland and Norway may intermittently disappear on the choropleth map. If you encounter this, a simple page reload (not the entire service) should resolve the problem.
Clicking on a country in the choropleth while downloading/preparing the new datasets may trigger an error requiring a complete restart. This behaviour is inconsistent, and varies wildly between machines (doesn't happen at all on some machines, happens semi-frequently on others).
The database calls the pipeline if the CSV population fails, and the CSV if the pipeline population fails. If both of these fail, it will be stuck in an infinite loop causing a recursion error. So far, this has never happened, but it is a theoretical issue.
Setting up and running the DataDetectives Dashboard is a straightforward process:
- Clone the repository to your local machine.
- Navigate to the root directory of the cloned project using the
cdcommand. - Execute the command
docker-compose buildin the project's root directory. - Once the build process is complete, launch the application with
docker-compose up.
The dashboard should now be accessible. By default, it should be available at 127.0.0.1:8080, but this may vary depending on your system. We recommend checking the terminal output to confirm the exact address.
The DataDetectives Dashboard utilizes a PostgreSQL database to query information. The interaction with this database is facilitated through a Database object. The database is populated by data pipelines, each dedicated to a specific type of data (Population data, Temperature data, Monthly Energy data, and Yearly Energy data). These pipelines download, clean, and prepare the corresponding data.
app.pyrepresents the Dash application, serving as the dashboard interface.Database.pycontains the code for the Database object, enabling seamless interaction with the PostgreSQL database.Electricity.pyis the pipeline for electricity data.Population.pymanages the population data pipeline.Temperature.pyhandles the temperature data pipeline.PerCapita.pycalculates per capita energy usage.Predictions.pyutilizes an optimized random forest regressor to predict GWH usage.Prediction-checkpoint.ipynbin the notebook folder shows the optimization parameters for the used forest tree regression algorithm
Our dashboard is powered by data from reputable sources to ensure accuracy and reliability. The temperature data is sourced from the KNMI Climate Explorer, a tool provided by the Royal Netherlands Meteorological Institute. Our population data is obtained from the World Bank API, which provides access to comprehensive global development data. Lastly, the energy data is derived from EUROSTAT, the statistical office of the European Union, which provides high-quality statistics at the European level. By integrating these diverse datasets, we aim to provide a holistic view of energy usage across the European Union.
The Jupyternotebook is no longer used in the deployment of this application. It's continued inclusion is for legacy reasons.
Ismaïl el Alami Dagmar van den Berg Laura Bröring Roy Schenk Thomas Valkenburg Casper van Wetten
This code has been licensed under the GNU General Public License V3.0.