Multifunctional Analysis of Regions through Input-Output

Following the documentation for the Eurostat-parsing functionality, I ran into below error ("non-corresponding years"). This is unexpected since I was strictly following the documentation, i.e. did everything as described there. Any ideas why that happens and how to get my SUTs?

`--------------------------------------------------------------------------- WrongInput Traceback (most recent call last) c:\Users\maximiko\IO-testbed\Eurostat_MARIO.py in ----> 18 italy_sut = parse_eurostat( 19 supply_path = supply_path, 20 use_path = use_path, 21 region = "Italy", 22 year = 2011

~\Anaconda3\envs\pymrio\lib\site-packages\mario\tools\parsersclass.py in parse_eurostat(supply_path, use_path, region, year, consumption_categories, factors_of_production, imports, model, name, calc_all, **kwargs) 432 table = "SUT" 433 if table == "SUT": --> 434 matrices, indeces, units = eurostat_sut( 435 supply_path, 436 use_path,

~\Anaconda3\envs\pymrio\lib\site-packages\mario\tools\tableparser.py in eurostat_sut(supply_path, use_path, region, selected_year, consumption_categories, factors_of_production, imports) 992 years = supply_years 993 else: --> 994 raise WrongInput("No correspondance between years of use and supply tables") 995 996 "Warning on supply in IxP format, Use in PxI format"

WrongInput: No correspondance between years of use and supply tables`

Now the Hybrid version of EXIOBASE is available on Zenodo at https://zenodo.org/record/7244919#.Y6Lr0NXMIQ8

A new parse function is needed in order to take advantage of a lighter format.

In the calc_all function, we should check at the beginning of the function whether the scenario exists or not so the keyerror won't be about the missing scenario.

The search function should not be case sensitive. For instance, if the user run this command:

db.search("Sector","electricity")

he will not be able to track for sectors named "Electricity..." with capital "E". It would be super-useful to improve this making the function not case-sensitive

pull request for merging the functions and methods added for mario vs pymrio integration.

Changes:

• Adding a new method for converting mario.Database to pymrio.IOSystem in Database class
• Adding the parser function to parsers that reshapes the pymrio.IOSystem and pymrio.Extensions to mario.Database
• miscellaneous changes in tools and constants to map the differences in two libraries

Every time a new database is printed-out there should be a way to access information on the:

• year
• table version
• added extensions
• ...

Printing out the metadata (as a txt file in case .to_txt or .to_csv is used; as an additional sheet in case .to_excel is used) can do the job

SESAM-Polimi/MARIO now has a Chat Room on Gitter

@LorenzoRinaldi has just created a chat room. You can visit it here: https://gitter.im/SESAM-Polimi/MARIO.

This pull-request adds this badge to your README.rst:

If my aim is a little off, please let me know.

Happy chatting.

PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

TLDR I tried to follow the Mario set up instruction and I believe the pymrio package is missing from requirements.txt. I am using this as an opportunity to try my first pull request on the MARIO repo just to make sure everything works properly.

REPRO

i tried to install MARIO and then run a simple script using it. Here is the script I tried: https://gist.github.com/AAmedeo/fdeb1f74413d109d3f2727fa1116e687

This is what happened before adding pymrio to requirements.txt

afa@afa-mbp MARIO % rm -r venv
afa@afa-mbp MARIO % python3 -m venv venv
afa@afa-mbp MARIO % source ./venv/bin/activate
(venv) afa@afa-mbp MARIO % pip3 install -r requirements.txt
(venv) afa@afa-mbp MARIO % python3 mario_hello_world.py
Traceback (most recent call last):
  File "mario_hello_world.py", line 1, in <module>
    import mario
  File "/Users/afa/Desktop/MARIO/mario/__init__.py", line 48, in <module>
    from mario.core.AttrData import Database
  File "/Users/afa/Desktop/MARIO/mario/core/AttrData.py", line 91, in <module>
    from mario.core.CoreIO import CoreModel
  File "/Users/afa/Desktop/MARIO/mario/core/CoreIO.py", line 15, in <module>
    from mario.tools.tableparser import dataframe_parser
  File "/Users/afa/Desktop/MARIO/mario/tools/tableparser.py", line 48, in <module>
    import pymrio
ModuleNotFoundError: No module named 'pymrio'

This is what happened after adding pymrio to requirements.txt

afa@afa-mbp MARIO % rm -r venv
afa@afa-mbp MARIO % python3 -m venv venv
afa@afa-mbp MARIO % source ./venv/bin/activate
(venv) afa@afa-mbp MARIO % pip3 install -r requirements.txt
(venv) afa@afa-mbp MARIO % python3 mario_hello_world.py
cvxpy module is not installed in your system. This will raise problems in some of the abilities of MARIO
name = IOT test
table = IOT
scenarios = ['baseline']
Factor of production = 3
Satellite account = 4
Consumption category = 1
Region = 2
Sector = 6

name = SUT test
table = SUT
scenarios = ['baseline']
Activity = 6
Commodity = 6
Factor of production = 3
Satellite account = 1
Consumption category = 1
Region = 2

['RoW', 'Italy']
['Taxes', 'Wages', 'Capital']

Using add_sectors function with pandas version 1.4.4 lead to the following error:

ValueError: Length of new names must be 1, got 3

The function works with pandas 1.3.3

I am requesting to merge these changes to main branch. The changes implement a new parser function for exiobase database, in particular referring to version 3.3.18, downloadable at this link

In particular, the new feature have been implemented by updating the already present function parse_exiobase_sut, by adding the unit, version and extensions attributes.

The unit attribute can be selected among the _acceptable_units list, saved in the parserid.py file, and is by default set to "monetary". In the default case, the function is run exactly in the same way as the one in the main branch. The version attribute is by default set to None and must be provided in case unit="hybrid" The extensions attribute is by default set to None and can be provided as a list including any of the keys of the _acceptable_extensions dictionay saved in the parserid.py file.

Regarding the case when unit="hybrid", then the new hybrid_sut_parser function (in tableparser.py) will be invoked (see commit 799443fd5b3e0944894238b0d4c5861fe8e18640).

Initially, this function was developed to parse an old version of the database, downloadable from the official exiobase website, which was saved as a zip file. This version had supply, use and final demand matrices saved into separate .xlsb files therefore the parser was significantly slower than other ones for other exiobase versions. The function was then modified in order to parse the database as the one available on zenodo (link reported previously), where the same matrices are available by default in .csv formats (much faster to be read).

Another improvement brought by this merge would be the fact that the aggregation excel file template obtained by the get_aggregation_excel function, automatically prints a new sheet showing the units of each entry of each set. This would ease the process of aggregation in case of hybrid databases.

I see the package depency states "xlsxwriter <= 1.3.7", whereas the current version is 3.0.3... Is this really required? When installing the package, it downgraded my package version, and broke functionnalities with other Python scripts I'm running. Thanks in advance...

The function "get_data" is not working if the user provide an integer as "scenarios" parameter. Not even if it is provided within a list. Moreover, the documentation for this function seems missing.

Check if actvity units is making sense, in general: do they need to be provided with units? Especially in hybrid database the answer is probably no, while for economic databases it is the opposite.

Also, hybrid database need for further checks when dealing with aggregation of activities due to the fact activities may produce commodities with different units