Problem

The problem is the website performance is so slow, and it's so hard to browse and I think there's code that is not readable at all, for ex. in card.js there's a lot of code that no need for (I've deleted a little bit of it during reading and understanding the code)!

Solution

I've designed the front end and made it easier to browse check it out

Notes

• I don't know what's the point of updating MaxRowLength every single time someone enters a new data point in google sheets.
• Performance is so slow because of google sheets because of the google sheets request that takes more than 3s to load!
• A lot of external libraries, such as Axios and jquery, both of them slow the website because it's a client-based website(HTML, CSS, JS) and there's an alternative built-in same exact functionality.

Thanks to @sudomaze for helping me with the review of the design.

Tools to include

• [ ] black - code formatter
• [ ] mypy - static type checker
• [ ] pycln - remove unused imports
• [ ] isort - sort imports

Skeleton to start from

# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.0.1
    hooks:
      - id: check-ast
      - id: check-builtin-literals
      - id: check-case-conflict
      - id: check-docstring-first
      - id: check-merge-conflict
      - id: check-json
      - id: check-toml
      - id: check-yaml
      - id: end-of-file-fixer
      - id: mixed-line-ending
      - id: trailing-whitespace
      - id: check-vcs-permalinks
      - id: check-shebang-scripts-are-executable
  - repo: https://github.com/pre-commit/pygrep-hooks
    rev: v1.9.0
    hooks:
      - id: python-check-mock-methods
      - id: python-no-log-warn
      - id: python-use-type-annotations
  - repo: https://github.com/hadialqattan/pycln
    rev: v1.2.5
    hooks:
      - id: pycln
        args: [--all]
  - repo: https://github.com/psf/black
    rev: 22.3.0
    hooks:
      - id: black
        additional_dependencies: ['click==8.0.4']
  - repo: https://github.com/PyCQA/isort
    rev: 5.10.1
    hooks:
      - id: isort
        args: ["--profile", "black"]

Per what @AliOsm has done in https://github.com/ARBML/masader-webservice, we will need to add the following to complete the required tasks for the paper submission:

• [x] Get the entire datasets at once
• [x] Get a single dataset by id
• [x] Get explore page data in the right format
• [x] Get graphs page data in the right format

Added filteration page.

You should consider the following

• [ ] Build the ability to share the link with the filteration applied without making the page reload each time filter updated
• [ ] Solve bunch of boilerplates
• [ ] Solve long pagination buttons ( Should be one row )
• [ ] Refactor the code make it more better and less lines to achieve happiness

We will need to have some way to prevent spam either by using reCAPTCHA or auth the account with GitHub account to prevent who will spam using the reporting functionality.

• Refactored the project structure and removed unused pages and scripts.
• Redesigned the pages according to the org. brand.
• Adding icon(Favicons).

issue : #6

Is your feature request related to a problem? Please describe. The current pie chart under stats page doesn't have labels which make it hard to track which piece belongs to which dialect. In addition, the pie chart shows all the dialects at once, making it hard to understand the separation of dialects based on region.

Describe the solution you'd like

• Including labels for each piece of the pie chart.
• Include other pie charts based on region (Gulf, Lev, North Africa, etc.).

Relevant #93

The dialects graph in https://arbml.github.io/masader/charting.html only shows the number of resources. It will be helpful if we can show the list of datasets that correspond to a chosen country.

For this issue, we will refactor the front-end code under this branch https://github.com/ARBML/masader/tree/dev/move_to_jekyll, so use it as a base for any of the tasks below:

Core changes:

• [x] Update js scripts to expect data from the API endpoint
  • [x] index.js
  • [x] card.js
  • [x] charts.js
  • [x] graphs.js
• [ ] Ensure that the website is responsive on mobile screens

Additions

• [ ] Add a simple filtering functionality for #17

Per ARBML's discord group meeting, we have concluded the following tasks to be added to improve Masadar for the Demo paper publication.

Changes

• Move the front-end to Jekyll to clean up the code and have it easy to contribute to
• Build an API endpoint to fix the issue of slowdown when loading the page due to fetching data from the database from the front-end and preprocessing them (#22)
• Add additional functionalities: better filtering, report data (#25)
• Updating data entry (#24)

Todos

• [x] #19
• [x] #29
• [x] #24
• [x] #45

Pushed for later

• Issue with DDoS for #25
• ~#18~
• ~#20~
• Refactor the entire codebase to be under one framework and web app (Flask/FastAPI)

Describe the bug Adding white spaces to begging or end of the search query (dataset name) prevents the search engine from matching it to already existing datasets in the catalogue.

To Reproduce

1. Go to : https://arbml.github.io/masader/search
2. Type name of dataset with a white space (or multiple) at the beginning or end (example: https://arbml.github.io/masader/search?name=artest+)
3. No results returned

Expected behavior Search engine should handle such typo issue from the user as white spaces are most probably not part of any dataset name. Search engine should return the dataset even if the user added a space to its name in the search query.

Desktop (please complete the following information):

• OS: Windows 8
• Browser: Chrome

Rather than having a simple text like this Having a more unique styling of the landing page's header like this (not black and white) On mobile (not black and white) ,

cc: @unifoliatus