Healthsea is a spaCy pipeline for analyzing user reviews of supplementary products for their effects on health.

Overview

Welcome to Healthsea

Create better access to health with spaCy.

Healthsea is a pipeline for analyzing user reviews to supplement products by extracting their effects on health.

Learn more about Healthsea in our blog post!

💉 Creating better access to health

Healthsea aims to analyze user-written reviews of supplements in relation to their effects on health. Based on this analysis, we try to provide product recommendations. For many people, supplements are an addition to maintaining health and achieving personal goals. Due to their rising popularity, consumers have increasing access to a variety of products.

However, it's likely that most of the products on the market are redundant or produced in a "quantity over quality" fashion to maximize profit. The resulting white noise of products makes it hard to find the right supplements.

Healthsea automizes the analysis and provides information in a more digestible way.


🟢 Requirements

To run this project you need:

spacy>=3.2.0
benepar>=0.2.0
torch>=1.6.0
spacy-transformers>=1.1.2

You can install them in the project folder via spacy project run install

📖 Documentation

Documentation
🧭 Usage How to use the pipeline
⚙️ Pipeline Learn more about the architecture of the pipeline
🪐 spaCy project Introduction to the spaCy project
Demos Introduction to the Healthsea demos

🧭 Usage

The pipeline processes reviews to supplements and returns health effects for every found health aspect.

You can either train the pipeline yourself with the provided datasets in the spaCy project or directly download the trained Healthsea pipeline from Huggingface via pip install https://huggingface.co/explosion/en_healthsea/resolve/main/en_healthsea-any-py3-none-any.whl

import spacy

nlp = spacy.load("en_healthsea")
doc = nlp("This is great for joint pain.")

# Clause Segmentation & Blinding
print(doc._.clauses)

>    {"split_indices": [0, 7],
>    "has_ent": true,
>    "ent_indices": [4, 6],
>    "blinder": "_CONDITION_",
>    "ent_name": "joint pain",
>    "cats": {
>        "POSITIVE": 0.9824668169021606,
>        "NEUTRAL": 0.017364952713251114,
>        "NEGATIVE": 0.00002889777533710003,
>        "ANAMNESIS": 0.0001394189748680219
>    },
>    "prediction_text": ["This", "is", "great", "for", "_CONDITION_", "!"]}

# Aggregated results
print(doc._.health_effects)

>    {"joint_pain": {
>        "effects": ["POSITIVE"],
>        "effect": "POSITIVE",
>        "label": "CONDITION",
>        "text": "joint pain"
>    }}


⚙️ Pipeline

The pipeline consists of the following components:

pipeline = [sentencizer, tok2vec, ner, benepar, segmentation, clausecat, aggregation]

It uses Named Entity Recognition to detect two types of entities Condition and Benefit.

Condition entities are defined as health aspects that are improved by decreasing them. They include diseases, symptoms and general health problems (e.g. pain in back). Benefit entities on the other hand, are desired states of health (muscle recovery, glowing skin) that improve by increasing them.

The pipeline uses a modified model that performs Clause Segmentation based on the benepar parser, Entity Blinding and Text Classification. It predicts four exclusive effects: Positive, Negative, Neutral, and Anamnesis.


🪐 spaCy project

The project folder contains a spaCy project with all the training data and workflows.

Use spacy project run inside the project folder to get an overview of all commands and assets. For more detailed documentation, visit the project folders readme.

Use spacy project run install to install dependencies needed for the pipeline.

Demo

Healthsea Demo

A demo for exploring the results of Healthsea on real data can be found at Hugging Face Spaces.

Healthsea Pipeline

A demo for exploring the Healthsea pipeline with its individual processing steps can be found at Hugging Face Spaces.

You might also like...
✨Fast Coreference Resolution in spaCy with Neural Networks
✨Fast Coreference Resolution in spaCy with Neural Networks

✨ NeuralCoref 4.0: Coreference Resolution in spaCy with Neural Networks. NeuralCoref is a pipeline extension for spaCy 2.1+ which annotates and resolv

🛸 Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy

spacy-transformers: Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy This package provides spaCy components and architectures to use tr

spaCy plugin for Transformers , Udify, ELmo, etc.

Camphr - spaCy plugin for Transformers, Udify, Elmo, etc. Camphr is a Natural Language Processing library that helps in seamless integration for a wid

NLP, before and after spaCy

textacy: NLP, before and after spaCy textacy is a Python library for performing a variety of natural language processing (NLP) tasks, built on the hig

✨Fast Coreference Resolution in spaCy with Neural Networks
✨Fast Coreference Resolution in spaCy with Neural Networks

✨ NeuralCoref 4.0: Coreference Resolution in spaCy with Neural Networks. NeuralCoref is a pipeline extension for spaCy 2.1+ which annotates and resolv

🛸 Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy

spacy-transformers: Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy This package provides spaCy components and architectures to use tr

spaCy plugin for Transformers , Udify, ELmo, etc.

Camphr - spaCy plugin for Transformers, Udify, Elmo, etc. Camphr is a Natural Language Processing library that helps in seamless integration for a wid

SpikeX - SpaCy Pipes for Knowledge Extraction

SpikeX is a collection of pipes ready to be plugged in a spaCy pipeline. It aims to help in building knowledge extraction tools with almost-zero effort.

Augmenty is an augmentation library based on spaCy for augmenting texts.
Augmenty is an augmentation library based on spaCy for augmenting texts.

Augmenty: The cherry on top of your NLP pipeline Augmenty is an augmentation library based on spaCy for augmenting texts. Besides a wide array of high

Comments
  • Minor change in the code

    Minor change in the code

    Hi!I noticed that in the following line of code in the preprocess_clausecat.py file, at line 61 in the for loop while splitting the dataset into train and test set

    for label in label_dict:
            split = int(len(label_dict[label]) * eval_split)
            train += label_dict[label][split:]
            dev += label_dict[label][:split]
            checksum += len(label_dict[label])
            table_data.append(
                (
                    label,
                    len(label_dict[label]),
                    len(label_dict[label][split:]),
                    len(label_dict[label][:split]),
                )
            )
    

    the train and dev assignment statements need to be interchanged. As per the existing assignment, The train set has fewer samples than the dev set. Shouldn't it be the other way round? Something like this?

    train += label_dict[label][:split]
    dev += label_dict[label][split:]
    
    opened by shrinidhin 2
  • Case with 2 conditions mix reviews

    Case with 2 conditions mix reviews

    Sample input: Yam is great for joint pain but bad for liver disease Code below:

    `import spacy
    
    nlp = spacy.load("en_healthsea")
    doc = nlp("Yam is great for joint pain but bad for liver disease")
    
    print(doc._.clauses)
    print(doc._.health_effects)`
    

    Output below:

    [{'split_indices': (0, 11), 'has_ent': True, 'ent_indices': (4, 6), 'blinder': '_CONDITION_', 'ent_name': 'joint pain', 'cats': {'POSITIVE': 0.9715917110443115, 'NEUTRAL': 0.004003751091659069, 'NEGATIVE': 0.01367472019046545, 'ANAMNESIS': 0.010729866102337837}, 'prediction_text': ['Yam', 'is', 'great', 'for', '_CONDITION_', 'but', 'bad', 'for', 'liver', 'disease']}, {'split_indices': (0, 11), 'has_ent': True, 'ent_indices': (9, 11), 'blinder': '_CONDITION_', 'ent_name': 'liver disease', 'cats': {'POSITIVE': 0.9650901556015015, 'NEUTRAL': 0.004530671052634716, 'NEGATIVE': 0.023644359782338142, 'ANAMNESIS': 0.0067347269505262375}, 'prediction_text': ['Yam', 'is', 'great', 'for', 'joint', 'pain', 'but', 'bad', 'for', '_CONDITION_']}]

    Expected 1 positive and another negative. Actually found 2 positive sentiments.

    opened by onlyrohits 2
  • Invalid links on HF Model card

    Invalid links on HF Model card

    opened by davidgibsonp 1
  • ValueError: [E853] Unsupported component factory name 'healthsea.aggregation.v1'. The character '.' is not permitted in factory names.

    ValueError: [E853] Unsupported component factory name 'healthsea.aggregation.v1'. The character '.' is not permitted in factory names.

    These are the errors I get when I run this code:

    import spacy
    
    nlp = spacy.load("en_healthsea")
    doc = nlp("This is great for joint pain")
    print(doc._.health_effects)
    

    These errors happen when I install spaCy 3.4.4

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    en-healthsea 0.0.1 requires spacy<3.3.0,>=3.2.0, but you have spacy 3.4.4 which is incompatible.
    
    Traceback (most recent call last):
      File "/home/my_pc/scispaCy/spaCy lessons/Entity_recognition.py", line 24, in <module>
        nlp = spacy.load("en_healthsea")
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/__init__.py", line 54, in load
        return util.load_model(
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 432, in load_model
        return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 467, in load_model_from_package
        cls = importlib.import_module(name)
      File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/home/vasya/.local/lib/python3.10/site-packages/en_healthsea/__init__.py", line 4, in <module>
        from . import clause_aggregation
      File "/home/vasya/.local/lib/python3.10/site-packages/en_healthsea/clause_aggregation.py", line 6, in <module>
        @Language.factory("healthsea.aggregation.v1")
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/language.py", line 470, in factory
        raise ValueError(Errors.E853.format(name=name))
    ValueError: [E853] Unsupported component factory name 'healthsea.aggregation.v1'. The character '.' is not permitted in factory names.
    

    These errors happen when I install spaCy 3.2.0

    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    spacy-transformers 1.1.9 requires spacy<4.0.0,>=3.4.0, but you have spacy 3.2.0 which is incompatible.
    en-core-web-sm 3.4.1 requires spacy<3.5.0,>=3.4.0, but you have spacy 3.2.0 which is incompatible.
    en-core-web-md 3.4.1 requires spacy<3.5.0,>=3.4.0, but you have spacy 3.2.0 which is incompatible.
    en-core-web-lg 3.4.1 requires spacy<3.5.0,>=3.4.0, but you have spacy 3.2.0 which is incompatible.
    
    Traceback (most recent call last):
      File "/home/my_pc/scispaCy/spaCy lessons/Entity_recognition.py", line 24, in <module>
        nlp = spacy.load("en_healthsea")
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/__init__.py", line 51, in load
        return util.load_model(
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 420, in load_model
        return load_model_from_package(name, **kwargs)  # type: ignore[arg-type]
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 453, in load_model_from_package
        return cls.load(vocab=vocab, disable=disable, exclude=exclude, config=config)  # type: ignore[attr-defined]
      File "/home/vasya/.local/lib/python3.10/site-packages/en_healthsea/__init__.py", line 14, in load
        return load_model_from_init_py(__file__, **overrides)
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 615, in load_model_from_init_py
        return load_model_from_path(
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 488, in load_model_from_path
        nlp = load_model_from_config(config, vocab=vocab, disable=disable, exclude=exclude)
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/util.py", line 525, in load_model_from_config
        nlp = lang_cls.from_config(
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/language.py", line 1755, in from_config
        nlp = lang_cls(vocab=vocab, create_tokenizer=create_tokenizer, meta=meta)
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy/language.py", line 162, in __init__
        util.registry._entry_point_factories.get_all()
      File "/home/vasya/.local/lib/python3.10/site-packages/catalogue/__init__.py", line 119, in get_all
        result.update(self.get_entry_points())
      File "/home/vasya/.local/lib/python3.10/site-packages/catalogue/__init__.py", line 134, in get_entry_points
        result[entry_point.name] = entry_point.load()
      File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
        module = import_module(match.group('module'))
      File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/__init__.py", line 1, in <module>
        from . import architectures
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/architectures.py", line 6, in <module>
        from .layers import TransformerModel, TransformerListener
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/layers/__init__.py", line 1, in <module>
        from .listener import TransformerListener
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/layers/listener.py", line 4, in <module>
        from ..data_classes import TransformerData
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/data_classes.py", line 12, in <module>
        from .util import transpose_list
      File "/home/vasya/.local/lib/python3.10/site-packages/spacy_transformers/util.py", line 9, in <module>
        from thinc.api import get_torch_default_device
    ImportError: cannot import name 'get_torch_default_device' from 'thinc.api' (/home/vasya/.local/lib/python3.10/site-packages/thinc/api.py)
    
    opened by Researcher-123 0
Owner
Explosion
A software company specializing in developer tools for Artificial Intelligence and Natural Language Processing
Explosion
Spacy-ginza-ner-webapi - Named Entity Recognition API with spaCy and GiNZA

Named Entity Recognition API with spaCy and GiNZA I wrote a blog post about this

Yuki Okuda 3 Feb 27, 2022
A full spaCy pipeline and models for scientific/biomedical documents.

This repository contains custom pipes and models related to using spaCy for scientific documents. In particular, there is a custom tokenizer that adds

AI2 1.3k Jan 3, 2023
A full spaCy pipeline and models for scientific/biomedical documents.

This repository contains custom pipes and models related to using spaCy for scientific documents. In particular, there is a custom tokenizer that adds

AI2 831 Feb 17, 2021
DaCy: The State of the Art Danish NLP pipeline using SpaCy

DaCy: A SpaCy NLP Pipeline for Danish DaCy is a Danish preprocessing pipeline trained in SpaCy. At the time of writing it has achieved State-of-the-Ar

Kenneth Enevoldsen 71 Jan 6, 2023
A machine learning model for analyzing text for user sentiment and determine whether its a positive, neutral, or negative review.

Sentiment Analysis on Yelp's Dataset Author: Roberto Sanchez, Talent Path: D1 Group Docker Deployment: Deployment of this application can be found her

Roberto Sanchez 0 Aug 4, 2021
NLP-based analysis of poor Chinese movie reviews on Douban

douban_embedding 豆瓣中文影评差评分析 1. NLP NLP(Natural Language Processing)是指自然语言处理,他的目的是让计算机可以听懂人话。 下面是我将2万条豆瓣影评训练之后,随意输入一段新影评交给神经网络,最终AI推断出的结果。 "很好,演技不错

null 3 Apr 15, 2022
Product-Review-Summarizer - Created a product review summarizer which clustered thousands of product reviews and summarized them into a maximum of 500 characters, saving precious time of customers and helping them make a wise buying decision.

Product-Review-Summarizer - Created a product review summarizer which clustered thousands of product reviews and summarized them into a maximum of 500 characters, saving precious time of customers and helping them make a wise buying decision.

Parv Bhatt 1 Jan 1, 2022
A CRM department in a local bank works on classify their lost customers with their past datas. So they want predict with these method that average loss balance and passive duration for future.

Rule-Based-Classification-in-a-Banking-Case. A CRM department in a local bank works on classify their lost customers with their past datas. So they wa

ÖMER YILDIZ 4 Mar 20, 2022
NLP, before and after spaCy

textacy: NLP, before and after spaCy textacy is a Python library for performing a variety of natural language processing (NLP) tasks, built on the hig

Chartbeat Labs Projects 2k Jan 4, 2023
NLP, before and after spaCy

textacy: NLP, before and after spaCy textacy is a Python library for performing a variety of natural language processing (NLP) tasks, built on the hig

Chartbeat Labs Projects 1.6k Feb 10, 2021