Open-source offline translation library written in Python. Uses OpenNMT for translations

Overview

Argos Translate

Docs | Website | Video intro | GitHub | Dependencies

Open-source offline translation library written in Python. Uses OpenNMT for translations, SentencePiece for tokenization, Stanza for sentence boundary detection, and PyQt for GUI. Designed to be used as either a Python library, command-line, or GUI application. LibreTranslate is an API and web-app built on top of Argos Translate.

Argos Translate supports installing model files which are a zip archive with an ".argosmodel" extension that contains an OpenNMT CTranslate2 model, a SentencePiece tokenization model, a Stanza tokenizer model for sentence boundary detection, and metadata about the model. Pretrained models can be downloaded here.

Argos Translate also manages automatically pivoting through intermediate languages to translate between languages that don't have a direct translation between them installed. For example, if you have a es ➔ en and en ➔ fr translation installed you are able to translate from es ➔ fr as if you had that translation installed. This allows for translating between a wide variety of languages at the cost of some loss of translation quality.

Models

Supported languages

Arabic, Azerbaijani, Chinese, Czech, Dutch, English, Finnish, French, German, Greek, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Swedish, Turkish, Ukranian, Vietnamese

Screenshots

GUI

Screenshot Screenshot2 Argos Translate macOS Screenshot

Installation

Install from PyPI

Argos Translate is available from PyPI and can be easily installed or updated with pip.

pip3 install argostranslate

Install GUI:

pip3 install argostranslategui

Install from Snap Store

Argos Translate is available from the Snap Store and auto installs a content snap to support translation between Arabic, Chinese, English, French, Russian, and Spanish. Additional languages can be installed from supplementary content snaps.

With snapd installed:

sudo snap install argos-translate

Get it from the Snap Store

Automatically installs and connects to argos-translate-base-langs snap to support translations between Arabic, Chinese, English, French, Russian, and Spanish.

Additional languages can be installed from *.argosmodel files or from supplementary content snaps:

  • argos-translate-de-en - German - English
  • argos-translate-en-it - English - Italian
  • argos-translate-en-pt - English - Portuguese

To connect automatically: sudo snap connect argos-translate:argos-packages argos-translate-en-it:argos-packages

To run command line interface on Snapcraft:

argos-translate.cli --help

Installation for macOS

  1. Download the latest macOS release.
  2. Extract the archive.
  3. Copy the .app file to the Applications directory.

Python source installation

Dependencies

Requires Python3, pip (which should come with Python3), and optionally virtualenv to keep Argos Translate's dependencies separate from other Python programs you have installed.

Python Installation Instructions

On Ubuntu:

sudo apt-get update
sudo apt-get install -y python3

Install

  1. Download a copy of this repo (this requires either installing git or downloading a zip from GitHub):
git clone https://github.com/argosopentech/argos-translate.git
cd argos-translate
  1. Make a virtual environment to install into (optional):
pip3 install --upgrade virtualenv # If virtualenv not already installed
virtualenv env
source env/bin/activate
  1. Install this package with pip:
pip3 install --upgrade pip
pip3 install -e .

Graphical user interface

Note that the GUI code is in a separate repo. To install the GUI perform the same steps with the code in GUI repo after installing Argos Translate.

GPU Acceleration

To enable GPU support, you need to set the ARGOS_DEVICE_TYPE env variable to cuda or auto.

$ ARGOS_DEVICE_TYPE=cuda argos-translate --from-lang en --to-lang es "Hello World"
Hola Mundo

The above env variable instructs CTranslate2 to use cuda. if you encounter any issues with GPU, please consider reading CTranslate2 documentation.

Build and install snap package

Run Argos Translate!

GUI

Run the GUI version of Argos Translate.

argos-translate-gui

When installing with snap a .desktop file should also be installed which will make Argos Translate available from the desktop menu.

Languages are chosen as drop down choices. More languages pairs can be installed.

The left text box translates into the right box.

Example workflow translating from Vietnamese into English:

  1. Set the left drop down to Vietnamese and the right drop down to English.
  2. Replace the default text Text to translate from in the left text box with some text in Vietnamese. A quick way to do this is to click in the left text box and press the keyboard shortcut CTRL+a to select all and then CTRL+v to paste.
  3. Wait patiently.
  4. When text appears in the right text box, read the translation!

If the output looks similar to the input, try changing the origin language as some languages appear similar if you are unfamiliar with them.

Command Line

Run the command line version of Argos Translate.

argos-translate

Translate a string from English to Spanish.

Note: If you do not have the language pair that you are calling installed, you will get a Traceback error.

argos-translate --from-lang en --to-lang es "Hello World."
Hola Mundo

Translate longer text piped into argos-translate.

echo "Text to translate" | argos-translate --from-lang en --to-lang es
Texto para traducir

Adding language pair models

Language pairs are on average 100MB each.

Installing new pairs through the GUI

  1. Open Argos Translate: argos-translate-gui
  2. Click on the Manage Packages menu item.
  3. Click on the Download packages button.
  4. Click on the down arrow beside a language pair that you want to add.
  5. Wait for the hourglass icon to change into a check mark icon.
  6. Repeat the last two steps until you have all of the language pairs that you want.
  7. Click on the X in the top right to close the Download packages window.
  8. Click on the X in the top right to close the Manage Packages window.

Note: The Download packages screen does not seem to have a scroll bar so you will probably need to follow the next set of instructions to import new pairs through the GUI.

Importing new pairs through the GUI

  1. Download or make new pairs. Model links can be downloaded from this page or this JSON file.
  2. Open Argos Translate: argos-translate-gui
  3. Click on the Manage Packages menu item.
  4. Click on the Install package file button.
  5. Navigate to where you downloaded the new language pairs, click on the .argosmodel file, and click on the Open button.
  6. Repeat the last two steps until you have all of the language pairs that you want.
  7. Click on the X in the top right to close the Manage Packages window.

Importing new pairs through the CLI

  1. Update list of available language pairs: argospm update
  2. List all available language pairs: argospm search
  3. Install new pair syntax: argospm install *lang_pair_name*

For example, install Turkish to English pair: argospm install translate-tr_en

Optionally, you could install all language pairs using BASH.

for i in $(argospm search | sed 's/:.*$//g'); do argospm install $i ; done

Removing a pair through the GUI

  1. Open Argos Translate: argos-translate-gui
  2. Click on the Manage Packages menu item.
  3. Click on the trash can icon besides the pair you want to remove.
  4. Click on the X in the top right to close the Manage Packages window.

Removing a pair through the CLI

  1. Remove the Turkish to English pair: argospm remove translate-tr_en

Optionally, you could remove all language pairs using BASH if you need to free space fast.

for i in $(argospm list); do argospm remove $i ; done

Examples

Python

>> '¡Hola Mundo!'">
from argostranslate import package, translate
package.install_from_path('en_es.argosmodel')
installed_languages = translate.get_installed_languages()

>>> [str(lang) for lang in installed_languages]
>>> ['English', 'Spanish']

translation_en_es = installed_languages[0].get_translation(installed_languages[1])
translation_en_es.translate("Hello World!")

>>> '¡Hola Mundo!'

LibreTranslate Web App (Demo)

Web App Screenshot

LibreTranslate API

const res = await fetch("https://libretranslate.com/translate", {
	method: "POST",
	body: JSON.stringify({
		q: "Hello!",
		source: "en",
		target: "es"
	}),
	headers: {
		"Content-Type": "application/json"}
	});

console.log(await res.json());

{
    "translatedText": "¡Hola!"
}

HTML Translation

The translate-html library is built on top of Argos Translate and Beautiful Soup and parses and translates HTML. The LibreTranslate API also has support for translating HTML.

Files Translation

The argos-translate-files library is built on top of Argos Translate and parses and translates files. The LibreTranslate API also has support for translating files.

Uninstall

If you want to uninstall Argos Translate, you may choose the applicable method.

Uninstall PYPI/pip package

If you installed Argos Translate via pip you can uninstall it using

python3 -m pip uninstall argostranslate

You may choose to also delete temporary and cached files:

rm -r ~/.local/cache/argos-translate
rm -r ~/.local/share/argos-translate

Uninstall Snap

The following command will uninstall the snap package.

sudo snap remove argos-translate argos-translate-base-langs

If you installed additional language packs, you might want to remove them as well, e.g.

sudo snap remove argos-translate-de-en

Contributing

Awesome Humane Tech

Contributions are welcome! Available issues are on the GitHub issues page.

Support

For support use GitHub Issues, Github Discussions, or the LibreTranslate Forum.

Custom models

Custom models trained on your own data are available for $1000/each (negotiable).

Donations

If you find this software useful donations are appreciated.

  • GitHub Sponsor
  • PayPal
  • Ethereum: 0x4E9017d8e275cA54C91E056381DAb9fe6ECC1AF6
  • Bitcoin cash: qzqklgjpgutdqqlhcasmdd2hkqcelw426sxzk5qtne
  • Bitcoin: 16UJrmSEGojFPaqjTGpuSMNhNRSsnspFJT
  • Filecoin: f1nrnpmjxn27amidyiqrzq5mxihdo2trh2oijw2sq
  • Basic Attention Token: 0x8a16f26D277f924B04FCA5ECec64b76B5410A06c
  • Cheap Eth: 0x996133E61b81c300a37ACa9b24898685eB872b61

Paid supporters receive priority support.

License

Dual licensed under either the MIT License or CC0.

Issues
  • Compilation error in FreeBSD

    Compilation error in FreeBSD

    I'd like to compile argos-translate in a FreeBSD system. It's supposed to be installed locally. Perhaps there are some packages missing that could be installed globally. Is there a list of packages to be installed, besides Python 3.8 and Qt5?

    python3 -m pip install argostranslate
    Defaulting to user installation because normal site-packages is not writeable
    Collecting argostranslate
      Using cached argostranslate-1.5.2-py3-none-any.whl (30 kB)
    Collecting PyQt5==5.15.4
      Using cached PyQt5-5.15.4.tar.gz (3.3 MB)
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
        Preparing wheel metadata ... error
        ERROR: Command errored out with exit status 1:
         command: /usr/local/bin/python3 /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp4cdt4aal
             cwd: /tmp/pip-install-953r8nsq/pyqt5_46a027e9f4f24884bf1b30ba03152645
        Complete output (31 lines):
        Traceback (most recent call last):
          File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 126, in prepare_metadata_for_build_wheel
            hook = backend.prepare_metadata_for_build_wheel
        AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'
    
        During handling of the above exception, another exception occurred:
    
        Traceback (most recent call last):
          File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
            main()
          File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
          File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 130, in prepare_metadata_for_build_wheel
            return _get_wheel_metadata_from_wheel(backend, metadata_directory,
          File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 159, in _get_wheel_metadata_from_wheel
            whl_basename = backend.build_wheel(metadata_directory, config_settings)
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/sipbuild/api.py", line 51, in build_wheel
            project = AbstractProject.bootstrap('pep517')
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/sipbuild/abstract_project.py", line 83, in bootstrap
            project.setup(pyproject, tool, tool_description)
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/sipbuild/project.py", line 484, in setup
            self.apply_user_defaults(tool)
          File "project.py", line 63, in apply_user_defaults
            super().apply_user_defaults(tool)
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/pyqtbuild/project.py", line 70, in apply_user_defaults
            super().apply_user_defaults(tool)
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/sipbuild/project.py", line 230, in apply_user_defaults
            self.builder.apply_user_defaults(tool)
          File "/tmp/pip-build-env-jmsxirts/overlay/lib/python3.8/site-packages/pyqtbuild/builder.py", line 66, in apply_user_defaults
            raise PyProjectOptionException('qmake',
        sipbuild.pyproject.PyProjectOptionException
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/8e/a4/d5e4bf99dd50134c88b95e926d7b81aad2473b47fde5e3e4eac2c69a8942/PyQt5-5.15.4.tar.gz#sha256=2a69597e0dd11caabe75fae133feca66387819fc9bc050f547e5551bce97e5be (from https://pypi.org/simple/pyqt5/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/local/bin/python3 /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp4cdt4aal Check the logs for full command output.
    
    
    opened by lpvm 21
  • Massively repeated characters on MacOS

    Massively repeated characters on MacOS

    argos-translate --from-lang en --to-lang ru "Hello World"
    >>>  З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З! З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З! З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З З
    

    or another example

    argos-translate --from-lang en --to-lang ru "Good to see you"
    >>>  У! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До! До!
    

    tried script agro-translate.py:

    import argostranslate
    from argostranslate import package, translate
    package.install_from_path('translate_en_ru.argosmodel')
    
    installed_languages = translate.get_installed_languages()
    for lang in installed_languages:
      print(lang)
     
    translation_en_ru = installed_languages[0].get_translation(installed_languages[1])
    translated = translation_en_ru.translate("How are you!")
    print(translated)
    

    and got:

    python agro-translate.py 
    >>> English
    >>> Russian
    >>> ?!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    

    what did i do wrong? Mac OS 11.1 Big Sur

    opened by exvayn 21
  • Command-line Usage

    Command-line Usage

    Would it be possible to support command-line usage? I searched the documentation but found nothing. I would like to automate translating texts and also text files into multiple languages.

    As an example I suggest the following: argos-translate -text "Hello World!" -from en -to de argos-translate -file Novel.txt -from en -to de

    enhancement help wanted 
    opened by fleschutz 12
  • Better model distribution

    Better model distribution

    Currently models are distributed by Google Drive (not ideal) and a slow BitTorrent, so there's lots of room for improvement:

    • More Torrent seeders
    • Create individual torrent files for each model
    • HTTP or FTP mirrors
    • I avoided git distribution because I was worried about running into GitHub limits but we may want to link to the LibreTranslate Git Mirror
    • Open to other ideas too

    The plan was to make a separate repo for storing model distribution information so let me know if your interested.

    enhancement help wanted good first issue 
    opened by PJ-Finlay 10
  • Add support Persian language

    Add support Persian language

    Is it possible to add support for Persian language?

    opened by peylight 10
  • OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already intialized

    OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already intialized

    Hi, Thanks for the amazing project. This was a fresh install on a new python 3.8.3 virtual environment using pip install and launching straight away. The web app launches, but after a few key strokes, the process crashes with the following error logged to the terminal: OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/. Abort trap: 6

    Any advice appreciated here, thanks again.

    opened by nikohunt 10
  • Better Handle HTML input

    Better Handle HTML input

    enhancement help wanted 
    opened by PJ-Finlay 8
  • Translation error

    Translation error

    opened by dingedi 8
  • Potential regression after updating to the latest version

    Potential regression after updating to the latest version

    I've noticed different output from LibreTranslate and the latest version of argos-translate for Chinese language. LibreTranslate uses argostranslate==1.1.4.

    I've tested this problem locally and got the same results. It looks like a regression to me. Both versions are using the same model.

    Original sentence (in English, translated using GoogleTranslate):

    Hi there, this is a Chinese sentence. How are you doing today?
    

    argostranslate==1.3.0

     echo "嗨,这是中文句子。 你今天过得怎么样?" | argos-translate --from-lang zh --to-lang en
    Pierre, the Chinese sentence. What you have done today?
    

    LibreTranslate (argostranslate==1.1.4):

    This is the Chinese sentence. What is you doing today?
    
    opened by rushter 7
  • Question about non-deterministic results

    Question about non-deterministic results

    While investigating using argos-translate as a library, I have noticed non-deterministic results when translating a short test string "Hello world!" using your pre-trained models. For English -> Russian, it returns "Здравствуй мир!" on some hosts, and "Здравствуй!" on others. The results on a given host are deterministic on repeated runs and environments (at least in my testing so far).

    I first tried to follow the advice here thinking it could be a random seed issue to no avail: https://github.com/OpenNMT/OpenNMT-py/issues/392 https://github.com/pytorch/pytorch/issues/7068#issuecomment-484918113

    I was not able to determine any significant differences between hosts (both running on cpu), and the output of ct2_verbose is identical:

    [ct2_verbose] CPU: GenuineIntel (SSE4.1=true, AVX=true, AVX2=true)
    [ct2_verbose] Selected CPU ISA: AVX2
    [ct2_verbose] Use Intel MKL: true
    [ct2_verbose] SGEMM CPU backend: MKL
    [ct2_verbose] GEMM_S16 CPU backend: MKL
    [ct2_verbose] GEMM_S8 CPU backend: MKL (u8s8 preferred: true)
    [ct2_verbose] Use packed GEMM: false
    

    Manually setting num_hypotheses=2 in the ctranslate2 Translator shows that it appears to be a score difference:

    Host #1:
    	{'tokens': ['▁З', 'д', 'рав', 'ству', 'й', '!'], 'score': -2.7840166091918945}
    	{'tokens': ['▁З', 'д', 'рав', 'ству', 'й', '▁мир', '!'], 'score': -2.841048240661621}
    
    Host #2:
    	{'tokens': ['▁З', 'д', 'рав', 'ству', 'й', '▁мир', '!'], 'score': -2.7670412063598633}
    	{'tokens': ['▁З', 'д', 'рав', 'ству', 'й', '!'], 'score': -2.7944717407226562}
    

    Setting beam_size=1 so it uses greedy search did produce the same result on both hosts, but I don't think that is a valid solution.

    I created a gist to provide some debugging output, and didn't notice any difference in the actual argos-translate parsing logic, so it seems to be much deeper: https://gist.github.com/mikemoritz/a5bf76193ccb16d018a1af9ec584fb41

    My questions are:

    • Are there other options you would recommend setting to increase the likelihood of deterministic results? If so, could these be surfaced as options within argos-translate?
    • Is it possible that "Hello world!" is a bad test string? If so, do you have any recommendations?
    • Do you think it could still be a random seed issue that may need to be implemented within argos-translate?
    • Is there additional debugging within ctranslate2 and/or torch that you would recommend to highlight differences between the hosts?

    Thanks!

    enhancement 
    opened by mikemoritz 6
  • Poor translation?

    Poor translation?

    I seem to get some completely unrelated translation results. I don’t know if my use method is wrong or the model effect is poor.

    opened by CatchDr 0
  • [Meta] How are language variations requested?

    [Meta] How are language variations requested?

    So #91 is about requesting languages, but how would I request a certain script? For example, Mandarin is is already implemented as "Chinese", but it defaults to simplified characters instead of the normal ones. Furthermore, if someone requests Cantonese, how would this be implemented (normal characters or the simplified ones?)

    I am not sure what other languages exist out there, but I guess there must be others where the phonetic is the same but the written part can differ...

    opened by null-von-sushi 0
  • [Feature request] gui: add option to sync scroll-bar on re-translation

    [Feature request] gui: add option to sync scroll-bar on re-translation

    While editing/typing a lot of text (cq. a scroll bar is appearing) the translated scroll bar is not in sync with the translated side which makes it difficult to track changes.

    Add an option to sync them on re-translation

    opened by bwakkie 0
  • ValueError when tokenize some inputs using `Vietnamese → English`

    ValueError when tokenize some inputs using `Vietnamese → English`

    A simple example:

    from argostranslate.translate import get_installed_languages
    
    languages_list = get_installed_languages()
    languages = {l.code: l for l in languages_list}
    
    trans = languages['vi'].get_translation(languages['en'])
    
    text = 'thuc luc di em trai <@!12345>'
    res = trans.translate(text)
    

    output:

    Traceback (most recent call last):
      File "test.py", line 9, in <module>
        res = trans.translate(text)
      File "/home/username/.local/lib/python3.7/site-packages/argostranslate/translate.py", line 52, in translate
        return self.hypotheses(input_text, num_hypotheses=1)[0].value
      File "/home/username/.local/lib/python3.7/site-packages/argostranslate/translate.py", line 275, in hypotheses
        paragraph, num_hypotheses
      File "/home/username/.local/lib/python3.7/site-packages/argostranslate/translate.py", line 160, in hypotheses
        self.pkg, paragraph, self.translator, num_hypotheses
      File "/home/username/.local/lib/python3.7/site-packages/argostranslate/translate.py", line 385, in apply_packaged_translation
        stanza_sbd = stanza_pipeline(input_text)
      File "/home/username/.local/lib/python3.7/site-packages/stanza/pipeline/core.py", line 166, in __call__
        doc = self.process(doc)
      File "/home/username/.local/lib/python3.7/site-packages/stanza/pipeline/core.py", line 160, in process
        doc = self.processors[processor_name].process(doc)
      File "/home/username/.local/lib/python3.7/site-packages/stanza/pipeline/tokenize_processor.py", line 88, in process
        no_ssplit=self.config.get('no_ssplit', False))
      File "/home/username/.local/lib/python3.7/site-packages/stanza/models/tokenize/utils.py", line 165, in output_predictions
        st0 = text.index(part, char_offset) - char_offset
    ValueError: substring not found
    

    The bug only occurs for vi->en, thus should be related to the model used by stanza.

    opened by AutumnSun1996 1
  • Markdown and AsciiDOC Translations support

    Markdown and AsciiDOC Translations support

    It would be nice to have Translations for Markdown and Asciidoc. I understand we can use translation-html, pre translation ie. prior to HTML would be better for review and approval can be done on translations.

    enhancement help wanted 
    opened by srbala 2
  • Segmentation fault, memory error? [There appear to be 1 leaked semaphore objects]

    Segmentation fault, memory error? [There appear to be 1 leaked semaphore objects]

    Hey there! First of all, thanks for your fantastic work here. Really love this project 😄

    I'm trying to run the python library of argos, something like doing this in my code:

    installed_languages = translate.get_installed_languages()
    print([str(lang) for lang in installed_languages])
    
    print("Installed argos languages:")
    fr = [lang for lang in installed_languages if lang.name == "French"][-1]
    en = [lang for lang in installed_languages if lang.name == "English"][-1]
    es = [lang for lang in installed_languages if lang.name == "Spanish"][-1]
    breakpoint()
    translation_fr_en = fr.get_translation(en)
    print(translation_fr_en.translate("Bonjour monde!"))
    

    But I'm facing some issues.

    To be more specific, I'm actually running with the Kedro framework inside of a pipeline module.

    When running the above code, I got the following error:

    23642 segmentation fault  kedro run --pipeline translate
    

    Upon closer inspection, when setting up breakpoints, the following happens:

    (Pdb) print(translation_fr_en.translate("Bonjour monde!"))
    [1]    23698 segmentation fault  kedro run --pipeline translate_news
    /Users/--/python/3.8.9/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown              
      warnings.warn('resource_tracker: There appear to be %d '
    

    Do you have any idea of what this could be? I've bashed my head quite a lot, but figuring there's something wrong?

    In a notebook, I can run the following code no problem...

    opened by duarteocarmo 7
  • Repetition penalty

    Repetition penalty

    opened by PJ-Finlay 6
  • Any language to English lowercase problem

    Any language to English lowercase problem

    Hi,

    When argostranslate translates from any language to English, always return the text in lowercase, this causes that if you do the translation back (English to the initial language) to be erroneous. Actually it works correctly from English to any language.

    For example, translations from Spanish to English Mi móvil es Apple => My mobile is apple and translation back is My mobile is apple => Mi móvil es manzana

    Correct translation should be Mi móvil es Apple => My mobile is Apple with the translation back My mobile is Apple => Mi móvil es Apple

    This occurs with any text that has uppercase characters. This is just an example.

    Fix this issue will improve a lot translations.

    Thanks!

    opened by linkses 0
  • Set default language in GUI

    Set default language in GUI

    • https://github.com/argosopentech/argos-translate/discussions/163
    • https://github.com/argosopentech/argos-translate/issues/24
    enhancement help wanted good first issue 
    opened by PJ-Finlay 1
  • Snapcraft CLI breaks when translating special characters

    Snapcraft CLI breaks when translating special characters

    $ argos-translate.cli --from-lang en --to-lang fr "Hello World"
    Traceback (most recent call last):
      File "/snap/argos-translate/10/bin/argos-translate", line 5, in <module>
        cli.main()
      File "/snap/argos-translate/10/lib/python3.6/site-packages/argostranslate/cli.py", line 60, in main
        print(translation.translate(text_to_translate))
    UnicodeEncodeError: 'ascii' codec can't encode character '\xe0' in position 8: ordinal not in range(128)
    

    https://github.com/argosopentech/argos-translate/discussions/163

    bug help wanted 
    opened by PJ-Finlay 0
Open Source Neural Machine Translation in PyTorch

OpenNMT-py: Open-Source Neural Machine Translation OpenNMT-py is the PyTorch version of the OpenNMT project, an open-source (MIT) neural machine trans

OpenNMT 5.4k Jan 10, 2022
Open Source Neural Machine Translation in PyTorch

OpenNMT-py: Open-Source Neural Machine Translation OpenNMT-py is the PyTorch version of the OpenNMT project, an open-source (MIT) neural machine trans

OpenNMT 4.8k Feb 18, 2021
voice2json is a collection of command-line tools for offline speech/intent recognition on Linux

Command-line tools for speech and intent recognition on Linux

Michael Hansen 881 Jan 11, 2022
Partially offline multi-language translator built upon Huggingface transformers.

Translate Command-line interface to translation pipelines, powered by Huggingface transformers. This tool can download translation models, and then us

Richard Jarry 6 Nov 13, 2021
Utility for Google Text-To-Speech batch audio files generator. Ideal for prompt files creation with Google voices for application in offline IVRs

Google Text-To-Speech Batch Prompt File Maker Are you in the need of IVR prompts, but you have no voice actors? Let Google talk your prompts like a pr

Ponchotitlán 1 Aug 19, 2021
This is the offline-training-pipeline for our project.

offline-training-pipeline This is the offline-training-pipeline for our project. We adopt the offline training and online prediction Machine Learning

null 1 Jan 8, 2022
Training open neural machine translation models

Train Opus-MT models This package includes scripts for training NMT models using MarianNMT and OPUS data for OPUS-MT. More details are given in the Ma

Language Technology at the University of Helsinki 100 Jan 6, 2022
Python-zhuyin - An open source Python library that provides a unified interface for converting between Chinese pinyin and Zhuyin (bopomofo)

Python-zhuyin - An open source Python library that provides a unified interface for converting between Chinese pinyin and Zhuyin (bopomofo)

null 2 Jan 13, 2022
An open source library for deep learning end-to-end dialog systems and chatbots.

DeepPavlov is an open-source conversational AI library built on TensorFlow, Keras and PyTorch. DeepPavlov is designed for development of production re

Neural Networks and Deep Learning lab, MIPT 5.6k Jan 16, 2022
An open-source NLP research library, built on PyTorch.

An Apache 2.0 NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks. Quic

AI2 10.7k Jan 12, 2022
An open source library for deep learning end-to-end dialog systems and chatbots.

DeepPavlov is an open-source conversational AI library built on TensorFlow, Keras and PyTorch. DeepPavlov is designed for development of production re

Neural Networks and Deep Learning lab, MIPT 5.6k Jan 11, 2022
An open-source NLP research library, built on PyTorch.

An Apache 2.0 NLP research library, built on PyTorch, for developing state-of-the-art deep learning models on a wide variety of linguistic tasks. Quic

AI2 9.7k Feb 18, 2021
An open source library for deep learning end-to-end dialog systems and chatbots.

DeepPavlov is an open-source conversational AI library built on TensorFlow, Keras and PyTorch. DeepPavlov is designed for development of production re

Neural Networks and Deep Learning lab, MIPT 5k Feb 18, 2021
A deep learning-based translation library built on Huggingface transformers

DL Translate A deep learning-based translation library built on Huggingface transformers and Facebook's mBART-Large ?? GitHub Repository ?? Documentat

Xing Han Lu 185 Jan 13, 2022
Code for papers "Generation-Augmented Retrieval for Open-Domain Question Answering" and "Reader-Guided Passage Reranking for Open-Domain Question Answering", ACL 2021

This repo provides the code of the following papers: (GAR) "Generation-Augmented Retrieval for Open-domain Question Answering", ACL 2021 (RIDER) "Read

morning 29 Dec 28, 2021
open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

中文开放信息抽取系统, open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

null 6 Sep 23, 2021
Saptak Bhoumik 15 Jan 13, 2022
nlpcommon is a python Open Source Toolkit for text classification.

nlpcommon nlpcommon, Python Text Tool. Guide Feature Install Usage Dataset Contact Cite Reference Feature nlpcommon is a python Open Source

xuming 2 Dec 31, 2021