Injector/automatic translator (using deepL API) for Tsukihime Remake

Overview

deepLuna

Extractor/Editor/Translator/Injector for Tsukihime Remake

About

deepLuna, from "deepL", the machine translation service, and "Luna", the name of the Roman Moon goddess, is a Python script with a GUI interface that works as an extractor/editor/translator (with deepL API)/injector for the text of the Tsukihime Remake game on Switch and possibly on PS4, while taking into account the whole internal structure of the script.

Installation

You just need a regular Python distribution, for example the last one from https://www.python.org/downloads/, and then launch the script deepLuna.py using a Python IDE, for example IDLE that you get by default, or some other (Spyder, Pyzo, etc.).

Usage

Preprocessing steps

  1. Dump/Find the NSP Tsukihime Switch ROM as well as the relevant keys (prod.keys and title.keys)
  2. Extract the NSP and the resulting (biggest) NCA file using for example hactool (https://github.com/SciresM/hactool)
  3. In the romfs folder, copy the script_text.mrg and allscr.mrg files and paste them in the same folder as deepLuna.py.

Extracting the text (first compulsory step)

After launching deepLuna, click on "Extract text" and wait until the extraction finishes.

IMPORTANT NOTE: This might take a bit of time, around a minute, and this is normal if you have the feeling that the graphical interface is lagging. The script is actually working at the command line level, so you can follow what is going on in the python console or in the console of the IDE you're using.

The main editor window should open. If the script complains (an error window will open), make sure that you placed both script files in the same folder as deepLuna and check that their respective names are "script_text.mrg" and "allscr.mrg" (these are the original names of the files).

Editing the text

Once the first extraction has been done, when launching deepLuna simply click on "Open translation". This might also take some seconds.

The main editor window contains three main elements:

  • On the left there is a tree view of the text files of the game, and it is the only active element at the beginning. This section is the organized by heroine and days ("Arcueid" and "Ciel"). The remaining three sections concern the bad end section "Teach Me, Ciel-sensei!", the part that is common to both Arcueid's and Ciel's route called "Common" and finally the section "Hidden text" that shows some text grabbed by the script that is not used ingame. The first thing you have to do is choose one of the scenes that appear as elements of the form "TEXT_WITH_UNDERSCORES_AND_NUMBERS", and then double-click on it. IMPORTANT NOTE: When clicking on one of the element, due to the structure of the script, you might have to wait a few seconds before the next content loads - this is perfectly normal.
  • The second element from the left, a scrolling list, activates. It shows the content of the selected scene in the form "a : b" where a is the page and b is the line on the page a. Double-click on any element. You might also note at this point that the two fields above activated, and will show up the percentage of translation done on the selected scene as well as for the global file, computed both in real time. We will come back to them in the section below.
  • Finally, you have the main editing part: a field that shows the original japanese selected line called "Original text", a field called "Translated text" below where you have to input your translation of the line above by replacing the word "TRANSLATION" that is there, and finally some buttons that will allow you to perform all the relevant actions.

IMPORTANT NOTE: When in some line you meet the pound/hashtag character "#", PLEASE do not delete it and leave as it is in the translation! This character symbolizes the fact that the line is shown ingame with a small break where the "#" is, and from a low-level perspective, symbolizes that the line is cut into two separate strings with altogether different pointers, so erasing it WILL break everything.

Commands

Validate: When you finish inputting the translation for a line, click on this button. The corresponding offset will become green to confirm the translation and the percentage of advancement of translation will update for the scene as well as for the total game.

Save: When you finish your translation, simply click on this button to save everything. IMPORTANT NOTE: This operation might also take a few seconds, this is also normal.

Search: Opens a window to search for some text. Enter the text in the first field, and click on "Search". Move between the results with "Previous"/"Next". If you want to replace some text (works only for the translated text for obvious reasons), simply use the last field in the window and then click on "Replace"/"Replace all" (the number of results will update automatically). This allows you to search for text only in the selected scene, not in the whole game script.

Translating the text

Select an offset or a range of offset (using Shift+double-click), then click on the "Translate" button in the main screen, a new window will open. The first time you execute the script, you'll have to configure three options (after that, they will be saved in a config.ini file created in deepLuna folder after the initial launch and load automatically everytime you launch deepLuna):

a) "deepL API link": Input here your API link(s) from deepL (replace the 0 that is there). If you have several of them, just separate them by a comma - the script will try them successively when a translation request fails. Currently works only for free deepL accounts (possible future update for a pro deepL API link). NOTE: If you copy/paste your API link and it doesn't work, make sure that there is not an additional space character that was copied at the end of the link, this might happen.

b) "Language": Put the initials of the language into which you want to translate the script (e.g. EN for english, FR for french, DE for german, IT for italian, RU for russian, PL for polish, ES for spansih, etc.)

c) "Block translation": If you keep this box unchecked, deepLuna will send each line separately to deepL for translation. If you check the box, deepLuna will glue all the lines from the selected range of offsets together, and send the whole block to deepL before splitting them again. From experience, the latter option might improve the quality of translation since you give more context to deepL, but on the other hand produces sometimes completely unrelated translated sentences in the middle if the block is really big. You have to play around to see what option is the best depending on context and chosen sentences.

IMPORTANT NOTE: The furigana are thrown away during translation at the moment since their structure prevent deepL from giving a consistent translation, and also because their meaning and positioning sometimes doesn't match at all the original word (Nasu writing style is really something...). Maybe I'll try to implement something to deal with this problem in the future, but for the moment it is as it is.

Now, it suffices to click on "OK" and wait until translation finishes. If something fails (too much deepL requests for example), the program will tell you and if you've chosen the line-by-line translation approach, will point you out the line where it stopped.

Injecting the text back

Easy as pie, simply click on the button "Insert" and wait a little bit (it might take up to ~20 seconds on some older computers). deepLuna will generate a new script file of the form "script_text_translatedDATETIME.mrg", with DATETIME being the date and time of insertion (to avoid overwriting files and keeping backups). NOTE: The script inserts everytime the full translation.

NEW: Exporting functions

Since the v2.0, deepLuna makes your life easier if you're working as a group on the translation, which is usually the case. How to keep everything up to date for all the people? Whenever you finish working on some scene, just press on "Export", and deepLuna will generate an update file in the update folder, in the same directory. If you look inside, you will see that this is a simple text file with the same name as the scene you were working on, and if you open it, you'll realise that this is exactly the case - the exported file contains the whole scene in a very nice and readable way, with the original sentences where they were, and the freshly translated sentences where the japanese ones were supposed to be. It suffices now to send this file to your colleagues, and they will simply have to put it in the "update" folder. When they launch deepLuna the next time they want to use it, it will update itself and include the new file in its own database.

If you need for some reason to export your whole personal database, this is also doable - simply click on "Export all", and a new window will open, asking you for confirmation. Note that this operation might take up to 10 mn on some older computers.

On another note, this update function makes it possible to have an alternative use of deepLuna: simply export the scene file, then edit it directly with your favorite text editor, and finally share it with your teammates or use the "Insert" function to inject it in the script yourself.

Additional comments

Feel free to raise an issue if something happens or if you wish to have some specific functionality, I'm always open to new possibilities and improvements! On another note, if you end up using this tool in some way or the other for your translation, please simply credit the script name itself, deepLuna, in some place or the other on your website/patched game/other.

This readme will get a more substantial update in the following days with screenshots and/or video tutorials.

Acknoledgments

I am really thankful to Hintay for his wonderful tools at https://github.com/Hintay/PS-HuneX_Tools. I relied on them heavily (unpack_allsrc.py, prep_tpl.py and the mzx decompression script) and included a modified version of them in deepLuna to make it perfectly operational.

Comments
  • Encoding question

    Encoding question

    Sorry to bother you again, and for the dumb question too; but is there a way to have the encoding supported for other languages? I was testing translations in my native language - which makes a lot of use of commas and accents - however, when executed the letters are always distorted since the format is not supported

    opened by Zac91281 13
  • ModuleNotFoundError (I do have the module installed)

    ModuleNotFoundError (I do have the module installed)

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in from PIL import ImageTk, Image ModuleNotFoundError: No module named 'PIL'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in install("Pillow") NameError: name 'install' is not defined

    The problem here is that when I try to run the script on IDE this errors occurs and I just don't know what to do since I already have everything needed installed.

    opened by Arthurfogo7 10
  • Strange Error

    Strange Error

    I've been doing the deepLuna translation in increments and have had minimal issues. I got up to 44% and now whenever I try to translate again, no matter the line or chunk of lines, I'm getting this.

    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\Jeremy\AppData\Local\Programs\Python\Python39\lib\tkinter\__init__.py", line 1892, in __call__
        return self.func(*args)
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 910, in translate_game
        if self.i < cs[-1]:
    IndexError: tuple index out of range
    
    opened by jeremoople 7
  • Not opening

    Not opening

    Whenever I try to open deepLuna, python opens and closes it instantly. I'm using the latest version and the files are placed with the needed extracted files from the game. My knowledge is pretty lay, so I don't know what to do.

    opened by Zac91281 5
  • Decode error on new build

    Decode error on new build

    I decide to check new build, and it's give a error when i try to press "Extract Text" or "Open translation". Traceback (most recent call last): File \luna\ui\start_window.py", line 166, in extract_database tl_db = TranslationDb.from_mrg( File \luna\translation_db.py", line 387, in from_mrg strings_by_offset[i] = data.decode('utf-8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

    opened by SkyAshpd 4
  • All of the .py files keep immediately closing?

    All of the .py files keep immediately closing?

    So for some reason every time I try to open deepLuna.py it keeps immediately closing with no text or anything at all. I'm quite confused as I have python downloaded.

    opened by Rescuro 2
  • Install error

    Install error

    Every time I try to run deepLuna.py in IDLE or Pyzo, I get this error:

    Traceback (most recent call last):
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in <module>
        from PIL import ImageTk, Image
    ModuleNotFoundError: No module named 'PIL'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "F:\tsuikihime\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in <module>
        install("Pillow")
    NameError: name 'install' is not defined
    >>> 
    

    I'm not used to installing programs using Python as opposed to like, a setup wizard, so I could very well just be making a basic newbie mistake, but still, I don't know where to go from here.

    opened by jeremoople 2
  • Issue with inserting translation

    Issue with inserting translation

    When I press the insert button on the translation window, I get this error in the console window:

    Saving file, please wait...
    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\fergu\AppData\Local\Programs\Python\Python38-32\lib\tkinter\__init__.py", line 1883, in __call__
        return self.func(*args)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1518, in function_insert_translation
        self.enregistrer_fichier()
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1718, in enregistrer_fichier
        self.table_file = self.reinsert_daytable(self.table_scr_file,self.table_file)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1510, in reinsert_daytable
        if self.splitDayTable[self.i][0] == self.newMainTable[self.j][0] and self.splitDayTable[self.i][2] != 'TRANSLATION':
    TypeError: 'type' object is not subscriptable
    

    Any ideas as to what I can do to fix the issue? I'm currently using python 3.8.5.

    opened by shingle-bells 1
  • Fix cursor accounting for _long_ split lines

    Fix cursor accounting for _long_ split lines

    Previously, when breaking lines, we would take the length of the final section of the broken line and add that to the cursor position, e.g.

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("20 characters of text")
    // Cursor position: 40
    MSAD("5 characters of text")
    // Cursor position: 45
    -> "20 characters of text20characters of text5 characters of text"
    

    however, this is wrong for lines where a MSAD causes a line break -

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("40 characters of text")
    // Gets wrapped to 35 chars + 5 chars
    // Cursor position: 20 + 5 (WRONG!)
    
    opened by rschlaikjer 0
  • Injecting Question

    Injecting Question

    Greetings. I know it isn't quite a issue, but I don't know where else to ask. But should the edited script go in some specific folder? Mine is within the DeepLuna folder, already replacing the original file, but the translation is not appearing. image

    opened by Zaldi556 0
Releases(v5.2)
  • v5.2(Jan 16, 2022)

  • v4.6(Jan 9, 2022)

    • deepLuna now uses new 'Literate' import/export script format
    • Internal database structure now JSON-based
    • Various performance improvements
    • Added luna_cli and luna_linter to facilitate scripting of simple workflows
    Source code(tar.gz)
    Source code(zip)
  • v3.1.1(Oct 18, 2021)

    • Major performance improvements, in particular importing update files and loading scenes takes now only split seconds
    • A new comment field has been added (see Readme for more details)
    • A new "swap character" option has been added for languages with special characters (see Readme for more details)
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v3.1.1.zip(224.48 KB)
  • v2.3(Oct 6, 2021)

  • v2.2.1(Sep 14, 2021)

  • v2.2(Sep 10, 2021)

  • v2.1(Sep 7, 2021)

    Major upgrade of deepLuna:

    • Extraction algorithm and main interface completely remodelled
    • Taking now into account the underlying day/heroine structure of the files
    • Proposes a new simple and intuitive function to share your updated translation with you teammates
    • Added many small improvements to the interface that enhances the global user experience
    • Corrected many different bugs
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v2.1.zip(215.62 KB)
  • v1.0(Sep 6, 2021)

    The is the initial release of deepLuna. It contains the following elements:

    • a linear extraction module of the Tsukihime Remake script coming from the script_text.mrg file
    • a graphical interface edition module with some basic usual functions, such as search and replace
    • a machine translation module using deepL API links as a professional translation possible helper
    • an injection module that injects the translated text in the script_text.mrg file
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v1.0.zip(203.09 KB)
Owner
null
Python library for the DeepL language translation API.

The DeepL API is a language translation API that allows other computer programs to send texts and documents to DeepL's servers and receive high-quality translations. This opens a whole universe of opportunities for developers: any translation product you can imagine can now be built on top of DeepL's best-in-class translation technology.

DeepL 535 Jan 4, 2023
A Python wrapper for the DeepL API

deepl.py A Python wrapper for the DeepL API installing Install and update using pip: pip install deepl.py A simple example. # Sync Sample import deep

grarich 18 Dec 12, 2022
Translator based on Google API

Yakusu Toshiko Translator based on Google API. Instance of this bot is running as @yakusubot. Features Add a plus to a language's name to show an orig

Arisu W. 2 Sep 21, 2022
A simple language translator with python and google translate api

Language translator with python A simple language translator with python and google translate api Install pip and python 3.9. All the required depende

null 0 Nov 11, 2021
Google translator bot using pyTelegramBotAPI

iTranslator-bot Super google translator bot using pyTelegramBotAPI A bot is a professional bot that automatically detects a language in texts or capti

Abdulatif 6 Nov 22, 2022
streamlit translator is used to detect and translate between languages created using gTTS, googletrans, pillow and streamlit python packages

Streamlit Translator Streamlit Translator is a simple translator app to detect and translate between languages. Streamlit Translator gets text and lan

Siva Prakash 5 Apr 5, 2022
A simple google translator telegram bot

Translator-Bot A simple google translator telegram bot Please fork this repository don't import code Made with Python3 (C) @FayasNoushad Copyright per

Fayas Noushad 14 Nov 12, 2022
A simple google translator telegram bot version 2

Translator-Bot-V2 A simple google translator telegram bot version 2 Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License

Fayas Noushad 15 Oct 21, 2022
a translator bot for discord

TranslatorBOT it is a simple and powerful discord bot, it been used for translating includes more than 100 language, it has a lot of integrated comman

Mear. 2 Feb 3, 2022
An advanced telegram language translator bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License -> https://github.com/FayasNoushad/Translator-Bot-V3/blob/main/LICE

Fayas Noushad 19 Dec 24, 2022
This is a translator that i made by myself in python with the 'googletrans' library

Translator-Python This is a translator that i made by myself in python with the 'googletrans' library This application completely made in python allow

Thadeuks 2 Jun 17, 2022
Simple Translator in Python

Simple Translator in Python Project Description: In this project, we'll be making a very simple translator in Python using some libraries. Requirement

Hassan Shahzad 3 Jan 23, 2022
EthSema - Binary translator for Ethereum 2.0

EthSema is a novel EVM-to-eWASM bytecode translator that can not only ensure the fidelity of translation but also fix commonly-seen vulnerabilities in smart contracts.

weimin 8 Mar 1, 2022
A Bot Telegram Anti Users Channel to automatic ban users who using channel to send message in group.

Tg_Anti_UsersChannel A Bot Telegram Anti Users Channel to automatic ban users who using channel to send message in group. Features: Automatic ban Whit

idzeroid 6 Dec 26, 2021
Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API.

Tg_PHub_Bot Telegram PHub Bot using ARQ Api and Pyrogram. This Bot can Download and Send PHub HQ videos in Telegram using ARQ API. OS Support All linu

TheProgrammerCat 13 Oct 21, 2022
Automatic SystemVerilog linting in github actions with the help of Verible

Verible Lint Action Usage See action.yml This is a GitHub Action used to lint Verilog and SystemVerilog source files and comment erroneous lines of co

CHIPS Alliance 10 Dec 26, 2022
A results generator and automatic token checker for Yandex Contest

Yandex contest Python checking tools A results generator and automatic token checker for Yandex Contest. Версия на русском языке Installation Clone th

Nikolay Chechulin 9 Dec 14, 2022
A script that writes automatic instagram comments under a post

Send automatic messages under a post on instagram Instagram will rate limit you after some time. From there on you can only post 1 comment every 40 se

Maximilian Freitag 3 Apr 28, 2022
Discord Voice Channel Automatic Online

Discord-Selfbot-voice Features: Discord Voice Channel Automatic Online FAQ Q: How can I obtain my token? A: 1. How to obtain your token in android 2.

Pranav Ajay 3 Oct 31, 2022