Simple app for visual editing of Page XML files

Overview

Name

nw-page-editor - Simple app for visual editing of Page XML files.

Version: 2021.02.22

Description

nw-page-editor is an application for viewing/editing ground truth or predicted information for diverse purposes related to the areas of document processing and text recognition. The edition is done interactively and visually on top of images of scanned documents. Additionally the app supports many keyboard shortcuts to allow more efficient editing, see section Application usage shortcuts.

The app is available in two variants. The first variant is as a desktop application based on the NW.js framework thus making it cross-platform. The second variant is as a web application that allows remote editing by multiple users and can be easily setup via a docker container.

Supported formats

The format used for storing the information is the omni:us Pages Format hightly based on the PRImA Page XML schema, but only supports a subset of elements and has a few minor extensions.

New XML files created by the app are saved as the latest version of the omni:us schema. Modifications to existing XML files using older omni:us schemas or PRImA schemas since version 2013-07-15 are saved using the original schema. Note that this might mean that the saved XMLs are no longer valid according to that schema. Additionally the app supports importing XMLs from other formats, namely: PRImA 2010-03-19, alto v2 and v3, tet and poppler's pdftotext xhtml. Many details from imported formats might be lost and there is no functionality to export back to those formats.

Desktop variant

After correct installation of the desktop app, the standard way in Linux and Mac to start the app is through the command line, normally specifying which file(s) to open. In windows currently it is not possible to open files directly from the command line, so you need to start the app and then select a file to open.

Command line synopsis

nw-page-editor [*page.xml*]+ [*pages_dir*]+ [--list *pages_list*]+ [--css *file.css*]+ [--js *file.js*]+

Example Page XML files

You can find example Page XML files in the nw-page-editor source code repository, the directory examples. Thus, you can open the examples as:

nw-page-editor examples/*.xml

Desktop variant installation

Linux/Mac:

  1. Download the SDK version of the NW.js package appropriate for your platform from http://nwjs.io/downloads. Extract it to a location where you store applications and add to your PATH the directory containing the nw|nwjs binary: root of package in Linux or nwjs.app/Contents/MacOS in OSX. Alternatively to setting the PATH it is also possible to simply symlink the nw|nwjs executable to a directory already in your path.

  2. Clone the nw-page-editor repository (using the --recursive option) to a location where you store applications and add the package's bin directory to your PATH. Like in the previous step, alternatively you can simply symlink the bin/nw-page-editor executable to a directory already in your path.

Windows:

In the github repository's releases there are for selected version prebuilt packages for windows. To install, go to https://github.com/mauvilsa/nw-page-editor/releases and download and extract the latest nw-page-editor-win-x64.zip file. To open the app, execute the nw-page-editor.exe and for convenience create a shortcut to open the app.

If you are interested in the latest version for which there is no prebuilt package, follow the following instructions:

  1. Download the SDK version of the NW.js package appropriate for your platform from http://nwjs.io/downloads. Extract it to a location where you store applications renaming the base directory to nw-page-editor.

  2. Move, copy or clone the files of this github repository such that the file package.json is in the same directory as nw.exe. If cloning be sure that you use the --recursive option.

  3. For convenience rename the nw.exe executable to nw-page-editor.exe and create a shortcut to it in your desktop to ease opening the app.

Notes:

  • The reason to install the SDK version of NW.js is to allow inspection of elements using the Chrome DevTools and for example be able to do Page XML modifications not implemented in the app.

Desktop variant startup CSS and JavaScript files

The --css and --js command line options can be used to modify the appearance or execute custom code on startup. For example, the following JavaScript code changes the position, color and size of overlayed text, and prints to a pdf.

setTimeout( function() {
    $.stylesheet('#page_styles { #xpg .TextEquiv }').css( 'fill', 'green' ); /* Set font color to green */
    $.stylesheet('#page_styles { #xpg .TextEquiv }').css( 'font-size', '60px' ); /* Set font size to 60px */
    pageCanvas.cfg.textPositionOffset = [ 10, 0 ]; /* Set text position x and y offsets */
    pageCanvas.util.positionText(); /* Update text position */
    nw.Window.get().print({pdf_path:loadedFile.replace(/\.xml$/,'.pdf')}); /* Print to pdf */
}, 500 );

To run this example save this code snippet to a file test.js and start the editor from the command line as

nw-page-editor --js test.js examples/lorem.xml

Web server variant

The page editor can also be used as a web server allowing multiple users to edit page xmls remotely. Moreover, the server can be configured so that all the history of changes of the page xmls are saved in a git repository with commits associated to the respective users. To ease the installation and usage of the web server version, docker is used.

Web server variant installation

The steps for installation are the following:

  1. Install docker in the server and for convenience configure it so that sudo is not required to run containers.

  2. Either pull the latest image of nw-page-editor-web from docker hub by choosing one of the available tags or build the docker image from the source.

## Pull from docker hub ##
TAG="YOUR CHOSEN TAG HERE"
docker pull mauvilsa/nw-page-editor-web:$TAG

## Build docker image from source ##
TAG="local"
docker build -t mauvilsa/nw-page-editor-web:$TAG .
  1. Create a directory for the data and copy the images and page xml that will be available to access remotely. It is highly recommended to make the data directory a git repository so that change history is kept.
## Create directory for data ##
mkdir data

## Copy documents, e.g. the examples in nw-page-editor source ##
cp $NW_PAGE_EDITOR_SOURCE/examples/* data

## Create list of xmls if use of lists is needed ##
ls data/*.xml | sed 's|^data/||' | head -n 2 > data/xmls.lst

## Init data directory as git repo ##
git init data
  1. By default the web app can be accessed without authentication in which case all git commits are associated to the anonymous user. Alternatively you can create users with passwords to restrict the access to the app and associate commits to different people. This is done by creating the file data/.htpasswd using the htpasswd tool. This tool might not be available in the server, but it is included in the nw-page-editor-web docker image. To ease the usage of htpasswd within this image, it is recommended that you use the docker-cli script. Just download it into some directory included in your PATH and then do the following.
## Create users and passwords ##
docker-cli -- mauvilsa/nw-page-editor-web:$TAG htpasswd -cb data/.htpasswd [email protected] pass1
docker-cli -- mauvilsa/nw-page-editor-web:$TAG htpasswd -b data/.htpasswd [email protected] pass2

## For more details on htpasswd usage ##
docker-cli -- mauvilsa/nw-page-editor-web:$TAG htpasswd --help
  1. Start a container exposing the web server port 80 to a port of your preference (e.g. 8080) and set the data directory as a volume.
docker run --rm -d -p 8080:80 \
  -v $(pwd)/data:/var/www/nw-page-editor/data \
  mauvilsa/nw-page-editor-web:$TAG
  1. The Page XMLs can be accessed using URLs like the example ones below. Three different GET parameters can be used to specify which documents to view. In all cases relative paths with respect to the data directory must be given. The three supported parameters are:
    • f: to specify a single xml file
    • l: to specify a list of xml files
    • d: to specify a directory
http://$SERVER_ADDRESS:8080/app?f=lorem.xml
http://$SERVER_ADDRESS:8080/app?l=xmls.lst
http://$SERVER_ADDRESS:8080/app?d=.

Data owner, group and permissions

By default the files created/modified by the web server will use a default umask and have the same owner and group as the data directory. These defaults can be overridden by defining the DATA_UMASK, DATA_UID and DATA_GID environment variables.

docker run --rm -d -p 8080:80 \
  -e DATA_UMASK=007 -e DATA_UID=1234 -e DATA_GID=5000 \
  -v $(pwd)/data:/var/www/nw-page-editor/data \
  mauvilsa/nw-page-editor-web:$TAG

Web server variant startup CSS and JavaScript files

Similar to the desktop variant, the web variant can also receive css and js files to modify the appearance or execute custom code on startup. To use this functionality the css and js files need to be included in some path inside the data directory. Then the docker container should be started setting as value to the CSS and JS environment variables, relative paths to the files separated by spaces. For example:

docker run --rm -d -p 8080:80 \
  -e CSS=mystyle.css -e JS=mycode.js \
  -v $(pwd)/data:/var/www/nw-page-editor/data \
  mauvilsa/nw-page-editor-web:$TAG

Application usage shortcuts

Shortcut Command
ctrl/cmd + q Quit application
ctrl/cmd + o Open file dialog
ctrl/cmd + s Save changes to file
ctrl/cmd + shift + s Save as new file dialog
ctrl/cmd + p Print
ctrl/cmd + enter Open/close menu
ctrl/cmd + n New window
pageup Load previous document
pagedown Load next document
shift + pageup Load previous 10th document
shift + pagedown Load next 10th document
ctrl/cmd + z Undo last change
ctrl/cmd + y Redo last change
mouse click Select or deselect element
esc Deselect the currently selected element
tab Select the next element
shift + tab Select the previous element
ctrl + tab Select the next dragpoint
ctrl + shift + tab Select the previous dragpoint
ctrl + f Enable/disable text filtering
mouse click Start creating or add point to polygon/polyline
ctrl/cmd + mouse click Add point to and finish creating polygon/polyline
ctrl/cmd + del Delete selected element
del Delete selected element (only when text not editable)
minus + . Delete selected dragpoint
plus + . Add dragpoint next to selected
ctrl/cmd + 0 View full document (initial zoom)
ctrl/cmd + 1 Zoom to page of selected element or hovered
ctrl/cmd + 2 Zoom to selected element or hovered
ctrl/cmd + plus/minus Zoom in/out
shift + mouse wheel Zoom in/out
ctrl/cmd + arrows Move (pan) the image in the respective direction
mouse wheel Move (pan) the image in the respective direction
mouse drag Move (pan) the image in the respective direction
ctrl + , Select the next edit mode element level
ctrl + shift + , Select the previous edit mode element level
ctrl + . Select the next edit mode type
ctrl + shift + . Select the previous edit mode type
ctrl/cmd + e Open property editor for selected element
ctrl/cmd + shift + pageup Increase bottom pane text font size
ctrl/cmd + shift + pagedown Decrease bottom pane text font size
ctrl/cmd + pageup Increase overlayed text font size
ctrl/cmd + pagedown Decrease overlayed text font size
ctrl/cmd + g Change gamma of image
alt + arrows Change table selected cell in the respective direction
plus + c/r Split selected table column/row
minus + c/r Remove selected table column/row
move dragpoint Modifies dragpoint and its opposite dragpoint(s)
shift+ move dragpoint Modifies only the dragpoint, not the opposite(s)
ctrl/cmd + r Toggle selected element protection

Copyright

The MIT License (MIT)

Copyright (c) 2015-present, Mauricio Villegas [email protected]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Issues
  • How to use it

    How to use it

    I download the sdk version and run ./nwjs 屏幕快照 2019-03-11 上午10 21 06 but i don't know how to use it , shortcuts not work

    opened by rockyzhengwu 8
  • other region types

    other region types

    Hi, if I understand correctly, the application validates against https://github.com/omni-us/pageformat/blob/fe1ca7f589dfd8aa9cc9abcc4a336e9afddd8aba/pagecontent_omnius.xsd

    I was wondering: The schema mentions several elements which I don’t find reflected in the editor’s GUI, namely ImageRegion, SeparatorRegion and CustomRegion. Especially ImageRegion would be very helpful for us. Are there plans to include these as well? Thank you!

    enhancement 
    opened by dasch124 8
  • Validation error on store

    Validation error on store

    We have installed the docker image mauvilsa/nw-page-editor-web:2019.01.28 from dockerhub (the latest image did not work), however when I try to edit a text box in the sample file at {ourmachine}/app/?f=lorem.xml&n=1 I receive the following validation error on saving:

    Problems saving file ../data/lorem.xml: 400: Page XML schema validation failed:
    ../data/lorem.xml~2:85: element Group: Schemas validity error : Element '{http://schema.primaresearch.org/PAGE/gts/pagecontent/2013-07-15}Group': This element is not expected. Expected is ( {http://schema.primaresearch.org/PAGE/gts/pagecontent/2013-07-15}Page ).
    ../data/lorem.xml~2 fails to validate
    SvgCanvas/[email protected]://page-editor.acdh-dev.oeaw.ac.at/js/svg-canvas.js?v=2019.01.28:248:15
    saveFile/<@https://page-editor.acdh-dev.oeaw.ac.at/js/web-app.js?v=2019.01.28:176:24
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:2:28017
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:2:28783
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:4:14035
    c/<@https://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:4:16323
    

    Anything we can do about it?

    bug 
    opened by dasch124 5
  • Convert pdf to page-xml

    Convert pdf to page-xml

    On 2019/09/01 23:02, deepseek wrote:

    @mauvilsa Thank you for your hard word,

    Is there a way to convert a pdf file into a page-xml file. perhaps convert the pdf to svg first, then to page-xml?

    opened by ghost 4
  • Installs and opens, but can't give any commands

    Installs and opens, but can't give any commands

    Hi, this might be a severe case of pebkac, in which case I apologize. After installing and opening nw-page-editor (trying with one of my own scans as well as your examples) I can neither create not modify any of the polygons. None of the commands from the README (except for Ctrl+O) seem to do anything at all. Wondering if the NW.JS version (83.0.4103.61) might be the problem?

    opened by sixtyfive 2
  • nw-page-editor: error: unable to resolve the nw-page-editor app location on OSX

    nw-page-editor: error: unable to resolve the nw-page-editor app location on OSX

    OSX 10.12.5

    added path in ~/.zshrc .

    export PATH=$PATH:/Applications/nwjs.app/Contents/MacOS
    export PATH=$PATH:/Applications/nw-page-editor/bin
    
    echo $PATH
    /Users/crokobit/.rvm/gems/ruby-2.2.2/bin:/Users/crokobit/.rvm/gems/[email protected]/bin:/Users/crokobit/.rvm/rubies/ruby-2.2.2/bin:/Users/crokobit/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/nwjs.app/Contents/MacOS:/Applications/nw-page-editor/bin:/Users/crokobit/.rvm/bin
    

    When I execute nw-page-editor /Users/crokobit/Projects/test.xml there is following error message:

    readlink: illegal option -- f
    usage: readlink [-n] [file ...]
    nw-page-editor: error: unable to resolve the nw-page-editor app location
    

    Many Thanks.

    bug 
    opened by crokobit 1
  • Add short-cut to go to single-page view in one go (without using the current zoom option).

    Add short-cut to go to single-page view in one go (without using the current zoom option).

    Viewing the separate pages of a multi-page document, is very slow on my computer. Could a short-cut be implemented, other than [ ctr/cmd +plus/minus / shift + mouse wheel ], to speed up the single-page view?

    enhancement 
    opened by didirus 1
  • Problems writing to temporal file

    Problems writing to temporal file

    Hi, I'm experiencing strange problem with the latest release of the web-version of the editor, namely on saving I receive the following error:

    Problems saving file ../data/lorem.xml: 400: Problems writing to temporal file
    SvgCanvas/[email protected]://page-editor.acdh-dev.oeaw.ac.at/js/svg-canvas.js?v=2021.02.22:254:15
    saveFile/<@https://page-editor.acdh-dev.oeaw.ac.at/js/web-app.js?v=2021.02.22:182:24
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:2:28017
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:2:28783
    [email protected]://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:4:14035
    c/<@https://page-editor.acdh-dev.oeaw.ac.at/js/jquery-3.2.1.min.js:4:16323
    

    Here's what the logs tell me:

    ==> error.log <==
    [Fri Apr 09 14:43:35.004832 2021] [php7:warn] [pid 26] [client 172.17.0.1:44382] PHP Warning:  DOMDocument::loadXML(): Start tag expected, '&lt;' not found in Entity, line: 1 in /var/www/nw-page-editor/app/saveFile.php on line 49, referer: https://page-editor.acdh-dev.oeaw.ac.at/app/?f=lorem.xml&n=1
    [Fri Apr 09 14:43:35.004937 2021] [php7:warn] [pid 26] [client 172.17.0.1:44382] PHP Warning:  XSLTProcessor::transformToXml() expects parameter 1 to be object, bool given in /var/www/nw-page-editor/app/saveFile.php on line 49, referer: https://page-editor.acdh-dev.oeaw.ac.at/app/?f=lorem.xml&n=1
    [Fri Apr 09 14:43:35.004953 2021] [php7:warn] [pid 26] [client 172.17.0.1:44382] PHP Warning:  DOMDocument::loadXML(): Empty string supplied as input in /var/www/nw-page-editor/app/saveFile.php on line 50, referer: https://page-editor.acdh-dev.oeaw.ac.at/app/?f=lorem.xml&n=1
    [Fri Apr 09 14:43:35.004960 2021] [php7:warn] [pid 26] [client 172.17.0.1:44382] PHP Warning:  XSLTProcessor::transformToXml() expects parameter 1 to be object, bool given in /var/www/nw-page-editor/app/saveFile.php on line 50, referer: https://page-editor.acdh-dev.oeaw.ac.at/app/?f=lorem.xml&n=1
    
    ==> access.log <==
    172.17.0.1 - user1:NvDjj+FV [09/Apr/2021:14:43:34 +0000] "POST /app/saveFile.php HTTP/1.1" 200 59 "https://page-editor.acdh-dev.oeaw.ac.at/app/?f=lorem.xml&n=1" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0"
    

    Do you have an idea what could be causing this? Thanks in advance!

    opened by dasch124 1
  • support newer PAGE-XML versions

    support newer PAGE-XML versions

    AFAICS the latest version nw-page-editor can open/import is the 2017-07-15 schema (although the import transform does not seem to do anything really). When I open a 2018 or 2019 PAGE-XML, I get: Expected as input a Page document.

    What would be required to get them to work? (I have spawned a new xslt/page_from_2019-07-15.xslt from xslt/page_from_2017-07-15.xslt and added this to js/web-app.js as well as js/nw-app.js, but still get the same error.)

    Maybe the XSLT needs to actually remove (or rename/map) elements and attributes that have been added to the schema since 2013? See here for a summary of what has changed.

    enhancement 
    opened by bertsky 3
Owner
Mauricio Villegas
I am a postdoc researcher/developer currently working at omni:us
Mauricio Villegas
Page to PAGE Layout Analysis Tool

P2PaLA Page to PAGE Layout Analysis (P2PaLA) is a toolkit for Document Layout Analysis based on Neural Networks. ?? Try our new DEMO for online baseli

Lorenzo Quirós Díaz 169 Nov 19, 2021
Text page dewarping using a "cubic sheet" model

page_dewarp Page dewarping and thresholding using a "cubic sheet" model - see full writeup at https://mzucker.github.io/2016/08/15/page-dewarping.html

Matt Zucker 1.1k Dec 2, 2021
Deep learning based page layout analysis

Deep Learning Based Page Layout Analyze This is a Python implementaion of page layout analyze tool. The goal of page layout analyze is to segment page

null 169 Nov 8, 2021
ocroseg - This is a deep learning model for page layout analysis / segmentation.

ocroseg This is a deep learning model for page layout analysis / segmentation. There are many different ways in which you can train and run it, but by

NVIDIA Research Projects 72 Nov 4, 2021
a deep learning model for page layout analysis / segmentation.

OCR Segmentation a deep learning model for page layout analysis / segmentation. dependencies tensorflow1.8 python3 dataset: uw3-framed-lines-degraded-

null 99 Sep 13, 2021
OCR-D-compliant page segmentation

ocrd_segment This repository aims to provide a number of OCR-D-compliant processors for layout analysis and evaluation. Installation In your virtual e

OCR-D 52 Nov 29, 2021
This repository lets you train neural networks models for performing end-to-end full-page handwriting recognition using the Apache MXNet deep learning frameworks on the IAM Dataset.

Handwritten Text Recognition (OCR) with MXNet Gluon These notebooks have been created by Jonathan Chung, as part of his internship as Applied Scientis

Amazon Web Services - Labs 375 Nov 29, 2021
Validate and transform various OCR file formats (hOCR, ALTO, PAGE, FineReader)

ocr-fileformat Validate and transform between OCR file formats (hOCR, ALTO, PAGE, FineReader) Installation Docker System-wide Usage CLI GUI API Transf

Universitätsbibliothek Mannheim 127 Nov 4, 2021
~1000 book pages + OpenCV + python = page regions identified as paragraphs, lines, images, captions, etc.

cosc428-structor I had an open-ended Computer Vision assignment to complete, and an out-of-copyright book that I wanted to turn into an ebook. Convent

Chad Oliver 40 Nov 11, 2021
A Tensorflow model for text recognition (CNN + seq2seq with visual attention) available as a Python package and compatible with Google Cloud ML Engine.

Attention-based OCR Visual attention-based OCR model for image recognition with additional tools for creating TFRecords datasets and exporting the tra

Ed Medvedev 880 Nov 22, 2021
Visual Attention based OCR

Attention-OCR Authours: Qi Guo and Yuntian Deng Visual Attention based OCR. The model first runs a sliding CNN on the image (images are resized to hei

Yuntian Deng 1.1k Nov 23, 2021
Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Image Detector and Convertor App created using python's Pillow, OpenCV, cvlib, numpy and streamlit packages.

Siva Prakash 10 Nov 14, 2021
PyQT5 app that colorize black & white pictures using CNN(use pre-trained model which was made with OpenCV)

About PyQT5 app that colorize black & white pictures using CNN(use pre-trained model which was made with OpenCV) Colorizor Приложение для проекта Yand

null 2 Nov 19, 2021
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched

OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched or copy-pasted. ocrmypdf # it's a scriptable c

jbarlow83 5.4k Dec 3, 2021
virtual mouse which can copy files, close tabs and many other features !

AI Virtual Mouse Controller Developed an AI-based system to control the mouse cursor using Python and OpenCV with the real-time camera. Fingertip loca

Diwas Pandey 23 Oct 5, 2021
Simple SDF mesh generation in Python

Generate 3D meshes based on SDFs (signed distance functions) with a dirt simple Python API.

Michael Fogleman 853 Nov 22, 2021
A simple document layout analysis using Python-OpenCV

Run the application: python main.py *Note: For first time running the application, create a folder named "output". The application is a simple documen

Roinand Aguila 96 Oct 29, 2021
A simple OCR API server, seriously easy to be deployed by Docker, on Heroku as well

ocrserver Simple OCR server, as a small working sample for gosseract. Try now here https://ocr-example.herokuapp.com/, and deploy your own now. Deploy

Hiromu OCHIAI 439 Nov 17, 2021
"Very simple but works well" Computer Vision based ID verification solution provided by LibraX.

ID Verification by LibraX.ai This is the first free Identity verification in the market. LibraX.ai is an identity verification platform for developers

LibraX.ai 42 May 12, 2021