JS Segment Annotator
Javascript image annotation tool based on image segmentation.
- Label image regions with mouse.
- Written in vanilla Javascript, with require.js dependency (packaged).
- Pure client-side implementation of image segmentation.
A browser must support HTML canvas to use this tool.
There is an online demo.
Importing data
Prepare a JSON file that looks like the following. The required fields are labels and imageURLs. The annotationURLs are for existing data and can be omitted. Place the JSON file inside the data/ directory.
{
"labels": [
"background",
"skin",
"hair",
"dress",
"glasses",
"jacket",
"skirt"
],
"imageURLs": [
"data/images/1.jpg",
"data/images/2.jpg"
],
"annotationURLs": [
"data/annotations/1.png",
"data/annotations/2.png"
]
}
Then edit main.js to point to this JSON file. Open a Web browser and visit index.html.
Known issues
Browser incompatibility
A segmentation result can greatly differ due to the difference in Javascript implementation across Web browsers. The difference stems from numerical precision of floating point numbers, and there is no easy way to produce the exact same result across browsers.
Python tips
Annotation PNG
The annotation PNG file contains label map encoded in RGB value. Do the following to encode an index map.
import numpy as np
from PIL import Image
# Decode
encoded = np.array(Image.open('data/annotations/1.png'))
annotation = np.bitwise_or(np.bitwise_or(
encoded[:, :, 0].astype(np.uint32),
encoded[:, :, 1].astype(np.uint32) << 8),
encoded[:, :, 2].astype(np.uint32) << 16)
print(np.unique(annotation))
# Encode
Image.fromarray(np.stack([
np.bitwise_and(annotation, 255),
np.bitwise_and(annotation >> 8, 255),
np.bitwise_and(annotation >> 16, 255),
], axis=2).astype(np.uint8)).save('encoded.png')
JSON
Use JSON module.
import json
with open('data/example.json', 'r') as f:
dataset = json.load(f)
Using dataURL
Do the following to convert between dataURL and NumPy format.
from PIL import Image
import base64
import io
# Encode
with io.BytesIO() as buffer:
encoded.save(buffer, format='png')
data_url = b'data:image/png;base64,' + base64.b64encode(buffer.getvalue())
# Decode
binary = base64.b64decode(data_url.replace(b'data:image/png;base64,', b''))
encoded = Image.open(io.BytesIO(binary))
Matlab tips
Annotation PNG
The annotation PNG file contains label map encoded in RGB value. Do the following to encode an index map.
% Decode
X = imread('data/annotations/0.png');
annotation = X(:, :, 1);
annotation = bitor(annotation, bitshift(X(:, :, 2), 8));
annotation = bitor(annotation, bitshift(X(:, :, 3), 16));
% Encode
X = cat(3, bitand(annotation, 255), ...
bitand(bitshift(annotation, -8), 255), ...
bitand(bitshift(annotation, -16)), 255));
imwrite(uint8(X), 'data/annotations/0.png');
JSON
Use the matlab-json package.
Using dataURL
Get the byte encoding tools.
Do the following to convert between dataURL and Matlab format.
% Decode
dataURL = 'data:image/png;base64,...';
png_data = base64decode(strrep(dataURL, 'data:image/png;base64,', ''));
annotation = imdecode(png_data, 'png');
% Encode
png_data = imencode(annotation, 'png');
dataURL = ['data:image/png;base64,', base64encode(png_data)];
Citation
We appreciate if you cite the following article in an academic paper. The tool was originally developed for this work.
@article{tangseng2017looking,
Author = {Pongsate Tangseng and Zhipeng Wu and Kota Yamaguchi},
Title = {Looking at Outfit to Parse Clothing},
Eprint = {1703.01386v1},
ArchivePrefix = {arXiv},
PrimaryClass = {cs.CV},
Year = {2017},
Month = {Mar},
Url = {http://arxiv.org/abs/1703.01386v1}
}
)
On the next picture below is annotated image from Pascal VOC 2012 for segmentation, containing RGB information + index for object and also the same for background.
1 Feb 2, 2022
2 Apr 18, 2022
48 Jan 5, 2023
141 Dec 13, 2022
52 Nov 29, 2022
12 Dec 23, 2022
1 Nov 13, 2021
3 Oct 25, 2022
2 Jan 28, 2022
2 Dec 27, 2021
39 Dec 20, 2022
6 Mar 31, 2022
8 Dec 8, 2022
3 May 16, 2022
8 Sep 21, 2022
228 Dec 17, 2022
24 Dec 20, 2022
238 Dec 30, 2022
199 Dec 8, 2022