The Doodle Master seeks to turn your UI mockups into real code.

Related tags

Miscellaneous DoodleMaster
Overview

Doodle Master

The Doodle Master seeks to turn your UI mockups into real code. Currently this repository just serves to demonstrate a Proof Of Concept of Artificially Intelligent Design Tools.

The demo supports the detection of 4 classes of UI components

  1. TextViews
  2. Headers
  3. ImageViews
  4. Buttons

One can view the different doodles that this software supports for classification in the classes directory. Note that the classifier is slightly overfit towards the text view, training again with greater regularization can help solve this problem.

This type of tool can be useful in a big organisation which adheres to a consistent design guide and has several reusable components on variety of platforms such as mobile and the web. For example a button on the Airbnb website follows the same styling across all platforms. This tool can simultaneously generate the same UI for Android and the Web.

Demo

alt text

Technologies

This demo uses a simple CNN for the classification of the doodles made. The software is able to localise the UI element through a combination of geometric techniques and simple mouse/finger position detection. Frameworks used are:

  1. Deep Learning Model = Pytorch
  2. UI = HTML/CSS

This demo was inspired by Airbnb's work on Sketching Interfaces. This demo was made while I was a frontend developer. I was inspired to make this as I saw a lot of developers spending time on coding the UI from the mockups, this time could be better spent in coding the functionality behind the UI. Doodle master seeks to build a painless and natural way of prototyping interfaces. This tool can be used by designers and developers alike to showcase ideas quickly in the form of real code.

Instructions for Building and Running

This project uses Python3.

  1. Download the weights from here and create a folder weights in the root of the repo, paste the file inside this weights folder.
  2. Build the project with these commands:
conda create -n doodle python=3.6
conda activate doodle
sh ./setup.sh
  1. Start the server with the command : python3 server.py
  2. Goto localhost:5000 to view the Doodle Master.
  3. Output HTML file is generated at the output folder
Note

This is not a prodution worthy piece of software, it is only meant for demo purposes. Please don't hesitate to contact me for more details on this project. This project sparked an interesting discussion on HackerNews on the advantages and disadvantages of "Codeless UI", one can read more about in this thread

Contributors

Comments
  • No distribution found for torch===0.4.0

    No distribution found for torch===0.4.0

    Hi! Found this issue when trying to build:

    Notes: macOS High Sierra 10.13.5 pip3 version: 18.0 python3 version: 3.6.5

    Stacktrace from running pip3 install -r requirements.txt

    $Ben [master] DoodleMaster =>  pip3 install -r requirements.txt
Requirement already satisfied: Flask>=0.12.3 in /usr/local/lib/python3.7/site-packages (from -r requirements.txt (line 5)) (1.0.2)
Requirement already satisfied: Flask_Cors==3.0.3 in /usr/local/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (3.0.3)
Requirement already satisfied: Flask_SocketIO==2.9.3 in /usr/local/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (2.9.3)
Requirement already satisfied: Pillow==5.2.0 in /usr/local/lib/python3.7/site-packages (from -r requirements.txt (line 15)) (5.2.0)
Collecting bcolz==1.1.2 (from -r requirements.txt (line 18))
  Using cached https://files.pythonhosted.org/packages/a8/6c/6351cab42fca7ba7e96191ee24181357e62e79afdad7cf74485c9053ea19/bcolz-1.1.2.tar.gz
Collecting beautifulsoup4==4.4.1 (from -r requirements.txt (line 22))
  Using cached https://files.pythonhosted.org/packages/cf/69/9abfdab06490af5e0233bcebe3f617ec128486d94ea987ad4f77b9332eef/beautifulsoup4-4.4.1-py3-none-any.whl
Collecting h5py==2.7.1 (from -r requirements.txt (line 25))
  Using cached https://files.pythonhosted.org/packages/41/7a/6048de44c62fc5e618178ef9888850c3773a9e4be249e5e673ebce0402ff/h5py-2.7.1.tar.gz
Collecting imageio==2.2.0 (from -r requirements.txt (line 28))
Collecting matplotlib==2.0.2 (from -r requirements.txt (line 31))
Collecting numpy==1.13.3 (from -r requirements.txt (line 36))
  Using cached https://files.pythonhosted.org/packages/bf/2d/005e45738ab07a26e621c9c12dc97381f372e06678adf7dc3356a69b5960/numpy-1.13.3.zip
Collecting scipy==0.19.1 (from -r requirements.txt (line 43))
  Using cached https://files.pythonhosted.org/packages/52/67/d9ef9b5058d4a9e3f0ae641ec151790622cbeb37f157de5773358e2bf3da/scipy-0.19.1.tar.gz
Collecting torch==0.4.0 (from -r requirements.txt (line 48))
  Could not find a version that satisfies the requirement torch==0.4.0 (from -r requirements.txt (line 48)) (from versions: 0.1.2, 0.1.2.post1, 0.4.1)
No matching distribution found for torch==0.4.0 (from -r requirements.txt (line 48))

    Let me know if there's anything I can do to help you debug this!

    opened by MrBenJ 11
  • How to build and run project?

    How to build and run project?

    DoodleMaster is a really great job! I love it. And I want to try it by myself, but I don't know how to run. My python knowledge is not good yet. If you can post a how-to-build readme, it'll be very helpful. Thanks!

    opened by CoderInOne 7
  • Incomplete method?

    Incomplete method?

    What's up with div method in layout.py ?

    def div(margin_top,h,w):
    hv

    Method isn't used anywhere. hv is unresolved.

    Can the whole layout.py be removed? I don't see it being used in any other module.

    opened by narenchoudhary 3
  • Possible problem (and solution) to Numpy import error

    Possible problem (and solution) to Numpy import error

    Hello! I created a virtualenv with python 3.6 using the next command: mkvirtualenv --python=/usr/local/Cellar/python3/3.6.2/bin/python3 DoodleMaster -r requirements.txt

    Part of the output was:

    ....
Collecting bcolz==1.1.2 (from -r requirements.txt (line 18))
  Downloading https://files.pythonhosted.org/packages/a8/6c/6351cab42fca7ba7e96191ee24181357e62e79afdad7cf74485c9053ea19/bcolz-1.1.2.tar.gz (1.3MB)
    100% |████████████████████████████████| 1.3MB 2.1MB/s
....
Collecting numpy==1.13.3 (from -r requirements.txt (line 36))
  Downloading https://files.pythonhosted.org/packages/75/06/faf181739f682da35f1310a904e650fc4706558b5657d8ec2f6b29c45220/numpy-1.13.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.5MB)
    100% |████████████████████████████████| 4.6MB 3.1MB/s
....
Building wheels for collected packages: bcolz, imageio
  Running setup.py bdist_wheel for bcolz ... done
  Stored in directory: /Users/jp/Library/Caches/pip/wheels/44/f7/16/4091780f261d71153e383dd4532ffbec14447fd43f3e68ea62
  Running setup.py bdist_wheel for imageio ... done
  Stored in directory: /Users/jp/Library/Caches/pip/wheels/93/41/b3/e4f5788551792cac4742308f077c32b91753b94b3010d44c5d
Successfully built bcolz imageio
Installing collected packages: Werkzeug, click, itsdangerous, MarkupSafe, Jinja2, Flask, Six, Flask-Cors, python-engineio, python-socketio, Flask-SocketIO, Pillow, numpy, bcolz, beautifulsoup4, h5py, imageio, pytz, python-dateutil, cycler, pyparsing, matplotlib, scipy, torch, torchvision
Successfully installed Flask-1.0.2 Flask-Cors-3.0.3 Flask-SocketIO-2.9.3 Jinja2-2.10 MarkupSafe-1.1.0 Pillow-5.2.0 Six-1.11.0 Werkzeug-0.14.1 bcolz-1.1.2 beautifulsoup4-4.4.1 click-7.0 cycler-0.10.0 h5py-2.7.1 imageio-2.2.0 itsdangerous-1.1.0 matplotlib-2.0.2 numpy-1.13.3 pyparsing-2.3.0 python-dateutil-2.7.5 python-engineio-2.3.2 python-socketio-2.0.0 pytz-2018.7 scipy-0.19.1 torch-0.4.0 torchvision-0.2.1

    Everything was right during the installation, but not at the moment of the execution: python server.py

    RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "server.py", line 7, in <module>
    from preprocessing import preprocess
  File "/Users/jp/Documents/DoodleMaster/preprocessing.py", line 6, in <module>
    from util import cmd_image_visualizer, brightenImage
  File "/Users/jp/Documents/DoodleMaster/util.py", line 6, in <module>
    import bcolz
  File "/Users/jp/.envs/DoodleMaster/lib/python3.6/site-packages/bcolz/__init__.py", line 76, in <module>
    from bcolz.carray_ext import (
ImportError: numpy.core.multiarray failed to import

    After that, I reinstall numpy 1.13.3 (as required in the requirements file) using pipenv:

    pipenv install numpy==1.13.3

    Installing numpy==1.13.3…
Looking in indexes: https://pypi.python.org/simple
Requirement already satisfied: numpy==1.13.3 in ./.envs/DoodleMaster/lib/python3.6/site-packages (1.13.3)

Adding numpy==1.13.3 to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (a7595d)!

    When trying again to execute it: python server.py

    RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "server.py", line 7, in <module>
    from preprocessing import preprocess
  File "/Users/jp/Documents/DoodleMaster/preprocessing.py", line 6, in <module>
    from util import cmd_image_visualizer, brightenImage
  File "/Users/jp/Documents/DoodleMaster/util.py", line 6, in <module>
    import bcolz
  File "/Users/jp/.envs/DoodleMaster/lib/python3.6/site-packages/bcolz/__init__.py", line 76, in <module>
    from bcolz.carray_ext import (
ImportError: numpy.core.multiarray failed to import

    I found (here)[https://stackoverflow.com/a/52717947/1674908] that a possible solution was to upgrade numpy to 1.15.2:

    pipenv install numpy==1.15.2

    Installing numpy==1.15.2…
Looking in indexes: https://pypi.python.org/simple
Collecting numpy==1.15.2
  Downloading https://files.pythonhosted.org/packages/0a/2b/726b7d4e4ba844d4805c52b8e05299a5f49bc16c69ca0fa8e1964c0871fe/numpy-1.15.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (24.5MB)
Installing collected packages: numpy
  Found existing installation: numpy 1.13.3
    Uninstalling numpy-1.13.3:
      Successfully uninstalled numpy-1.13.3
Successfully installed numpy-1.15.2

Adding numpy==1.15.2 to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (c8f2af)!

    The numpy upper version solves my problem.

    opened by jpchavat 3
  • ModuleNotFoundError: No module named '_tkinter'

    ModuleNotFoundError: No module named '_tkinter'

    Here is the console error message

    Traceback (most recent call last):
  File "server.py", line 21, in <module>
    import matplotlib.pyplot as plt
  File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 115, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/usr/local/lib/python3.6/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
    globals(),locals(),[backend_name],0)
  File "/usr/local/lib/python3.6/site-packages/matplotlib/backends/backend_tkagg.py", line 6, in <module>
    from six.moves import tkinter as Tk
  File "/usr/local/lib/python3.6/site-packages/six.py", line 92, in __get__
    result = self._resolve()
  File "/usr/local/lib/python3.6/site-packages/six.py", line 115, in _resolve
    return _import_module(self.mod)
  File "/usr/local/lib/python3.6/site-packages/six.py", line 82, in _import_module
    __import__(name)
  File "/usr/local/lib/python3.6/tkinter/__init__.py", line 36, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
ModuleNotFoundError: No module named '_tkinter'
    opened by kuharan 2
  • Two undefined names in layout.py

    Two undefined names in layout.py

    flake8 testing of https://github.com/karanchahal/DoodleMaster on Python 3.7.1

    $ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

    ./layout.py:56:5: F821 undefined name 'hv'
    hv
    ^
./layout.py:81:30: F821 undefined name '_id'
            color = class_id[_id]
                             ^
2     F821 undefined name 'hv'
2
    opened by cclauss 1
  • Broken Change.

    Broken Change.

    Originally posted by @karanchahal in https://github.com/karanchahal/DoodleMaster/issues/14#issuecomment-441159269

    I am getting this now. Collecting matplotlib==2.0.2 (from -r requirements.txt (line 31)) Using cached https://files.pythonhosted.org/packages/f5/f0/9da3ef24ea7eb0ccd12430a261b66eca36b924aeef06e17147f9f9d7d310/matplotlib-2.0.2.tar.gz Complete output from command python setup.py egg_info: ============================================================================ Edit setup.cfg to change the build options

    BUILDING MATPLOTLIB
            matplotlib: yes [2.0.2]
                python: yes [3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018,
                        14:05:16) [MSC v.1915 32 bit (Intel)]]
              platform: yes [win32]

REQUIRED DEPENDENCIES AND EXTENSIONS
                 numpy: yes [not found. pip may install it below.]
                   six: yes [six was not found.pip will attempt to install
                        it after matplotlib.]
              dateutil: yes [dateutil was not found. It is required for date
                        axis support. pip/easy_install may attempt to
                        install it after matplotlib.]
           functools32: yes [Not required]
          subprocess32: yes [Not required]
                  pytz: yes [pytz was not found. pip will attempt to install
                        it after matplotlib.]
                cycler: yes [cycler was not found. pip will attempt to
                        install it after matplotlib.]
               tornado: yes [tornado was not found. It is required for the
                        WebAgg backend. pip/easy_install may attempt to
                        install it after matplotlib.]
             pyparsing: yes [pyparsing was not found. It is required for
                        mathtext support. pip/easy_install may attempt to
                        install it after matplotlib.]
                libagg: yes [pkg-config information for 'libagg' could not
                        be found. Using local copy.]
              freetype: no  [The C/C++ header for freetype (ft2build.h)
                        could not be found.  You may need to install the
                        development package.]
                   png: no  [The C/C++ header for png (png.h) could not be
                        found.  You may need to install the development
                        package.]
                 qhull: yes [pkg-config information for 'qhull' could not be
                        found. Using local copy.]

OPTIONAL SUBPACKAGES
           sample_data: yes [installing]
              toolkits: yes [installing]
                 tests: no  [skipping due to configuration]
        toolkits_tests: no  [skipping due to configuration]

OPTIONAL BACKEND EXTENSIONS
                macosx: no  [Mac OS-X only]
                qt5agg: no  [PyQt5 not found]
                qt4agg: no  [PySide not found; PyQt4 not found]
               gtk3agg: no  [Requires pygobject to be installed.]
             gtk3cairo: no  [Requires cairocffi or pycairo to be installed.]
                gtkagg: no  [Requires pygtk]
                 tkagg: yes [installing; run-time loading from Python Tcl /
                        Tk]
                 wxagg: no  [requires wxPython]
                   gtk: no  [Requires pygtk]
                   agg: yes [installing]
                 cairo: no  [cairocffi or pycairo not found]
             windowing: yes [installing]

OPTIONAL LATEX DEPENDENCIES
                dvipng: no
           ghostscript: no
                 latex: no
               pdftops: no

OPTIONAL PACKAGE DATA
                  dlls: no  [skipping due to configuration]

============================================================================
                        * The following required packages can not be built:
                        * freetype, png

----------------------------------------

    Command "python setup.py egg_info" failed with error code 1 in C:\Users....\AppData\Local\Temp\pip-install-pfl82zhk\matplotlib\

    opened by kuharan 8
Owner
Karanbir Chahal
.
Karanbir Chahal
GitHub
Turn your IPad into a Screen-Slaver with 1 simple Pythonista script

ScreenSlaver Turn your IPad into a Screen-Slaver with 1 simple Pythonista script

null 6 Jul 9, 2022
Turn a raspberry pi into a Bluetooth Midi device

PiBluetoothMidSetup This will change serveral system wide packages/configurations Do not run this on your primary machine or anything you don't know h

MyLab6 40 Sep 19, 2022
A python script to turn tabs into spaces the right way.

detab A python script to turn tabs into spaces the right way. detab turns all tabs into spaces, not just leading tabs. Not all tabs have the same leng

null 1 Jan 26, 2022
This is an online course where you can learn and master the skill of low-level performance analysis and tuning.

Performance Ninja Class This is an online course where you can learn to find and fix low-level performance issues, for example CPU cache misses and br

Denis Bakhvalov 1.2k Dec 30, 2022
Whatsapp Messenger master

Whatsapp Messenger master

Swarup Kharul 5 Nov 21, 2021
EFB Docker image with efb-telegram-master and efb-wechat-slave

efb-wechat-docker EFB Docker image with efb-telegram-master and efb-wechat-slave Features Container run by non-root user. Support add environment vari

Haukeng 1 Nov 10, 2022
Master Duel Card Translator Project

Master Duel Card Translator Project A tool for translating card effects in Yu-Gi-Oh! Master Duel. Quick Start (for Chinese version only) Download the

null 67 Dec 23, 2022
Turn crypto miner on/off depending on powerwall charge level

Mining Crypto with Tesla Solar and Powerwalls This script turns a crypto miner on and off when the Tesla Powerwall level drops/rises above a certain t

Matt 1 Nov 9, 2021
Use a real time weather API to apply wind to your mouse cursor.

wind-cursor Use a real time weather API to apply wind to your mouse cursor. Requirements PyAutoGUI pyowm Usage This program uses the OpenWeatherMap AP

Andreas Schmid 1 Feb 7, 2022
Transform your boring distro into a hacking powerhouse.

Pentizer Transform your boring distro into a hacking powerhouse. Pentizer is a personal project that imports Kali and Parrot repositories in any Debia

Michail Tsimpliarakis 2 Nov 5, 2021
A Python script to convert your favorite TV series into an Anki deck.

Ankiniser A Python3.8 script to convert your favorite TV series into an Anki deck. How to install? Download the script with git or download it manualy

null 37 Nov 3, 2022
A python tool that creates issues in your repos based on TODO comments in your code

Krypto A neat little sidekick python script to create issues on your repo based on comments left in the code on your behalf Convert todo comments in y

Alex Antoniou 4 Oct 26, 2021
An assistant to guess your pip dependencies from your code, without using a requirements file.

Pip Sala Bim is an assistant to guess your pip dependencies from your code, without using a requirements file. Pip Sala Bim will tell you which packag

Collage Labs 15 Nov 19, 2022
About A python based Apple Quicktime protocol，you can record audio and video from real iOS devices

介绍 本应用程序使用 python 实现,可以通过 USB 连接 iOS 设备进行屏幕共享 高帧率（30〜60fps） 高画质 低延迟（<200ms） 非侵入性 支持多设备并行 Mac OSX 安装 python >=3.7 brew install libusb pkg-config 如需使用 g

YueC 124 Nov 30, 2022
pyRTOS is a real-time operating system (RTOS), written in Python.

pyRTOS Introduction pyRTOS is a real-time operating system (RTOS), written in Python. The primary goal of pyRTOS is to provide a pure Python RTOS that

Ben Williams 96 Dec 30, 2022
An unofficial python API for trading on the DeGiro platform, with the ability to get real time data and historical data.

DegiroAPI An unofficial API for the trading platform Degiro written in Python with the ability to get real time data and historical data for products.

Jorrick Sleijster 5 Dec 16, 2022
use Notepad++ for real-time sync after python appending new log text

FTP远程log同步工具 使用Notepad++配合来获取实时更新的log文档效果 适用于FTP协议的log远程同步工具,配合MT管理器开启FTP服务器使用,通过Notepad++监听文本变化,更便捷的使用电脑查看方法注入打印后的信息 功能 过滤器 对每行要打印的文本使用回调函数筛选，支持链式调用

Liuhaixv 1 Oct 17, 2021
Virtual webcam that takes real webcam footage and replaces the background in order to have Virtual Backgrounds in MS Teams for Linux where the feature is unimplemented.

Background Remover The Need It's been good long while since Microsoft first released a Teams version for Linux and yet, one of Teams' coolest features

Dylan Turner 80 Dec 20, 2022
A Linux program to create a Windows USB stick installer from a real Windows DVD or image.

WoeUSB-ng A Linux program to create a Windows USB stick installer from a real Windows DVD or image. This package contains two programs: woeusb: A comm

Longinus 1 Nov 19, 2021