PyAutoEasy is a extension / wrapper around the famous PyAutoGUI, a cross-platform GUI automation tool to replace your boooring repetitive tasks.

Dingu Sagar

Last update: Oct 27, 2022

Overview

PyAutoEasy

PyAutoEasy is a extension / wrapper around the famous PyAutoGUI, a cross-platform GUI automation tool to replace your boooring repetitive tasks.

When PyAutoGUI offers powerful ways to control keyboard and mouse movements, PyAutoEasy makes it super easy to grab the cursor positions on the screen and give meaningful names to them and generate the boilerplate program which we can easily modify to suit our needs.

How to install

Thanks to pip! its as simple as :

pip install --upgrade pip
pip install pyautoeasy

Example Usage

Step 1 : Generate initial code after grabbing the required locations in the screen

Open a terminal / command prompt and run

pyautoeasy

Move your cursor to the desired location and press alt+r in the keyboard to record the coordinates of a point in the screen.
This opens up a dialog box, Now give a name to this point and click OK.
Repeat the above 2 steps for all the cursor locations that you want to grab.
Press alt+s to generate your initial automation script. (sample.py)

Step 2 : Modify the generated sample script to suit your needs.

Lets say you wanna create a login automation. Follow step 1 to record all desired locations in the screen like email field, password field, submit button etc. The generated sample program would look something like this.

from pyautoeasy import ScreenPoint

email_field = ScreenPoint(pos=(952, 309))
email_field.click_here()

password_field = ScreenPoint(pos=(934, 438))
password_field.click_here()

submit_button = ScreenPoint(pos=(1127, 597))
submit_button.click_here()

Now you can take above generated file and easily modify it to something like this. Super easy and intuitive!

from pyautoeasy import ScreenPoint

email_field = ScreenPoint(pos=(952, 309))
email_field.click_here()
email_field.type_here("[email protected]")

password_field = ScreenPoint(pos=(934, 438))
password_field.click_here()
password_field.type_here("password@123")

submit_button = ScreenPoint(pos=(1127, 597))
submit_button.click_here()

More about ScreenPoint

A ScreenPoint object represents a point in your screen with (x,y) cordinates specified in the pos argument in the constructor.

point1 = ScreenPoint(pos=(34, 78))

We can use the ScreenPoint object to move the curser there, click on that point, type something there, right click on that point etc. These operations can be done easily by using the following methods :

point1 = ScreenPoint(pos=(34, 78))
point1.click_here() # click on this point
point1.right_click_here() # right click on this point
point1.cursor_here() # move the cursor to this point
point1.type_here("text to type") # type something after selecting this point.
point1.double_click_here() # double click on this point
point1.triple_click_here() # triple click on this point

By default all the operations in ScreenPoint class is done after a delay of `ScreenPoint.Config.delay' (defaults to 1 second). We can change this property gloabally as follows

ScreenPoint.Config.delay = 0.5

Alternatively we can even have a delay specified at each operation by passing the argument after_sleeping_for.

point1 = ScreenPoint(pos=(34, 78))
point1.click_here(after_sleeping_for=0.5) # click on this point after a delay of 0.5 seconds
point1.right_click_here(after_sleeping_for=0) # right click on this point after a delay of 0 seconds.

type_here() method can take an optional argument enter=True which would press an enter key after typing the text. By default enter=True

Comments

Add dragTo funtionality between points.
point1 = ScreenPoint(pos=(23,67)) point2 = ScreenPoint(pos=(239, 88)) point1.dragTo(point2)

something like this wold be useful. we can use the dragTo function present in pyautgui
good first issue hactoberfest
opened by dingusagar 25
Alternate way to execute the interactive program to grab cursor locations.
Currently we just need to run pyautoeasy in terminal/cmd to start this. But there are cases when the path to this program is not currently set while users are trying to install this package. So as a backup option, we should be also able to call the interactive program like this.

python -m pyautoeasy
opened by dingusagar 1
default shortcuts not working on windows

the default shortcuts while running pyautoeasy - alt+r and alt+s are not working in windows. This is because in windows, the alt key is detected as Key.alt_l
bug

opened by dingusagar 1
Test this library on macOS and report any issues if found.

This issue is really to test if its working as expected in a macOS. i have tested it on linux and windows. But if you can create any interesting use-case for testing, you can add the automation script to examples folder and create a pull request.
good first issue hactoberfest

opened by dingusagar 0
General Feedback on PyAutoEasy

This thread can be used to provide general feedback on this library. It can be positive or negative. Any form of general suggestions/ improvements can also be discussed here. Any specific discussions on new feature requests or resolving bugs can be done in seperate issues. Treat this as a casual thread to throw in any feedbacks. :)
feedback

opened by dingusagar 0
Cool Feature to have : Initalise ScreenPoint from text using an ocr engine.
Somthing like this.

point1 = ScreenPoint.from_text("Search")

Internally it should

grab the screenshot of the screen (can be done using pyautogui.screenshot())

do OCR to identify the texts in the screen. (can tesseract or any other library ?)

find the bounding box which best matches the text. get the center of this box and use that as coordinates to initialise ScreenPoint object.

enhancement
opened by dingusagar 0