This is a personalized fork of the original image recognition library for Robot Framework. It extends the functionality with additional keywords and features required for an ongoing desktop image recognition testing. Keywords such as Click With Offset or Wait For And Click Image have new advanced arguments for full control mouse control. These include, for example:
| argument | type | default | description |
|---|---|---|---|
x_offset |
int | 0 | offset in the horizontal plane in both positive and negative directions (right / left) |
y_offset |
int | 0 | offset in the vertical plane in both positive and negative directions (up / down) |
button |
str | 'left' | specified mouse button to click with (possible are 'right', 'left', 'middle') |
clicks |
int | 1 | number of clicks (in interval) of the specified button |
interval |
float | 0.0 | interval of clicks in seconds |
Keywords have also been added for more convenient usage, such as Does Not Exist or Scroll Window.
Planned features include the integration of new image processing options (such as blurring and edge detection) advanced image comparison settings, OCR options to get text from images/screenshots and the ability to define custom search frames for more precise recognition.
- Click With Offset
- Click With Offset From Location
- Scroll Window
- Click Image (enhanced with optional arguments)
- Click With Offset From Image
- Does Not Exist
- Wait For And Click Image
- Click Image And Wait For
- Click Image If Exists
- Set Timeout
The library import accepts a timeout option (in seconds, defaults to
10). It is used by every timeout-aware keyword (e.g. Wait For) whenever
no explicit timeout is passed to the keyword. An explicit value on the
keyword always takes precedence, and the default can be changed at runtime
with Set Timeout.
*** Settings ***
Library ImageHorizonLibrary timeout=30
*** Test Cases ***
Example
Wait For dialog # waits up to 30 s (library default)
Wait For dialog timeout=5 # waits up to 5 s (explicit wins)This Robot Framework library provides the facilities to automate GUIs based on image recognition similar to Sikuli. This library wraps pyautogui to achieve this.
For non pixel perfect matches, there is a feature called confidence level that comes with a dependency OpenCV (python package: opencv-python). This functionality is optional - you are not required to install opencv-python package if you do not use confidence level.
Original Keyword Documentation
- Python >= 3.6
- pip for easy installation
- pyautogui and it's prerequisites
- Robot Framework
On Ubuntu, you need to take special measures to make the screenshot functionality to work correctly. The keyboard functions might not work on Ubuntu when run in VirtualBox on Windows.
If you have pip, installation is straightforward:
$ pip install robotframework-imagehorizonlibrary
This will automatically install dependencies as well as their dependencies.
ImageHorizonLibrary should work on Windows "out-of-the-box". Just run the commands above to install it.
NOTICE ImageHorizonLibrary does not currently work with XCode v.8. Please use a previous version.
You additionally need to install these for pyautogui:
$ pip install pyobjc-core pyobjc
For these, you need to install XCode
You additionally need to install these for pyautogui:
$ sudo apt-get install python-dev python-xlib
You might also need, depending on your Python distribution, to install:
$ sudo apt-get install python-tk
If you are using virtualenv, you must install python-xlib manually to the virtual environment for pyautogui:
- Fetch the source distribution
- Install with:
$ pip install python-xlib-<latest version>.tar.gz
$ python tests/utest/run_tests.py [verbosity=2]
Additionally to unit test dependencies, you also need OpenCV, Eel, scrot and Chrome/Chromium browser. OpenCV is used because this tests are testing also confidence level. Browser is used by Eel for cross-platform GUI demo application. scrot is used for capturing screenshots.
$ pip install opencv-python eel
To run tests, run this command:
$ python tests/atest/run_tests.py
To regenerate documentation (doc/ImageHorizonLibrary.html), use this command:
$ python -m robot.libdoc -P ./src ImageHorizonLibrary doc/ImageHorizonLibrary.html