
Discover Sikuli, an image recognition tool that automates anything visible on screen, with open source, cross platform Python usage, and limitations like needing an active UI and scaling considerations.
Install sikuli on mac by enabling execution from security and privacy, approving installation prompts, granting IDE permissions, and launching the SikuliX IDE to begin automation.
Install sikuli on linux (ubuntu 16.04 x64) by updating the system, installing java, opencv, and tesseract, then download and run the sikuli id setup jar to launch the ide.
Discover how scaling and zoom affect Sikuli automation visuals and image matching. Maintain consistency by using 100% scaling and checking display settings and resolution when transferring scripts between machines.
Navigate the sikuli script folder, create and save a demo script with image references, and learn how python paths and hmo previews help share and visualize the automation.
Master Sikuli's basic automation functions—click, type, wait, drag and drop, and find—capture screenshots, use highlight, and run scripts with delays, saving organized task scripts and images.
Learn to automate actions with Sikuli and Python by taking screenshots, waiting for images, clicking labels, and handling timeouts and hover interactions in real examples.
Learn to use the find function to locate an image, highlight it with a rectangle for two or three seconds, and see what's going on in the script with Sikuli.
Explore matching preview in Sikuli, learning how to adjust width and similarity to reliably identify buttons. 70 to 90 percent similarity helps prevent misclicks and handle duplicates and missing buttons.
Master Sikuli's click function with image-based and left panel methods, handling missing buttons and errors. Name screenshots clearly, tune similarity, and verify clicks with precise image matching for reliable automation.
Automate tasks with Sikuli and Python by mastering drag and drop: match images, adjust similarity, and control a slider across web or Windows apps.
Master Python basics by exploring data types, variables, conditional statements, and for and while loops, with a gentle introduction and a focus on practical programming fundamentals.
Learn Python basics, data types, variables, conditionals, and loops, with an introduction to Python and Jython. Discover why Python is cross platform, expressive, and supported by rich libraries.
Explore how if-elif-else control flow evaluates true or false conditions, stopping the checks when a true condition is found, and how to execute the corresponding block or else branch.
Explore how a for loop iterates over a string or list, assigning each item to a variable and executing code for every element, including automating clicks on images and checkboxes.
Explore the find function in Sikuli with Python, learn to assign images to variables, highlight matches, and perform clicks to automate on-screen tasks.
Learn to use find all to locate all unchecked checkboxes from a single image, store them in a list, and click each with a Python loop.
Learn to automate a UI with Sikuli and Python using wait vanish to detect an image disappearing and press stop when the target image vanishes.
Explore the exists function in Sikuli to detect image presence, return true or false, and automate login flows with Python by scrolling with page down when the button isn't visible.
Learn to create a PopAsk pop-up, capture the user’s yes or no as a boolean, assign to a variable, and print the result.
Hello and welcome to the Amazing world of Sikuli Automation.
This course is an introduction to Sikuli, a tool that lets you automate anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI's internal or source code. You can easily Automate the Boring Stuff with Python Programming. The course is designed with a lot of real examples. By the end of this course, you'll be able to write scripts that will dramatically increase your productivity.
What is SikuliX?
Sikuli automates anything visible on the screen of your computer running Windows, Mac, or some Linux/Unix
It uses image recognition to identify and control GUI components, like a real user.
It can run the mouse and the keyboard to interact with the identified GUI elements through image recognition
Sikuli comes with basic text recognition that can be used to search text in images.
Sikuli was developed at the User Interface Design Group at MIT
Sikuli has an OCR-mode, in which image matches are performed after converting those image patterns to text. The OCR-functionality is powered by Tesseract, an open-source optical character recognition engine whose development is sponsored by Google.
P.S.: Thanks to my mentor, colleague, and friend Neven Dinev who showed me this incredible tool years ago.