Desktop Operation Simulation API

CukeTest provides a set of objects in the leanpro.common library to simulate mouse, keyboard, and screen operations:

Since these operations are implemented via simulation, they can be used across all platforms.

Purpose

In CukeTest, there are advanced APIs for direct object recognition and interaction. However, in some cases, automation still requires a combination of keyboard and mouse operations. To address this, CukeTest provides Mouse and Keyboard objects.

They allow handling scenarios where:

  • The target position for mouse movement does not correspond to a UI control.
  • Built-in object APIs cannot complete the task.

For example, in some applications, menus are hidden at the edge of the screen and only expand when the mouse moves to that edge. In such cases, direct mouse and keyboard operations are required.

An automation script using mouse and keyboard operations can be written as follows:

JavaScript
Python
const { Keyboard, Mouse } = require('leanpro.common');
    Mouse.move(1920, 1080);

    Keyboard.keyDown("control");
    Keyboard.keyTap("a");
    Keyboard.keyUp("control");

from leanproAuto import Keyboard, Mouse
    Mouse.move(1920, 1080)
    Keyboard.keyDown("control")
    Keyboard.keyTap("a")
    Keyboard.keyUp("control")

Additionally, since Mouse operations (movement, clicking) are based on screen coordinates, CukeTest also provides the Screen object. This object not only retrieves screen properties (such as resolution) but can also capture screenshots:

JavaScript
Python
Screen.capture();
Screen.capture()

Note: These methods are synchronous and do not require the use of await.