Screen Elements

(Available from version 7.0 onwards.)

This section explains how to capture and manage screen elements.

To learn about screen element assets, including methods, properties, and events, see Screen Element Assets.

Screen elements are objects that represent elements displayed in the user interfaces of the applications you want to control through your automation project.

For example, if the process you want to automate includes entering text into a search field and then clicking a search button, you will need two screen elements: one for the search field and one for the search button.

Every screen element is assigned its own methods, properties, and events that you can then use within you workflows, event handlers, callouts, etc.

The Screen Element Workspace

The management and capture of all screen elements is performed from the screen element workspace.

The video below introduces that workspace.

Screen Element Types

Automation Studio provides two different technologies for capturing screen elements.

  • Screen Element Capture: Automation Studio captures and identifies screen elements using APIs and hooks. A range of Connectors are provided: each connector enables capturing screen elements from a different application type, for example, there are connectors for Microsoft Office applications, .Net applications, Java applications, web applications, and more. Screen elements can also be captured and identified using their accessibility properties.

  • IBA (Image Based Analysis): Automation Studio captures and identifies screen elements visually, based on their appearance, location on the screen, and proximity to other items on the same screen. For more information, see IBA Elements.

Screen element capture is generally recommended. IBA capture is recommended when screen element capture cannot be performed.

Overview of Working with Screen Elements

The process of working with screen elements is summarized in the table below. The process is similar for both screen elements and IBA elements.

Step

Description and Example

For Screen Elements

For IBA Elements
1

Launch the screen element capture process,

and capture the screen element:

 Capture a Screen Element Capture an IBA Element
2

Modify the screen element name and other settings.

 Edit a Screen Element

Edit an IBA Element and Edit an IBA Frame

 
3

Use the screen element's methods, properties, and events in your automation project.

 Screen Element Assets  
4

View and modify the values of a screen element's properties in execution mode.

 Monitor Screen Elements in Execution Mode