Interactive Segmentation of User Interface Logs

Robotic Process Automation (RPA) is an emerging technology that relies on software (SW) robots to automate intensive and repetitive tasks (i.e., routines) performed by human users on the application’s User Interface (UI) of their computer systems. RPA tools are able to capture in dedicated UI logs the execution of many routines of interest. A UI log consists of user actions that are mixed in some order that reflects the particular order of their execution by the user, thus potentially belonging to different routines. In the RPA literature, the challenge to understand which user actions contribute to which routines and cluster them into well-bounded routine traces is known as segmentation. In this paper, we present a novel approach to the discovery of routine traces from unsegmented UI logs, which relies on: (i) a frequent-pattern identification technique to automatically derive the routine behaviors (a.k.a. routine segments) as recorded into a UI log, (ii) a human-in-the-loop interaction to filter out those segments not allowed (i.e., wrongly discovered from the UI log) by any real-world routine under analysis, and (iii) a trace alignment technique to cluster all those user actions belonging to a specific segment into routine traces. We evaluate our approach showing its effectiveness in terms of supported segmentation variants.