A User Evaluation of Process Discovery Algorithms in a Software Engineering Company

Process mining methods allow analysts to use logs of historical executions of business processes in order to gain knowledge about the actual behavior of these processes. One of the most widely studied process mining operations is automated process discovery. An event log is taken as input by an automated process discovery method and produces a business process model as output that captures the control-flow relations between tasks that are described by the event log. In this setting, this paper provides a systematic comparative evaluation of existing implementations of automated process discovery methods with domain experts by using a real-life event log extracted from an international software engineering company and four quality metrics: understandability, correctness, precision, and usefulness. The evaluation results highlight gaps and unexplored trade-offs in the field and allow researchers to improve the lacks in the automated process discovery methods in terms of usability of process discovery techniques in industry.