Table of Contents
Open Table of Contents
Summary
Between August 2020 and August 2021, I contributed 20 hours per week to a research project alongside my role at Allplan Österreich GmbH, in partnership with the PEEC research group at the University of Applied Sciences Upper Austria and in collaboration with Microsoft Research Cambridge. My main responsibility was the implementation of a prototype for the semi-automated analysis of collaborative interactions using C# and Python, supported by various AI services. The project culminated in a research paper published in 2022.
How I joined the project
During my master’s thesis at the University of Applied Sciences Upper Austria, I came across a posting for software developers in this initiative. Since remote and hybrid collaboration had become indispensable during the COVID-19 pandemic, the project addressed a highly relevant topic at the time. I therefore negotiated reduced hours at Allplan to participate, and after completing my degree in July, I joined the project in August.
Challenges
Automatically analyzing collaboration in images and video proved to be deceptively complex. Even human annotators struggled to distinguish fine-grained interaction types, making manual analysis highly time-consuming. Hybrid and remote work further blurred the boundaries between collaboration modes, complicating reliable automated recognition.
Project work
We collaborated closely with researchers at Microsoft Research Cambridge, iterating on ideas, methods, and evaluation strategies. My primary responsibility was end-to-end system development: designing, implementing, and integrating components while assessing the feasibility of automatic detection of distinct collaboration processes. To this end, we evaluated a range of AI services and frameworks, including Microsoft Cognitive Services (now Azure AI Services) and Google Cloud, as well as experimental technologies such as Microsoft Platform for Situated Intelligence (PSI) and Microsoft Project Tokyo, to benchmark performance against our objectives.
By the end of the year, we delivered a working prototype and conducted a comprehensive user study to validate its functionality and usefulness. Our findings informed the final design and were published in December 2022, documenting the prototype, methodology, and key lessons learned.
Paper
T. Neumayr, M. Augstein, J. Schönböck, S. Rintel, H. Leeb and T. Teichmeister, “Semi-Automated Analysis of Collaborative Interaction-Are We There Yet?”, Proceedings of the ACM on Human-Computer Interaction (PACM HCI), vol. 6, no. ISS, pp. 354-380, 2022, doi: https://doi.org/10.1145/3567724.