Since the 1960s, robots have been used in many manufacturing applications, with the first industrial robot, the Unimate, installed by General Motors in 1961. In the present day, the use of robots within manufacturing has continued to advance, with the development of collaborative robots designed to work alongside humans in shared spaces.
Within the UK, BAE Systems are pioneering the use of collaborative robots in aircraft manufacturing and have established a new experimental factory called the Factory of the Future (FotF), where such technologies are being tested. One important aspect of this testing is to understand the way in which humans interact within this environment and to demonstrate that humans and robots can work safely together.
Considering this, the primary aim of this project was to evaluate the effectiveness of how this visual information is conveyed to the operator and to review how the systems work as whole. As a secondary aim, the project also assessed the level of confidence the overall approach to safety has instilled in the operator.
A mixed methods study was performed collecting quantitate data via user testing and qualitative data from semi-structured interview and questionnaire. Testing was split into three phases:
Phase 1 – User Testing of Colour Coding
The first phase of testing assessed the intuitiveness of the use of colour. Individual participants were positioned within the RAAC and taken through a set scenario where each visual indication and colour option was shown. Participants were then asked to choose the colour meaning from a pre-defined list, which included a single correct answer and the remaining incorrect answers.
A baseline test was performed using 6 existing operators trained on the current meaning of colour. An identical test was repeated with 6 untrained participants. The performance of trained and untrained participants was compared, thus providing insight into the intuitiveness of the use of colour and the reliance placed on training.
The next test used the same 6 untrained participants and repeated the test with a modified colour philosophy. The new philosophy was based on analysis of data from the previous test and included consideration of associated ergonomic principles. The aim was to try and improve participant performance and produce an optimal colour philosophy.
Phase 2 – User Testing of Installation Effects
The second phase of testing assessed the effectiveness of the installation of the visual indications, e.g., can operators see each indication from their typical working position. 6 participants were positioned at set locations within the RAAC, each marked for location and direction of orientation, and were asked to declare when they saw a change in operating state. The change in operating state was triggered by the test co-ordinator and user response time was recorded. Participants were not told which indication was about to change, and to simulate working conditions, participants were asked to perform a basic robot movement operation, either from the pendant or workstation interface controls.
A baseline reaction time test was completed to record how quickly each participant would respond when no other cognitive processing is required. In this case, each participant was informed of the change to take place and were positioned in line-of-sight of the indication, thus removing installation effect. The baseline reaction time and the reaction times recorded at locations A, B and C were compared for each participant, therefore to assess if the installation has negatively impacted on participant reaction time.
Phase 3 – User Interview / Questionnaire.
Following completion of phases 1 and 2, the group of trained operators were asked to attend an individual semi-structured interview. A thematic analysis of the recorded data was conducted using NVivo data analysis software and participants were asked to complete both System Usability Scale (SUS) and Trust in Automation Systems Test (TOAST) questionnaires.
Analysing participant performance data via Null Hypothesis Statistical Testing (NHST) showed a significant difference, with trained participant’s outperforming untrained when attempting to identify the meaning of colour. Opportunities for improvement were also identified, with certain colours not producing the intended participant response. Testing of the revised colour philosophy showed a significant increase in performance.
Operator response times were generally acceptable, whilst in selected cases were impacted by physical obscuration within the facility. Finally, user feedback suggested that operators do feel safe when working within the RAAC environment and have developed confidence in the safety systems implemented.
