Varian Medical Systems was facing new FDA requirements for safety and usability human factors testing for all medical devices and software. EchoUser was tasked to develop a new process for testing medical devices and piloted this process through usability tests on the new version of Varian’s TrueBeam radiation cancer treatment machine.
The methodology that EchoUser developed became standardized for testing a wide range of medical devices and software across Varian.
Approach
EchoUser collaborated with the Varian team for a year-long project to develop, pilot, and implement an FDA-compliant, rigorous, repeatable process to measure, diagnose, classify, and report on the use safety and experience of its medical products. Our goal was to assess how well-designed, productive, and safe the interfaces were for conducting tasks. To achieve this, we utilized benchmark usability tests to evaluate the product’s level of usability and use safety.
Our team conducted the pilot on the TrueBeam radiotherapy machine, which is used to treat cancer by delivering targeted radiation to cancerous cells with pinpoint accuracy. This complex imaging system encompasses both hardware and software, and both patient and operator considerations. We tested the process from start to finish on a large, highly complex, and safety-critical system.
During the usability tests, we tested both current and potential users with possible scenarios to gauge their perceptions of the device’s current safety. The test results helped us identify areas for improvement as well as areas that performed beyond expectations.
Outcome
We categorized research measures across four basic Key Performance Indicators (KPIs) to determine the product’s effectiveness, use safety, efficiency, and satisfaction. We used this testing methodology and measurement classification to evaluate the TrueBeam machine's usability, and identified areas that needed improvement and areas that exceeded expectations.
Our usability testing process helped Varian enhance the safety and usability of their products, improve patient experience, and maintain compliance with regulatory requirements. Through our partnership with the Varian team, we were able to provide valuable insights that helped them create safer, more effective products for cancer patients.
