Augmented reality (AR) is a tricky field in many regards. One area where it is especially challenging is AR application testing. Guidelines for augmented reality testing applications are still being developed, and testing engineers often have to find the right approach by trial and error.
Looking for Augmented Reality Software? Check out SoftwareSuggest’s list of the best-augmented reality software solutions.
5 Common Challenges in Augmented Reality Projects
1. Non-standard interfaces and 3D space
Checklists for GUI testing cover such basic elements as windows, icons, menus, and pointing devices (WIMP). However, augmented reality apps go far beyond the standard WIMP interface model, allowing users to interact with the app in a wide range of creative ways, including interactions in 3D space. This also makes standard heuristics for usability testing inefficient in the case of AR app testing.
Some research groups are already working on the problem, striving to create new standards in GUI and usability testing that would reflect all the peculiarity of multidimensional relations between the user and the imaginary world of augmented reality. But there’s a growing avalanche of AR apps that should be tested here and now. So, for the time being, testing engineers have to invent their own heuristics.
QA consultants from A1QA recommend starting with studying relevant research papers – for example, here, one can find a checklist for usability testing of AR applications. It can be used as a basis for your own checklist tailored to a particular QA project.
2. Motion Testing
Motion testing means assuring accurate superimposition of digital data onto an image of the real world in motion. To achieve that, the movement and rotation of a smartphone have to be perfectly translated from the physical camera to its virtual representation.
“This sounds simple; it’s actually really, really hard. While your phone gyroscope is really great for rotation, it can drift over time. And while your accelerometer is great for those instantaneous inputs, it’s not so great to figure out actual position,” Google AR Developer Tom Slater says. Further on, he explains how Google coped with this development challenge in their ARCore framework by applying the technique known as concurrent odometry and mapping (COM).
As you can see, motion testing seems to be a goldmine of bugs in AR applications. So, it’s crucial to check for augmented objects starting to float from their intended positions when the user moves the device in different, unexpected ways.
3. Variability of hardware
With any new technology, there is not going to be much standardization. There are no central governing authorities and no industry standards that have been set over time. On the other hand, there are a plethora of different devices, operating systems, screen sizes, camera resolutions, and numerous other variables that can make testing AR applications something of a nightmare.
An AR application renders differently on each screen for each device. While this is a typical challenge for software testing in general, with AR applications, cross-platform testing is aggravated by the variety of possible devices. All smartphones are different, but they are still smartphones. But AR apps can also run on smart glasses, trade show displays, interactive kiosks, and whatever else (remember Pepsi’s Monster Mirror?).
There are few emulators to test augmented reality apps, so testing engineers usually need access to the device itself to test all components of the new technology. As augmented reality works in real-time interacting with physical objects, it is often necessary to walk with the actual device in a physical location.
This represents an obvious budget and convenience limitation. Emulators such as iOS Simulator or Android Studio do not really replicate the experience of an actual user, and they do not address application performance when used on a specific device. Device clouds are possibly the best type of emulator, allowing remote access to physical devices. However, testers are still not interacting with the device in the same way that users would. So it is best to do testing with a mixture of real devices and emulators.
4. Legal Implications
A comprehensive legal framework with regard to AR application testing has not been created yet. The physical nature of the application aside, it will soon become necessary to test how one AR application interacts with another so that privacy is maintained. Data privacy is an important legal issue that is increasing in prominence as the world becomes more digitalized and cybercrime continues to increase.
The legal implications have not yet surfaced, but this will be a huge issue as the industry becomes more developed.
5. More Testing Required
Right now, testers are ensuring that no obvious bugs are visible and that the application runs smoothly. However, they may need to check that the application is an accurate representation of reality and that dangerous areas render a warning to the user. This is more relevant to VR headsets and pertains more to the developers than the testers. However, an augmented reality testing application is going to need far more testing than a regular mobile or web application.
Because there is a physical element involved, testers will have to assess how the movement will affect user experience. Testing engineers may need to take into account how usable the device is to the physically impaired or those who suffer from motion sickness.
Bottom Line
In general, the more sophisticated and futuristic the software, the more extensive the testing will need to be. This is especially true for augmented reality apps, which deal with reality so closely that many real-world augmented reality challenges enter them and complicate testing. The lack of generally approved augmented reality testing standards adds to the problem. However, drawing upon the state of the art in software testing research, paying attention to the weakest points of AR apps, and ensuring wide-scale device coverage is a good start for testers striving to assure the impeccable quality of an AR app.
A digital marketing and content curator at SoftwareSuggest, a software recommendation platform. I am passionate about Digital Marketing, and all things digital. Also interested in technology and innovation. In my spare time, enjoy playing cricket, and Badminton.
