No software is perfect. Rockets are send to Mars and they crash because of bugs. Some airplanes shut down if they are not “rebooted” at least every 248 days (and you ask me why I am afraid of flying).

The list goes on and on, there is even a Wikipedia article for this - https://en.wikipedia.org/wiki/List_of_software_bugs.

Having and fixing a bug in one central place like an SAP system can already be a pain (system restart, down time, unhappy users, lost data etc.). Fixing a bug in an application installed on many computers (like for example SAP GUI) is a bit trickier, but still manageable if all machines are in-house and connected to the corporate network.

It gets a lot more difficult to ship a new version of an enterprise offline application that is installed on smartphones and tablets outside the company office. Even if you have a device management solution like Afaria, SAP Mobile Secure, Mobile Iron or Airwatch, it is still a lot of work. Because of this fact, it is important to test enterprise offline apps very carefully. It does not matter which platform you use – Agentry, SMP, MobiLink or something else. Testing must be performed on all layers, including client, middleware and backend. The following is a list of tests that should be performed for every application. 

Functional tests

• Test of all functionality in both directions - create data in SAP and synchronize it to the mobile device. Create data on the mobile device and send it to SAP. Update records on both sides, delete data and check that it is removed on the opposite system as well.
• Test validation on the mobile device - is the user getting the right warning/error message when entering wrong data?

Performance tests - synchronization and on device

• Synchronise with the maximum data volume that you expect. Also take into consideration that the data volume could increase over time.
• Test with the maximum number of concurrent devices to see how the system can handle the load.
• Test the device with the maximum amount of data - how is the performance of lists and detail dialogs? How fluent is the navigation within the app? How fast is the startup time?

Test against the UI/UX Guidelines of the mobile operating system

• If you have a native application, is it following the UX guidelines from Apple, Google and Microsoft?
• If you have a hybrid/platform independent application, is it following common guidelines (for example the SAP Fiori UX guide)?

Usability Tests

• Give the application to users and let them test it. Is it intuitive? Can they figure out how to use it even without or with only little documentation?
• Test your solution with users experienced with smartphones/tablets and with those who are not used to modern technology. 
• If you roll out internationally, test with users from different countries to see how they react to the software.
• Test with different languages and location settings
• Let the translation be reviewed and tested by a native speaker
• Listen to your test users!

Test behind the curtain - not only UI, also the middleware and backend integration

• Test client, middleware (if you have any) and backend
• Make sure the communication between the three components is working fine.

Test the positive and the negative cases

• Test your solution with data that you expect to work
• Test your solution with data that you expect to fail
• Make sure both cases are covered successfully
• Test the application when the middleware server is down? Does it behave like expected? Test it when SAP is not available as well.
• Hard kill the app and check if any data is lost

Test outside of the lab/with the real network

• Test your application with the real network your users will use later. Is the performance still good enough with GPRS or Edge?
• Test the software on the real device, not just a simulator
• Test the software with the real device under real conditions. Can you read the screen in bright sunlight?
• Test the solution with all the other apps installed - are there any side effects?
• Test it in low memory situations – is the app still working? How is the performance in this case? What happens if the app is removed from memory?
• Test the app with older versions of the operating system. Also test it with beta versions that will be released in the future!

Write automated test scripts

• Let your developers create automatic test cases that can be run before every build.

Test on different devices

• If you have a BYOD (bring your own device) setup, try to test the application on different devices. Is the app working on all screen resolutions? If platform independent, is it working on all platforms?

Test the installation/update

• Test the installation and upgrade of your mobile app. Can the APK, IPA or XAP be downloaded over the phone connection?

Test the security of the app

• Use a network sniffer to see if the encryption that you switched on is active.
• Try to open the device database to see if it is really encrypted.
• Test the secure storage like for example the iOS keychain – is this data only local or is it send to the cloud? 

Test the administration part of the app

• Can you create new users, devices etc. in your mobile landscape?
• Can you unregister/delete devices
• Are the logs setup correctly? Do they have enough information to analyse issues? Is it set low enough that the log is not decreasing the performance?

No matter how long and well you test, there will be bugs slipping through the cracks. The more complex the app is, the more likely that will be. Because of that you should always have a fall-back solution - this could be either an Excel sheet or just a piece of paper. Ideally you never need to use it. 

Testing is a hot topic - how much testing is enough? How much is too much? You could argue that one can never test enough, however somebody must pay for it. Like with everything, you need to find the right balance. 

Good luck with your testing!