Why Multimodal UI is the future of mobile

At Waracle, nothing gets us more excited than talking about digital technology and in particular mobile trends such as multimodal UI. As Scotland’s digital products specialist, we love to work with our clients to ensure their customers or employees can interact with technology in order to do something better than they did before, or do it faster than they did it before. Digital transformation is about transforming an existing process into a much more efficient process.

More often than not, this now involves multimodal UI. This simply means that the user has multiple ways of interacting with the application e.g. input text and use voice control and view text and listen to responses. Multimodal UI enables faster and more free and natural communication. It can offer a flexible and efficient interface allowing users to interact through voice, text, hand gesture and even gaze, and to receive information from the app through output modalities, such as speech synthesis, smart graphics and more.

What does this all mean for app development?

International Data Corporation (IDC) forecasts worldwide spending on digital transformation technologies to reach $1.3 trillion in 2018. As Waracle’s business expands with the opportunities emerging from technologies such as voice, Internet of Things, blockchain and augmented reality we know that multimodal UI plays a key role in the future of mobile.

Traditionally apps have been self-contained experiences. At Waracle, we anticipate a world where multimodal UI’s become the norm, making implementation much more complex, but importantly allowing for vastly improved user experiences.

Our UX and development teams are already taking into account multiple entry points to the digital experience so as app users interact with a number of interfaces. We work with Amazon Alexa, Siri, Google Assistant and more to create some incredibly smart user experiences.

The future of mobile lies in the speed of mobile

The future of mobile is pretty much determined by the benefits of speed. By this we simply mean that the mobile UI needs to drive the means to the purpose of the task. For example whether the purpose of the task is communication, entertainment, commerce, or basically anything we do on a mobile device, the speed of the mobile UI will determine the future success of the app.

The UX of the app will continuously be pushed to maximise the speed of information sharing. For example, if humans can type at 40 words per minute and speak at around 130 words per minute, then this sets the optimal experience for interaction with a mobile device.

Now you can understand exactly why Apple, Google and Microsoft are spending billions on developing better speech recognition and AI!

Multimodal UI

At Waracle, we know the future lies in a multimodal interface such as one allowing a user to speak via a variety of entry points – for instance, within an app, within an assistant like Siri, or within a chatbot. The user will then receive information visually through the app or through dedicated messenger or bot applications. Again, now you can understand why Apple and Google are adding UI capabilities to their messaging apps!

Multimodal UI Office Purchase App

So, how might an experience be improved by using multimodal UI? Let’s say, for example, you’re thinking about buying a new laptop. You might start by asking your mobile phone voice assistant “What are the best laptops?”

This would generate a list of laptops, which you can read.

Imagine if this data could display within your Office Supplies Mobile App, within a messaging app, or within an assistant – so that the data is separate from the basic application.

As you select a laptop (perhaps first by looking at reviews, what your colleagues have seen and liked, asked if anyone has any advice), you will constantly go between speaking and reading – i.e. between voice – and screen-based interfaces. So how could this work better using a multimodal UI?

  • Spoken interaction – I want a new laptop and ask my phone via Google Assistant or Siri to look up latest laptops.
  • Reading interaction – The Office Supplies app launches and shows me options, displaying them within the Siri/Chatbot interface.
  • Spoken and Reading Interactions – I ask for reviews of 2 laptops and see them.
  • Spoken interaction – I ask to send the laptops and the reviews to 2 colleagues via Slack and my director via email.
  • Typing and Reading interaction – After I hear back from my colleagues and director, I go into Siri/Google Assistant and order the laptop.
  • Touch ID – The app launches and I confirm purchase with a touch ID.
  • Spoken interaction – I tell the app to send a copy of the order to my director.

This is the future of mobile UI! This is just one business use example, but as you can see, our lives are about to get a whole lot easier with Multimodal UI.

IDC state that by 2021, at least 50% of global GDP will be digitised, with growth driven by digitally-enhanced offerings. Isn’t it time your business started to think about digital transformation? If you would like to learn more about other trends affecting mobile app development, download one of our free whitepapers on mobile trends and mobile marketing.

About Waracle Mobile App Development

As digital professionals, our job at Waracle is to continue exploring the design and technical challenges of bringing such multimodal experiences to life. Waracle is one of the UK’s leading mobile app development companies and Scotland’s digital products specialist. We’ve established the largest digital skills capability in Scotland with offices in Glasgow, Edinburgh and Dundee. We offer full lifecycle mobile app and digital product services through conception, design, development and post launch optimisation.

We specialise in a widening range of technology: Mobile Web, iOS, Android, Hybrid, Native, Internet of Things (IoT), Voice and Augmented Reality (AR) and we have achieved global recognition from independent research companies Clutch and AppIndex.

Subscribe to Our Thoughts