Skip to main content

Using Speech with iOS and Android: SiriKit, Voice Capabilities, Google Assistant

SiriKit
SiriKit enables your iOS apps and watchOS apps to work with Siri, so users can get things done using just their voice. Your content and services can be used in new scenarios including access from the lock screen and hands-free use.

Apps adopt SiriKit by building an extension that communicates with Siri, even when your app isn’t running. The extension registers with specific domains and intents that it can handle. For example, a messaging app would likely register to support the Messages domain, and the intent to send a message. Siri handles all of the user interaction, including the voice and natural language recognition, and works with your extension to get information and handle user requests.

Apple Developer


Adding Voice Capabilites

Voice actions are an important part of the wearable experience. They let users carry out actions hands-free and quickly. Wear provides two types of voice actions:

System-provided
These voice actions are task-based and are built into the Wear platform. You filter for them in the activity that you want to start when the voice action is spoken. Examples include "Take a note" or "Set an alarm".

App-provided
These voice actions are app-based, and you declare them just like a launcher icon. Users say "Start " to use these voice actions and an activity that you specify starts.

Android Developer


Get Started with System Voice Action

  1. Define an intent filter
  2. Handle the intent in your app
  3. Update your app completion status


Overview of the Voice Interaction API

Whether your app uses system or custom voice actions, there might be times when the app would like to ask the user a follow-up question before performing the action. For example when a user launches a music app by saying “play some music”, the app may want to ask the user “what genre?” Or when a home automation app hears the user say “OK Google, turn on the lights”, it might want to ask “which room?” The Voice Interaction API lets apps ask follow-up questions like these.




The Google Assistant and Media Apps

The Google Assistant lets you use voice commands to control many devices, like Google Home, your phone, and more. It has a built-in capability to understand media commands ("play something by Beyonce") and supports media controls (like pause, skip, fast forward, thumbs up).

Android Developer



Comments

Most Favorite Posts

Lean prioritization matrix

  We are sorting the stream of stakeholder request and product ideas in a KPI-weighted table. The Matrix This 2x2 matrix low effort vs. high effort high value vs. low value is a nice and easy consumable visualization model. The Weights Suggested weights are Reach How many customers does the feature impact? Customers New/existing target groups Revenue Will it drive revenue either direct or via extended CLV? Acquisition Will the feature help drive new customers? Efficiency Does the feature help drive efficiency in customers’ lives – be that internal customers (colleagues), or external (paying) customers? Brand Does the feature enhance your brand awareness? The classification/quadrants Top left to bottom right Q1 Do it now! Q2 Break it down and put in prio sequence Q3 Gap filler Q4 Forget about it... for now. Andy Wicks - Mind the product

Alpha Apps vs. App Unbundling

Aktuell wird viel über das Modell der "Alpha Apps" und "App Unbundling" gesprochen. Hier kurz eine Übersicht und meine 5 cents: Alpha Apps Die chinesische App WeChat geht noch weiter: Neben einem Messenger, vergleichbar mit WhatsApp, bietet sie einen Lieferdienst à la Lieferando, die Möglichkeit etwa das eigene Konto zu checken (wie sonst bei der Bank-App) und gleichzeitig die Chance etwa Promis zu folgen, wie es Twitter bietet. Solche Alpha-Apps können dadurch verschiedene Aspekte und Möglichkeiten des Internets verbinden und werden so zum idealen Zugangsportal zum Netz – so wie traditionell der Browser am Computer. Den Tod des Browsers bedeutet das aber noch lange nicht. Der Browser ist tot, es lebe der Browser! Wirtschafts Woche App Unbundling Unbundling steht für das Unterteilen von Apps oder verschiedener Funktionen in mehrere, eigenständige Applikationen. Aber nicht jede Unbundling Aktion wird positiv von Usern aufgenommen. Facebook Messenger ...

Comcast - Automatic Testing of instable network connections

Comcast improves your test coverage by letting you simulate shitty network connections so you can build better systems. Testing distributed systems under hard failures like network partitions and instance termination is critical, but it's also important we test them under less catastrophic conditions because this is what they most often experience. Comcast is a tool designed to simulate common network problems like latency, bandwidth restrictions, and dropped/reordered/corrupted packets. Usage: $ comcast --device=eth0 --latency=250 --target-bw=1000 --packet-loss=10% GitHub

KSCrash

Another crash reporter? Why? Because all existing solutions fall short. PLCrashReporter comes very close, but not quite: It can't handle stack overflow crashes. It doesn't fill in all fields for its Apple crash reports. It can't symbolicate on the device. It only records enough information for an Apple crash report, though there is plenty of extra useful information to be gathered! As well, each crash reporter service, though most of them use PLCrashReporter at the core, has its own format and API. KSCrash is superior for the following reasons: It catches ALL crashes. Its pluggable server reporting architecture makes it easy to adapt to any API service (it already supports Hockey and Quincy and sending via email, with more to come!). It supports symbolicating on the device. It records more information about the system and crash than any other crash reporter. It is the only crash reporter capable of creating a 100% complete Apple crash report (including thre...

Server-driven UI (SDUI): Meet Zalandos AppCraft and AirBnB Lona

A short WTF: Joe Birch:  SERVER DRIVEN UI, PART 1: THE CONCEPT Zalando seems to follow the SDUI principle as well - defining a common design language and construct the screens on the backend while displaying them natively on the clients. They even go one step further; they implemented a mighty toolset to enable non-technical stakeholders to define their own native app screens Compass: Web tooling to create screens and bind data Beetroot: Backend service that combines the screen layout definition with the data Lapis/Golem: iOS/Android UI render engines Crazy cool! Good job, guys (when you do an open-source release?) To even move faster a Flutter based UI render engine implementation was great! See also AirBnB Lona SDUI approach Building a Visual Language Why Dropbox sunsetted its universal C++ mobile project and AirBnB its React Native implementation

Swift on Android

Running Swift code on Android The biggest issue here is going to be a missing SwiftCore library. Right now Apple is shipping one for iOS, OS X and Watch OS. But that's it - and obviously they don't ship an Android version. However, not all Swift code requires the SwiftCore library, just like not all C++ code requires the STL. So as long as we use the subset of Swift that doesn't hit SwiftCore, we should be ok. Romain Goyet How to Use Apple Swift to Make an Android App Our best course of action to get something working quickly was to take Swift code and compile it into Java source code — and then take it into the Android development environment to finish building it into an installable app. So we started in Xcode, where we wrote the Swift code. Then we tested it, including running it on an iPhone or in the iPhone Simulator, and ran a custom compiler that parsed the Swift code and output Java source code (adjusting any necessary syntax, and substituting appropriate A...

Android with Kotlin, iOS with Swift, Kotlin Native, flutter.io, React Native, PWA, Xamarin, Hybrid - which way to go?

Currently there are tons of frameworks how to get your business model to the user... and in the app store Full Native Android with Kotlin, iOS with Swift Deepest integration Single way to make sure that you have no lock-in effect with a framework, and you are f**ed, when Apple or Google disallows the usage of a specific technology... Two teams required 2x code PWA (Progressive Web App) Write offline- and push-capable PWA with web-technologies only Some native features might require hybrid native development and bridging (like In-App purchases, AR, ...) In best case: One web team only for website and app Maybe some native specialists for special features Kotlin Native Develop a shared framework with or without UI using Kotlin Native Additional native code will most probably be required Big Android team, small iOS specialists flutter.io (React Native | Xamarin | ... ) One codebase (flutter: Dart, React Native: JavaScript, Xamarin: C#) Additional native code ...