Skip to main content

Using Speech with iOS and Android: SiriKit, Voice Capabilities, Google Assistant

SiriKit
SiriKit enables your iOS apps and watchOS apps to work with Siri, so users can get things done using just their voice. Your content and services can be used in new scenarios including access from the lock screen and hands-free use.

Apps adopt SiriKit by building an extension that communicates with Siri, even when your app isn’t running. The extension registers with specific domains and intents that it can handle. For example, a messaging app would likely register to support the Messages domain, and the intent to send a message. Siri handles all of the user interaction, including the voice and natural language recognition, and works with your extension to get information and handle user requests.

Apple Developer


Adding Voice Capabilites

Voice actions are an important part of the wearable experience. They let users carry out actions hands-free and quickly. Wear provides two types of voice actions:

System-provided
These voice actions are task-based and are built into the Wear platform. You filter for them in the activity that you want to start when the voice action is spoken. Examples include "Take a note" or "Set an alarm".

App-provided
These voice actions are app-based, and you declare them just like a launcher icon. Users say "Start " to use these voice actions and an activity that you specify starts.

Android Developer


Get Started with System Voice Action

  1. Define an intent filter
  2. Handle the intent in your app
  3. Update your app completion status


Overview of the Voice Interaction API

Whether your app uses system or custom voice actions, there might be times when the app would like to ask the user a follow-up question before performing the action. For example when a user launches a music app by saying “play some music”, the app may want to ask the user “what genre?” Or when a home automation app hears the user say “OK Google, turn on the lights”, it might want to ask “which room?” The Voice Interaction API lets apps ask follow-up questions like these.




The Google Assistant and Media Apps

The Google Assistant lets you use voice commands to control many devices, like Google Home, your phone, and more. It has a built-in capability to understand media commands ("play something by Beyonce") and supports media controls (like pause, skip, fast forward, thumbs up).

Android Developer



Comments

Most Favorite Posts

Validity Time Auto-Renewables in Sandbox

The subscription durations, sandbox durations and incentive durations of auto-renewables. Hint: In Sandbox the validity time differs from live environment!!! Durations Sandbox Duration Incentive Durations (optional) 7 days 3 minutes 7 days 1 month 5 minutes 7 days, 1 month 2 months 10 minutes 7 days, 1 month 3 months 15 minutes 1 month 6 months 30 minutes 1 month, 2 months 1 year 1 hour 1 month, 2 months, 3 months After 6 extensions the abo is cancelled automatically in the sandbox environment.

Backend-driven native UIs

Backend-drive native UIs John Sundell  Slide Share Using Back-End Design to Create Customizable Front-End Mobile Experiences By controlling the front end of mobile apps from the back end we can build customized experiences at runtime, creating cleaner interfaces and reducing load times. Nithin Rao UX Magazine The Hub Framework Welcome to the Hub Framework - a toolkit for building native, component-driven UIs on iOS ( no Android support released yet ). It is designed to enable teams of any size to quickly build, tweak and ship new UI features, in either new or existing apps. It also makes it easy to build backend-driven UIs. The Hub Framework has two core concepts - Components & Content Operations. Spotify LeeGo: Build UI without UIView LeeGo is a lightweight Swift framework that helps you decouple & modularise your UI component into small pieces of LEGO style's bricks, to make UI development declarative, configurable and highly reusable. Wang Shen...

UI Automation

There is a known issue when using the performTaskOnHost API in a UI Automation script. If the task being performed with the API outputs excessively (say, thousands of lines of text) to standard out or standard error, the task may deadlock until the timeout is reached, at which point it will throw a javascript exception. The lock() and unlock() functions of UIATarget have been replaced with the lockForDuration( ) function. Starting in iOS 5 beta 2, you can now trigger the execution of an UI Automation script on an iOS device from the host terminal by using the instruments tool. The command is: instruments -w device id -t template application When using the cli instruments for UI Automation you can now target the default Automation Template and pass the script and results path into the tool as environment variable options. For example: instruments -w device_id -t /Developer/Platforms/iPhoneOS.platform/Developer/Library/Instruments/PlugIns/AutomationInstrument.bundle/Contents/...

Microservice Micro-Feature architecture for mobile Apps

RIBs is the cross-platform architecture framework behind many mobile apps at Uber. The name RIBs is short for Router, Interactor and Builder, which are core components of this architecture. This framework is designed for mobile apps with a large number of engineers and nested states. The RIBs architecture provides: Shared architecture across iOS and Android. Build cross-platform apps that have similar architecture, enabling iOS and Android teams to cross-review business logic code. Testability and Isolation. Classes must be easy to unit test and reason about in isolation. Individual RIB classes have distinct responsibilities like: routing, business, view logic, creation. Plus, most RIB logic is decoupled from child RIB logic. This makes RIB classes easy to test and reason about independently. Tooling for developer productivity. RIBs come with IDE tooling around code generation, memory leak detection, static analysis and runtime integrations - all which improve developer product...

iPad HTTP Debugging with Charles

After noticing that the caching in iPad Safari seemed a little funky, I made an effort to decipher some of the logic used by the browser cache. I didn’t get very far, but in the process I figured out how to route my iPad HTTP traffic through a web debugger on my laptop. It turns out it was very easy to do (although I’m sure there is a more complicated way to go about it). What follows is a simple step-by-step for connecting your iPad to an HTTP debugging proxy. The main requirement is that your desktop/laptop and iPad be on the same wireless network. Then it’s just a matter of telling your iPad to use your desktop as an HTTP proxy. iPad HTTP Debugging with Charles

Starting UIAutomation via command line

You can do it now, starting with XCode 4.2 for iOS5 beta 4 From command line, you can run instruments pointing to the automation template and specify as environment variables the test script you want to execute and destination path for results: instruments -w device_id -t /Developer/Platforms/iPhoneOS.platform/Developer/Library/Instruments/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate application -e UIASCRIPT script -e UIARESULTSPATH results path DevForums Apple instruments Mac OS X Developer Tools Manual Page

Git Workflows by Yan Pritzker

Why use git? Git is an extremely powerful source control system. Its power lies in its speed and flexibility, but this can also be a point of confusion for many new users. Git is unfortunately quite inconsistent in its syntax, and exposes many of its not-so-friendly internals to the outside world, sometimes to the detriment of usability. As many systems built by hardcore engineers (Git came from Linux kernel hackers), if wielded by a wizard, it can be used to achieve many great things, but can be initially confusing even for seasoned developers. This book bypasses the ugly internals of git and gets to the heart of improving your development workflows by using git. If you've picked up this book, it's likely you're already convinced that git is great. For a great overview, check out http://whygitisbetterthanx.com. To sum it up, the following are my favorite features of git, which this book will focus on. Git Workflows Book