App Actions on Enterprise Devices (Part 2)

This is the second of two blogs exploring the newly announced Shortcuts feature of App Actions and whether it will work well on Enterprise devices. A sample app to accompany this series is available at https://github.com/darryncampbell/App-Actions-Demo/

This blog considers custom intents and builds on the first part, which covered built-in intents. I recommend you read the first part to get the context and setup before diving into this blog.

I implemented a Custom Intent to return the quantity of a scanned product at a particular store. The use case might be that the user asks, 'Is this product available at the London store?' and Android will recognise that you are trying to find the product availability at a defined store (in this case, London). Custom Intents are similar to built-in Intents with the following additions:

You need to define the queries that a user might ask. These take the form of a String array with placeholder text. You can define multiple queries for the same capability and the more you define, the better the Assistant will be able to determine that the user is trying to invoke your app, though there is an upper limit of 100. The type of expected placeholder data is also included, in this case 'text'. You can also specify 'number' or 'date' but you cannot define your own types.

<string-array name="StockAvailabilityQueries">
    <item>Lookup stock in $text1</item>
    <item>Find this item at $text1</item>
    <item>Stock check for $text1</item>
    <item>Is this item in stock at $text1</item>
    <item>Find this item at $text1</item>
</string-array>

Then define the custom Intent capability in your shortcuts.xml file and reference the user queries under the app:queryPatterns property. Just like my built-in Intents sample discussed in part one, I had this capability fulfilled by sending an explicit Intent to my application but notice how this custom intent contains a parameter which is the location of the store. Per the documentation, the android:name cannot start with actions.intent.

<capability
    android:name="custom.actions.intent.STOCK_AVAILABILITY"
    app:queryPatterns="@array/StockAvailabilityQueries">
    <intent
        android:action="android.intent.action.VIEW"
        android:targetPackage="com.darryncampbell.appactionsdemo"
        android:targetClass="com.darryncampbell.appactionsdemo.MainActivity">
        <parameter
            android:name="store_location"
            android:key="store_location" />
    </intent>
</capability>

Note that I had a build error when I added the app:queryPatterns property to my capability and I needed to update the minSdkVersion of my project to 25 to fix it. I'm not sure if 25 was the actual number, it was just the first value I tried

I also defined a new shortcut to invoke the new, custom intent for stock availability:

<shortcut
    android:shortcutId="STOCK_AVAILABILITY"
    android:shortcutShortLabel="@string/stockAvailabilityShort">
    <capability-binding android:key="custom.actions.intent.STOCK_AVAILABILITY">
        <parameter-binding
            android:key="store_location"
            android:value="@string/storeLocation"/>
    </capability-binding>
</shortcut>

I saw some strange behaviour when invoking the custom intent, even though I had defined the same target package and class to fulfill the capability the Android Intent received by my app came through in onCreate and not onNewIntent like I expected. I am unsure whether this was something I had configured incorrectly in my app (my activity has its launchMode set to singleTask) or whether this is an issue with custom Intents.

I know my custom Intent example will always have a 'store_location' extra, so I use that in my generic Intent handler. As I mentioned in part one, Google recommend you segregate these capability handlers into separate modules but this is just a demo.

fun handleIntent(intent: Intent)
{
    ...
    else if (intent.hasExtra("store_location"))
    {
        //  STOCK_AVAILABILITY Intent entry point (App Actions)
        storeLocation = intent.getStringExtra("store_location").toString()
        Log.d(LOG_TAG, "Looking up stock availability in $storeLocation")
        stockAvailabilityCheck = true;
        dwInterface.sendCommandString(applicationContext, 
            DWInterface.DATAWEDGE_SEND_SET_SOFT_SCAN, "START_SCANNING")
    }
}

The application will then perform a (mocked) product availability lookup on the next scan:

fun handleIntent(intent: Intent)
{
    ...
    else if (intent.hasExtra(DWInterface.DATAWEDGE_SCAN_EXTRA_DATA_STRING)) {
        if (stockAvailabilityCheck)
        {
            //  For demo purposes only(!)
            val date = Calendar.getInstance().time
            val df = SimpleDateFormat("dd/MM/yyyy HH:mm:ss")
            val dateTimeString = df.format(date)
            val availabilityLookup = Scan(storeLocation + " stock: " + Random().nextInt(10), 
                "Stock Availability", dateTimeString)
            scans.add(0, availabilityLookup)
            stockAvailabilityCheck = false;
        }
        ...
    }
}

Testing the Custom Intent

Please see the Video below to show the custom intent being invoked from the Android Studio Google Assistant plugin

I mentioned in part one that to show App Actions running in response to a real invocation from the Google Assistant, it must first be approved according to the Play Store Policy Asssitant review process.

The limitation of not being able to test with the real Google Assistant is even more evident for custom Intents because you want to test the effectiveness of the model Google created from your specified queryPackages, but to do that you need to install the (reviewed) app from the app store. Also, the Android Studio plugin appears to test the <capability/>, not the <shortcut/> from your shortcuts.xml file.

Google also requires the user to manually enable your voice shortcut. You can suggest these shortcuts to your users using the In-App Promo SDK but it would require manual intervention from the user to enable your shortcut.

The dialog shown to the user looks similar to what is shown below:

Conclusions

The App Actions feature is interesting and potentially powerful but I would not personally recommend it to any Enterprise app developers today.

The required integration with Google Assistant and manual approval of shortcuts would not work on a shared, dedicated enterprise device. I am unsure whether App Actions would work with Android Enterprise Google accounts.

There would be definite advantages to using App Actions in the enterprise, notably the ability to harness natural language processing on-device through a simple and approachable API is compelling but the technology is today definitely focused on consumer end users. If you are looking for on-device speech recognition today, I would probably direct my customers towards the DataWedge Voice Input feature