Import WearML-Engine

The latest version of the WearML engine can be downloaded from the RealWear HTML GitHub Repository, or by installing via NPM.

npm install wearml

After downloading, include the javascript file to your web page. The script will only execute on Android based devices with a screen size of 854×480.

Defining Speech Commands

Once you have added the javascript file to your web page you can start adding speech commands to your HTML elements. There are two methods for associating a voice command with its intended application functionality:

Adding DOM Attributes to the HTML

For simple controls like input fields or buttons that already have predefined onClick functionality, the easiest way to add a voice command is by setting the ‘data-wml-speech-command’ attribute to the desired phrase.

<input data-wml-speech-command="Enter Username" class="form-control" type="text" placeholder="Username"/>

Adding a Callback Command

Sometimes there may need to be a voice command that is not associated with any particular UI element, in which case you can add a Callback Command to run arbitrary JS code when the voice command is recognized.

wearML.addCallbackCommand("Refresh", null, function(){ 

//Refresh the UI... 

 }); 

..... 

var refreshFunc = function() { 

//Refresh the UI... 

 }; 

var commandSets = 'main|loginpage'; 

wearML.addCallbackCommand("Refresh", commandSets, refreshFunc);

You can also remove a Callback Command individually:

wearML.removeCallbackCommand("Refresh")

In addition, the voiceCommandsCallback method is called every time a command is successfully recognized. You can use this to selectively perform additional actions upon successful command recognition.

Updating Voice Commands

In order to update voice commands the wearML engine will use a MutationObserver to detect the adding and removing of html dom elements. If you are finding that speech-commands and overlays are not correctly updating when changing your html content you can call the public method:

wearML.getCommands()

This will force the WearMLEngine to reload grammar and overlays.

Using CommandSets

WearHF interprets the entire DOM tree at once to build its list of commands, which sometimes can lead to a mismatch between the commands WearHF detects and the commands the user expects to be able to interact with. For example, if a menu drawer is overlaid over the main UI pane, the user expects to only be able to interact with the menu items, not the the UI elements underneath, which are obscured to the user but are still present on the DOM. This issue can be circumvented by associating one or more ‘Command Sets’ with each command and programatically switching between them.

In the DOM

A CommandSet can be defined for a DOM element by setting the DOM attribute ‘data-wml-commandsets’ to a string value referencing one or more CommandSets, separated by a pipe character.

<a href="#" data-wml-speech-command="Open Menu" data-wml-style=".wmlMenuBtn" data-wml-commandsets="main|favorites" 
 onclick="wearML.setCommandSet('menu')" class="link icon-only open-panel"><i class="fa fa-bars"></i></a>

This element definition will tell WearHF to register the command ‘Open Menu’ when the active CommandSet is set to ‘main’ or ‘favorites’.

With a Callback Command

The CommandSets for a Callback Command is defined using the second argument of the wearML.addCallbackCommand function.

wearML.addCallbackCommand("Open Menu", "main|favorites", function(){ menu.open(); });

Switching the Active CommandSet

The wearML object contains the variable wearML.commandSet, which contains a single String defining the active CommandSet. This variable defaults to null, in which case all commands detected by WearHF will be registered upon detection, regardless of their CommandSet. This variable can be manipulated using the wearML.setCommandSet() function at any time, after which it will change the active CommandSet and subsequently refresh its list of commands.

<script type="text/javascript"> 
     document.addEventListener("DOMContentLoaded", function(event) { 

          wearML.setCommandSet('main'); //Set Active CommandSet to 'main' as soon as the document loads 
          wearML.addCallbackCommand("Refresh", "main|favorites", refreshFunc); 
     }); 
 </script>

WearML overlays

WearML Overlays are the hints provided by WearHF to help the user with navigating the current screen. WearML overlay’s have many attributes and options to choose from in order to provided helpful hints to the user. There are many features contained in the wearML overlays and the API can be found below for reference.

HTML WearML Overlay

In order to customize the wearML overlay first create a CSS style and add custom css attributes using “–” and then appended the WearML Style attribute you wish to use.

<style> 
     .nativeSpeechRightAligned{ 
     --overlay_show_number:false; 
     --overlay_show_dot:true; 
     --overlay_persists:true; 
     --overlay_anchor_hv:"110,50"; 
     } 
</style>

Add the style to your dom element:

<input data-wml-style=".nativeSpeechRightAligned" data-wml-speech-command="Enter Username" 
 class="form-control" type="text" placeholder="Username"/

Updating the ‘Show Help’ Menu

For commands that have no obvious visual indication for what to say, the best practice is to update the ‘Show Help’ menu with hints so the user can figure out what to do.

WearHF will automatically add any command to the ‘Show Help’ Menu if the CSS attribute ‘–include_help’ is a part of its style.

... 
<style> 
          .wmlSearch{ 
                --overlay_show_number:false; 
                --overlay_show_text:true; 
                --overlay_persists:true; 
                --include_help:true; 

     }
</style> 
<body> 
     <a href="#" data-wml-speech-command="Search" data-wml-commandsets="menu" data-wml-style=".wmlSearch" class="search-link item-content list-panel-all close-panel item-link"> 
</body>
...

Callback Commands are automatically added to the Show Help menu upon being registered.

API Definitions

DOM Attributes

AttributeData TypeDescription
data-wml-speech-commandStringtext/content_description/no/xxxx Optional: Defines the source for the speech command. text will take the text attribute from the component. content_description will use that attribute from the component. no will turn the voice command off on the component all together. xxxx You are also able to provide a custom voice command here, e.g. xxxx
data-wml-styleStringOptional: References to a CSS style using the classname.
data-wml-commandsetsStringDefines which CommandSets this command will be available to. Can be one or more string values, separated by a pipe character ( | ).

CSS Attributes

AttributeData TypeDescriptionNotes
rootBooleanAll elements below this will inherit the attributes provided to this dom tag unless otherwise specified.
overlay_show_textBooleanOptional: Turns a text label on or off. Text on the label will be taken from the speech_command that is set. (default = no)
overlay_persistsBooleanOptional: Number and/or overlay won’t fade away. (default = no, fades away)
overlay_orientationStringleft,right,top,bottom Optional: Text overlay direction (default = right)
overlay_background_colorStringOptional: Changes the background color of the element String is represented as HEX
overlay_text_colorStringOptional: Changes the background color of the element String is represented as HEX
overlay_border_colorStringOptional: Changes the background color of the element String is represented as HEX
overlay_anchor_hvString as “H,V”Optional:Specifies an anchor point for the WearML indicator represented as an ordered pair of integers 0-100, corresponding to the horizontal and vertical axes.

When H=0, the indicator is anchored to the left of the element. When H=100, the indicator is anchored to the right.

Similarly, when V=0, the indicator is anchored to the bottom of the element, and when V=100, the indicator is anchored to the top.

overlay_offsetString as “H,V”Optional: Offsets the WearML indicator by the specified number of pixels, corresponding to the horizontal and vertical axes.

For example, a value of ’50,-25′ would offset the indicator right by 50 pixels, and down by 25 pixels. 

overlay_show_dotBooleanOptional: Turns purple dot icon on or off for the element. Off by default
overlay_show_iconBooleanOptional: Turns microphone icon on or off for the element. On by default if there is a text overlay
hf_scrollString“None” = switches off headtracker

“Horizontal” = Headtracker only works horizontally

“Vertical” = Headtracker only works vertically

text_fieldStringFor text field elements, this will indicate what keyboard should be opened.

“dictation” = keyboard will open in Dictation mode 

“barcode” = keyboard will open in ‘Scan Code’ mode

“keyboard” = keyboard will open in default text entry mode

barcodeStringany, qr , code128, up, ceanOptional: Will define which type of barcode is being scanned. Ignored if the text_field isn’t set to barcde. (default = any)
global_commandsBooleanOptional: Disables all global commands, doesn’t matter which component it is applied to.
include_helpBooleanOptional: If true, the command associated with this element will be automatically added to the ‘Show Help’ Menu when detected by WearHF and part of the active CommandSet.
hide_helpBooleanOptional: Hides the Show Help Menu, doesn’t matter which component it is applied to.