Context Sensing for Mac OS X
Training Pennyworth
In order for Pennyworth to be useful, you must train it to recognize your activity, location, and social context. The best way to train Pennyworth is to correct it while it runs.
Running Pennyworth for the first time
If you downloaded and installed Pennyworth, you should now have an application with a yellow bell icon. Double-click this icon to launch it:
Pennyworth's application icon
When Pennyworth launches, it installs a small status bell status icon in the system menu bar. Clicking this icon will invoke the menu that exposes the application's functionality:
The Pennyworth menu
To display the context that Pennyworth currently predicts, select the
Predictions... item from the menu. If this is your first time running
Pennyworth, your context will be unknown.
To begin training Pennyworth, click the icon in the system menu bar, and select the
Correct Current Prediction... item. Pennyworth will display a small window
where you can describe your context:
Correcting & training Pennyworth
In this window, you can enter new labels that describe your context or select
previously used labels. If this is your first time running Pennyworth, there will be no
previously used labels available. After you describe your context, click the
Set button.
If you left the Prediction window open, the labels displayed will be
updated to reflect your context:
Pennyworth's current predictions
This window will be automatically updated as Pennyworth's context predictions change.
You may close this window and bring it back at a later time by selecting the
Predictions... item from Pennyworth status icon in the system menu bar.
Using Pennyworth with Growl
Pennyworth includes support for the Growl notification system. Pennyworth will emit Growl notifications when it detects your context changing:
Pennyworth's Growl notifications
Using Growl with Pennyworth is a useful way to understand how and when Pennyworth detects changes in your context.
Making Pennyworth smarter
As you use Pennyworth, it will make incorrect assessments about your context. (This
will be frequent at first, and less so as you train the system.) When you are in a new
context that you previously did not train Pennyworth to recognize, you can teach
Pennyworth about your new location, activity, or social context by selecting
Correct Current Prediction... from the Pennyworth menu. Change the specific
elements that are incorrect and click the Set button as described
before.
On an abstract level, each time you correct Pennyworth, you are providing more examples of your context that it uses to generate a model to predict your context in the future. As you provide more examples, its "mental mode" of you and your context becomes more sophisticated and accurate.
"Why does Pennyworth think that I'm doing X, when I am doing Y?
Pennyworth uses an
information theoretical
heuristic to decide which sensor readings or observations are useful for predicting your
context. For example, the system may learn that your current active application is a
reliable indicator of your current activity. To view all of the observations that
Pennyworth is using to make its predictions, select Observations... from
the Pennyworth menu:
Pennyworth's current observations
Pennyworth ships with a variety of sensors that uses the capabilities of your computer to detect things such as your current wireless network, running applications, and the time of the day. (This set of sensors may be augmented with other sensors. See the AppleScripting and developer pages for more information.) When you correct Pennyworth's predictions and provide it with context labels, you are associating particular observations and sensor readings with those labels.
Pennyworth uses a decision tree to organize these observations and detect which observations are relevant to your context. For example, a decision tree that predicts your location might look like
if "Current Wireless Network" == "My Work Network" then if "Hour of Day" == "12" then Location is "Lunchroom" else Location is "Office" else Location is "Home"
In this example the location of the user is determined using the current wireless network and the time of the day.
If you would like to "see" Pennyworth's mental model, select Learners
from the Pennyworth menu to see a representation of the internal decision trees:
Pennyworth's mental model (as a nested set diagram)
In the visible portion of the representation above, when the user is running PowerPoint on Wednesday, they are in a meeting. However, on Thursday, they are teaching.
Using this feature, you can identify where the system is making a mistake. To correct the mistake, simply correct Pennyworth when it makes incorrect predictions. That's all there is to it.