Mono Accessibility 2.1
Last Tuesday, we presented Mono Accessibility 2.1. We worked really hard on this release. Our main goals were, among other things, to improve our UI Automation Client API implementation, polish the interaction with at-spi2, better Moonlight accessibility and to handle custom and client-side providers. The great work made by all the contributors was the reason this release accomplished those goals.
What happened since last time?
Explaining this release is kind of hard if you are not involved in Mono Accessibility, and even more difficult if you’re not involved in Accessibility at all. However, if you’re familiar with Mono and .NET you are, also, aware of the possibility of developing .NET-based applications that run on both platforms with minor* changes or no changes at all. But, before the implementation of Mono Accessibility, there was something missing: all those applications were not accessible.
This meant two things. First Atk-based Assistive Technologies (ATs) were not able to access Windows Forms nor Silverlight applications. Second .NET-based ATs using UI Automation were not able to run on Mono. Making them useless for people requiring Accessibility to interact with software running Mono on Linux**.
After the implementation of Mono Accessibility, we have:
- UI Automation API Framework
- Bridge between UI Automation and ATK
This way ATs using either Accessibility Technology, Atk or UI Automation, will be able to interact with Gtk+, Windows Forms and Silverlight applications, with no changes at all.
This is what happened
In the following video: (if you are using a RSS reader click this link) gcalctool is launched and an application, using UI Automation Client API, interacts with it, clicking buttons “2″, “+”, “3″ and “=”.
Then UIA Explorer, a .NET- based application using UI Automation Client API, also interacts with gcalctool, if you pay attention you will notice that UIA Explorer also lists all other Gtk+ based applications, such as Nautilus and the GNOME Panel.
Finally Mozilla Firefox is executed (both Novell Moonlight with Accessibility Support and Novell Moonlight Accessibility Extensions plugins were installed prior running it) and an Atk-based Accessility Debugger, Accerciser, is launched to interact with the Moonlight application.
The source code used to test GCalctool is the following:
// gmcs -pkg:mono-uia gcalc.cs && mono gcalc.exe
using System;
using System.Linq;
using System.Windows.Automation;
namespace Mono.A11y.Examples {
public class GCalc {
public static void Main (string []args)
{
AndCondition cond
= new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Window),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculator"));
var gcalc = AutomationElement.RootElement.FindFirst (TreeScope.Children, cond);
cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Button),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Add"));
var addButton = gcalc.FindFirst (TreeScope.Descendants, cond);
cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Button),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Numeric 2"));
var Button2 = gcalc.FindFirst (TreeScope.Descendants, cond);
cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Button),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Numeric 3"));
var Button3 = gcalc.FindFirst (TreeScope.Descendants, cond);
cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Button),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculate result"));
var calcButton = gcalc.FindFirst (TreeScope.Descendants, cond);
cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
ControlType.Button),
new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculate result"));
var resultText = gcalc.FindFirst (TreeScope.Descendants, cond);
// Clicking buttons
InvokePattern addInvoke = (InvokePattern) addButton.GetCurrentPattern (InvokePattern.Pattern);
InvokePattern Invoke2 = (InvokePattern) Button2.GetCurrentPattern (InvokePattern.Pattern);
InvokePattern Invoke3 = (InvokePattern) Button3.GetCurrentPattern (InvokePattern.Pattern);
InvokePattern calcInvoke = (InvokePattern) calcButton.GetCurrentPattern (InvokePattern.Pattern);
System.Console.WriteLine ("Pressing \"2\"");
System.Threading.Thread.Sleep (500);
Invoke2.Invoke ();
System.Console.WriteLine ("Pressing \"+\"");
System.Threading.Thread.Sleep (500);
addInvoke.Invoke ();
System.Console.WriteLine ("Pressing \"3\"");
System.Threading.Thread.Sleep (500);
Invoke3.Invoke ();
System.Console.WriteLine ("Pressing \"=\"");
System.Threading.Thread.Sleep (500);
calcInvoke.Invoke ();
}
}
}
Where do I get it?
If you installed Mono Accessibility 2.0 an option to upgrade should be available now, if not Mono Accessibility is available for a variety of Linux distributions, including:
- openSUSE 11.2 1-Click Install
- Ubuntu Karmic Koala Package Archive on Launchpad
- Fedora 12 openSUSE Build Service
Moonlight applications require the following extra steps:
- Install the updated xulrunner package from the above repositories. (Required until patch on bug #480317) gets approved.
- Install Novell Moonlight with Accessibility Support for 32 bit or 64 bit.
- Install Novell Moonlight Accessibility Extensions
Have fun and if you find any bug with this release, please file it. If you want to contribute or need specific assistance, please join our mailing list, or drop in #mono-a11y on irc.gimp.org.
* Of course this depends on what APIs you are using. If you want to know how compatible is your application try MoMA.
** Notice however, even though Mono is muti platform and UI Automation is included in both Mono and Silverlight, in order to allow other ATs running on other platforms, different than Linux, you will require to implement an specific bridge to talk to you OS Accessibility Layer.
