supporting multiple pointing devices in microsoft windowssummer research workshop 2002 1/22...
Post on 21-Dec-2015
220 views
TRANSCRIPT
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
1/22
Supporting Multiple Pointing Devices in Microsoft Windows
Michael Westergaard
Department of Computer Science
University of Aarhus
Denmark
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
2/22
Outline
• Why multiple pointing devices?
• Requirements
• Architecture– Framework
– High-level API
– Low-level API
– Device driver
– Example
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
3/22
Why Multiple Pointing Devices
• Demonstration
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
4/22
Why Multiple Pointing Devices (2)
• Advanced interaction techniques– Multiple hand interactions
• Two-handed resize/zoom
• Two handed rotation of 3D objects
• Three handed zoom + move + rotate
– Different mice for different tasks
– Floating palettes
– Tool glasses
• Move work from one hand to another reduce risk of cumulative trauma disorders
• Cooperative work (for example on a wall PC)
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
5/22
Requirements
• Earlier: External application directly polling hardware. This requires exclusive access to the (serial) port
• Support arbitrarily many mice
• Treat USB, serial, and PS/2 mice alike
• Wish to use mouse in “old” Windows applications
• Generic solution not tied to a specific application or framework
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
6/22
Architecture
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse
subsystem
Framework
Application Low-level API
Main topic of article
Octopus
CPN Tools
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
7/22
ArchitectureFramework
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse
subsystem
Framework
Application Low-level API
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
8/22
Framework
• Provide widget-like interface (for example one can override the OnMouseDown event of a Button )
• Must also provide information about what mouse was clicked (this can be obtained by adding a Mouse property to the MouseEventArgs along with information about all mice on the system )
• Must also provide information about other mice working with an element (just another property of MouseEventArgs ) in order to allow exclusive access to an element and to allow multiple hand actions
• Give mouse back to Windows when it leaves application window (only when no native support is available, of course)
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
9/22
ArchitectureDevice Driver
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse
subsystem
Framework
Application Low-level API
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
10/22
Device Driver
• Windows does not distinguish between different mice
• Install filter between hardware dependent driver and Windows Mouse subsystem– We do not have to deal with hardware differences
– We can select whether to send a mouse event to Windows
• Keep driver as simple as possible to avoid bugs in kernel-mode code– Use callback to send data to user-mode
– Communicate by creating files in the drivers namespace
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
11/22
Device Driver (2)
• Interface
• A finite state machine is implemented
Name Description
Get Hook up a mouse for use by an application, register a callback, stop sending Windows mouse events
UnGet Release a mouse, drop the callback and start sending Windows events again
Suspend Temporarily suspend a mouse, keep the callback but start sending Windows events
UnSuspend Stop sending Windows events
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
12/22
Device Driver (3)
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
13/22
ArchitectureLow-level API
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse
subsystem
Framework
Application Low-level API
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
14/22
Low-level API
• Hide operating system specific support (for multiple mice) from higher levels– Simple C interface rather than operating system specific calls
to communicate with device driver
– If generic Windows support is developed only the low-level API needs to be changed
• Simplify identification of mice
• Add simple error handling
• Calls are not thread-safe
• Designed to be statically linked to a specific application
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
15/22
Low-level API (2)• Interface
• The provided callback does not have to be thread-safe
Name Description
RegisterCallback Register a callback
GetMice Hook up a specified number of mice
HasMouse Query if we have hooked a mouse with a specified number
UnGetMouse Release a specified mouse
SuspendMouse Passed to the device driver
UnSuspendMouse Passed to the device driver
UnRegisterCallback Drop a previously registered callback
UnGetAllMice Release all hooked mice
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
16/22
ArchitectureHigh-level API
Hardware dependent driver
Device driver
Low-level API
High-level API
Framework
Windows mouse
subsystem
Framework
Application Low-level API
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
17/22
High-level API• Hide the underlying operating system from the
framework• Build on top of low-level API• Spawn a thread to listen for callbacks from the low-
level API• Send “mouse events” to application• Calls are thread-safe• Draw mouse cursors• Accelerate mouse• Offer polling interface• Like low-level API designed to be statically linked to
an application
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
18/22
High-level API (2)• Interface
Name Description
Initialise Start up the listener thread and do some other initialising
Cleanup Kill the listener thread and cleanup other allocated resources
SuspendMouse Thread-safe version of corresponding low-level functionsUnsuspendMouse
GetRelativePosition Get mouse movement since last call
GetAbsolutePosition Get the position of the mouse
SetAbsolutePosition Set the position of the mouse
SetCursor Set the cursor for a specified mouse
LockCanvas Used for drawing cursors
UpdateCursors Used for drawing cursors
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
19/22
ArchitectureExample -- Startup
Application Framework H-API L-API Driver User
StartStart
Initialise
GetMice
Get
Getn times
n
spawn
n
ok
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
20/22
ArchitectureExample -- Suspend
Application Framework H-API L-API Driver User
Move mouseMouse movedcallbackMouse moved
callbackMouse movedevent
Mouse ismoved outof window Suspend
Suspend
Suspend
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
21/22
Future Work
• More control of what mice are hooked by GetMice function (by number, name, position on bus, or capabilities)
• Be able to handle multiple applications sharing mice
• Higher-level API: DLL/System service
• Port the needed layers to also support Windows 9x/ME/NT, Unix, and Mac
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
22/22
Conclusions
During this talk we have…
• seen how multiple mice can be useful (advanced interaction techniques, cooperative work)
• seen requirements for an architecture supporting multiple mice (work with any kind of mouse, backwards compatibility, not tied to an application)
• seen requirements for a framework supporting multiple mice
• seen a device driver and two supporting APIs that can satisfy the needs of a framework supporting multiple mice
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
23/22
FrameworkWidgets With Events
public class MyButton: Button { protected override void OnMouseDown(MouseEventArgs e) { base.OnMouseDown(e);
if (e.Button == left) { MessageBox.Show("Button pressed", ""); } }}
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
24/22
FrameworkKnowledge of Active Mouse
public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e);
if (e.Mouse == Mice[1]) { MessageBox.Show("Mouse 1 pressed", ""); } }}
September 10, 2002
Supporting Multiple Pointing Devices in Microsoft Windows Summer Research Workshop 2002
25/22
FrameworkKnowledge of Other Mice
public class MyButton: Button { protected override void OnMouseDown(MyMouseEventArgs e) { base.OnMouseDown(e);
if (e.OtherMice.Length == 0) { MessageBox.Show(“Only one mouse pressed the button", ""); } }}