Windows 7 Multitouch Using WPF 3.5

Update: If you’re using .NET 4.0, be sure to check out my posts about the new touch events.

Finally!  Another post about programming!  I know!  And Windows 7, too!  That ever sure is topical!  This one is for all of you developers running the Win7 beta on the HP TouchSmart (moneyhat go).

Win7 is supposed to woo and wow you with fixing Vista’s many shortcomings new features like multi-touch support.  If you’re curious on how it all works, I’d suggest you watch this great PDC 2008 video on the subject.  Windows will give you everything you need to fancify your touch application, once you’ve set it up to do so.  They’ll tell you how to get multi-touch working in unmanaged code.  There’s also some examples out there showing how to use interop to use this method in C#.  We’ve bridged the gap from unamanaged to managed code – what am I still writing this post for?

Well, WPF is a bit different.  Not only is native multi-touch not present in WPF right now (look forward to .NET 4.0 some time after Win7 releases), but you actually can’t use interop to support multi-touch in your applications.  Yeah, I know.  Something’s amiss when interop fails.

Actually, it is just that WPF doesn’t accept the WM_Touch messages that are sent to windows when the user touches the screen.  Since you don’t get this notification, you can’t capture information regarding Win7 gestures or raw data using interop.

Hold on!  Don’t run to make your shiny, new, and intuitive application in C++ just yet.

As Anson Tao alludes to in the Q&A session after the presentation in the video above, you can recieve the raw data from stylus events in WPF 3.5 SP1, which is already released.  However, you have to do just a tad bit of fidgeting to get it working.

Here’s an application that will show you how to access this information. It just moves two rectangles to the two points you touch on the window.

I’ll start with the simple XAML:

<Window x:Class="MultitouchTest.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window1" Height="800" Width="1200">
    <Canvas>
        <Rectangle Canvas.Left="-20" Canvas.Top="0" Height="20" Name="Touch1" Stroke="Black" Fill="Black"  Width="20" />
        <Rectangle Canvas.Left="-20" Canvas.Top="0" Height="20" Name="Touch2" Stroke="Red" Width="20" Fill="Red" />
    </Canvas>
</Window>

Now, the business logic:

public partial class Window1 : Window
{
    #region Class Variables
 
    private int Touch1ID = 0;    // id for first touch contact
    private int Touch2ID = 0;    // id for second touch contact
 
    #endregion
 
    #region P/Invoke
 
    // just a little interop.  it's different this time!
    [DllImport("user32.dll")]
    public static extern bool SetProp(IntPtr hWnd, string lpString, IntPtr hData);
 
    #endregion
 
    #region Constructors/Initialization
 
    public Window1()
    {
        InitializeComponent();
 
        // here's the first thing you need to do.  upon window load, you want to set the tablet
        // property to receive multi-touch data.  you need to the window loaded to ensure the handle is created.
        this.Loaded += new RoutedEventHandler(
           delegate(object sender, RoutedEventArgs args)
           {
               var source = new WindowInteropHelper(this);
 
               SetProp(source.Handle,
                   "MicrosoftTabletPenServiceProperty", new IntPtr(0x01000000));
 
           });
 
        // then, simply subscribe to the stylus events like normal.  you'll get an event for each contact.
        // so, when you move both fingers, you get a StylusMove event for each individual finger
        this.StylusDown += new StylusDownEventHandler(Window1_StylusDown);
        this.StylusMove += new StylusEventHandler(Window1_StylusMove);
        this.StylusUp += new StylusEventHandler(Window1_StylusUp);
    }
 
    #endregion
 
    #region Touch Events
 
    void Window1_StylusDown(object sender, StylusDownEventArgs e)
    {
        Point p = e.GetPosition(this);   // get the location for this contact
 
        // attribute an id with a touch point
        if (Touch1ID == 0)
        {
            Touch1ID = e.StylusDevice.Id;
            // move the rectangle to the given location
            Touch1.SetValue(Canvas.LeftProperty, p.X - Touch1.Width / 2);
            Touch1.SetValue(Canvas.TopProperty, p.Y - Touch1.Height / 2);
        }
        else if (Touch2ID == 0)
        {
            Touch2ID = e.StylusDevice.Id;
            // move the rectangle to the given location
            Touch2.SetValue(Canvas.LeftProperty, p.X - Touch2.Width / 2);
            Touch2.SetValue(Canvas.TopProperty, p.Y - Touch2.Height / 2);
        }
    }
 
    void Window1_StylusMove(object sender, StylusEventArgs e)
    {
        Point p = e.GetPosition(this);
        // determine which contact this belongs to
        if (Touch1ID == e.StylusDevice.Id)
        {
            // move the rectangle to the given location
            Touch1.SetValue(Canvas.LeftProperty, p.X - Touch1.Width / 2);
            Touch1.SetValue(Canvas.TopProperty, p.Y - Touch1.Height / 2);
        }
        else if (Touch2ID == e.StylusDevice.Id)
        {
            // move the rectangle to the given location
            Touch2.SetValue(Canvas.LeftProperty, p.X - Touch2.Width / 2);
            Touch2.SetValue(Canvas.TopProperty, p.Y - Touch2.Height / 2);
        }
    }
 
    void  Window1_StylusUp(object sender, StylusEventArgs e)
    {
         // reinitialize touch id and hide the rectangle
        if (e.StylusDevice.Id == Touch1ID)
        {
            Touch1.SetValue(Canvas.LeftProperty, -Touch1.Width);
            Touch1ID = 0;
        }
        else if (e.StylusDevice.Id == Touch2ID)
        {
            Touch2.SetValue(Canvas.LeftProperty, -Touch2.Width);
            Touch2ID = 0;
        }
    }
 
    #endregion
}

Just a Bit: Loading Images Asynchronously in WPF

Multithreading using WPF is a little tricky at first (I’m planning a more thorough post on that later), but the basic thing you need to know is that you can’t access WPF elements outside of the UI thread, because they aren’t thread safe.  There are some things you can do to speed loading up, however.  Here’s a method for loading images on a separate thread:

private void LoadImage(string fname)
{
	// instantiate and initialize the image source
	BitmapImage bmi = new BitmapImage();
	bmi.BeginInit();
	bmi.UriSource = new Uri(fname, UriKind.Relative);
	bmi.EndInit();
 
	bmi.Freeze();		// freeze the image source, used to move it across the thread
 
	// this method tells the separate thread to run the following method to run on the UI thread
	// the (ThreadStart)delegate(){ } notation is a shorthand for creating a method and a delegate for that method
	TheImage.Dispatcher.BeginInvoke(System.Windows.Threading.DispatcherPriority.Normal, (ThreadStart)delegate ()
	{
		TheImage.Source = bmi;
	});
}

The meat of the method is the BitmapImage.Freeze method – this is specifically useful for loading images.  The Dispatcher.BeginInvoke is also important, but I’ll leave more explanation on that for the threading post.

Introduction to the Windows Presentation Foundation (WPF)

WPF is a graphical subsystem to the .NET framework.  It is essentially used to make UI programming easier by separating the UI from the business logic.  By doing this, it is easy to maintain and allows designers to easily change the UI without futzing with too much programming.  This, of course, makes it easier for the programmers because they don’t have to convert the design elements into application assets.  How does this all fit together?  I’ll go through a sample application in order to show how one might use the strengths of WPF to make application development a breeze.

For our example, I’ll pretend I’m making a browser application.  To keep it nice and simple, I’m only interested in the controls.  Being an expert graphic artist, I’ve crafted a stunning visual masterpiece:

Design

In order to create this amazing experience, all I had to do was drag-and-drop and edit the details using Visual Studio 2008’s design view:

For designers, Visual Studio 2008 will make it much easier.  However, it could still be a little daunting.  With WPF, the entire UI is laid out using XAML, which is an XML-based language that describes the layout, properties, and actions of the UI elements (I’ll go over this in more detail later).  Because of this, any program can be made to more easily lay out the elements of your window in the way the designers need (here‘s some examples) – all it has to do is output the result in XML-based XAML.

The best part for the programmer is that he doesn’t have to take the bitmaps created by the designer in photoshop, break them apart, and create buttons.  All he has to do is add the XAML and add the code behind the elements (aka business logic).  Oh, and say the design completely changes?  While the XAML changes, the business logic remains the same – no need to change things on the back end (yeah, that’s how it should be).  This makes maintaining the application marginally better – major UI revisions could leave all of the business logic exactly the same.  Though you should probably considering adding some features if all your doing is adding a new coat of paint.

Programming

“But Andrew,” says the imaginary, whiney, no-designing programmer that presumably exists in my subconscious.  “What does all of this mean to me!?”  Well, imaginary programmer, maybe this blog isn’t JUST FOR YOU.  Maybe you should just go read man pages for the rest of the day…or you can just read the rest of this post, since it is for you.  I’d say the best idea would be to stop existing in my imagination and quit telling me to start fires.

XAML

I’ll start with XAML, since it’s the basis behind WPF (and I’ve already introduced it to you).  Like I mentioned earlier, Extensible Application Markup Language (XAML, typically pronounced ‘zammel’) is based on XML.  This makes laying out an application as easy as setting up your first Geocities webpage.  Like XML, it has the main tags (like <Window> or <Button>) with properties that define the tag.  Here’s the XAML for the browser control window:

 
<Window x:Class="WPFIntroduction.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Browser Controls" Height="121.38" Width="775" Keyboard.KeyDown="Window_KeyDown">
    <Grid>
        <Button Height="48" HorizontalAlignment="Left" Margin="32.844,18.564,0,0" Name="BtnBack" VerticalAlignment="Top" Width="50" Click="BtnBack_Click">
            <Image Source="Images\back.png" />
        </Button>
        <Button Height="48" HorizontalAlignment="Left" Margin="102.816,18.564,0,0" Name="BtnForward" VerticalAlignment="Top" Width="50" Click="BtnForward_Click">
            <Image Source="Images\forward.png" />
        </Button>
        <Button Height="48" HorizontalAlignment="Left" Margin="172.788,18.564,0,0" Name="BtnHome" VerticalAlignment="Top" Width="50" Click="BtnHome_Click">
            <Image Source="Images\home.png" />
        </Button>
        <Button Height="48" HorizontalAlignment="Left" Margin="242.76,18.564,0,0" Name="BtnStop" VerticalAlignment="Top" Width="50" Click="BtnStop_Click">
            <Image Source="Images\stop.png" />
        </Button>
        <TextBox Height="38.164" Margin="309.876,23.24,79.254,0" Name="AddressBar" VerticalAlignment="Top" />
        <Button Height="48" HorizontalAlignment="Right" Margin="0,18.564,12.138,0" Name="BtnRefresh" VerticalAlignment="Top" Width="50" Click="BtnRefresh_Click">
            <Image Source="Images\refresh.png" />
        </Button>
    </Grid>
</Window>

Notice the XAML namespaces defined in the root element – these are required for XAML files. These are the namespaces used in defining the various XAML elements.

You might be wondering just how the XAML text gets converted into a user interface. The XAML is actually converted into code. You might notice above that all of the tags in the XAML are actually control classes. Window, Button, and Image are all classes, and WPF actually instantiates objects of these classes at runtime. So, for each Button tag, we have a Button object. The object names are as defined: BtnBack, BtnForward, etc.  You can actually reference these objects in your code – and no need to instantiate or set them up in the constructor.

Business Logic

Now, this isn’t entirely automatic.  You do have to include a call to a method, namely InitiataiteComponent(), to create the UI elements contained in the XAML.  Where do you call this method?  Well, each XAML window has a file that contains the code behind the XAML.  The Window created by the XAML is actually an inherited class.  In our case, the new class is Window1.  In this class’ constructor, the call to InitializeComponent() is called.  This class also houses the business logic, or the code behind the application.

This is also the best place to add event handlers for your application.  Since I’m not making a full browser, we’ll just stick with dialog boxes when buttons are clicked.  Here’s the whole class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
 
namespace WPFIntroduction
{
    ///
    /// Interaction logic for Window1.xaml
    ///
    public partial class Window1 : Window
    {
        public Window1()
        {
            InitializeComponent();
        }
 
        private void BtnBack_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Back button clicked");
        }
 
        private void BtnForward_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Forward button clicked");
        }
 
        private void BtnHome_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Home button clicked");
        }
 
        private void BtnStop_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Stop button clicked");
        }
 
        private void BtnRefresh_Click(object sender, RoutedEventArgs e)
        {
            MessageBox.Show("Refresh button clicked");
        }
 
        private void Window_KeyDown(object sender, KeyEventArgs e)
        {
            if (e.Key == Key.Enter &amp;&amp; AddressBar.IsFocused)
                MessageBox.Show("New address entered");
        }
    }
}

Notice in the Window_KeyDown event, I reference the AddressBar object.

This is all pretty standard, so no need to really explain anything here – it’s simply some simple event handlers.  So, you have a basic WPF application to reference when creating your own WPF application.  While this, obviously, won’t get you to be able to create a brand new application in its entirety, it’ll give you a start to the mentality behind WPF.  I use this quite a bit, so don’t be surprised if I go into more in-depth examples in the future.  In the meantime, check out MSDN‘s resources on WPF…or feel free to ask questions. Lord knows I could have left plenty of details out.

Just a Bit: Copying Text to the Clipboard

A bit is the difference between true and false

So when working on this blog sometimes I spend a little too much time getting code together. I’m sure I’ll get better, faster, and care less about what it looks like over time.

Until then, I’ll use these ‘Just a Bit’ posts. The idea being that I post a tiny bit of code that I found useful recently. Sure, about 400,000,000 pages can be found elsewhere with this information, but they don’t have clever wordplay in their titles!

So, the first “Just a Bit” is using .NET to add text to the Windows clipboard.  First, put together a string using the StringBuilder class – the reason for this is that any formatting characters (‘\n’, ‘\t’, etc.) won’t necessarily translate correctly onto the clipboard correctly if you simply put them into a string.  Finally use the System.Windows.Clipboard static method SetDataObject to copy the text to the clipboard.

// build the string, with new lines between entries
StringBuilder clipStr = new StringBuilder();
foreach (string itm in TheItemCollection.Items)
{
	clipStr.Append(itm);
	clipStr.Append("\n");
}
// copy to clipboard, clear the box, and notify user
Clipboard.SetDataObject(clipStr.ToString(), true);

Enjoy that one.

Sending Commands/Events to Another Application

In order to send a keystroke to another application, you can just utilize Win32’s PostMessage.

Windows is very event-driven.  Anytime anything changes within a window, an event is fired.  If you click your mouse button, an event is fired.  If you resize the window, an event is fired.  If you press a key, an event is fired.

Quiz time.  What happens when you move your mouse within a window?
If you guessed “The pointer moves”, you are technically correct but not getting what I’m trying to say.
If you guessed “A bobcat will charge out of my monitor and eat my childhood friend”, you’re way off.
If you guessed “An event is fired”, you’ve somehow retained the knowledge I bestowed upon you 1.2 seconds ago!

All pretty easy – but how is your window notified of these events?  Windows sends messages to the window with codes regarding the type of command that has occurred, along with any information related to that command.

PostMessage() essentially does the same thing.  Alternatively, you can use SendMessage (to post the message, then wait until the message is received), SendMessageTimeout (to do the same but timeout after a given time).  Here’s PostMessage’s syntax:

BOOL PostMessage(
    HWND hWnd,        // the handle of the window the message is to be posted to
    UINT Msg,            // the message code
    WPARAM wParam,  // message-specific information
    LPARAM lParam      // message-specific information
);

The window handle is how Windows references application windows.  If you’re using .NET, you can use the System.Diagnostics.Process.MainWindowHandle property with the application’s process.

The message code is of course specific to the type of message that you want.  For example:

  • If you want to close a window, use WM_COMMAND (0x0112), with wParam = WM_CLOSE (0xF060).
  • If you want to send keystrokes to a window, use a call to PostMessage with WM_KEYDOWN (0x100) and WM_KEYUP (0x101) as message codes with the VK key code as the WPARAM (for a list of VK key codes, check this page out).

For more Windows Messages, check out this random list of Windows Messages.

So, say we want to close Notepad if it is open.  Using C#, here’s a complete example:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
using System.Diagnostics;
 
namespace TestCmd
{
    class Program
    {
        // Import the library containing PostMessage
        [DllImport ("user32.dll")]
        public static extern bool PostMessage(IntPtr hWnd, int Msg, int wParam, int lParam);
 
        public const int WM_COMMAND = 0x0112;	   // Code for Windows command
		public const int WM_CLOSE = 0xF060;		 // Command code for close window
 
        static void Main(string[] args)
        {
            try
            {
                IntPtr hWndNotepad = Process.GetProcessesByName("notepad")[0].MainWindowHandle;
                Console.WriteLine("Ready to tell Notepad to close.  Press enter to send the command.");
                Console.ReadLine();
                if (hWndNotepad != null)
                {
                    // Close Window
                    PostMessage(hWndNotepad, WM_COMMAND, WM_CLOSE, 0);
                }
                Console.WriteLine("Command sent.  Press enter to close.");
                Console.ReadLine();
            }
            catch (Exception e)
            {
                Console.WriteLine ("Notepad not running.  Press enter to close.");
                Console.Read();
            }
        }
    }
}

Take that, Notepad! Now to make a tray application to close iTunes everytime it starts.

Wait…

No.  Please, code responsibly.

–Andrew