Stacking Code

This is post #32 in the Building Neno series. Please click here for a description of the Building Neno project and instructions on how to access the source code for this post.

In my last post I made a quick and dirty screenshot generator for my Pivot collection. Which works, but it's an unpleasant little application. This morning I'm going to attempt to improve it by, one, adding date pickers for selective screenshot generation, and two, addressing the unresponsiveness of the application by adding multithreading.

Date Pickers

To get the small change out the way, I'm adding two System.Windows.Forms.DateTimePicker controls to the top of the window, one for a "From" date and one for a "To" date. These will show a checkbox indicating whether they have a value selected or not -- and will default to false. The user can ignore them completely and hit the "Go" button to generate screenshots for all posts like before.

if (FromDateTimePicker.Checked)
    posts = posts.Where(post => post.PublishDate >= FromDateTimePicker.Value.Date);

if (ToDateTimePicker.Checked)
    posts = posts.Where(post => post.PublishDate < ToDateTimePicker.Value.Date.AddDays(1));

Identifying Work

Now for the real improvement. To make the UI of this application more responsive I plan to move as much work as I can out of the main thread into worker threads. Identifying which work can be moved is the first step, so here's a quick summary of the three things the main thread did in the first iteration of the application.

1. The form (when loading) had to initialise everything, including IOC.
2. The "Go" button (when clicked) had to:
   1. Retrieve all posts from the application and place them in a queue.
   2. Direct each WebBrowser control to navigate to their first post from the queue.
3. The WebBrowser controls (after loading a page) had to:
   1. Capture the screenshot from the WebBrowser control.
   2. Save the screenshot to a file and/or to the application.
   3. Navigate to the next post in the queue.

Of all of those, the most expensive are 2.1 and 3.2. It doesn't make a lot of sense to move 2.1 since the application doesn't have anything to do until it completes. 3.2 on the other can be moved out into worker threads -- there's no dependency on the result, and it will allow the WebBrowser control to move onto the next post while the worker thread completes.

The logic for 3.2 is part of the CaptureScreenshot method which also handles 3.1, the capture of the screenshot from the WebBrowser control. One of the things to be aware of when adding multithreading to a Windows application is that it's considered a no-no to access controls directly from outside the main thread. In an effort to keep this application as simple as possible, I'll leave the screenshot capturing logic in the main thread, and pass the rest of the work off to a worker thread. This saves me from having to worry about Invoke, etc.

private void CaptureScreenshot(WebBrowser webBrowser, string url)
{
    AddMessage("{0} loaded {1}.", webBrowser.Name, url);
    var bitmap = new Bitmap(1024, 1024);
    var targetBounds = new Rectangle(0, 0, 1024, 1024);
    webBrowser.DrawToBitmap(bitmap, targetBounds);
    int postId = WebBrowserPostIdHash[webBrowser.Name];
    // Truncate this method here and throw the rest into a worker thread.
    new Thread(() => SaveScreenshot(url, bitmap, postId)).Start();
}

// Worker Thread
private void SaveScreenshot(string url, Bitmap bitmap, int postId)
{
    // Save to file.
    string path = "C:\\Users\\Adam Boddington\\Desktop\\Screenshots\\" + url.Replace(":", "-").Replace("/", "-").Sluggify() + ".png";
    bitmap.Save(path, ImageFormat.Png);
    AddMessage("Worker thread saved screenshot to {0}", path);

    // Save to post.

    var memoryStream = new MemoryStream();
    bitmap.Save(memoryStream, ImageFormat.Png);

    DomainModel.Post post = Moja.InversionOfControl.Container.Get<IPostService>().GetPost(postId);
    Attachment attachment = post.Attachments.Where(a => a.Slug == "pivot-screenshot").SingleOrDefault();

    if (attachment == null)
    {
        attachment = new Attachment();
        post.Attachments.Add(attachment);
    }

    attachment.Name = "Pivot Screenshot.png";
    attachment.Slug = "pivot-screenshot";
    attachment.ContentType = "image/png";
    attachment.Store(memoryStream.ToArray());
    Moja.InversionOfControl.Container.Get<IPostService>().UpdatePost(post);
    AddMessage("Worker thread uploaded screenshot for {0}.", url);
    WebBrowsersInProgress--;
}

NHibernate Session per Thread

Now that I'm using worker threads, I've switched to one NHibernate session per thread. This is the conventional wisdom when working with threads and saves me from having to worry about locking sessions, etc. Ninject makes it easy with the InThreadScope method.

public class NinjectModule : Ninject.Modules.NinjectModule
{
    #region Overrides of NinjectModule

    public override void Load()
    {
        // Services

        Bind<IPageService>().To<PageService>().InThreadScope();
        Bind<IPostService>().To<PostService>().InThreadScope();
        Bind<ITagService>().To<TagService>().InThreadScope();
        Bind<IUserService>().To<UserService>().InThreadScope();

        // Repositories

        // ...

        // Moja

        Bind<IContext>().To<Context>().InThreadScope();

        Bind<ISession>()
            .ToMethod(context => context.Kernel.Get<SessionFactoryWrapper>().OpenSession(
                new IInterceptor[]
                {
                    context.Kernel.Get<StaleInterceptor>(),
                    context.Kernel.Get<InvalidInterceptor>()
                }))
            .InThreadScope();

        Bind<StaleInterceptor>().ToSelf().InThreadScope();
        Bind<InvalidInterceptor>().ToSelf().InThreadScope();
        Bind<SessionFactoryWrapper>().ToSelf().InSingletonScope(); // Singleton
    }

    #endregion
}

Messages

My various methods are pushing messages out to the MessageTextBox control as a kind of running log of what's happening. Doing that from a worker thread means I have to think about using Invoke (see the link given earlier), or I can push the messages into a central List<string> instead. By running a message timer, I can grab the last 100 messages from the list and display them in the MessageTextBox control once a second.

private void AddMessage(string format, params object[] args)
{
    string message = string.Format(format, args);

    lock (MessageList)
        MessageList.Insert(0, message);
}

private void MessageTimer_Tick(object sender, EventArgs e)
{
    IEnumerable<string> messages;

    lock (MessageList)
        messages = MessageList.Take(100);

    MessageTextBox.Text = string.Join(Environment.NewLine, messages);
}

This will save some processing time on the main thread when working with large collections of posts and lots of messages.

Looking for Completion

Now that worker threads are handling the trailing end of the workload, I need a way to know when they're done, and when the entire process is done. I'm going to do that with a simple counter and a timer that watches for when the counter hits zero. Here's where my counter, WebBrowsersInProgress, is initialised -- and where the timer is enabled.

private void GoButton_Click(object sender, EventArgs e)
{
    // Posts

    IQueryable<Post> posts = Moja.InversionOfControl.Container.Get<IPostService>().GetPosts()
        .Where(post => post.IsPublished)
        .Select(post => new Post { Id = post.Id, PublishDate = post.PublishDate, Slug = post.Slug });

    if (FromDateTimePicker.Checked)
        posts = posts.Where(post => post.PublishDate >= FromDateTimePicker.Value.Date);

    if (ToDateTimePicker.Checked)
        posts = posts.Where(post => post.PublishDate < ToDateTimePicker.Value.Date.AddDays(1));

    // PostQueue
    lock (PostQueue)
        PostQueue = new Queue<Post>(posts);

    AddMessage("{0} posts queued.", PostQueue.Count);
    WebBrowsersInProgress = 0;

    // Kick them off.
    NavigateToNextPost(WebBrowser1);
    NavigateToNextPost(WebBrowser2);
    NavigateToNextPost(WebBrowser3);
    NavigateToNextPost(WebBrowser4);
    NavigateToNextPost(WebBrowser5);
    NavigateToNextPost(WebBrowser6);
    NavigateToNextPost(WebBrowser7);
    NavigateToNextPost(WebBrowser8);

    // Watch for completion.
    ProcessCompleteTimer.Enabled = true;
}

Note, I'm only selecting the information I need to make this initial retrieve from the application faster. Hence the tiny DTO. Now that the application is more responsive and faster, I've also added four more WebBrowser controls to cut down on the load latency even more.

Here's where WebBrowsersInProgress is incremented, decremented and watched for a zero value.

private void NavigateToNextPost(WebBrowser webBrowser)
{
    Post post = null;

    lock (PostQueue)
        if (PostQueue.Count > 0)
            post = PostQueue.Dequeue();

    if (post == null)
        return;

    WebBrowsersInProgress++;
    WebBrowserPostIdHash[webBrowser.Name] = post.Id;
    string url = BaseUrl + string.Format("blog/{0}/{1:00}/{2:00}/{3}", post.PublishDate.Year, post.PublishDate.Month, post.PublishDate.Day, post.Slug);
    webBrowser.Navigate(url);
    AddMessage("{0} navigating to {1}.", webBrowser.Name, url);
}

// Worker Thread
private void SaveScreenshot(string url, Bitmap bitmap, int postId)
{
    // ...
    WebBrowsersInProgress--;
}

private void ProcessCompleteTimer_Tick(object sender, EventArgs e)
{
    if (WebBrowsersInProgress == 0)
    {
        AddMessage("Process complete.");
        ProcessCompleteTimer.Enabled = false;
    }
}

Embedded Objects

One last thing to worry about. The document load event on the WebBrowser control fires after the HTML is rendered and the JavaScript is executed, but before embedded objects have a chance to render. So things like the embedded Flash video in one of my recent posts won't have rendered when I take the screenshot. Enter a quick hack to get Ross Geller's face back into my Pivot collection instead of a big blank square.

// Hack, hack, hackity hack!
// Wait a few seconds for things like embedded objects to render.
if (webBrowser.DocumentText.Contains("<object"))
{
    DateTime waitUntil = DateTime.Now.AddSeconds(3);

    while (DateTime.Now < waitUntil)
        Application.DoEvents();
}

webBrowser.DrawToBitmap(bitmap, targetBounds);

This causes each WebBrowser control with an object tag to sit still for three seconds before it captures its screenshot. The call to Application.DoEvents allows the rest of the WebBrowser controls to continue on as normal.

Conclusion

The application is nicer to run, responsive to moving and resizing, and noticeably faster thanks to some parallel processing.