Stacking Code

This is post #35 in the Building Neno series. Please click here for a description of the Building Neno project and instructions on how to access the source code for this post.

Back when I was a young whippersnapper sitting at the foot of my grandfather's chair, he would regale me with tales of computing back in federation times while he carved new mainframe terminals from chunks of eucalyptus.

Okay, not really, this mainframe tale is mine -- and not from that long ago, only the mid-nineties (it just feels like a century ago). Back then, in the company I was working for at the time, all the little bits of data that a mainframe job would need (besides the actual data about to be crunched) would often be held in a global lookup table. This table would have all kinds of stuff in it, for a variety of environments, not just production. And it was all inconsistently keyed, so if a job didn't choose a unique enough key for its lookup value, it could collide with another batch job using the same key. Over time the table would accrete to a size where things would start to slow down and someone would attempt a cleanup. Documentation being imperfect, this would often lead to jobs failing horribly the next night. The poor schmuck on call after one of these cleanups usually got little to no sleep, and a burning desire to update the old resume. Ah, the joys of shared database programming.

Lookup Class

As painful as that particular environment was, there is some merit to the idea of a lookup table. The applications I build today are very different to the PL1 jobs that I used to maintain, but applications still need to store lots of little pieces of data. These small pieces of data can be represented by a variety of very small classes in modern OO languages, but storing each of these classes in their own table can be a lot of work and end up over complicating at least the database schema.

Enter the abstract Lookup class. This is a base class with a number of protected properties covering the most common data types. The idea is for each small lookup class to inherit from the abstract Lookup class and store its small amount of data in the already defined protected properties. A simple enough idea, but a powerful one. Here's a crack at what it could look like.

namespace StackingCode.Moja.DomainModel
{
    public abstract class Lookup<TId> : Entity<TId>
    {
        protected bool? _boolA;
        protected bool? _boolB;
        protected int? _intA;
        protected int? _intB;
        protected DateTimeOffset? _dateTimeOffsetA;
        protected DateTimeOffset? _dateTimeOffsetB;
        protected string _stringA;
        protected string _stringB;

        [Required]
        [StringLength(100)]
        public virtual string Name { get; set; }

        [StringLength(500)]
        public virtual string Description { get; set; }

        [DisplayName("Display Order")]
        public virtual short DisplayOrder { get; set; }

        protected virtual bool? BoolA
        {
            get { return _boolA; }
            set { _boolA = value; }
        }

        // ... and so on
    }
}

All the virtual methods are a side effect of using an ORM, NHibernate in this case.

The database table for the Lookup class is pretty much what you would expect, except for the Discriminator column. This is where a value specifying the concrete lookup class is placed (more on that later).

CREATE TABLE [dbo].[Neno_Lookup] (
    [Id]                    INT            IDENTITY (1, 1) NOT NULL,
    [Discriminator]         NVARCHAR (50)  NOT NULL,
    [Version]               INT            NOT NULL,
    [Name]                  NVARCHAR (100) NOT NULL,
    [Description]           NVARCHAR (500) NULL,
    [DisplayOrder]          SMALLINT       NOT NULL,
    [BoolA]                 BIT            NULL,
    [BoolB]                 BIT            NULL,
    [IntA]                  INT            NULL,
    [IntB]                  INT            NULL,
    [DateTimeOffsetA]       DATETIMEOFFSET NULL,
    [DateTimeOffsetB]       DATETIMEOFFSET NULL,
    [StringA]               NVARCHAR (200) NULL,
    [StringB]               NVARCHAR (200) NULL,
    PRIMARY KEY CLUSTERED ([Id] ASC))

Ah, a table full of general purpose columns... Poppy assorted 1990's colleagues would be proud.

The NHibernate mapping file is pretty straight forward too. All the protected properties are mapped, although I use the backing field for setting in case a concrete lookup class overrides one of the protected virtual properties (a paranoid precaution, not necessarily required -- this is the only reason why I have backing fields in the first place).

<class xmlns="urn:nhibernate-mapping-2.2" name="StackingCode.Neno.DomainModel.Lookup, StackingCode.Neno" abstract="true" mutable="true" table="Neno_Lookup">
    <id name="Id">
        <generator class="identity" />
    </id>
    <discriminator column="Discriminator" />
    <version name="Version" />
    <property name="Name" />
    <property name="Description" />
    <property name="DisplayOrder" />
    <property name="BoolA" access="nosetter.camelcase-underscore" />
    <!-- ... and so on -->
</class>

Example

Before I go too much further, this post could benefit from a real world example, so one of the things in my Neno "list of things to do" is store some metadata about the Pivot collections I generate. Doing so will help me find and clean up old Pivot collections automatically when the need arises. In my PivotCollection domain class I want to store the date and time the Pivot collection is generated, as well as its path on the server.

namespace StackingCode.Neno.DomainModel
{
    public abstract class Lookup : Lookup<int>
    {
    }

    public class PivotCollection : Lookup
    {
        public PivotCollection()
        {
            _dateTimeOffsetA = DateTimeOffset.UtcNow;
        }

        [DisplayName("Create Date, Time and Offset")]
        public virtual DateTimeOffset CreateDateTimeOffset
        {
            get { return DateTimeOffsetA ?? DateTimeOffset.UtcNow; }
            set { DateTimeOffsetA = value; }
        }

        [Required]
        [StringLength(200)]
        public virtual string Path
        {
            get { return StringA; }
            set { StringA = value; }
        }
    }
}

The properties on this domain class store their values in the underlying protected properties. Some validation attributes have been added, which don't override the underlying validation rules but rather add to them instead.

I can tell NHibernate about this new class by adding a subclass element to the NHibernate mapping file I just defined. Here is where I specify a discriminator value for the concrete subclass too (the abstract base class doesn't need one -- it's never instantiated).

<class xmlns="urn:nhibernate-mapping-2.2" name="StackingCode.Neno.DomainModel.Lookup, StackingCode.Neno" abstract="true" mutable="true" table="Neno_Lookup">
    <id name="Id">
        <generator class="identity" />
    </id>
    <discriminator column="Discriminator" />
    <version name="Version" />
    <!-- ... and so on -->
    <subclass name="StackingCode.Neno.DomainModel.PivotCollection, StackingCode.Neno" discriminator-value="PivotCollection" />
</class>

When pulling PivotCollection objects out of the database, I may want to use some of its properties in my query. I can see myself at least sorting by the CreateDateTimeOffset property in a query. Unfortunately NHibernate doesn't know about the two public properties I've put on the PivotCollection class, only the underlying protected properties. That can be remedied like this...

<subclass name="StackingCode.Neno.DomainModel.PivotCollection, StackingCode.Neno" discriminator-value="PivotCollection">
    <property name="CreateDateTimeOffset" column="DateTimeOffsetA" insert="false" update="false" />
</subclass>

I've defined the public CreateDateTimeOffset property and told NHinernate to use the same column as the underlying protected DateTimeOffsetA property. This is fine for queries, but it's important to specify insert="false" and update="false" lest the DateTimeOffsetA column appear twice in an INSERT or UPDATE SQL statement.

Repositories and Services

So what does all that give me besides a way to avoid making an extra table and an NHibernate mapping file for every lookup class? Well that's usually enough, especially when building an enterprise application with 30 odd lookup classes. But I can expand on the idea a bit more. I can build a single repository and a single service to handle every lookup in my system.

Back in the StackingCode.Moja namespace...

namespace StackingCode.Moja.DomainModel.Repositories
{
    public interface ILookupRepository<TLookupId> : IRepository<Lookup<TLookupId>, TLookupId>
    {
        TLookup Get<TLookup>(TLookupId id) where TLookup : Lookup<TLookupId>;
        TLookup GetByName<TLookup>(string name) where TLookup : Lookup<TLookupId>;
        IQueryable<TLookup> GetQueryable<TLookup>() where TLookup : Lookup<TLookupId>;
    }
}

The ordinary Get method in IRepository will of course still work, but it will return the lookup as Lookup. A typed Get to return the real lookup will be handier. Similarly for GetByName and GetQueryable.

Here's a good starting point for the lookup service without going overboard.

namespace StackingCode.Moja.DomainModel.Services
{
    public interface ILookupService<TLookupId>
    {
        TLookup GetLookup<TLookup>(TLookupId id) where TLookup : Lookup<TLookupId>;
        TLookup GetLookupByName<TLookup>(string name) where TLookup : Lookup<TLookupId>;
        IQueryable<TLookup> GetLookups<TLookup>() where TLookup : Lookup<TLookupId>;
        void CreateLookup(Lookup<TLookupId> lookup);
        void UpdateLookup(Lookup<TLookupId> lookup);
        void DeleteLookup(Lookup<TLookupId> lookup);
    }
}

I have enough information to implement most of these interfaces in the StackingCode.Moja namespace right now.

namespace StackingCode.Moja.Repositories.NHibernate
{
    public abstract class LookupRepository<TLookupId> : Repository<Lookup<TLookupId>, TLookupId>, ILookupRepository<TLookupId>
    {
        protected LookupRepository(IContext context, ISession session)
            : base(context, session)
        {
        }

        #region ILookupRepository<TLookupId> Members

        public TLookup Get<TLookup>(TLookupId id) where TLookup : Lookup<TLookupId>
        {
            return Session.Get<TLookup>(id);
        }

        public TLookup GetByName<TLookup>(string name) where TLookup : Lookup<TLookupId>
        {
            return Session.Query<TLookup>()
                .Where(lookup => lookup.Name == name)
                .SingleOrDefault();
        }

        public IQueryable<TLookup> GetQueryable<TLookup>() where TLookup : Lookup<TLookupId>
        {
            return Session.Query<TLookup>()
                .OrderBy(lookup => lookup.DisplayOrder);
        }

        #endregion
    }
}

NHibernate lets me load objects by type and id, so implementing Get<TLookup> is insanely easy.

The service will pass through straight to the repository for now. There is still nothing special happening in my service layer, like security checks.

namespace StackingCode.Moja.Services
{
    public abstract class LookupService<TLookupId, TLookupRepository> : Service, ILookupService<TLookupId> where TLookupRepository : ILookupRepository<TLookupId>
    {
        protected LookupService(IContext context, TLookupRepository lookupRepository)
            : base(context)
        {
            LookupRepository = lookupRepository;
        }

        protected TLookupRepository LookupRepository { get; private set; }

        #region ILookupService<TLookupId> Members

        public TLookup GetLookup<TLookup>(TLookupId id) where TLookup : Lookup<TLookupId>
        {
            return LookupRepository.Get<TLookup>(id);
        }

        public TLookup GetLookupByName<TLookup>(string name) where TLookup : Lookup<TLookupId>
        {
            return LookupRepository.GetByName<TLookup>(name);
        }

        public IQueryable<TLookup> GetLookups<TLookup>() where TLookup : Lookup<TLookupId>
        {
            return LookupRepository.GetQueryable<TLookup>();
        }

        public void CreateLookup(Lookup<TLookupId> lookup)
        {
            LookupRepository.Create(lookup);
        }

        public void UpdateLookup(Lookup<TLookupId> lookup)
        {
            LookupRepository.Update(lookup);
        }

        public void DeleteLookup(Lookup<TLookupId> lookup)
        {
            LookupRepository.Delete(lookup);
        }

        #endregion
    }
}

Now I'm pretty much liberated from having to implement a repository for each lookup class, and from implementing separate service methods for them too.

Back to the Example

Before I get back to my "real world" example, in StackingCode.Neno all my Id properties are integers, so I can make the above interfaces and classes less generic, like this...

namespace StackingCode.Neno.DomainModel.Repositories
{
    public interface ILookupRepository : ILookupRepository<int>
    {
    }
}

namespace StackingCode.Neno.Repositories.NHibernate
{
    public class LookupRepository : LookupRepository<int>, ILookupRepository
    {
        public LookupRepository(IContext context, ISession session)
            : base(context, session)
        {
        }
    }
}

This just makes life a little easier with things like IOC containers.

Right, so, in an earlier post, I described how I use an MVC controller/action to generate a new Pivot collection whenever I need one. I'm going to modify that action to also create a PivotCollection domain object. Then I'm going to create a new action to use that information to clean up old Pivot collections from the disk.

[UserIsAnAdministrator]
public ActionResult GenerateCollection()
{
    try
    {
        IEnumerable<PivotPost> items = Container.Get<IPostService>().GetPosts()
            .Where(post => post.IsPublished)
            .Where(post => post.Attachments.Any(attachment => attachment.Slug == "pivot-screenshot"))
            .ToArray()
            .Select(post => new PivotPost(post));

        var pivotCollection = new PivotCollection();
        pivotCollection.Name = pivotCollection.CreateDateTimeOffset.ToString("yyyyMMddHHmmss");
        pivotCollection.Path = Path.Combine(Server.MapPath("~/content/pivot"), pivotCollection.Name, "Collection.cxml");

        var creator = new PivotCollection<PivotPost>("Stacking Code", items, 1024);
        creator.WriteToFile(pivotCollection.Path);

        Container.Get<ILookupService>().CreateLookup(pivotCollection);
        Messages.Add("Pivot collection successfully generated.");
    }
    catch (Exception exception)
    {
        Messages.Add(exception);
    }

    return RedirectToAction("removeoldcollections", "pivot");
}

By the way, the action that serves up the PivotViewer control directs it to load the Pivot collection with the newest name -- it does that by looking at the directory names on the disk (although after this it can look in the database instead).

In the above action I create the domain object early and use the information contained within to help with the actual Pivot collection generation. Grabbing the ILookupService from my IOC container, all I need to do is call CreateLookup to persist the domain object. Note, I'm redirecting to my new cleanup action as well, which goes like this.

[UserIsAnAdministrator]
public ActionResult RemoveOldCollections()
{
    try
    {
        PivotCollection[] pivotCollections = Container.Get<ILookupService>().GetLookups<PivotCollection>()
            .OrderByDescending(pivotCollection => pivotCollection.CreateDateTimeOffset)
            .ToArray();

        if (!pivotCollections.Any())
            throw new Exception("There are no Pivot collections.");

        PivotCollection currentPivotCollection = pivotCollections.FirstOrDefault();

        // Remove Pivot collections older than hour.

        DateTimeOffset anHourAgo = DateTimeOffset.UtcNow.AddHours(-1);

        IEnumerable<PivotCollection> pivotCollectionsToRemove = pivotCollections
            .Where(pivotCollection => pivotCollection.CreateDateTimeOffset < anHourAgo)
            .Where(pivotCollection => pivotCollection.Name != currentPivotCollection.Name);

        if (pivotCollectionsToRemove.Any())
        {
            // If the current Pivot collection is less than an hour old, keep one Pivot collection older than an hour.
            if (currentPivotCollection.CreateDateTimeOffset > anHourAgo)
                pivotCollectionsToRemove = pivotCollectionsToRemove.Skip(1);

            foreach (PivotCollection pivotCollection in pivotCollectionsToRemove)
            {
                string path = Path.GetDirectoryName(pivotCollection.Path);

                if (Directory.Exists(path))
                    Directory.Delete(path, true);
            }

            Container.Get<ILookupService>().DeletePivotCollections(pivotCollectionsToRemove);
        }

        Messages.Add(string.Format("{0} Pivot collections kept and {1} removed.", pivotCollections.Count() - pivotCollectionsToRemove.Count(), pivotCollectionsToRemove.Count()));
    }
    catch (Exception exception)
    {
        Messages.Add(exception);
    }

    return RedirectToAction("index", "home");
}

The premise of the cleanup code is to remove Pivot collections that are older than an hour. Anything under an hour old could still be in use by a user and I don't want their Pivot experience to suddenly cease mid-stream. (Hypothetically users could still be using Pivot collections more than an hour old, but I need to draw the line somewhere.) I don't want to remove the current (newest) Pivot collection. And if the current Pivot collection is less than an hour old, then that means one of the Pivot collections older than an hour could still be in use, so I'll keep the newest one of those too. (Regardless of how old it is, a user could have accessed it 50 minutes ago and be totally unaware of the newer Pivot collections created 45, 29 and 5 minutes ago.)

So once all the directories have been deleted -- which is much better than removing the thousands of files in each Pivot collection by FTP, I can tell you -- I need to remove the corresponding PivotCollection domain objects as well. Enter a new service method to do that inside a transaction.

namespace StackingCode.Neno.DomainModel.Services
{
    public interface ILookupService : ILookupService<int>
    {
        void DeletePivotCollections(IEnumerable<PivotCollection> pivotCollections);
    }
}

namespace StackingCode.Neno.Services
{
    public class LookupService : LookupService<int, ILookupRepository>, ILookupService
    {
        public LookupService(IContext context, ILookupRepository lookupRepository)
            : base(context, lookupRepository)
        {
        }

        #region ILookupService Members

        public void DeletePivotCollections(IEnumerable<PivotCollection> pivotCollections)
        {
            ITransaction transaction = Context.BeginTransaction();

            try
            {
                foreach (PivotCollection pivotCollection in pivotCollections)
                    LookupRepository.Delete(pivotCollection);

                transaction.Commit();
            }
            catch
            {
                transaction.Rollback();

                throw;
            }
        }

        #endregion
    }
}

I knew that service layer was there for a reason.

Conclusion

Hopefully I've demonstrated how a one-size-fits-all lookup class can make managing all the tiny classes in an application a bit easier, and reduce a large amount of very repetitive code. Less code can mean less errors and hopefully greater maintainability.

Don't feel like it needs to stop here either. The concept could be extended to making a base MVC controller to handle all the CRUD views for lookups. (I might do that if I ever get around to creating an administration area for this application.)