Thursday 30 July 2015

Books, mostly

Hola.

My last set of posts about Using the Model-Store pattern in Inversion took a large heap of my blogging-concentration, but as they've been going since March it's good that they are finally out of the way.

Work's going well and I'm getting my teeth stuck into producing a cloud-based platform for digital library services for digitised collections, services like deep-zoom image serving, discovery, managing metadata and work with the emerging IIIF standard for viewers. Work days, I'm at Wellcome, which is a bit like being on a spaceship, although the food is probably better than NASA serve.

I'm cycling to work which is moderately terrifying. Drivers are terrifying. Cyclists are terrifying. Especially cyclists, frankly, considering some of the whacked-out shit I've seen them do in just a few weeks of riding.

At the moment I'm studying for the Microsoft Programming in C# Exam (70-483) which is on Monday afternoon. It helps out Digirati and definitely doesn't hurt me to go for the certification. However, the study has kinda precluded me making much progress with the stuff I'm reading.

Oh, I bought a Microsoft Band the other day, partly because geek and partly to mitigate an RSI I'm getting in my right index finger. Reaching for my phone in jeans pockets every 30 seconds is really causing a problem, it seems, so the Band picks up the notifications for me. The cycle / run tracking is really good too, although I was tipped over into buying one when I learned that they'd opened up the SDK to allow arbitrary RSS feeds to be added to the tiles.



Go get one! They're really cool.

Currently reading:

Assassin's Apprentice (Farseer Trilogy part 1)
Robin Hobb




Building Microservices
Sam Newmann


(Well, I lie - I just finished it the other day - really bloody good, though.)

And my exam book:
Programming in C# Exam Ref 70-483
Wouter de Kort


Which has got a whole bunch of text inconsistencies (calling things modules, for example), some really bad practice (if you generally mark your classes as sealed then you, whomever you are, are a sociopath), and some answers missing. So that's nice.


Tuesday 21 July 2015

Using the Model-Store pattern in Inversion (part 5)

This post is the fifth in a series about using a Model-Store pattern in the Inversion behavioural micro-framework for .Net (see: https://github.com/guy-murphy/inversion-dev).
The first part can be found here.
The second part can be found here.
The third part can be found here.
The fourth part can be found here.

Aftermath


Looking back, I might as well have titled this essay - "I hate ORMs".

Firstly, because I really, really do - I think they are at the root of many evils in .Net software architecture. That's not to say that its the ORMs fault, because it really isn't. However, time after time we are told to "just use" Entity Framework or N-Hibernate because that's what Microsoft or the big boys have blogged about, and time after time I see people getting into the same mess with a half-cocked Repository pattern, absurdly heavyweight models and bloody DTOs springing up all over the place. What I want is a clean, quick, safe and lightweight pattern of data access, and an ORM has never convinced me that it is any of these things because people tend to abuse the functionality it provides without thinking about the consequences or that there might be other ways to approach data access.

Secondly, I didn't actually need to say that this was using Inversion, because Model-Store can be done in any .Net framework. However, it's been useful to write the foundations of the Inversion.Data namespace by considering the paths that writing this article has led me down.

The views in this set of articles almost feel heretical when looking at the current "best practice" of .Net application development. Having an immutable model POCO and a lightweight store (which eschews ORM-like functionality) shifts complexity away from the data access implementation, forcing any business logic back to the application where (in my opinion) it belongs. Data access should be dumb, MUST be dumb, if the application is to be agnostic about the storage technology to be used. This massively increases the platform and language options available for porting and the development of new features. I would counsel avoiding relying upon the features of a particular storage medium, because you never know when that medium could become suddenly unsuitable or unavailable.

Model objects are best designed very closely to the underlying table or document structures. This is for the simplicity of maintenance and the ease of introduction to new development staff. A Model should remain at a level of abstraction such that the properties would be the same across common types of database representation. Therefore, an application that uses this pattern can achieve granular control over the strategies used for reading, writing and updating individual data classes whilst also remaining agnostic to storage technology choices.

When developing the MyOffers.co.uk data analytics engine, we tended to write Stores for at least two storage mediums, e.g. MS Sql Server and Redis and/or MongoDB. This forced us into keeping data access straightforward while letting us concentrate on what the application actually needed to accomplish. These same patterns helped us again when writing a cloud-based quiz engine that would operate at scale and the new version of the website application. It forms part of a discipline that assists what I believe to be a fast and clean way of considering modern system architecture.

Immutable Models and CQRS/Event-Sourcing


There are other benefits that immutable Models can bring. If you take the basic tenet that a Model is immutable, then it follows that any change that is required to a Model can be represented as a separate thing. This is the basis of an architecture which gives CQRS-like benefits to an application that uses immutable Models in combination with read-only, optimised Model Stores, with changes persisted in the form of Events to write-capable Stores which could have a completely different storage medium that is optimised for write, thus separating the reads (a Query) from the writes (a Command).

This helps support eventual consistency, because any changes must be persisted explicitly and the source that the read-optimised Store fetches from can be lazily updated by a mechanism that is in the background. With this architecture, concurrent threads are guaranteed to read the same value when referencing an object, and modifications can be made without having to worry about locking or making a local copy.

Ending


It is not necessary to use the whole of Model-Store to get these advantages. However, the Store part of the pattern provides an efficient way of packaging data access which can use the immutable Model objects, and shows us that when implementing a DAL in the .Net ecosystem there are other options than Repository patterns and ORMs like Entity Framework or N-Hibernate.

If you haven't agreed with me so far, then perhaps the most compelling thing I can offer is that, most of the time, an application need only read what the values of a Model object are; updates and writes are scarce by comparison. Why weigh your application down with the ability to track changes in an object when you don't use it in most operations?

If you design your application to make changes explicitly then you can take advantage of the benefits of lightweight data access.

I've wanted to document some of my thoughts about this pattern for a while, and I do hope that some of you have stuck through to the end. There are many things that can be pointed to in this set of arguments which other developers may not agree with, and that's fine. As I said, this style of pattern may not be for you. However - I have seen it work in small, medium and production-scale systems, making them fly, whilst also providing a great deal of freedom as a developer.

At the very least - please check out the Inversion framework. Full disclosure being that I'm a major contributor for it and I am using it very happily in the production of a large, automatically scaling cloud-based system for a major institution at the moment, including the bits and pieces you will find in the Inversion.Data namespace.

Dream large.

Sunday 12 July 2015

Using the Model-Store pattern in Inversion (part 4)

This post is the fourth in a series about using a Model-Store pattern in the Inversion behavioural micro-framework for .Net (see: https://github.com/guy-murphy/inversion-dev).

The code that I have been developing as part of this article series can be found at https://github.com/fractos/inversion-data

The first part can be found here.
The second part can be found here.
The third part can be found here.

Using the Store


In this instalment, I'm going to show you what using the pattern looks like. I'll put together a few example functions that use the IUserStore interface to access and manipulate data stored in an underlying MongoDB instance.


Configuring and Instantiation



We are going to use the MongoDBUserStore class that I showed you in the previous article to perform operations in MongoDB. To instantiate it, we will need a few details that describe the MongoDB connection-string, the database name and the name of the collection used for users.

I am using a local MongoDB server, which has a database called 'test' and I want to access model objects in a collection called 'users':



Those details can then be passed to the MongoDBUserStore constructor:



But since this is an IDisposable and we only actually care about the interface methods, then more correctly we should be calling it like this:



At which point we are off to the races.

However, the Store could be gained by dependency injection behind an interface, or fetched from a service container (such as Inversion.Naiad, which implements Inversion.Process.IServiceContainer). I'll try and cover the use of IServiceContainer in a separate article, but the call might look something like this:



By using interfaces alone, you can keep your code free from references to the assemblies needed for accessing MongoDB, keeping the main application as agnostic as possible. All the code cares about is the IUserStore interface.

Asking a service container for an object of type IUserStore which has the label "store-user" will cause the container to perform any configuration required according to the instructions that are associated with the service label. The application doesn't have to care about the configuration of the Store because it is handled by the container. This is a form of loose coupling by service location.

The more agnostic your code is about its dependencies, the more flexible your application can be.

Creating a collection of Users in MongoDB


This method assumes that you have a list of User models which are ready to be written to the database. I've included the basic version of instantiating the Store just to keep it simple.



After this call, you should find that the User models have been written into MongoDB, complete with Object IDs.

Read a list of Users from MongoDB


This method will return all the users from MongoDB as a list of User models.

Note that the IEnumerable is converted to an IList with ToList() before the using statement has been terminated (and therefore the Store and its resources disposed) in order to avoid the enumeration code reading a closed connection.



Updating a list of Users


This function iterates through a passed list of User models, modifying their Password property and then writing them back into MongoDB. It makes use of the Mutate method on the User object in order to change an immutable object. The ID of the User model will remain unchanged, however, so the act of writing the data into MongoDB will perform an update rather than an insert.




Deleting a list of Users


This function will delete each of a list of User models from MongoDB.



Next time ...


In the next (and final, so far ...) instalment, I'll wrap up with a conclusion and some thoughts about how this process went.