Traditional Valid-Time State Table

This is certainly a valid and widely used method, but I find that when coding this creates difficulties in coding, especially when working with these records as definitions of different methods in a continuous piecewise  calculation.

I find it preferable, and more efficient to model these tables after life. In the case of the employee, the logic works out like this:

1. If no record exists for a given Employee  ID, then the Employee does not exist. This is analogous to a birth certificate, we could not have a record of a birth that has not yet happened. This is consistent with the traditional model.
2. An employee’s salary is effective beginning with the time recorded and ending when another record is encountered with an effective time > than the record being examined.
3. An employee ceases to exist when a death certificate is filed, (In this example NULL is used to denote the termination of an employee.)

The table below shows how this implementation would work with the same data from the previous table.

Existence Model Valid-Time State Table

The existence model clearly would be beneficial from a maintenance point of view, as it requires a new record only for each change in salary, unlike the traditional model which would require an update for each contract extension, new year, etc. Additionally it is always possible to calculate the same information stored in from date and to date by collecting the desired record and a more recent record if available.

The two models are very similar, but consider the existence model in the case of a real time system (not to say that this would be efficient for millisecond level state recording). The current state of an entity can always be found by finding the most recent record. The previous state can always be found by querying for the second most recent record and so on. This also simplifies the logic necessary to find the state at any given time. A sample query would look like this:

SELECT * FROM SALARY WHERE EMPLOYEE ID = 4 AND EFFECTIVE DATE <= INPUT DATE ORDER BY DATE DESC LIMIT 1

Because we are modeling this after existence, a result is guaranteed to be returned as long as an employee exists.

The same query in the traditional model would look like this

SELECT * FROM SALARY WHERE EMPLOYEE ID = 4 AND FROM_DATE <= INPUT DATE AND END_DATE >= INPUT DATE

Which provides no guarantee of a result and is thus prone to misconfiguration by the end user(assuming salary renewals are not automated).

Neither model is perfect for every use case, but I believe and I think you will find, that the existence model makes much more sense logically from a developer and end user perspective for a majority of applications.

Here is a post from the fantastic blog EXPLAIN EXTENDED on how to adapt existing range based tables to produce time gap results.

The solution to this problem is beginning your application design with the Naked Object architecture in mind. Naked Objects is a relatively new design pattern for implementing business logic in a flexible way. Richard Pawson formalized the concept in his PHD thesis in 2004 (pdf). I recommend reading it if you have the time.

The idea is a simple one, traditional business logic patterns like MVC (Model View Controller) have been misconstrued by their implementers into architectures in which the design limits the abilities of the user. (Trygve Reenskaug, the man behind MVC, wrote a great introduction the Pawson’s thesis describing this.) The Naked Objects Pattern removes the controller and places all of the logic on the object thus allowing the object to define the view. (There are frameworks built around this which automatically generate GUIs).

Naked Object Architecture

The real benefit of implementing your design in this way is that it allows for quick development of new features and application layers. There is currently no PHP framework for implementing true Naked Objects design (it looks like this attempt has been abandoned), but you can apply the basic philosophy to your application.

The immediate argument that comes to mind for its use, besides the obvious development speed increase for GUI features, is easily implementing an API on top of your sites data. Think about Twitter for example. You have three basic objects user, feed, and post. If all of the interaction is defined in an MVC type pattern then you would have to invoke several controllers to authenticate a user and retrieve a feed of the 10 most recent posts. The architecture is too limiting and difficult to be easily adapted to support basic API requests. Getting the 10 most recent posts from naked objects would be simple. Authenticate against a method from the user object, call the most recent method from the feed given the user specified, and request the content of the posts given from the post object. The actual implementation of this API call would likely be ~10 – 15 lines of code (not including all of the protocol stuff).

The other major advantage is easy unit and performance testing. Supposed you wanted to unit test the new post / retrieve post logic for Twitter. In MVC you would invoke the controller which in turn invokes the model. Hunting down a problem in a simple storage and retrieval like this could be simple. But how would you know where exactly, the model or controller, your code broke down in a controller function with 100 lines of code and a model with 50 lines of storage logic? You end up with stack traces and that’s just a mess (though always useful). With the methods defined on the objects you can simply implement and unit test these calls (assuming you have a well tested data storage layer). This seriously helps in avoiding problems with feature creep if you use a standard GET / SET methodology on your data members. More complicated functions call on simple ones and you can unit test up the complexity tree.

There’s an argument against over coding your objects though, especially in PHP. This is due mostly to a lack of inlining / compilation. Every function all has considerable overhead when you’re thinking on a microsecond scale. I think however you would find that the savings in development costs far outweigh any costs for extra resources needed to support the extra capacity in terms of page views and response times. As a developer it seriously decreases the amount of days high on the headache index when your boss comes running down the hall asking for new features.

Horizontally Scalable DB Layer Using Gearman and Shard Query

Obviously more job servers and worker nodes can be added to this structure to meet demand.

Take as an example a data set with a DATETIME field. A sample query over this data set would be

SELECT * FROM EXAMPLE_TABLE WHERE DATE > ’2009-01-01′ AND DATE < ’2010-01-01′

MySQL handles this as a single process. Shard-Query, however, automatically partitions the query across Gearman workers.

With two worker nodes the query would become two queries.

SELECT * FROM EXAMPLE_TABLE WHERE DATE > ’2009-01-01′ AND DATE < ’2009-6-01′

SELECT * FROM EXAMPLE_TABLE WHERE DATE > ’2009-06-01′ AND DATE < ’2010-01-01′

Connections would be established to the DBMS from each worker and when both data sets had been returned, Shard-Query combines them and returns them as a single result set.

See the graph below for a performance comparison between Shard-Query and standard query execution.

Shard Query Parallelism