In our latest blog, Software Developer Henry explains how we created Mabdeck’s latest new feature – validation.
By Henry Pye
Processing meter readings sits right at the core of what Mabdeck does. Performing logic on received meter readings really is the heart of the system. Mabdeck transforms a tangled messy web of numbers into beautiful charts and statements that our clients can use to bill their consumers and analyse their overall energy consumption. It’s fair to say that Mabdeck is a winner when it comes to providing a solution for converting confusing documents containing meter reads into interpretable data that anybody can understand.
In an ideal world, this process would be seamless. However, we aren’t quite living in Utopia yet! Why is there a problem? Let’s say a faulty reading came through with a value of -20 kWh. This can cause multiple problems; a bill could be generated that would credit the consumer, there could be a faulty meter on a network that remains undetected and so on. What happens if a property just stops consuming energy? Has the resident gone on holiday, or is there a genuine problem with their system or meter? It’s clear that a solution was needed to help tackle these. Queue, The New Validation Engine!
The New Validation Engine was originally proposed 6 months ago and has been in planning since then. Initially, the decision was made to develop it in C Sharp, which is the language Mabdeck is developed in. This was discussed in detail and, after many coffees and headaches, it was scrapped, with all the developers moving towards using SQL, or Structure Query Language.
SQL was originally developed in the 1970s by two guys who wanted to interact with databases in a simpler way than was available at the time. Now, I know, business logic in the database?! Had we lost our minds?! Maybe, but, with careful planning, we all felt this could be done well! We needed a language that would allow us to deal with large amounts of data and fast, which is exactly what SQL does. The outcome of all the validation discussions was that incoming readings would need to be validated for the following:
- Negative Readings
- Negative Consumption
- Zero Consumption
- Unusual Temperature Difference – Mabdeck deals with Flow Temperatures (water coming out of the tap) and Return Temperatures (water being received again at the other end). It was clear we would need to ensure the return was lower than the higher for hot water and vice-versa for cold water!
Development of the Validation Engine progressed well. Using SQL was a delight and it’s fair to say that we were very surprised and getting quite excited at just how quick it was churning out validated readings!
We decided to develop it using our Test-Driven-Development (TDD) pattern which was something we were yet to implement in SQL. This was beneficial as it grew our confidence in what we were developing more than we could’ve imagined. The use of TDD allowed us to instantly test what we were creating. Once we had created the Negative Reading function, we instantly tested to ensure it worked and thankfully it did! Without us having to touch any data (the test suite we used allowed us to generate data to run the tests with).
Because SQL is data-driven, it has certain properties that set it apart as a winner from other languages in the industry. SQL allows us to interact with the data we receive directly, without slowing anything down. Other languages on the other hand, bring a lot of bulky extras with them (libraries, package managers, installers, need I go on!?), SQL simply has the queries you need to interact with the data you want. It’s the minimalist’s language! Great!
One of the hiccups we encountered whilst developing was something called table locking. A few of the operations SQL allow you to perform on data are SELECT, UPDATE and INSERT. Each of these operations do exactly as the name would suggest. When one of these operations are performed on a table, the tables will ‘lock’ itself to prevent anything happening that could damage the data within it. Initially, we used a trigger, which allowed us to call a function whenever one of the aforementioned operations was executed. However, this trigger locked the table down! Oh no! A solution was needed quickly. We solved this by removing the trigger and moving to a scheduled task, which runs on a timer – every 30 seconds in this case, which proved much more effective in the long run!
Overall, The Mabdeck Validation engine will provide a high quality of readings to the end user, which will be more usable and much more reliable. We used the best languages for the job, which allowed us as developers to learn and work with something new. We are confident that a high-quality solution has been added, using the most appropriate design patterns and programming languages, that will enhance every interaction users have with Mabdeck.