Many years ago – in 2004 to be exact – I deployed my very first production software. Together with a bunch of friends, we had thought of developing a school management information system (SMIS) during our finalist year at university and had subsequently set up an outfit to work on this and other projects. I happened to be the lead developer on the SMIS project and shortly after we graduated, I had a prototype ready. A demo of the – admittedly rather crude but impressively functional – prototype at a certain international school netted us our first order but the school was only interested in the Library Management module. I therefore put in a few sleepless nights enhancing the said module and adapting it to the international school’s requirements. Within a couple of weeks, it was ready for deployment. Oh the joy!
Our agreement with the school stipulated that we would handle the initial data entry and so after deployment and testing with the school librarian, we arranged for a small team of super-skilled data entrants (read a bunch of friends who had some time on their hands) to go and get the data entry done. After an hour or so, we had close to 100 book titles in the system. I was anxiously testing the system every few minutes to confirm that the book listing was working fine. With the 80 or 90 books, the system felt quite fast – practically as snappy as it was on my development PC even though the database was on a different machine and the data was thus being accessed over the local area network. Satisfied that all was going smoothly, I left the data entry team on site and went off to go and get a few more development tasks done.
I returned to the school some two days later to check on the team’s work. The data entry was going on well and the team had posted a few thousand titles. We had estimated that the data entry exercise would take about a week based on the progress registered after the first day and we were on course to achieve this. However, I was in for a rude shock when I fired up the application on one of the client PCs. Listing books was no longer snappy. Rather, a simple listing of, say – all “Fiction” books was taking more than 30 seconds to complete. To say I was horrified and completely perplexed is an understatement. My first thought was there must have been something wrong with the network. But posting new titles was working very well off the same client machine. I hastily took a backup of the database to test offline on my development machine. For the nerds out there, I was using a database called Firebird, an open-source derivation of Borland‘s Interbase. Here’s a fun fact for the nerdy: Borland’s founder Philippe Kahn happens to be the inventor of the camera phone. I truly loved Firebird – not only did it have an incredibly small footprint (2.5MB installer) but it was lightning fast and packed databases into single self-contained files so a backup essentially meant copying a single file to whatever backup medium you had (usually a floppy disk back then). Unlike MS Access, it had no single-user locks and could comfortably handle millions of records. In my opinion, it was (and still is) one of the best database products ever produced even though the open source market eventually settled for MySQL and PostgresSQL.
Back to the story: When I tested on my local machine, my worst fears were confirmed. Book listing while using a filter such as book category or accession number was terribly slow. Curiously however, listing all the books was relatively quick. This is the point in the story where I must confess that this was the very first database I was deploying to actual production in real life, to be used by real human beings and yet I didn’t know very much about databases. The little I knew about databases was entirely self-taught and was mostly about how to design tables, insert, update and retrieve data. Not only had I not done the kind of stress testing that would have been representative of the customer’s environment (that would have required some 10,000 records) but I was blissfully unaware of the need to optimise the data tables to ensure consistently speedy retrieval of data especially as the number of records increased. Luckily enough, a little bit of research (these were the pre-mobile internet, pre-Stack Overflow days so this kind of research was decidedly not as easy as it is now) revealed that I needed to define indexes on the various book information fields (category, status, accession number etc) that I was filtering by. Note to any pedant reading this: “indexes” as opposed to “indices” is correct.
On defining the various indexes, the book listing against the database with a few thousand records was immediately responsive again. If anything, it was even quicker than my original testing involving a handful of records with any filter returning data almost instantaneously. On returning to the school, I quickly recreated the indexes on the system database and sighed with immense relief when they worked as expected. That was my introduction to real-life optimisation and trust me, it opened my eyes a great deal. Over the course of my career, I have gone on to design and/or manage a number of important, high-performance database systems and the lessons from my first deployment have stuck with me throughout: test using representative loads during development and optimise continuously.
I thought a lot about that indexless database this week when I came across this tweet and the subsequent thread and responses:
It is obvious that the number of motor vehicles in Uganda in general and Greater Kampala in particular has increased a lot over the past two decades. It is also obvious that this increase is largely driven by an increase in the number of people with sufficient income to afford a secondhand vehicle. Typically though, such a vehicle will be a Japanese make aged between 10 and 15 years old on importation. The upper limit of 15 years is now fixed in the country’s regulations with initial attempts to fix it at 8 years having failed after a public outcry. On average, such a “new” car will cost between $4,000 and $12,000. There are of course outliers on the upper side of this scale (and – thanks to shipping and exorbitant taxes – none on the lower side). Thus one will see a number of brand new or relatively new Toyota Landcruisers, pickups of all makes (Ford, Toyota, Nissan, Mitsubishi etc), various Lexus, Mercedes, Jeep and even Porsche models on our roads that cost 10 times as much as the average price range indicated above. Car enthusiasts will actually remind you that a couple of Bentleys, Rolls Royces and even a Lamborghini Urus have been spotted on our streets. There are also a number of secondhand vehicles between 5 and 10 years. However, all these are vastly outnumbered by the “entry level” secondhand vehicles. There is therefore a debate to be had as to whether buying 12 year old, $5K Japanese vehicles is actually a sign of prosperity or a sign that Japan has an excessive supply of unwanted used vehicles that they must get rid of in order to make space for the latest and greatest in their garages. I hesitate to call this dumping given how useful and long-lived these imports tend to turn out but it is definitely something like that. This however is not the main point and it is definitely not the reason I thought about that indexless database.
Rather, it was the notion that our horrible (and it is terribly horrible nowadays) jam is down to the number of vehicles. Unfortunately, it is quite hard to get up-to-date statistics on car ownership in Kampala/Uganda. I am therefore unable to provide current per-capita ownership figures (I’ll include them in a subsequent post if I get them). I did come across this page with 2015 data however and it was an interesting read especially for the sake of comparison with other countries.
My problem with Andrew’s argument is that he completely ignores the fact that, just like my indexless database, our city road network is completely unoptimised for smooth vehicular traffic flow. I remember there used to be jam through Wandegeya in the early 2000s before the now-cited explosion in the number of vehicles. Ditto for other places like Nakawa, Kireka and Ntinda. On my way to the CBD from Kira Municipality, I encounter so many obvious bottlenecks that cause jam even for a handful of vehicles that I struggle to take anyone placing blame on the vehicle count seriously. Consider the following route: Kira -> Kyaliwajjala -> Kireka -> Banda -> Kyambogo -> Nakawa -> Jinja Road/Kampala Road/Bombo Road up to Wandegeya. This is a journey of about 20km which one should be able to do in 20 minutes moving at an average speed of 60km/h. Yet, this routinely takes about 2 hours even out of rush traffic times. Why? Because along the way are so many bottlenecks that one seriously wonders why the municipality and city employ planners. Here is a non-exhaustive list of bottlenecks: Kyaliwajjala junction, various feeder roads from Mbalwa (many of which carry motorists fleeing from Jinja Road), the Kireka junction, feeder roads off the main Jinja Road in Banda and Kyambogo and then the insufferable Spear Motors junction. None – and I mean none – of these bottleneck junctions before the Nakawa traffic lights towards Lugogo has any electronic traffic signalling to manage traffic flow equitably and so they are manned by traffic policemen/women who I’ll admit I admire very much for their dedication and tenacity. They do a thankless job in the dust and variable weather conditions and they always turn up. However, no matter how much they try, they cannot beat modern junction design and infrastructure like over-and-under passes. Meanwhile, the Kira-Kireka portion of this journey features a road that is too narrow for motorcyclists to safely share with 4 wheeled traffic and so accidents between the two forms of traffic are a constant occurrence on the road. The conditions described above can be copy/pasted to practically all the other routes from Kira to the Kampala CBD i.e. through Naalya or Najeera/Ntinda and you’d still be accurate. Going through Naalya means part of the journey will be along the Northern Bypass – the kind of infrastructure development that we desperately need if we’re to ease the current debilitating jam. However, this project started in 2004, is only 23 km long and the first phase took 5 years to deliver quite a bit less than what was needed i.e. full dual-carriage way with seamless interchanges and slip roads for traffic joining along the way. Instead, we got a single modern interchange (Bwaise), a single dual-carriageway section (Kalerwe – Namungoona) and large roundabouts everywhere else. As a result, the Bwaise section showed what we expected/needed and the roundabout chaos (Kalerwe and Namungoona in particular) showed us what the shortsightedness delivered. The second phase, intended to finally bring the Bwaise vision is currently ongoing. Having started before the last election, it is likely to be completed after the next one.
Infrastructure like the Northern Bypass, the Entebbe Expressway and the recently kicked off Kampala Flyover project will definitely improve traffic flow and reduce jam. The problem is that these projects are not only too few but the execution speed is such that we will only see their benefits decades from now. Read Daniel Bwambale‘s excellent take on some of the bewildering decisions that have been made in this area here. Even smaller, cheaper infrastructure projects like installing traffic lights often (read always) have to wait for donors. So, whereas synchronised traffic lights along Jinja Road from Nakawa up to Namanve or even Mukono would no doubt help ease city traffic flow, we opt to hide behind the “there are too many cars” defeatist and simplistic argument. Even simple solutions like “mini bypasses” at junctions are rarely thought about. A good example of such a “mini bypass” is the one at the Kati-Kati junction for vehicles coming down from Kololo and intending to turn left onto the Lugoto Bypass towards Bukoto. Before the mini bypass at the junction was implemented, drivers turning left had to wait endlessly for those turning right to Lugogo to navigate the traffic. That simple measure significantly improved the traffic flow at that particular bottleneck before the now-present traffic lights were installed. Another example of simple but brilliant and impactful measures is the climbing lanes that have been included in recent highway reconstruction projects. These simple, highly convenient additions mean that relatively fast traffic is not stuck behind slow-moving vehicles going up slopes and has no doubt saved many lives. If we were a scientific, data-driven nation, we would no doubt have commissioned and published studies into their impact. Alas, we are not. The equivalent of database indexes would be the implementation of such simple measures in all the places where it is possible to have them.
Is it possible to completely eliminate traffic jam through such optimisation? Of course no. At some point, the number of vehicles will indeed cause jam in some places. It is also impossible to eliminate all bottlenecks. Indeed, in the subsequent Twitter discussion, Andrew and others pointed out that cities like London, New York and Dubai also experience jam. But there are two crucial differences between Kampala and those other cities: First, those cities have made huge steps in improving and optimising their road infrastructure so they are genuinely at or near the point where the limiting factor is the number of vehicles. Further, they strictly enforce things like land use and urban planning policies and they don’t tolerate road reserve encroachment and blatant violation of traffic rules (another contributor to our traffic mess). Secondly, all these cities offer decent alternatives to motor vehicle traffic. It is possible for city residents to use those alternatives entirely. To use our database optimisation analogy, even with all the indexes required and other measures in place, your database may slow down once it contains a large amount of data. A good example of a city where the limits of road infrastructure implementation and optimisation have already been reached is Singapore whose government announced a little while back that it would be capping the net increase in the number of cars on their roads to 0% (down from an annual increase of 0.25%). Reason? They no longer have enough land to dedicate to road expansion. But we are nowhere near this point yet. Appropriate, fit-for-purpose infrastructure and alternative public mass transit options would greatly reduce the jam we experience every day. To put it differently: if an urban planner from Singapore or Dubai (or any of the cities mentioned) came to Kampala and studied our traffic congestion, I really doubt he/she would list the number of cars as the top cause. That would be too simplistic. All it is is a convenient excuse for the technocrats and politicians who have failed us and the prosperity argument is just some strange kind of consolation logic – we should endure our traffic misery and the associated negative impact to quality of life since it is a sign that our wallets are better off. After all, you’d think that kind of prosperity would translate into greater tax revenue that would then fund infrastructure development at the same rate as the increase in the number of vehicles. Vehicles, fuel products and spare parts are all heavily taxed by the way. But then again, I doubt our hypothetical Singaporean technocrat would take the abundance of $5K, 12 year old secondhand cars as a serious sign of prosperity. Given this observation will likely annoy my NRM friends who believe our president is a better leader than Lee Kuan Yew, I’ll stop here..