One of my motivations in amateur radio is DX, but the other is keeping my skills sharp in IT. It’s rare that I do any coding in a professional sense – because I am not a professional software engineer – so my hobby time is shared between DXing or coding. This is a blog post about choosing where to invest my hobby time next! It is from the perspective of Club Log now reaching a natural stable point where the development work is falling off.

Club Log retrospective

A few years ago, when I first started writing Club Log as a very simple tool, I had a notion that more features would be added and that I wanted to work quite quickly, so I used PHP as my language. This turned out to be a good choice: PHP is versatile, rather easy to use, and lends itself well to database-backed web sites. Club Log is almost all database. PHP also happened to be a language I already knew, but that was a secondary consideration.

Back then in 2008, I thought Club Log was a web application, but it turns out what I really needed – and didn’t yet realise – was to learn more about was SQL, database scalability and caching. Club Log has forced me to learn about these areas in great depth. In terms of hobby time, this has been absolutely perfect – a huge reward gained for the time spent.

Only in the past 9 to 12 months or so could I say that the new bits – things I didn’t know before or didn’t realise existed – have started to dry up. With the exception of RabbitMQ (still on my to-do list!) it’s more or less a mature application where most of the software work is now maintenance and bug fixes.

Future directions

Looking back on the choices involved in getting Club Log this far is one thing, but looking forward is quite different. Far from being finished, there is an endless supply of work to do, and much of it seems to me greatly fascinating. Actually, the end objective for me is not what I want Club Log to do, although I have it in mind. Since this is my hobby, the real question is “what do I want to learn?”

I’ve thought about this on and off a lot, but I started exploring my options in detail in the past few days because I’m pretty sure the time is right to start looking for some new inspiration.

A Javascript application?

I could tear down the Club Log user interface (which is rather old-school in that all of the pages are generated server side). Modern applications are written in Javascript and consume APIs to backend services; a profound difference to how applications used to be. I have looked into certain MVC frameworks, especially Ember.JS, with the thought that this might be worth a shot. Club Log itself doesn’t feel like the best place to start but it could be worth the invested time if more UI is going to be added in future.

Is Club Log “Big Data”?

I could start exploring MapReduce and MongoDB. These data abstractions and worker models might be the key to far more advanced data analysis. In fact, if I were to use a MongoDB approach (which favours a JSON storage engine) this could be paired up with the Javascript idea since JSON is the data presentation layer of choice in JS frameworks. I’d definitely learn a lot if I tried to convert Club Log to these new paradigms that have sprung up in the past few years, although having solved these problems once in SQL I’m not sure the change is necessary.

API first and foremost?

Most developers now accept that API is the top priority if you want to work with others (and as a hobbyist coder, you surely do!). Club Log has quite a few APIs already. They’re mainly for convenience of access to reports, but also the ability to upload QSOs in realtime. They don’t go far enough though: modern applications tend to be almost all about API because that is how Javascript applications tend to function. After all, if the program is in the user’s browser and the data is on the server, you need an API to connect them. So, a worthwhile project could be to make every page in Club Log support API representations of the data (probably with OAuth and JSON).

Replace PHP?

Probably not. There are too many indicators that PHP is more or less the perfect choice for Club Log!

However, one technology that really grabs me today, writing in late summer 2013, is Google’s Go programming language (see www.golang.org). Go is an elegant and rather ingenious language which is somewhat like C, but it comes with native parallel processing features that aren’t impossible to work with (pretty much the holy grail!). Go is more convenient and more productive than C – maybe closer to Python in that sense. I’m not the only one fascinated by Go. In fact, it’s the biggest craze in programming out there today. I really want to learn it.

My idea is to use Go to solve a problem I have in Club Log that I can’t solve any other way. This is the problem of rapidly importing ADIF files, with the correct DXCC data looked up from the callsign and date.

Why use Go to solve the same problem all over again?

There is some truth in this (but remember, I really want to learn Go, so that is one of the reasons).

In fact, there is a good reason to look again at the ADIF and DXCC parser. The basic concept of extracting ADIF fields is trivial, but in Club Log I’m interested in how things scale. It’s important to realise that in most ADIF files, the country (DXCC) claim made by the logging software is of dubious worth. In Club Log, we ignore it and calculate the DXCC entity purely from the callsign (the hard way). This callsign-to-DXCC lookup is the focus of my interest.

Club Log is known for its DXCC mapping accuracy, thanks to Alan 5B4AHJ. His research is the cornerstone of almost every part of Club Log that involves DXCC (which is, in turn, almost everything in Club Log). We convert callsigns into DXCC mappings via a software procedure involving hundreds of re-entrant branches of logic, all built in PHP. This code is neither simple nor difficult, but there is so much of it to run for every DXCC lookup – and there are so many DXCC lookups to do! – that it really needs to be fast. A basic tenet of coding a fast application is to make the common case fast (see Amdahl’s law).

Yet, the process of converting a callsign into a DXCC mapping is painfully slow today. Club Log can do it about 100 times a second, per CPU. When I profile the ADIF uploader in Club Log, almost 90% of the elapsed time processing a log is spent in the DXCC lookup routine. This is where a fast, compiled language with parallelism (like Go, basically) can come to the rescue. I want to get that number up – how far, I don’t yet know. Perhaps an order of magnitude.

So, from 1 September 2013, my aim is to learn Go and replace the DXCC lookup code in Club Log with a brand new set of code that is as fast as possible. I’ll put a new REST API around this so anyone who wants to use it can do so. While I’m learning Go I probably won’t do much else in Club Log, but it’ll be a new outlet for my hobby time that I am looking forward to exploring. Depending on how it goes, I may then work on more Go code or I might circle back and look at the user interface (Javascript) project.

Closing remark: A few people have asked me the question “How do you find the time?” but I think a better question is “What motivates you?” Every hobby is about indulgence in the end!