One of my motivations in amateur radio is DX, but the other is keeping my skills sharp in IT. It’s rare that I do any coding in a professional sense – because I am not a professional software engineer – so my hobby time is shared between DXing or coding. This is a blog post about choosing where to invest my hobby time next! It is from the perspective of Club Log now reaching a natural stable point where the development work is falling off.
Club Log retrospective
A few years ago, when I first started writing Club Log as a very simple tool, I had a notion that more features would be added and that I wanted to work quite quickly, so I used PHP as my language. This turned out to be a good choice: PHP is versatile, rather easy to use, and lends itself well to database-backed web sites. Club Log is almost all database. PHP also happened to be a language I already knew, but that was a secondary consideration.
Back then in 2008, I thought Club Log was a web application, but it turns out what I really needed – and didn’t yet realise – was to learn more about was SQL, database scalability and caching. Club Log has forced me to learn about these areas in great depth. In terms of hobby time, this has been absolutely perfect – a huge reward gained for the time spent.
Only in the past 9 to 12 months or so could I say that the new bits – things I didn’t know before or didn’t realise existed – have started to dry up. With the exception of RabbitMQ (still on my to-do list!) it’s more or less a mature application where most of the software work is now maintenance and bug fixes.
Looking back on the choices involved in getting Club Log this far is one thing, but looking forward is quite different. Far from being finished, there is an endless supply of work to do, and much of it seems to me greatly fascinating. Actually, the end objective for me is not what I want Club Log to do, although I have it in mind. Since this is my hobby, the real question is “what do I want to learn?”
I’ve thought about this on and off a lot, but I started exploring my options in detail in the past few days because I’m pretty sure the time is right to start looking for some new inspiration.
Is Club Log “Big Data”?
API first and foremost?
Probably not. There are too many indicators that PHP is more or less the perfect choice for Club Log!
However, one technology that really grabs me today, writing in late summer 2013, is Google’s Go programming language (see www.golang.org). Go is an elegant and rather ingenious language which is somewhat like C, but it comes with native parallel processing features that aren’t impossible to work with (pretty much the holy grail!). Go is more convenient and more productive than C – maybe closer to Python in that sense. I’m not the only one fascinated by Go. In fact, it’s the biggest craze in programming out there today. I really want to learn it.
My idea is to use Go to solve a problem I have in Club Log that I can’t solve any other way. This is the problem of rapidly importing ADIF files, with the correct DXCC data looked up from the callsign and date.
Why use Go to solve the same problem all over again?
There is some truth in this (but remember, I really want to learn Go, so that is one of the reasons).
In fact, there is a good reason to look again at the ADIF and DXCC parser. The basic concept of extracting ADIF fields is trivial, but in Club Log I’m interested in how things scale. It’s important to realise that in most ADIF files, the country (DXCC) claim made by the logging software is of dubious worth. In Club Log, we ignore it and calculate the DXCC entity purely from the callsign (the hard way). This callsign-to-DXCC lookup is the focus of my interest.
Club Log is known for its DXCC mapping accuracy, thanks to Alan 5B4AHJ. His research is the cornerstone of almost every part of Club Log that involves DXCC (which is, in turn, almost everything in Club Log). We convert callsigns into DXCC mappings via a software procedure involving hundreds of re-entrant branches of logic, all built in PHP. This code is neither simple nor difficult, but there is so much of it to run for every DXCC lookup – and there are so many DXCC lookups to do! – that it really needs to be fast. A basic tenet of coding a fast application is to make the common case fast (see Amdahl’s law).
Yet, the process of converting a callsign into a DXCC mapping is painfully slow today. Club Log can do it about 100 times a second, per CPU. When I profile the ADIF uploader in Club Log, almost 90% of the elapsed time processing a log is spent in the DXCC lookup routine. This is where a fast, compiled language with parallelism (like Go, basically) can come to the rescue. I want to get that number up – how far, I don’t yet know. Perhaps an order of magnitude.
Closing remark: A few people have asked me the question “How do you find the time?” but I think a better question is “What motivates you?” Every hobby is about indulgence in the end!