This article appears in the Autumn 2015 edition of FOCUS, the journal of the First Class Operators’ Club (FOC). It is reproduced here as I felt it would be of wider interest.

The Origins of Club Log

Late in 2014 at the FOC Annual Dinner in Milton Keynes, I was caught completely unawares when the G3FXB trophy was awarded to Club Log. Last month, I was also inducted into the CQ DX Hall Of Fame, again a surprise and a moment of great pride. Prompted by Gabor, I have realised that it is high time I wrote a piece for FOCUS to try to put things into perspective!

My aim here is to give a little bit of history and insight into how Club Log came to be. Here is a quick overview to start us off:

1. Club Log is a web site – http://www.clublog.org – to which DXers may upload their logs. Club Log reads the contents of those logs and from them generates feedback, information and statistical reports (somewhat like a desktop logging tool).
2. The data in Club Log is also put to work in a number of other ways. For example, Club Log helps identify QSLing gaps, finds DX cluster spots for needed band slots, offers propagation tips, great circle maps, automatic Twitter announcements when you work a new one, and much more besides.
3. Club Log also builds league tables of how many entities and zones have been worked by all users: it is an aggregator. Much of its power is found here, for example in finding propagation patterns or building most-wanted leagues from its vast troves of empirical data.
4. Club Log is commonly associated with big expeditions. Through its special expedition service it helps the team leaders and DXers coordinate their activity through graphical depictions of the bands, modes and times of day when QSOs are being made. It is capable of matching logs for the purpose of QSL verification, much like Logbook of the World, but also integrates with PayPal so that QSL cards can be paid for. This cuts 50% of the paper QSLing cycle out for these ‘one way’ QSLs.

It is difficult to sum up every feature, but to finish setting the scene, Club Log is currently storing 300 million QSOs that are owned by 45,000 users from every corner of the globe. Club Log is free of charge, and free from commercial influences such as advertising or sponsorship. Instead, users can choose to make a donation from time to time. This is an effective, alternative model to charging that keeps Club Log viable.

How did all this start?

Although I was first licensed in 1994 when I was 16 years old, I did not really do much serious amateur radio. I was of course interested in computers and the internet. I never really thought to push on to the full class ‘A’ license and the rewards of HF back then, and in the intervening time, I was a student at Cambridge studying engineering. After completing that course, I founded a software company (so time was somewhat in short supply!).

Thankfully, in the summer of 2005 or so, with a push from Martin G3ZAY, I got much more involved with the Cambridge University Wireless Society (G6UW), and began to take a keen interest in the HF bands and DXing. By 2007 the bug had really bitten, and I was considering writing some software to help me keep track of the DXCC entities I had worked.

Even though I had good logging tools on my own PC I could see that it could be great fun to establish a league table with some other locals. Rather naïvely I felt sure I could knock this problem on the head in a weekend or two by writing some code, and with a few tips from Dominic M0BLF (of RSGB IOTA fame) I decided to crack on and write Club Log v1.0.

The objective was simple: using the definitions of DXCC entities and the prefixes for those entities, I would read in the contents of an ADIF log file and convert every QSO into a database record. A database is a powerful analytical tool. It would mean I could make league tables and analyse the aggregate data, and maybe begin with some simple reports.

It was actually quite a delight to be able to combine my professional and radio interests into one project. I do consider myself very fortunate in that respect as it quickly became apparent that this little venture was going to need far more work than a weekend or two!

A bit of trivia – why is it called Club Log? Well, I felt it was fundamental that users should declare themselves to be members of clubs. Clubs would make it possible to create league tables that were smaller and more focused, and perhaps reset annually to reward activity. The word Club in the name is about this philosophy of making smaller, more regional user groups. It seems to work well.

The appeal of statistics

The potential of all the data in Club Log is tremendous. In our hobby, there is a fascination with statistics, mainly I think due to our awards and our tendency to be competitive. We are also interested in substantiating with facts our beliefs about the state of our hobby and its future.

Well, I had perhaps 2 million QSOs to analyse by 2008, and this figure doubled every year after that. Each new QSO sharpens up the database, and from the mass of QSOs piling up even at an early stage, I could see trends were emerging. I could find which bands were busy at which times of day, which DXCCs were most active, which modes were most popular – pretty much any question had an answer hiding in the raw data, and now it was in a database I could quickly get those answers. I was inspired by the possibilities for analysis.

Firstly, I added an empirical most wanted league. It works on the basis that when an entity has not been worked, it is ‘wanted’. This means that entities with fewer QSOs will rise up the most wanted league by counting the QSOs and ranking the results (with a little bit of care about the way each user is counted). I remember being thrilled when I first ran this report, and out came an almost perfect most wanted list! It matched my expectations as a DXer so well, yet it was the product of empirical data.

Later, I added the ability to study the activity between two entities, as a graph of time against each band. It is not a propagation chart but looks very much like one due to the smooth curves in the data. This evolved from work Marios 5B4WN was doing with expedition charts at the time, and now in 2015 it also covers filtering by solar flux or comparing to VOACAP predictions. The underlying data is sprawling and intractable on its own, but this tool makes it feel simple and manageable to explore.

I must say, it is great fun bring the data to life with tools or reports. I have gladly collaborated with one or two like-minded DXers, such as Wayne N7NG and Roger G3SXW, to perform interesting analyses from time to time. These reports can reveal the state of our hobby and how it might be changing. For example, I made a graph of the proportion of QSOs are made in CW each year – pleasingly it seems to be SSB that gives way to data modes, while CW remains as popular as ever! See http://g7vjr.org/2015/03/proportion-of-modes-used-on-the-air-update/ if you are interested.

Building the team

Although I made very little attempt to promote Club Log back in 2008 (knowing it had a few problems with DXCC lookups at the time), it quickly obtained a bit of a following, especially in the UK. There were some aspects of this that I could handle, such as hardware, hosting and databases, but in other areas I was going to need some help. By good fortune, I was able to team up with two very important people:

• Marios, 5B4WN who had invented the now-ubiquitous expedition charts, leaderboards and propagation tables. First used by 3B7C and FSDXA, these charts were inspired by the designs of FOC member John, G3WGV, in his StarSuite system. Before Club Log, Marios would help individual expeditions to run the expedition charts code, but it was hit and miss and a lot of effort each time, with sometimes disappointing results if the server buckled under load. I worked with Marios to provide him with a way to make the expedition charts into a product – one that could be used by appropriate users of Club Log on demand, rather than needing special arrangements each time. We were able to make it reliable and fast under stressful conditions, at last. Marios has recently gone on to write the immensely popular Online QSL Request System (OQRS) that you see in Club Log. I must say his contribution to Club Log is extraordinary and it is a joy to collaborate with such an innovative, smart guy.
• Alan, 5B4AHJ, who possesses a wealth of knowledge about callsigns, DXCC and IOTA, and has a background in software (having written Shacklog). Alan was keen to improve on the situation with the way Club Log was choosing DXCC values from the callsigns in people’s logs, and so back in 2008 or so – somewhat by accident I suspect – he began the long, long task of computerising the historical prefixes and entities from 1945 onwards. Alan has written extensively on how he does it. I will say that it is quite simply a mammoth task, and requires pouring through historical records and piecing together the geopolitics of amateur radio callsigns right through to the present day. Alan continues to manage this and improve upon Club Log’s precision every day. Only because of this work can Club Log generate credible information. I simply cannot imagine Club Log without Alan’s input.

Many others have helped, too. Gary ZL2IFB got things kicked off with our documentation project, which is hugely helpful to new users, and Jim KE8G also stepped in to give Alan and I some support dealing with Help Desk queries. You can sense, I hope, that it is only with such great volunteers that the real potential of Club Log has flourished.

Taking stock – 2007 to 2015

Club Log is about 8 years old – it really has whizzed by!

Learning new skills, databases and programming languages has been rewarding. As my professional life was turning to commercial and management responsibilities, Club Log gave me back the right to toy with computers and play with ideas that gripped me (but with a real purpose and focus, too). I have found this to be just right, and it is part of my reason for spending so much time on the project.

I must say, too, that cooperating with great people – other programmers, DXers and expedition teams – is a privilege. When a Club Log user provides a scan of a QSL card from the 1950s, it can be the last long-missing piece of a jigsaw puzzle for Alan’s research. When an expedition team logs 200,000 QSOs and Club Log calmly handles 2 million log lookups in a few weeks, that is truly gratifying. When we find a new DXCC in someone’s log, the shared sense of delight is amazing.

However, the truth is that 40,000 users can create a lot of correspondence, too. If you imagine that each user asks just one question every three years, it’s still over thirty emails sent to the Club Log helpdesk each day. We try our best to manage expectations on this front (and largely we are treated with exceptional tolerance and patience if we fall behind!). I think perhaps we are excused by the fact that Club Log is free. Had we charged even $1 for its use, I suppose our helpdesk would be mired in correspondence.

Perhaps the other consequence of Club Log is that I do not get to do quite as much on the air as I would like; instead, a lot of my hobby time is spent at the keyboard either maintaining existing systems that need performance enhancements as Club Log expands, or working towards future developments. One way or the other, to me this is all amateur radio and I am contented!

Come and have a chat

I hope this gives you the flavour of the origins of Club Log. The actual specifics have been glossed over, but I hope that any FOC member who is curious about Club Log will feel free to contact me.

I will also be doing a talk about the Club Log OQRS system at the RSGB Convention in Milton Keynes on Saturday 10 October 2015. This is a great chance to see how to use online QSLing in practice, if you are curious; alternatively you can ask me at the FOC Annual Dinner in Milton Keynes on 31 October.

I would like to thank publicly the many members of FOC who have actively helped and supported Club Log from its earliest days. Without their friendly encouragement and practical support to persevere, Club Log might not have succeeded – and there is still more to do!

161, Michael G7VJR