Whenever a newsdesk is short of a story, one tried and tested method is to use the results of a survey to fill a page or a minute or two of airtime. Surveys come in various forms: some are expected and respected, such as the National Rail Passenger Survey from Transport Focus, while others are compiled by organisations which have their own reasons for promoting a particular point of view. These views may seem more convincing when they are supported by the opinions of a couple of thousand people, but Sim Harris asks whether those opinions are always quite what they seem.

THE present emergency may have temporarily stopped many things, but compiling surveys does not seem to have been one of them.

The ‘science’ of compiling surveys, if that is the word, rests on the principle of samples. The argument is that a sufficiently large number of people – sometimes no more than one or two thousand – can provide a realistic picture of what the rest of the population of the United Kingdom thinks.

However, whether the resulting picture is actually very realistic depends on a number of things, the details of which are often kept under the covers.

People who make their living from running surveys – members of the public relations business – know quite well that surveys are often commissioned by organisations which require a certain result.

One way of easing the ‘respondents’ – the members of the sample – into a chosen direction is to choose and phrase the questions with care.

When you see the published results of the survey, you will not always be shown the questions. Neither will any details always be given of how the sample was chosen.

Let us take a fictitious example. The Daily Klaxon is a mid-market, right-wing newspaper whose readers are mostly over 50 years old. There happens to be a wider debate in progress about whether capital punishment should be restored for some kinds of murder, such as the killing of a police officer on duty. The Klaxon wants to contribute to this debate, and its readers seem likely to believe that it’s a good idea.

How to make the point? A survey! A sample of 2,000 people is chosen, and they are all known to be readers of the Klaxon. The former editor Sir Max Hastings memorably once said that ‘people choose the newspaper which best reflects their existing beliefs and prejudices’.

So we can reasonably assume that the majority of Klaxon readers are conservatively inclined.

Even so, the way that the questions are phrased can help things along.

Leaving the Klaxon on one side for a moment, consider a survey of young single men which wants to know about their social lives and relationships. The designer of the survey wants to ask them if they have a girlfriend. If the designer wants to show (for whatever the reason) that most young men do have a ‘significant other’, then the question could be: ‘Do you have a girlfriend?’

It looks straightforward, and so it is. But what about the young men who do not have a girlfriend? Maybe there really is no one in their lives, or perhaps their ‘significant other’ is another male.

Even in these enlightened times, not every young man wants to tell a stranger that he doesn’t have a girlfriend, or that his choice is not a girl. So this question seems likely to prompt a number of untruthful answers, with some respondents claiming that they have a relationship with someone who doesn’t exist. The survey will ‘demonstrate’ that most young men have a girlfriend.

But what if the designer of the survey is honest (they do exist), and wants as many truthful answers as possible? In this case, asking ‘do you have a girlfriend at the moment?’ makes it easier for the unattached men to be truthful. The question is, if you like, giving the respondent permission not to have a girlfriend. Last week, yes; next week, very probably. But just for a few days, like an unemployed actor, the respondent is ‘resting’.

To take it further, the question could bridge another issue as well, by asking ‘Is there anyone special in your life at the moment?’ This is a little twee, perhaps, but it neatly sidesteps the sexuality of the respondent, which is actually not the questioner’s business anyway unless the survey had openly set out to ask such questions. Presumably, those who felt shy about discussing such a thing would have refused to take part in the first place.

The tendency of respondents to say what they think the questioner wants to hear, or perhaps what they want the questioner to hear, is well known. One classic example, from the USA, concerned a survey of magazine readers. The survey was intended to discover which titles were the most popular.

When the results were collated, ten times as many people claimed to read a particular literary journal, of high academic repute, as those who said they bought ‘romantic’ story magazines.

In fact, the circulation figures had already revealed that the reverse was the case: the romantic stories won every time, and such magazines had much larger print runs than the literary journal.

The respondents, or at least many of them, had said what they wanted to the questioner to hear, rather than the truth. Pride takes many forms.

Back to the Klaxon, whose editor wants to be able to report that most people are in favour of capital punishment. The relevant question in its survey reads: ‘Are you in favour of clamping down on the shocking rise in murders by restoring hanging for the worst offenders – those who kill police officers, for example?’

As it happens, the editor of a left-wing newspaper, The Daily Torch, wants to achieve the opposite verdict, and this newspaper’s parallel survey includes a similarly weighted question: ‘Nearly all civilised countries have abolished capital punishment. Do you agree that such a practice has no place in a modern society?’

Notice, if you will, the trigger phrases in both questions. ‘Clamping down’ and ‘shocking rise’ from the Klaxon, and ‘civilised countries’ and ‘no place in a modern society’ in the case of the Torch.

Both the Klaxon and the Torch may well achieve the results they desire, but the net effect, as with so many real surveys, will be meaningless.

All surveys are certainly not equal, but they fall into one of two main classes. The first type asks people’s opinions about what has already happened, while the other seeks their predictions about what they (or others) are likely to do in the future.

The National Rail Passenger Survey from Transport Focus (which has a comparatively large sample of some 25,000 respondents in each of its two annual surveys) falls into the first class. It also tries to be as specific as possible, because it doesn’t seek general opinions about public transport but asks its respondents to concentrate on the last journey they made. Its questions are not designed to produce any particular answer. As a result, the results of these surveys are rightly reported and their conclusions respected.

Another survey has just appeared, from the SYSTRA transport consultancy.

This concentrates on the thorny issue of how the present coronavirus emergency could change travel patterns – particularly commuting – in the future.

There are many more or less haphazard predictions floating around, most of which agree that more people are likely to work from home. The evidence supporting such predictions is thin, and so perhaps we should welcome any attempt to ask some structured questions.

This particular survey (sample size: 1,500) concludes that 20 per cent of respondents say they will use public transport less, even after travel restrictions are lifted. This figure rises to 27 per cent for those who travel to and from work by rail.

Of these 20 per cent, 49 per cent are reported to be worried about getting ill, 24 per cent said they will work from home more than they have done before and 14 per cent said they have found another way to travel.

In a separate calculation (rather confusingly), 17 per cent of workers believe they will work from home ‘far more’ after travel restrictions have ended. Maybe this figure includes car travellers as well?

Many of these percentages are percentages of a percentage anyway – that original 20 per cent who will use public transport less. So, to put it another way, almost 10 per cent of all commuters are concerned about the health implications of using public transport, around 5 per cent say they will work from home more in future and 2.8 per cent of all commuters by public transport have found ‘another way to travel’.

We are, let us remember, talking about predictions, and predictions made at the height of a national medical emergency as well.

Most commuters travel by car. For the country as a whole, according to the DfT, around 68 per cent of commuters use their own car or van. Rail of all kinds (including light rail and trams) has a much smaller share, of 10 per cent.

This masks tremendous differences between the big cities and elsewhere. More than 80 per cent of people travelling to work in central London use rail (Underground and National Rail).

In 2017, 1.25 million people travelled into the centre of London in the morning peak by all modes of transport (1.02 million by Underground and National Rail and another 98,000 by bus, making a public transport total of 1.12 million).

Assuming they all went home again that night, that’s 2.24 million commuting journeys by public transport each day. If 20 per cent no longer travelled, it would be a reduction of just over 400,000 to  around 1.8 million a day, or 900,000 in each direction – which was the public transport total in 1999.

On the face of it, this survey is fair enough. The questions have been made available to Railnews, and it should be made clear that they are not of the ‘let’s trigger the answer we want’ variety.

What is a little depressing are some of the interpretations placed on it by the media. A report on the BBC website claims that ‘The number of people using public transport in Britain’s cities could be 20% lower than normal after the end of the coronavirus lockdown.’

The survey doesn’t say that. It is plainly concerned with commuting – not other types of travel. (If season tickets indicate a commuter, than at least 45 per cent of all rail passengers are commuters, although they do clock up more passenger kilometres than the leisure or occasional business travel markets. The DfT reckons that 57 per cent of rail journeys are made for commuting or business reasons.)

Even worse, although this is a survey, and the BBC copy says ‘The number of people using public transport in Britain's cities could be 20 per cent lower’ the headline turns that ‘could’ into a certainty: ‘Transport usage will change after lockdown’. Who says?

The BBC also claims: ‘It’s also challenging for public transport operators, which will face a sharp drop in income until public confidence returns.’

Again, they face nothing of the sort in the longer term, so long as services can be trimmed to match the lower demand. Catering for the extreme peaks is very expensive and is far from being a cash cow, although it seems almost impossible to convince commuters (and some journalists) of this fact.

Incidentally, we are told by SYSTRA that 17 per cent of the respondents believe they will work from home ‘far more’ after travel restrictions have ended. Belief is one thing. Has anybody asked their employers yet?

 

‘Tomorrow’s Railway’ will be the subject of the main feature in the May print edition of Railnews, RN279, to be published on 7 May. Copies of current or recent editions can be obtained by calling 01438 281200 from UK numbers or +44 (0)1438 281200 internationally, and selecting Option 2.