The Satisfaction Gap due to buses running on time is the difference in average rating for Overall Transit Service between those people who rated On Time as Excellent or Good and those people who rated it Fair or Poor, or had No Opinion. This is illustrated in Figure 1. This satisfaction gap may also be interpreted as a measure of the Importance of the service attribute On Time. In other words, the importance to riders of the attribute On Time is the degree to which this attribute affects overall satisfaction of the average rider. This is a measure of the correlation between individual attribute satisfaction and overall satisfaction.

Figure 1
Measuring Importance Using the Satisfaction Gap

The second step in the method combines the Satisfaction Gap with the percentage of riders who rated the service attribute less than Good or Excellent. For example, the impact score for On Time is calculated as follows:

Impact Score =

(Satisfaction Gap) x (% rating On-time as Fair/Poor/No Opinion) =

(0.81) x (25%) = 0.20

This formula expresses the intuitive connection that the potential impact of improving on-time performance depends on the importance of this attribute and the percentage of riders who currently find it less than Good or Excellent. For example, if on-time performance is very important (highly correlated to overall satisfaction) but 95 percent of riders currently rate it as Good or Excellent, then further improvements will have little impact. Similarly, suppose some other attribute is very weakly connected to overall satisfaction. Then, even if it were currently rated Good or Excellent by only 50 percent of riders, improvements to this attribute would still have little impact. The biggest impact comes for attributes that are important and currently have only moderate or low levels of satisfaction.

Mathematically, the impact score shows how much the average rating for Overall Transit Service for all senior respondents combined would be improved if they all rated On Time as Excellent or Good. This calculation assumes that, when riders' satisfaction with On Time improves, their overall satisfaction improves to match that of riders who are already satisfied with On Time. An equivalent calculation would be:

Impact score =

(Overall satisfaction of respondents rating On Time as Good or Excellent) - (Overall satisfaction of all respondents)

Note that impact scores for various improvements are not generally additive. For example, improving both On Time and Frequency of Service would not generally cause the average rating for Overall Transit Service to increase by the sum of their impact scores. This is because there are significant correlations among the satisfaction ratings for the various service attributes.

The impact score method can be contrasted with a method sometimes used in market research, in which respondents are asked directly to rate the importance of various attributes. For each attribute, say on-time performance or reliability, respondents are asked:

How satisfied are you now with this aspect of service?

How important is this aspect of service to you?

Those items with low-to-medium satisfaction and high importance are given the highest priority for action. A pitfall of this method is that respondents may have difficulty separating out importance from their current satisfaction level. Further, from a practical point of view, this method doubles the number of questions that must be asked. By comparison, the impact score method does not provide a measure of importance for any one respondent, but it does provide a measure of average importance that does not require a second, fairly abstract, judgment by respondents.

The impact score method requires the researcher to make a somewhat arbitrary decision on how to divide the attribute ratings. For example, in the example just used, respondents were given a four-point scale (Excellent, Good, Fair, Poor) plus No Opinion for each attribute. The impact score was computed by dividing the responses into two groups (Excellent/Good vs. Fair/Poor/No Opinion) and comparing the overall satisfaction ratings of the two groups. It would be possible to divide the responses differently, for example Excellent and Not Excellent. How to make this division is a matter of judgment, depending on the actual distribution of responses and on expectations about what kind of improvements can be targeted reasonably.

Data Sources

The research analyzed data from rider surveys conducted in 1988 by three West Coast transit agencies: the King County Department of Transportation (King County Metro), the Orange County Transportation Authority (OCTA), and the Santa Clara Valley Transportation Authority (VTA). The results include information about current satisfaction rates at all three systems. Because it is not the intention of this research to compare service quality among systems, when results are given in this report these systems are referred to as System A, System B, and System C (not necessarily in the order listed).

The surveys included questions asking riders to rate service attributes such as on-time performance, drivers, cleanliness, customer information, security and safety, as well as "overall transit service." Riders were also asked a question of the type, "What improvements would make you use transit more often?" The survey types and sample sizes, based on the final, cleaned data files, were:

EXPECTATIONS

It would be expected that satisfaction ratings of various service attributes would vary considerably among the transit systems. Further, since seniors and non-seniors use the same services, their relative ratings of various attributes would not necessarily vary. For example, if a transit system has friendly drivers but poor on-time performance, both seniors and non-seniors might be expected to rate drivers more highly than on-time performance. At a second transit system, the pattern might be reversed.

In the case of importance ratings, however, some consistency among transit systems might be expected, but less consistency between seniors and non-seniors. If the analysis method is capable of separating out importance from current satisfaction, then seniors might, for example, consider friendly drivers and on-time performance of equal importance, while non-seniors might see on-time performance as significantly more important. Further, these importance ratings might be expected to show some consistency among transit systems. That is, seniors might be expected to have similar priorities in different regions.

Figure 2. Analysis Results for System A

(See Appendix A)

Figure 3. Analysis Results for System B

(See Appendix A)

Figure 4. Analysis Results for System C

(See Appendix A)

Impact scores should show the combined influence of current satisfaction and importance scores. Since importance may differ between seniors and non-seniors, and relative satisfaction may be different among transit systems, there is no reason to expect any consistency in impact scores between seniors and non-seniors or among transit systems.

CURRENT SATISFACTION

For the most part, seniors are more satisfied with most service attributes than non-seniors at all three transit systems. However, there is broad consistency in relative satisfaction with various attributes between the two groups. For the samples sizes at Systems A and B, differences of 5 percent or more are statistically significant at the 95 percent confidence level. For the sample size at System C, differences of 8% or more are statistically significant at the 95 percent confidence level.

System A. System A had the least consistency between ratings of seniors and non-seniors. Between 62 percent and 69 percent of seniors rated every attribute as Excellent or Good. By comparison, less than 60 percent of non-seniors gave Excellent or Good ratings for three items:

Fares

Vehicle cleanliness

Bus stop information

The last two items were also among the least well rated by seniors. In the case of fares, it is not surprising that seniors are more satisfied than others because they pay the discounted fare, which is about one-third the regular adult fare.

System B. At System B, there is general consistency between seniors and non-seniors in relative satisfaction with the various attributes. The highest-rated items by seniors are also highly rated by non-seniors and the lowest-rated items among seniors are also the poorest-rated items by non-seniors. For both groups, three items stand out as having poorer ratings than other items:

Telephone information

Frequency of service

Information at bus stops

IMPORTANCE

As anticipated, there are some significant differences between seniors and non-seniors in the importance of various service attributes. At all three systems, importance ratings for seniors show much greater variation than they do for non-seniors. For example, at System A, the importance ratings for non-seniors range between 0.53 for Telephone Information and 0.85 for System Easy to Understand, a spread of 0.32. (These figures mean, for example, that improving telephone information so that all riders rate it as Excellent or Good would improve the overall satisfaction score for the transit system by 0.53 points on a scale from 1 to 4.) By comparison, importance ratings for seniors range between 0.36 for Ease of Transfer to 1.04 for System Easy to Understand, a spread of 0.62. Similar differences in variability can be seen for the other two systems.

Since sample sizes for seniors are not large, and importance is a difference of two means, statistical uncertainly is an issue in considering importance differences. These are shown in Table 1.1 Confidence intervals for seniors are much wider than for non-seniors because seniors comprise a small part of the total samples.

Approximate 95% Confidence Intervals for Importance

 

	Seniors	Non-seniors
Systems A and B	+/− 0.20	+/− 0.03
System C	+/− 0.22 to +/− 0.29	+/− 0.09

 

Figure 5 illustrates the ranking of importance by seniors at Systems A and B. Items that had the highest importance ratings are at the top of the list for each transit system. The connecting lines are provided to help judge the extent to which there is consistency in the importance ratings among the three systems. The figure shows that there is general but not complete consistency between Systems A and B. There were too few comparable questions on System C's survey with sufficient sample sizes to make a meaningful comparison. Systems A and B used the same method of survey administration and similar survey instruments.

Figure 5. Comparison of Importance Rankings

 

Both surveys were conducted by the same contractor. System C used a very different instrument and method of administration.

Three items were near the top of importance ratings at Systems A and B:

Drivers

On time

Equipment reliable

 

Similarly, three items were near the bottom of importance ratings at Systems A and B:

Bus stop information

Telephone information

Vehicle cleanliness

 

Two items fell in the middle of the rankings for both Systems A and B:

Schedule information/bus book

Service frequency

 

Three items stand out as having notably different importance at Systems A and B. Security and Safety and Ease of Transfer were low ranked at System A but ranked near the middle at System B. System Easy to Understand was top-ranked at System A but only mid-ranked at System B.

Overall, the importance scores for seniors at Systems A and B show broad consistency. Using a statistical test of rank correlation (Mosteller and Rourke, 1973), the importance ranks at Systems A and B are correlated with better than 95 percent confidence (there is less than a 5% chance the similarity of rankings could have occurred by chance). A similar test was performed for ranking of importance by non-seniors. The similarity in rankings at Systems A and B are different from chance, with better than 97.5 percent confidence.

IMPACT SCORES

The impact scores show the combined influence of importance and current satisfaction. Because current satisfaction varies among systems, impact scores do too and would even if importance were completely consistent among systems. Table 2 shows the service attributes with the top impact scores at each system for seniors and non-seniors.

Service Attributes with Greatest Impact on Overall Satisfaction

At System A, the highest impact items are completely different for seniors and non-seniors. Since there is relatively little spread among attributes in the current satisfaction ratings for seniors, the impact scores mostly reflect the influence of the importance ratings. Four of the top five impact items are also among the items with the five highest importance scores. The remaining item, Telephone Information, had the poorest current satisfaction ranking of all the attributes.

At System B, a different pattern emerges. Four out of five items with the highest impact overlap for seniors and non-seniors. In this case, it appears that the impact scores are dominated by the current satisfaction ratings rather than the importance score. This reflects the greater spread of current satisfaction ratings at System B compared to System A. Recall that impact = (importance) x (percent of respondents in the lower satisfaction group). If 60 percent of respondents rate Attribute X as Excellent or Good, and 80 percent rate Attribute Y as Excellent or Good, then Attribute X has twice the percentage of respondents who gave a rating of less than Excellent or Good.

A mixed pattern emerges at System C. The list of high-impact items contains some quite different items than at System A or B. However, this is at least partly because some attributes that might be important to seniors were excluded from analysis due to small samples sizes. The differences between seniors and non-seniors reflect both importance and current satisfaction. For seniors, there is close correspondence between satisfaction and importance ratings--three safety-related items with low current satisfaction are also those with high importance. As a result, these have the highest impact scores as well. Time Between Buses, which has a low satisfaction rating by seniors, has a lower-than-average importance for seniors, so it is not among the items with the highest impact scores. For non-seniors, there is relatively little spread in the importance ratings, so the impact scores largely reflect current satisfaction. Note that non-seniors are much less satisfied than seniors with Time Between Buses and On Time Performance, which is why these items are among the highest-impact items for non-seniors but not for seniors.

Direct Questioning About Improvements

All three transit systems included questions that asked respondents directly about whether various named improvements to transit service would result in respondents riding more. Systems A and B allowed respondents to check up to three suggested improvements on a list of nine or ten improvements. System C read improvements and asked respondents to judge how likely each would be (on a scale from 1 to 4) to make the respondent ride the bus more. Figures 6, 7, and 8 show the results for the three transit systems.

Only a few service attributes were included in the satisfaction questions and these lists of improvements. At all three systems, the improvements judged most likely to increase ridership were additions to the overall level of service, especially more frequent service and, at System C, direct service to destinations. Frequency of service was among the highest-impact items at System B, but not at System A. At System A, seniors already rated frequency of service highly (76 percent rated it Good or Excellent compared to 61 percent of non-seniors) and assigned it only medium importance. In this case, it appears that ridership is not the same thing as satisfaction. It stands to reason that if total service levels could be significantly increased, ridership of all types would go up. Even in this case, however, riders would not necessarily like the service more.

 

Figure 6. What Improvements Would Make You Ride More Often?

 

Figure 7. What Improvements Would Help You Choose to Ride the Bus More Often?

 

 

Figure 8. Would You Be More Likely to Ride the Bus If....

Conclusions

The impact analysis appears to provide valid indications of the relative importance that riders assign to various aspects of service, enabling calculation of the impact that improvements would have on service satisfaction. Many of the results are specific to individual transit systems; however, several general patterns were observed:

While adding service would probably have the greatest impact on ridership, the impact analysis shows that other improvements can have a major impact on satisfaction.

In general, seniors appear to rate service attributes more highly than do non-seniors.

The analysis showed a wider range of importance scores for seniors than for non-seniors. While importance scores for non-seniors tended to cluster together, the results for seniors appear to indicate that certain service attributes are significantly more important than others.

At the two systems that used a similar method of survey administration and question format, there is broad consistency in importance ratings for seniors. Among the most important attributes at both systems were drivers, reliable equipment, and on-time performance.

The items for which improvement would have the greatest impact on satisfaction are not necessarily the same as those that would result in the greatest increase in ridership.

Note that the satisfaction questions at Systems A and B did not include a number of items that would be interesting from a policy point of view, such as access to destinations, distance from home to a bus stop, and the number of transfers needed to complete a trip. System C did include questions about these issues, but the sample size for seniors was too small for our analysis. The method appears to separate current satisfaction from importance. However, note that importance is not necessarily independent of current satisfaction.

The analysis is limited to results from three West Coast transit systems. Since the surveys were all large-sample, professionally administered, systemwide surveys, the seniors who were surveyed are probably reasonably representative of seniors who ride these transit systems. However, we cannot be certain that seniors who ride other systems would have similar preferences.

Despite this limitation, the analysis indicates that the impact analysis method provides a valid way of assessing the improvements, short of adding service, that will have the greatest impact on seniors or the general public.

Appendix A--Detailed Satisfaction, Importance, and Impact Data

System A
	Percent Good/Excellent
	Seniors	Non-Seniors
Fares System easy to understand Ease of transfer Service frequency Equipment reliable On time Security and safety Drivers Schedule information Vehicle cleanliness Bus-stop information Telephone information	79% 78% 76% 76% 75% 75% 73% 73% 72% 68% 67% 62%	54% 74% 61% 61% 73% 64% 63% 70% 70% 55% 51% 60%
	Importance
	Seniors	Non-Seniors
Ease of transfer Security and safety Vehicle cleanliness Telephone information Bus-stop information Fares Service frequency Schedule information Equipment reliable On time Drivers System easy to understand	0.36 0.46 0.53 0.53 0.57 0.60 0.67 0.74 0.74 0.81 0.82 1.04	0.78 0.71 0.65 0.58 0.66 0.74 0.76 0.71 0.79 0.75 0.72 0.85
	Impact Score
	Seniors	Non-Seniors
Ease of transfer Security and safety Fares Service frequency Vehicle cleanliness Equipment reliable Bus-stop information Schedule information On time Telephone information Drivers System easy to understand	0.08 0.12 0.13 0.16 0.17 0.18 0.19 0.20 0.20 0.21 0.22 0.23	0.30 0.26 0.34 0.30 0.30 0.22 0.32 0.22 0.27 0.23 0.21 0.22

 

System B
	Percent Good/Excellent
	Seniors	Non-Seniors
Bus book Drivers System easy to understand Ease of transfers Reliable equipment On time Security/safety Cleanliness of vehicle Telephone information Frquency of service Information at bus stops	86% 86% 82% 80% 80% 79% 76% 74% 60% 54% 48%	85% 75% 78% 72% 79% 70% 73% 73% 60% 58% 57%
	Importance
	Seniors	Non-Seniors
Cleanliness of vehicles Telephone information Information at bus stops Frequency of service Ease of transfer Bus book System easy to understand Security/safety Drivers On time Reliable equipment	0.77 0.77 0.78 0.88 0.99 1.01 1.06 1.11 1.13 1.28 1.35	0.67 0.53 0.58 0.68 0.77 0.93 0.81 0.71 0.70 0.71 0.75
	Impact Score
	Seniors	Non-Seniors
Bus book Drivers System easy to understand Ease of transfers Cleanliness of vehicles Reliable equipment Security/safety On time Telephone information Frequency of service Information at bus stops	0.15 0.17 0.18 0.21 0.22 0.27 0.28 0.30 0.31 0.40 0.42	0.13 0.17 0.17 0.20 0.18 0.15 0.19 0.22 0.21 0.28 0.25

 

 

System C
	Percent Very Satisfied
	Seniors	Non-Seniors
Safety waiting for bus in daytime On-time performance Safety on bus during daytime Safety of park-ride lots Where bus routes go Safety on bus after dark Time between buses Security of car at park-ride lots Safety waiting for bus after dark	69% 65% 61% 51% 50% 40% 36% 30% 25%	67% 37% 50% 43% 39% 29% 24% 34% 23%
	Importance
	Seniors	Non-Seniors
Safety waiting for bus in daytime On-time performance Safety on bus during daytime Time between buses Where bus routes go Safety on bus after dark Safety waiting for bus after dark Security of car at park-ride lots Safety of park-ride lots	0.45 0.54 0.55 0.60 0.65 0.71 0.73 0.87 0.96	0.62 0.52 0.54 0.59 0.64 0.49 0.52 0.47 0.51
	Impact Score
	Seniors	Non-Seniors
Safety waiting for bus in daytime On-time performance Safety on bus during daytime Time between buses Where bus routes go Safety on bus after dark Safety waiting for bus after dark Security of car at park-ride lots Safety of park-ride lots	0.14 0.19 0.21 0.32 0.39 0.43 0.47 0.55 0.61	0.21 0.33 0.27 0.39 0.45 0.35 0.29 0.40 0.31

Bibliography

Burkhardt, J.E. Transportation Needs and Problems Among the Elderly. Bethesda, MD: Ecosometrics, Inc., 1996.

Ecosometrics, Inc. Mobility and Independence: Changes and Challenges for Older Drivers. Bethesda, MD: Ecosometrics, Inc., 1998.

Morpace International, Inc. A Handbook for Measuring Customer Satisfaction and Service Quality, Transit Cooperative Research Program Report 47. Washington, D.C.: National Research Council, 1999.

Mosteller F., and E.K. Rourke. Sturdy Statistics . Reading, MA: Addison-Wesley, 1973.

Rosenbloom, S. "Travel by the Elderly," in Nationwide Personal Transportation Survey: Demographic Special Reports. Washington, D.C.: U.S. Department of Transportation, 1995.

Rosenbloom, S. "The Mobility Needs of the Elderly," in Special Report 218, Transportation in an Aging Society: Improving Mobility and Safety for Older Persons, Volume II. Washington, D.C.: Transportation Research Board, National Research Council, 1988.

Transportation Research Board, Special Report 218, Transportation in an Aging Society: Improving Mobility and Safety for Older Persons, Vol. I. Washington, D.C.: National Research Council, 1998.

U.S. Bureau of the Census. 65+ in the United States: Current Population Reports. Special Studies, pp. 23-190. Washington, D.C.: U.S. Government Printing Office, 1996.

Yalch, R. A Handbook: Using Market Segmentation to Increase Transit Ridership, Transit Cooperative Research Program Report 36. Washington, D.C.: National Research Council, 1998.

Yalch, R. A Handbook: Integrating Market Research in Transit Management, Transit Cooperative Research Program Report 37. Washington, D.C.: National Research Council, 1998.

About the Authors