Attachment 1- 2008 SPPA Planning Study Final Report 08.31.07.pdf

PDF 181 KB Posted Mar 18, 2010

Attached to: Conduct a Planning Study for the 2012 Survey of Public Participation in the Arts.
Solicitation number: NEARFP10-07
Issued by: National Endowment for the Arts

About this file

From the source

Attachment 1 2008 SPPA Planning Study Final Report 08.31.07 (51 pages)

View the file

Other files

File	Type	Posted
RFP 10-07 Amendment 0003 Pg2 Questions Answers	—	Apr 16, 2010
SF30 Amendment 0003 Q A's RFP 10-07 to Conduct a Planning Study for the 2010 SPPA.pdf	PDF	Apr 16, 2010
RFP 10-07 Amendment 0002 Pg2 Questions Answers	—	Apr 7, 2010
SF30 Amendment 0002 Q A's RFP 10-07 to Conduct a Planning Study for the 2010 SPPA.pdf	PDF	Apr 7, 2010
Attachment 2 - Report on SPPA Planning Meeting 10.21.05.pdf	PDF	Mar 18, 2010
SF30 Amendment 0001 RFP 10-07 to Conduct a Planning Study for the 2010 SPPA.pdf	PDF	Mar 18, 2010
Attachment 3 - Results of the Pilot Survey to Test the SPPA Instrument 06.12.07.pdf	PDF	Mar 18, 2010
RFP 10-07 Conduct a Planning Study for the 2010 Survey of Public Participation in the Arts for ORA.pdf	PDF	Mar 17, 2010

On GovTribe

Work with this file

Download the original file
Contacts named in this file
Similar government files
Ask GovTribe AI about this file

Try for free Login

Text version

The file's text, extracted by GovTribe without its formatting.

Summary Report for the 2008 Survey of Public Participation in the Arts Planning Study

Prepared by Timothy Triplett

Urban Institute

August 31, 2007

Table of Contents

Section 1: Introduction Section 2: Overview of the Planning Study Section 3: Recommended 2008 Data Collection Methodology Section 4: Recommended 2008 Sample Design and Respondent Selection Procedures

Weighting Adjustment:

Section 5: Key Changes made to the 2002 SPPA Questionnaire

Changes to the Core Arts Attendance Questions Module A: Additional Reading Questions and Music Preferences Module B: Participation via Internet & Other Media Module C: Leisure Activities Module D: Arts Learning and Additional Demographic Questions

Section 6: Additional Pre and Post-Data Collection Recommendations Appendix A

Recommended 2008 SPPA questionnaire

CORE QUESTIONS

MODULE A: ADDITIONAL READING QUESTIONS & MUSIC PREFERENCE 25

MODULE B: PARTICIPATION VIA INTERNET & OTHER MEDIA

MODULE C: LEISURE ACTIVITIES

MODULE D: ARTS LEARNING

ADDITIONAL DEMOGRAPHIC QUESTIONS

Section 1: Introduction

In 2008, the National Endowment for the Arts will field the sixth Survey of Public Participation in the Arts (SPPA). The SPPA is the field’s premiere repeated cross sectional survey of individual attendance and involvement in arts and cultural activity.

The data are circulated to interested researchers, and they are the basis for a range of NEA reports and independent research publications. The SPPA provides primary knowledge on the extent and nature of participation in the arts in the United States.

Continuous improvement of the SPPA so that its estimates are as accurate as possible is a worthy goal. The planning study for the 2008 SPPA focused on three major concerns.

The first concern was the mode of data collection. Mode of data collection for the preceding SPPAs has shifted between stand-alone survey and attachment to other surveys. Such shifts result in well-documented differences in estimates, with uncertainty about how much of differences between years is attributable to change in the nation’s habits and how much is attributable to methodological differences in survey mode.

Consequently, as the NEA considered its options for fielding the next SPPA, it had to weigh how potential options both influence the accuracy of the resulting estimates and the ability to compare survey results across years. Section 3 of this report details the recommendation that the SPPA be conducted as a supplement to the Current Population Survey (CPS).

The second concern was the sample design and selection of respondents. Issues of the sample design became a mute point once it was decided that the 2008 SPPA would again1 be a supplement to the CPS with interviews conducted in approximately 1 households that are in the exit round of their CPS sample rotation. The SPPA has generally attempted to interview all persons 18 years or older within a household.

However, the CPS interviews a non-randomly selected respondent who provides information on everyone 16 or older in the household. In 2002, the Census Bureau interviewers attempted to complete the SPPA supplement with all persons 18 or older, but after 4 call attempts they accepted proxy reports. The Census Bureau felt that both the length of the questionnaire combined with these procedures were too burdensome. In addition, it was difficult to measure the impact of proxy reports given that they were collected out of necessity rather than by design. Therefore, to use the CPS again, a different set of rules are needed to decide who gets interviewed and whether or not to accept proxy interviews. Section 4 of this report details the recommendation that for the 2008 SPPA, Census collect proxy responses for spouse or partners and complete a second random interview in households that have additional adults besides the CPS respondent and their spouse or partner.

5,000

The third issue concerns content selection, question wording, order, and survey structure.

Some areas of inquiry benefit from varied replications of survey content, resulting in a

1 The 2002 SPPA was also conducted as a supplement to the Current Population Survey convergence of consistent and effective ways of asking questions to measure social phenomena. However, the SPPA has few other comparative research projects that help to hone its survey content and structure. Consequently, the planning study included a review from a panel of experts and pilot study to test the questions with actual respondents. Section 5 of this report details the changes that were made to the 2008 SPPA questionnaire primarily based on the advisory committee recommendations and the results from the pilot study.

The concluding section of this report contains some recommendations based on the planning effort that should be done both pre and post data collection.

Section 2: Overview of the Planning Study

The planning study began in June 2005 and the first major task was to solicit the participation of well balanced group of technical experts. The mix of experts that we were able to recruit consisted of mostly people who were very familiar with the past Survey for Public Participation in the Arts (SPPA) studies. This group included experts in survey methodology, including the former director of the census bureau, Ken Prewitt.

Furthermore, John Robinson who designed the very first (1982) SPPA was among the participants. All members of the technical advisory group attended an all day advisory meeting in October 2005 and continued to provide feedback at various stages of the planning process.

The next task involved pulling together information to send out to advisors in advance of the October advisory meeting. This included reviewing and summarizing relevant methodological documentation from previous SPPA studies. Searching for books, journals, dissertations, and newspaper articles published since 1992 that have used SPPA results. Reviewing user comments solicited previously by the Endowment and reviewing and summarizing options for fielding the SPPA as a supplement to other surveys versus fielding it as a stand-alone survey.

The advisory meeting was held on October 21, 2005. Prior to the meeting each advisor received a copy of the 2002 SPPA report; a detailed timeline that provided a historical overview; copies of prior SPPA questionnaires; a write-up describing data collection alternatives for the upcoming SPPA; and a copy of the bibliography of recent research that used the SPPA data. The meeting was divided into two major parts, starting with a discussion about the options for collecting the SPPA data, followed by a section by section review of the SPPA questionnaire. A full discussion summary of the meeting can be obtained from the National Endowment of the Arts upon request. The participants at the advisory meeting are listed below;

o Kelly J. Barsdate, Director of Research, Policy and Evaluation, NASAA o Tom Bradshaw, Senior Research Officer, NEA o Paul DiMaggio, Professor of Sociology, Princeton University o Mary McIntosh, Principal and President, Princeton Survey Research Associates o Bonnie Nichols, Research Analyst, NEA o Francie Ostrower, Senior Research Associate, Urban Institute o Ken Prewitt, Professor of Public Affairs, Columbia University o John Robinson, Professor of Sociology, University of Maryland o Marc Scorca, President and CEO, OPERA America o Peter Hildick-Smith, President, Codex-Group LLC o Daniel Swenson, Research Associate, Urban Institute o Steven Tepper, Associate Director, Curb Center at Vanderbilt University o Timothy Triplett, Survey Associate, Urban Institute

In addition to the technical experts who provided advice throughout the planning study there were additional conversations held with SPPA data users on the general topic of survey content, including queries regarding desirable content not covered in previous SPPAs. Based on feedback from the advisory committee meeting and these additional conversations with users a first draft of the 2008 SPPA questionnaire was written. This first draft was sent to all members of the advisory committee for feedback and based on these comments and suggestions the questionnaire was further revised

Besides the questionnaire our SPPA advisors were asked to think about what questions could be accurately reported by a spouse or partner? Based on the feedback we recommended that only the core questions and the leisure activities module accept proxy reports. The advisors were also asked about the proposed sampling of adult respondents within the CPS household. In addition, sampling procedures were discussed with several prominent survey methodologists who had a background in survey sampling. After carefully reviewing of all recommend changes, an updated questionnaire was developed.

While we received many suggested additions, we only incorporated those suggestions that improved the existing questionnaire and could be accommodated without exceeding the 10 minute average questionnaire length permitted for a supplement to the CPS.

The next step was submitting the project to the Office of Management and Budget for approval. We received OMB approval on the first pass through which meant that they we were not required to make any changes to the survey instrument or study design.

Once OMB approved the project the next step was to do a pilot study. The Urban Institute (UI) and Princeton Survey Research Associates International (PSRAI) conducted a pilot study of 302 adults to evaluate question wording and survey design for the 2008 Survey of Public Participation in the Arts (SPPA). The 302 interviews were conducted in English by Princeton Data Source, LLC from April 19 to May 2, 2007.

These were all telephone interviews with one randomly selected adult (18+) interviewed in each sampled household. The telephone households were selected using a random digit dial sample design that gave most continental United States telephone households a chance of selection. The Urban Institute worked with Princeton Survey Research Associates International to conduct a pilot test using a nationwide, random digit dial telephone survey. The “live test” survey included proxy reporting by respondents who had a spouse or partner. In addition, all interviews were recorded, by subsequently listening to the recorded interviews, we were able to identify a small number of issues that respondents had when answering the questions. A full report on the result of the pilot study can be obtained from the National Endowment of the Arts upon request.

Based on the results of the pilot study the final questionnaire and survey design were sent to the Census Bureau for review. After several discussions with the Census Bureau the final survey design and questionnaire was agreed upon ending the planning stage of the 2008 SPPA project.

Section 3: Recommended 2008 Data Collection Methodology

Currently, telephone surveys are suffering from decreasing frame integrity, falling response rates, and increasing costs, so the telephone random digit dial (RDD) approach used to field the 1997 SPPA is no longer a reasonable alternative. Some surveys, such as the National Election Survey, have gone away from telephone interviewing in favor of personal interviewing. The SPPA, however, requires a large sample to ensure a sufficient number of responses to examine the characteristics of participants in activities such as opera and ballet, which have relatively small participation rates. The large sample, unfortunately, makes personal interviewing too expensive. So, in effect, the possibility of conducting the 2007 SPPA as a stand-alone telephone or in-person interview was ruled out.

An additional problem in choosing a data collection methodology is that the Census Bureau would be busy in 2007 implementing a major software redesign of both the Current Population Survey (CPS) and the National Crime Victimization Survey (NCVS).

Hence, the earliest that either of these two surveys could host the SPPA as a supplement would be the later part of 2008.

So after eliminating most data collection methods due to either costs or compatibility issues we carefully considered the following three potential alternatives for fielding the sixth SPPA:

1) Wait until 2008 and conduct the SPPA as a supplement to the CPS. Given the sample size needs of the SPPA, using the NCVS would likely not be an option until late 2008 or early 2009.

2) Conduct the SPPA as a supplement to the American Time Use Survey (ATUS). The ATUS study subject matter fits well with the SPPA. The response rate for the ATUS, however, is substantially lower than the CPS response rate and the sample consists of households who recently completed their CPS panel rotation commitment.

3) Conduct the 2007 SPPA using an On-line Internet panel that has been recruited through acceptable probability sampling procedures2. This alternative would certainly be a dramatic change for the SPPA, and this mode of data collection does have both response rate and coverage concerns. On the other hand, it is an evolving mode of data collection and substantial resources have been spent in identifying ways of improving the panel recruitment and producing estimates that benchmark well against the CPS estimates. The biggest advantage of the On-line panel is the flexibility this mode would offer in changing or lengthening the SPPA questionnaire.

At the advisory meeting it seemed clear that most of our technical advisors would prefer that the SPPA remain a supplement to the CPS even if that means a delay in the data collection. However, given the current Census bureau climate and their schedule, an alternative or some contingency planning should be further explored. If there were to be a major design change toward Internet data collection, some additional pilot work and benchmark testing would need to be done. Since a SPPA supplement to the American Time use Survey would likely take a full year of data collection in order to complete enough interviews, there doesn’t seem to be any major advantage of switching from CPS to the ATUS. In addition, conversations held with the people in charge of the ATUS suggest that ATUS supplements are not allowed to exceed 5 minutes in length (SPPA averages close to 10 minutes) and should be an extension of the time diary study.

So given the major comparability concerns of using an internet panel and the restrictions on being a supplement to the ATUS, the strong recommendation from the planning study is to wait and take the earliest available time in which the Census Bureau could conduct the SPPA as a supplement to the CPS. This turns out to be May 2008.

Section 4: Recommended 2008 Sample Design and Respondent Selection Procedures

The Census Bureau will conduct the May supplement in conjunction with the CPS for which the universe is the more 100 million housing units in the 3,141 counties and independent cities the make up the United States. From this universe, they will select a sample of approximately 72,000 households each month, which approximately 60,000 households are eligible for interview (most of ineligible housing units are non-residential or unoccupied). They actually interview about 55,500 households each month. They will administer the SPPA supplement to a random ¼ of the 60,000 eligible households (which consists of the approximately 15,000 households who are in the exit round of their CPS sample rotation).

The big challenge is that the respondent selection procedures for SPPA supplement differ from the CPS procedures. CPS is administered to any person 16 or older who is able to report employment information on all persons 16 years or older in the household. The

2 Prior to the October advisory meeting representatives of Knowledge Networks gave an over view at the National Endowment for Arts on how a internet panel study could be used to collect the SPPA data.

SPPA needs to collect information on all adults 18 or older. In addition, the nature of many SPPA questions require that the survey be based on self reports rather than proxy reports. To make the SPPA a better fit for the CPS protocol we tested the possibility of allowing proxy responses from spouse or partners. Results from the SPAA pilot test of 300 respondents found that respondents in the pilot study seem to easily recall the activities of their partner or spouse. So we are recommending that respondents who have a spouse or partner also report on their activities. In addition, we recommend conducting a second interview in households that have adults in addition to the CPS respondent and their spouse or partner. The second interview would be done with a randomly selected adult from among the remaining non-interviewed adults. The following is a outline the recommended within household respondent selection strategy

1) If the May CPS respondent is 18 or older then

a. Interview the CPS respondent.

i. If the CPS respondent has a spouse/partner living in the household who is 18 or older then

1. Ask the CPS respondent to also answer the proxy questions for their spouse partner.

b. If there are one or more other adults (18+) besides the CPS respondent and their spouse/partner living in the household– randomly choose one of these other adults to be interviewed. (Hence, if there is only one other adult other than the CPS respondent and their spouse/partner that person should be interviewed.)

i. If the randomly selected adult has a spouse/partner living in the household who is 18 or older then

1. Ask the random adult to also answer the proxy questions for their spouse/partner.

2) If the May CPS respondent is not 18 or older then

a. Do not interview the CPS respondent.

b. Randomly choose one adult from amongst all adults (18+) in the household to be interviewed.

i. If the random adult has a spouse/partner living in the household who is 18 or older then

1. Ask the random adult to answer the proxy questions for their spouse/partner.

NOTE: No more than two interviews will be conducted per household (not including proxy interviews). In households where the May CPS respondent is not 18 or older, only one interview will be conducted (not including proxy interviews).

Weighting Adjustment:

For this sampling plan the supplement weight must be adjusted (prior to any post-stratification to census population estimates) for the different probability each adult has of being interviewed or interviewed via proxy. This weighting adjustment would simply be the inverse of an individual’s probability of selection.

1) If the May CPS respondent is 18 or older then

a. CPS respondent’s probability of selection is 1/1 or 1

b. CPS respondent’s age eligible spouse/partner would also have a probability selection of 1/1 or 1

c. Other adults (18+) who do not have an age eligible spouse/partner would have a probability of selection of 1/(number of other adults 18+)

d. Other adults (18+) who have an age eligible spouse/partner would have a probability of selection of 2/(number of other adults 18+)

2) If the May CPS respondent is not 18 or older then

a. Adults (18+) in the household who do not have an age eligible spouse/partner would have a probability of selection of 1/(total adults 18+)

b. Adults (18+) who have an age eligible spouse/partner would have a probability of selection of 2/(total adults 18+)

NOTE: It will be necessary to track which respondents were proxy respondents and whether their data came from a CPS respondent or a random adult interview. Also the Census bureau will need to keep track of whether or not the CPS respondent was eligible (i.e. 18+).

Section 5: Key Changes made to the 2002 SPPA Questionnaire

While the Census Bureau has agreed again to administer the SPPA as a supplement to the CPS, they expressed strong concerns about the length of time it took to conduct the 2002 survey. Hence, the biggest challenge in revising the 2002 SPPA was to ensure that it took less time to administer. This seemed to be a worthy task given how strongly everyone felt about administering the SPPA as an attachment to the CPS.

There were two major changes that reduced the average burden were:

1. Rather than attempt to interview all adults in the household (as in 2002), the 2008 SPPA would accept proxy responses for spouses or partners (for many of the questions).

While proxy responses were accepted in 2002, this time the questions will be properly worded so that it is clear who the question is referring to and proxy response will be clearly identified in the data file prepared by the Census Bureau.

2. Rather than administer the entire SPPA survey to all respondents, the questionnaire was separated into modules, so that any one respondent would only answer the core arts attendance questions and 2 of the 4 other modules. (Modules will be rotated and overlapped such that all variables can be linked and sample sizes will be sufficient for analysis.)

The following is a list of the key changes made to the 2002 SPPA questionnaire, the proxy versions of questions are not included in the changes, but the full questionnaire is included as Appendix A.

Changes to the Core Arts Attendance Questions

1. Added a new core arts participation question – [With the exception of elementary or high school performances] Did you go to a live Latin, Spanish, or Salsa music performance [during the last 12 months?] - If yes, how many times did you do this during the last 12 months?

a. This questions and the follow-up with number of times were added to reflect the growing size of the Hispanic population and the level of participation in this type of arts activity.

2. The phrase “or an Operetta” was dropped from the live musical stage play question

3. The dance question was modified to include “Broadway style” - [With the exception of elementary or high school performances] Did you go to a live dance performance other than ballet, such as modern, folk, tap, or Broadway style [during the last 12 months?]

4. The visit an art fair or festival, or craft fair or festival question was replaced with the following two questions and there is no longer a how many times follow-up question asked.

a. [During the last 12 months] Did you visit a crafts fair or a visual arts festival?

b. [During the last 12 months] Did you visit an outdoor festival that featured performing artists?

5. The follow-up question asking the number of times visiting a Historic Park … was dropped

6. The category order for the “Did you read any” question was changed so that “plays” is read first and “novels or short stories” is read last.

7. All core questions now have spouse or partner proxy worded versions.

Module A: Additional Reading Questions and Music Preferences

This section of the questionnaire drops a previous question about interest in attending more arts events, revises and shortens the number of response categories in the music preference question, and introduces new reading questions including the following new question about reading preferences:

Now I’m going to ask about your reading preference. Please tell me if you like to read each of the following. Do you like to read (INSERT)?

a. Mysteries

b. Thrillers

c. Romance

d. Science Fiction or Fantasy

e. Other Fiction

f. Health, Fitness, or Self-Improvement books

g. Religious texts or books about Religion or Spirituality

h. History or Political books

i. Biographies or Memoirs

j. Other Non-Fiction

k. Anything Else I have not mentioned – Specify_________________________

Module B: Participation via Internet & Other Media

This section has been revised to reflect the growing use of the internet and a variety of electronic devices to watch or listen to arts performances. The following new questions are included to capture information on Internet use; each of these questions includes a follow-up question about the frequency of use.

• [During the last 12 months] did you use the Internet to watch, listen to, or download live or recorded music, theater or dance performances?

• [During the last 12 months] did you use the Internet to view visual art online, such as paintings, sculpture, or photography?

• [During the last 12 months] did you use the Internet to obtain information about music, theater or dance performances or art exhibits including purchasing tickets online?

• [During the last 12 months] did you use the Internet to create or post your own art online including design, music, photography, films, video, or creative writing?

The following question was added to capture additional arts related activity enjoyed through electronic media:

• [During the last 12 months], did you watch or listen to a program about artists, art works, or art museums?

• [During the last 12 months], did you watch or listen to a program about books or writers?

In addition, the following introductory language to this section was updated to cover the broader range of electronic devices now being used:

These next questions are about watching or listening to a recorded or live broadcasted arts performances on Television, Radio, or on your Computer, including watching or listening on a portable media devices such as a IPOD, Cell phone, or portable DVD player.

And finally, several questions that asked about the number times each type of arts activity was viewed and on which specific devices were dropped.

Module C: Leisure Activities

This section has been revised to not only capture information on participation in an array of leisure activities, but also the extent to which adults attend arts activities in schools and religious institutions and additional information on their participation in civic activities.

This module is also the only module that collects spouse or partner proxy reports. The new questions added are:

• [During the last 12 months], did you attend an elementary, middle, or high school music, theater, or dance performance?

• [During the last 12 months], did you attend a music, theater or dance performance at a church, synagogue or other religious institution?

• [During the last 12 months], did you play a musical instrument?

• [During the last 12 months], did you participate in any community activities, meetings, or events?

• Did you vote in the last [2004] presidential election?

The new question about playing a musical instrument was also used as a filter question, so, if you answer no, you will not be asked about performing or rehearsing at a jazz or classical music performance.

Several questions that proved to have limited analytic utility were dropped. The dropped questions are:

• During the LAST 12 MONTHS, did ... go to an amusement or theme park, a carnival, or a similar place of entertainment?

• Did ... make repairs or improvements on your own home during the LAST 12

MONTHS?

• Also dropped was a follow-up question for respondents who had participated in a specific performing arts activity that asked whether it was done in a public performance or as a rehearsal for a public performance.

Module D: Arts Learning and Additional Demographic Questions

Most of the arts learning questions have been revised in order to make them less burdensome for respondents and interviewers, yet still provide the information needed for analysis. The following is an Example of the question format used in 2008 versus 2002:

2008 SPPA:

Have you ever taken lessons or classes in music — either voice training or playing an instrument?

If yes, did you take these lessons or classes when your were

a) A child under 18

b) an adult 18 or older

c) or both as an adult and child?

[if person chooses a or c] Excluding lessons or classes offered in elementary or high school, were any of the music lessons or classes you took as a child private lessons?

2002 SPPA:

Have/Has ... EVER taken lessons or classes in music - either voice training or playing an instrument?

Did ... take these lessons when ... was - Less than 12 years old?

Did ... take these lessons when ... was - 12-17 years old?

Did ... take these lessons when ... was - 18-24 years old?

Did ... take these lessons when ... was - 25 or older?

Were these lessons or classes offered by the elementary or high school ... were attending or did ... take these lessons elsewhere?

Besides the core question and four modules that 2008 SPPA includes a few additional demographic items that are asked of everyone but are not proxy reported. This section includes the revised parent’s education question that is comparable to the prior versions of the question but has fewer answer categories in 2008. Also the following three new questions have been added to the 2008 SPPA and are asked only for respondents who have school aged children (5 to 17):

• With the exception of lessons or classes offered in their school, have any of your school aged children ever taken any private art lessons or arts classes?

• If yes to having a child who took private art lessons/classes - Were any of these private lessons or classes taken in the past year?

• With the exception of elementary, middle, or high school performances, did any of your school aged children attend a live music, theater or dance performance during the last 12 months?

Section 6: Additional Pre and Post-Data Collection Recommendations

Prior to the data collection there are a few things that should be carefully checked to ensure things go smoothly. First, given the combination of the complex respondent selection procedures, proxy reporting, and the module design of the 2008 SPPA a careful checking of the Census cati (computer assisted programming instrument) by someone familiar with the project is critical. Secondly, it would also be important to check over the interviewer instructions or training manual to ensure that interviewers are being properly instructed on how to administer the supplement. Finally, checking to make sure that there is a variable in place that flags proxy interviews and distinguishes between spouse or partner proxy interviews and interviews that are not part of the design.

After the SPPA data has been collected, the biggest issue is making sure that the Census Bureau correctly produces population weights. There should be two population weights, one for analyzing the questions that include spouse or partner proxy reports and a second population weight that is used when analyzing questions that are only based on self reports. Finally, many SPPA data users and members of the advisory groups requested more geographic variables and variables from the other CPS supplements that are in the same exit rotation. While the Census Bureau may or may not be willing to do this, it would useful to include on the SPPA data file the appropriate linkage variables and documentation that would give SPPA data users the ability to add additional variables themselves.

Appendix A