Expectation and experience of ‘nonspecific’ (whole person) feelings elicited by acupuncture: Content validity of a set of questionnaires – Web version1

Erwartungshaltung und Wahrnehmung „unspezifischer’ (die ganze Person betreffender) Empfindungen – ausgelöst durch Akupunktur: Inhaltliche Validität einer Reihe von Fragebögen – Web-Version1

D. F. Mayor Footnote for footer of p 1: Mr David F Mayor 86 Handside Lane Welwyn Garden City Herts AL8 6SJ UK Tel.: +44 1707 320782 davidmayor@welwynacupuncture.co.uk Honorary research fellow, Department of Physiotherapy, University of Hertfordshire 1. This Web version includes tables and text additional to those in the printed version (Deutsche Zeitschrift für Akupunktur 2014; xxxxxx). ABSTRACT Background. Two 32-item questionnaires designed to assess expectation and experience of the relatively nonspecific feelings (whether bodily, emotional or mental) that may arise in response to acupuncture-type interventions have been developed and tested on acupuncture and other complementary health practitioners and students (N=204). Objectives. To conduct a survey to assess content validity (CV) for the questionnaires, following two methods popularised by Lawshe and Lynn, and to compare the results. A secondary objective was to reduce questionnaire length from 32 to 20 items. Methods. Two surveys (Phase 1, 48 items; Phase 2, 18 items) were conducted among 20 experienced acupuncture practitioners and researchers, who scored items as either ‘essential’, ‘useful but not essential’, or ‘not necessary’. Lawshe’s and Lynn’s methods were applied to the returned data, the former using Wilson’s recent modification, the latter with ‘useful but not essential’ rescored as either ‘essential’ or ‘not necessary’. Some subgroup analysis was carried out. In addition, Randolph’s multi-rater kappa was used to assess inter-rater reliability as a further measure of content validity. Results. There were 17 usable responses in Phase 1, 13 in Phase 2. In both surveys, women considered more items as ‘essential’ than men, but fewer as ‘not necessary’, and vice versa (p<0.001). Lawshe’s content validity ratio (CVR) was significant for only 3 items, and the resulting content validity index (CVI) therefore meaningless. When ‘useful’ was rescored as ‘essential’ and Lynn’s Scale CVI (S-CVI) calculated as the average of the CVI only for those items considered appropriate for retention (S-CVIAV-UA), acceptable values were found for subsets of the full list of items, although only for the women respondents. Inter-rater reliability was low for most items, but on the basis of the content validity analysis, together with respondent reservations about particular items and analysis of actual questionnaire usage, a 20-item list was created and circulated

to the survey respondents for further feedback. Discussion. This shorter scale is now being tested on electroacupuncture training courses, and may be useful in determining whether different feelings are elicited by different types of acupuncture, other CAM modalities, or even other, unrelated activities. Conclusions. Using a modified content validity assessment method, 32-item questionnaires on expectation and experience of nonspecific feelings were shortened to more manageable 20-item instruments. In general, CV was greater among women than among men. KEYWORDS Expectation, nonspecific feeling response, content validity, questionnaire reduction, electroacupuncture ZUSAMMENFASSUNG Hintergrund. Es wurden zwei 32 Punkte umfassende Fragebögen entwickelt und getestet, um Erwartungshaltung und Erleben relativ unspezifischer Empfindungen (sei es körperlich, emotional oder mental) zu erheben, die in Reaktion auf akupunkturartige Interventionen auftreten können. Als Testpersonen hatten sich Praktizierende und Studenten der Akupunktur und anderer komplementärer Verfahren bereit gefunden (N= 204). Ziele. Das erste Ziel war, mittels einer Erhebung die inhaltliche Validität der Fragebögen zu überprüfen, wobei die von Lawshe und Lynn eingeführten Methoden angewandt wurden, und die Ergebnisse zu vergleichen. Zweites Ziel war die Verkürzung der Fragebögen von 32 auf 20 Punkte. Methoden. Es wurden zwei Untersuchungen (Phase 1: 48 Punkte, Phase 2: 18 Punkte) unter Mitwirkung von 20 erfahrenen Praktizierenden und Forschern im Bereich Akupunktur durchgeführt. Sie bewerteten die einzelnen Punkte als „wesentlich“, „nützlich, aber nicht wesentlich“ oder „unnötig“. Die erzielten Daten wurden mit den Methoden von Lawshe und Lynn analysiert. Ersterem folgend wurde die jüngste Modifikation von Wilson benutzt, nach Letzterem wurde „nützlich, aber nicht wesentlich“ entweder als „wesentlich“ oder als „unnötig“ gewertet. Es wurde eine Analyse einiger Untergruppen vorgenommen. Zusätzlich wurde Randolphs Multi-Rater kappa eingesetzt, um die Interraterreliabilität als ein weiteres Maß für inhaltliche Validität zu bestimmen. Ergebnisse. Es gab 17 verwertbare Antworten in Phase 1 und 13 in Phase 2. In beiden Untersuchungen stuften Frauen mehr Punkte als „wesentlich“ ein als Männer, jedoch weniger als „unnötig“ – und umgekehrt (p<0.001). Der Lawshe Quotient für inhaltliche Validität (CVR – content validity ratio) war nur für 3 Punkte signifikant und der resultierende Index für inhaltliche Validität (content validity index – CVI) folglich bedeutungslos. Wenn „nützlich“ als „wesentlich“ neu eingestuft und Lynns Skalen-CVI (S-CVI) als Durchschnittswert des CVI nur für solche Punkte berechnet wurde, die als geeignet für die Beibehaltung betrachtet wurden (S-CVIAV-

UA), ergaben sich akzeptable Werte für Teilmengen der Gesamtliste, jedoch nur für die weiblichen Befragten. Die Interraterreliabilität war für die meisten Punkte gering. Basierend auf der Analyse der inhaltlichen Validität und unter Berücksichtigung der Vorbehalte der Befragten gegenüber einigen Punkten sowie der Analyse der derzeitigen Anwendung von Fragebögen wurde eine 20 Punkte umfassende Liste aufgestellt, an die Befragten verteilt und diese um Feedback gebeten. Diskussion. Die kürzere Liste wird jetzt in Ausbildungskursen für Elektroakupunktur getestet. Sie könnte helfen zu bestimmen, ob unterschiedliche Akupunkturverfahren, andere CAM-Verfahren oder sogar völlig andere Aktivitäten verschiedene Empfindungen auslösen. Schlussfolgerung. Unter Zuhilfenahme einer modifizierten Bewertungsmethode für inhaltliche Validität wurden 32 Punkte umfassende Fragebögen zu besser handhabbaren Instrumenten mit 20 Punkten umgearbeitet. Insgesamt war die CV bei Frauen höher als bei Männern

SCHLÜSSELWÖRTER Erwartungshaltung, nicht- spezifische Gefühlsreaktion, inhaltliche Validität, Fragebogenkürzung, Elektroakupunktur “If you expect to find a Higgs boson, you’ll find a Higgs boson” (anon) “Happiness equals Reality minus Expectations” (attributed to Tom Magliozzi) BACKGROUND The placebo effect has been described as evidence for or an activation of our self-healing capacities [1-3]. Expectation of positive outcome (‘response expectancy’) is often thought to be a major contributor to this nonspecific effect [2, 4], in part because it alters how bodily sensations are identified [5]. In response to placebo acupuncture, bodily sensations of warmth, tingling, pulsing, flow (spreading, radiating) and electricity have been elicited [6, 7]; warmth and tingling may be particularly associated with treatment efficacy [7]. Such sensations are also reported in other CAM modalities such as ‘biofield energy healing’ [8], and have been interpreted as resulting from the flow of Qi [9]. There do not appear to be any existing questionnaires that can be used to assess both expectation and experience of these relatively nonspecific feelings (whether bodily, emotional or mental) that may arise in response to acupuncture-type interventions. Therefore two ‘Expectation of feelings’ questionnaires (EXPre and EXPost) were developed for this purpose [10] (table 1). Table I. Items used in EXPre and EXPost questionnaires Aliveness Being at ease Being blue Being in control Being spaced out Calmness Cheerfulness Clarity

Connectedness with others Contentment Excitement Heaviness Hunger Inner awareness Inner flow Intestinal rumblings

Mental energy Mental focus Nervousness Pain Peacefulness Physical vitality Receptivity Relaxation

Restlessness Sensory acuteness Sleepiness Suppleness Tension Tingling Warmth or coolness Worry

In EXPre, respondents were asked whether they ‘expect to experience a change in the feeling of …’, and in EXPost, whether they ‘experienced a change in the feeling of …’. Responses were limited to ‘Yes’ (Y), ‘No’ (N) or ‘Don’t know’ (DK), but in EXPost they were also asked to asterisk ‘those changes ... noticed most’ (*). There is some overlap between EXPre (expectation) and the Acupuncture Beliefs Scale [11, 12] (four of the 36 items), but only one item overlaps with the 7-item or 4-item Acupuncture Expectancy Scale [13-15]. General measures such as the credibility/expectancy questionnaire [16] were developed for quite different purposes, and do not correspond at all.

There is minimal overlap between EXPost (experience) and scales designed to assess the specific sensations of acupuncture needling such as the Acupuncture Sensations Scale [17-19] and the related Acupuncture Sensation Questionnaire [20], Southampton Needle Sensation Questionnaire [21, 22], Subjective Acupuncture Sensation Scale [23] and Massachusetts General Hospital Acupuncture Sensation Scale [24, 25]. Interestingly, there is more overlap with the recently published Austrian Treatment Experience Questionnaire (8 Affective state items, 3 positive Bodily sensation items) than with any of the other acupuncture-specific instruments mentioned here [26]. The EXP questionnaires have been tested on 204 participants, familiar with acupuncture, in three separate cohorts. Their validity and reliability have been assessed, a cluster analysis conducted, and the association between expectation and experience analysed, along with those items most frequently found, association with other trait and state measures, and the influence of various aspects of treatment on experienced changes in feelings. Salient results are reported and discussed elsewhere [10]. OBJECTIVES The objectives of this study were to conduct a survey to assess content validity for the questionnaires, following two methods, that first described by Lawshe [27] and a different method popularised by Lynn [28], and to compare the results. A secondary objective was to examine patterns in the original questionnaire responses and survey respondents’ additional comments for indications on which items could comfortably be removed to reduce questionnaire length from 32 to 20 items. METHOD Given the evanescence and subjectivity of ‘feelings’, it is difficult if not impossible to establish the validity of a scale to assess their expectation or experience. As no existing measure of ‘expectation of feelings’ could be found, criterion-related validity could not be established. In addition, because these questionnaires were not designed to capture a specific construct (other than ‘expectation/experience of change’), their construct validity could not be assessed either. However, with the confidence of two years’ experience with the questionnaires, in Phase 1 of the survey 20 experienced acupuncture practitioners and researchers (18 known to the author) were invited to take part in a survey to rate 48 items for inclusion in a list of the ‘nonspecific effects’ of acupuncture (tables 1, 2). They were asked to indicate each item as ‘essential’, ‘useful but not essential’, or ‘not necessary’ [27]. They were also given the opportunity to comment on each item scored, add further items if desired, and were asked nine other brief questions about themselves and potential applications for the questionnaires. Table 2. Additional items used in Content validity survey Comfort Empowerment Expansion Floating

Harmony Looseness Melting Numbness

Optimism Positivity Pulsation Relief

Satisfaction Vibration [& Warmth or coolness split into 2 items]

In Phase 2 of the survey, 18 additional items suggested by respondents in Phase 1 were rated in the same way (table 3*).

Table 3. Additional items used in Phase 2 Content validity survey Appetite Being able to cope Being cleared out Being grounded Being stressed Bothersome heat

Happiness Heat Heightened acuity Mobility Needing medication Self-awareness

Self-confidence Self-esteem Sleep quality Something being lifted Tiredness Wellbeing

A content validity ratio (CVR) and content validity index (CVIL) were calculated using Lawshe’s method [27], with the correction by Wilson et al. [29], and also item CVI (I-CVI) and scale CVI (S-CVI) using Lynn’s method [28, 30]. In Lawshe’s method, items are scored as either ‘essential’, ‘useful but not essential’, or ‘not necessary’. Lynn’s I-CVI is usually scored on a four-point scale (1 = Irrelevant, 2, 3, 4 = Extremely relevant), which is then collapsed into a binary score (0 or 1, Inessential or Essential). Here Lawshe’s three scores were collapsed into a dichotomy, with ‘useful but not essential’ considered alternatively as either Lynn’s Inessential or Essential. Lawshe’s and Lynn’s CVIs (each of which has been used in prior acupuncture-related studies [20, 25]) were then compared. Lawshe’s CVR for each item is calculated according to the formula: CVRitem = (ne – N/2)

N/2 where ne is the number of respondents scoring the item as ‘essential’, and N is the total number of respondents. CVR may thus be negative , indicating < 50% agreement, zero (50% agreement) or positive, up to a maximum of 1 (complete agreement). Items are retained according to their significance in Schipper’s table of critical values as used by Lawshe [27] and later modified by Wilson et al. [29]. CVIL for the whole scale is then calculated as the average of the CVR for retained items. Lynn’s I-CVI is defined simply as the ratio of ne to N for that item, where ne is again the number of respondents scoring the item as Essential, but this time in Lynn’s dichotomy. S-CVI can then be calculated in two ways [30], either: (1) as the proportion of the total number of items which are deemed content valid (the ‘universal agreement’ method, S-CVIUA), or: (2) by calculating the average I-CVI across all items (the averaging method, S-CVIAV). A further possibility is to adopt the method used to calculate CVIL and take the average of I-CVI only for those items considered appropriate for retention (S-CVIAV-UA). This novel approach allows a comparison of Lawshe’s CVIL and Lynn’s S-CVI. In addition, Randolph’s multi-rater kappa [31] was used to assess inter-rater reliability as a further measure of content validity [30]. RESULTS Participants In Phase 1, of the 20 invitees from eight different countries, nine were women and eleven men. However, one of the former passed her invitation on to a male colleague. Of the women invited, one did not reply, and of the men, one was on vacation over the survey period, and one declined to

take part. Of the remaining 17 invitees, 14 were acupuncture practitioners (11 of whom were, or had been, also actively involved in acupuncture research) and three experienced acupuncture researchers but not practitioners (although one of these was, at the time, also studying acupuncture according to an ‘East Asian medical’ model). One of the non-practitioner researchers (male) did not feel competent to rate the 48 items, so only answered the more general questions. Of the 14 practitioners, 12 used predominantly an ‘East Asian medical’ model, two both ‘Western medical’ and ‘East Asian medical’ models, and one ‘wouldn’t agree with the use of such binary divisions’. Original acupuncture training was for 1-6 years (mean 3.1, SD 1,1); two respondents specified that their training has been full time. Participants had been in practice between 5 and 40 years (mean 20.9, SD 10.9). All except one participant (coincidentally, the one in practice for only 5 years) had read some of the research literature on nonspecific effects of acupuncture (and one added ‘and written a lot on the topic’). The 17 Phase 1 respondents were invited to participate in the Phase 2 survey, as were two from the original cohort who had not responded. Of these, one who had not replied to the first survey did not respond again, two were unable to participate because of time constraints, two stated they would try to complete the survey form but did not do so, and one who had not fully completed the first survey was only sent one general question but not the list of additional items. Thus 13 returned usable data for the purpose of content validity assessment. Of these, 11 (6 men and 5 women) were acupuncture practitioners. Two other women were experienced researchers (one a non-practitioner and the other currently training as an acupuncturist). Two men and one woman had not previously been actively involved in acupuncture research. 48-item and 18-item responses Respondent scores for all items, together with comments on individual items, may be found in tables 4-7 and under the headings Phase 1 and Phase 2 below. Table 4. Phase 1 Respondent scores for 48 items Item essential (1) useful (2) not nec (3) no response (0) Aliveness 4 9 2 2 Being at ease 8 6 2 1 Being blue / down in the dumps 3 8 5 1 Being in control 4 5 6 2 Being present 5 8 3 1 Being spaced out 6 5 5 1 Calmness 10 5 1 1 Cheerfulness 3 9 2 3 Clarity 4 8 3 2

Comfort 7 4 5 1 Connectedness with others 3 4 8 2 Contentment 6 6 3 2 Coolness 4 10 2 1 Empowerment 6 6 4 1 Excitement 0 4 11 2 Expansion 0 9 6 2 Floating 3 7 6 1 Harmony 5 6 4 2 Heaviness 7 7 2 1 Hunger 2 6 8 1 Inner bodily awareness 5 8 3 1 Inner bodily flow 2 8 6 1 Intestinal rumblings 2 7 7 1 Looseness 1 8 7 1 Melting 1 3 12 1 Mental energy 5 9 2 1 Mental focus 6 9 1 1 Nervousness 4 9 3 1 Numbness 4 9 3 1 Optimism 4 6 4 3 Pain 14 1 1 1 Peacefulness 7 6 3 1 Physical vitality 8 5 3 1 Positivity 6 6 4 1 Pulsation 2 8 6 1 Receptivity 3 7 5 2 Relaxation 12 3 1 1 Relief 10 2 3 2 Restlessness 3 6 7 1 Satisfaction 7 3 5 2 Sensory acuteness 6 4 6 1 Sleepiness 8 6 2 1 Suppleness 4 3 8 2 Tension 9 3 4 1 Tingling 6 6 4 1 Vibration 3 6 7 1 Warmth 4 9 3 1 Worry 4 8 4 1 Table 5. Phase 2 Respondent scores for 18 items Item essential (1) useful (2) not nec (3) no response (0) Appetite 4 6 2 1 Being able to cope 5 7 0 1 Being cleared out 2 3 7 1 Being grounded 2 3 7 1 Being stressed 4 5 3 1

Bothersome heat 1 3 8 1 Happiness 4 4 4 1 Heat 2 7 3 1 Heightened acuity 3 5 4 1 Mobility 5 4 3 1 Needing medication 1 4 7 1 Self-awareness 2 5 5 1 Self-confidence 1 8 3 1 Self-esteem 1 6 5 1 Sleep quality 6 5 1 1 Something being lifted 2 5 5 1 Tiredness 7 5 0 1 Wellbeing 9 3 0 1 Table 6. Most frequently (‘top five’) and least (‘bottom five’) frequently scored items (Phase 1) essential (1) useful (2) not nec (3) no response (0) Most often scored (‘top five’)

Pain Relaxation Relief Calmness Tension

Coolness; Mental energy Mental focus Warmth Numbness etc. (8 ties)

Melting Excitement Hunger Connectedness to others Suppleness

Optimism Cheerfulness; Excitement etc. (12 tied items)

Least often scored (‘bottom five’)

Excitement Expansion Looseness Melting; Hunger etc. (4 ties)

Pain Relief; Melting Suppleness Satisfaction etc. (5 ties)

Pain Relaxation Calmness Mental focus; Being at ease etc. (7 tied items)

34 tied items

Table 7. Most frequently (‘top three’) and least (‘bottom three’) frequently scored items (Phase 2) essential (1) useful (2) not nec (3) no response (0) Most often scored (‘top three’)

Wellbeing Tiredness

Self-confidence Being able to cope Heat

Bothersome heat Being cleared out Being grounded etc. (3 ties)

Appetite Being able to cope Being cleared out etc. (18 tied items)

Least often scored (‘bottom three’)

Bothersome heat Needing medication Self-confidence etc. (4 ties)

Being cleared out Being grounded Bothersome heat etc. (4 ties)

Being able to cope Tiredness Wellbeing

18 tied items

Note: In Phase 1, with 48 items, the ‘top/bottom five’ were selected, but in Phase 2, with only 18 items, only the ‘top/bottom three’. Phase 1 Comments on individual items Each item received between one and five comments (27-129 words per item). The ‘top five’ for number of comments were: ‘Inner bodily flow’ and ‘Expansion’ (5 comments each), ‘Satisfaction’, ‘Optimism’ and ‘Intestinal rumblings’ (4 each). The ‘top five’ items for comments wordage were (in order) ‘Satisfaction’, ‘Inner bodily flow’, ‘Tingling’. ‘Optimism’ and ‘Expansion’. Of these, most dissatisfaction on meaning was expressed about ‘Inner bodily flow’ and ‘Expansion’.

In addition to these two, several items were singled out as ambiguous or polysemous, such as ‘Being in control’, ‘Cheerfulness’, ‘Comfort’, ‘Connectedness with others’, ‘Contentment’, ‘Excitement’, ‘Harmony’, ‘Receptivity’, ‘Relief’ and ‘Satisfaction’. Others were considered as probably meaningless to the average patient (e.g. ‘Being present’, ‘Looseness’, ‘Melting’, ‘Mental focus’ and ‘Pulsation’). Yet others were thought to be synonymous, or at least so similar that not all were needed: ‘Aliveness’ and ‘Physical vitality’ ‘Being spaced out’ and ‘Floating’ ‘Being at ease’, ‘Calmness’, ‘Peacefulness’ and ‘Relaxation’ (and sometimes ‘Sleepiness’) ‘Being in control’ and ‘Empowerment’ ‘Inner bodily flow’ and ‘Vibration’ ‘Looseness’ and ‘Suppleness’ ‘Optimism’ and ‘Positivity’. Clearly, interpretations of similarity between some of these items, like the items themselves, will be highly individual. One respondent considered seven feelings as overly negative or possibly indicative of an adverse effect if experienced: ‘Being blue’, ‘Melting’, ‘Mental energy’, ‘Nervousness’, ‘Restlessness’, ‘Tension’ and ‘Worry’. Another considered ‘Positivity’ as more (positively) loaded than the other items. However, those who completed the EXPre and EXPost questionnaires were being asked whether they expected to experience, or experienced, a change in feelings, not necessarily the feeling itself, so these reservations may be partly based on a misapprehension. Phase 2 Comments on individual items Three respondents commented on individual items. Two, for example, stated that whether an item is considered nonspecific or useful would depend on the condition treated (or treatment context). ‘Something capturing contentment’ (whether ‘Happiness’ or another term) was thought to be essential by the most prolific commentator. In contrast, ‘Bothersome heat’ was not considered necessary if ‘Heat’ was already included, and ‘Something being lifted’ did not appear to be well understood. Content validity analysis There were some interesting differences in respondent style, summarised in table 8. In both surveys, women considered more items as ‘essential’ than men, but fewer as ‘not necessary’. Ratios were reversed for ‘useful but not essential’ items. Table 8A. Ratios of mean numbers of items scored ‘essential’, ‘useful’ and ‘not necessary’ for women and men in the two surveys. F/M ratios Initial survey (48 items) Second survey (18 items) Essential 18.8/10.0 (1.88) 5.3/4.0 (1.33) Useful but not essential 19.6/15.9 (1.23) 6.3/7.3 (0.86) Not necessary 1.9/5.4 (0.35) 3.9/6.7 (0.58)

Table 8B. Total numbers of items scored ‘essential’, ‘useful’ and ‘not necessary’ for women and men in the two surveys, with ratios and p values for the Binomial (proportion) test. Total numbers rated Initial survey (48 items) Second survey (18 items) Essential 150/90 (1.67, p<0.001) 37/24 (1.54, ns) Useful but not essential 157/143 (1.10, ns) 44/44 (1.00, ns) Not necessary 62/150 (0.41, p<0.001) 27/40 (0.68, ns) CVR, CVIL, I-CVI and S-CVI To test the effect of respondent style on Lawshe’s CVR and CVIL, respondents were either analysed either (1) together (Phase 1, N=17; Phase 2, N=13 ), or (2) by removing the non-practitioner researchers (leaving N=14 and N=11, respectively), or (3) by removing those who did not score any item as ‘essential’ (N=14, N=10, respectively) (tables 9, 10). In addition, the group was split between women (N=8, and N=7, respectively, in the two Phases) and men (N=9, N=6, respectively) (tables 11-14). Table 9. Lawshe’s CVR and CVIL for items (with 2-tailed significance). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded. (Phase 1) Item CVR (all) CVRL (no res) CVRL (no 0’s) Aliveness – (ns) – (ns) – (ns) Being at ease – (ns) 0 (ns) 0.14 (ns) Being blue / down in the dumps – (ns) – (ns) – (ns) Being in control – (ns) – (ns) – (ns) Being present – (ns) – (ns) – (ns) Being spaced out – (ns) – (ns) – (ns) Calmness 0.18 (ns) 0.14 (ns) 0.43 (ns) Cheerfulness – (ns) – (ns) – (ns) Clarity – (ns) – (ns) – (ns) Comfort – (ns) – (ns) 0 (ns) Connectedness with others – (ns) – (ns) – (ns) Contentment – (ns) – (ns) – (ns) Coolness – (ns) – (ns) – (ns) Empowerment – (ns) – (ns) – (ns) Excitement – (ns) – (ns) – (ns) Expansion – (ns) – (ns) – (ns) Floating – (ns) – (ns) – (ns) Harmony – (ns) – (ns) – (ns) Heaviness – (ns) – (ns) 0 (ns) Hunger – (ns) – (ns) – (ns) Inner bodily awareness – (ns) – (ns) – (ns) Inner bodily flow – (ns) – (ns) – (ns) Intestinal rumblings – (ns) – (ns) – (ns) Looseness – (ns) – (ns) – (ns) Melting – (ns) – (ns) – (ns) Mental energy – (ns) – (ns) – (ns) Mental focus – (ns) – (ns) – (ns) Nervousness – (ns) – (ns) – (ns)

Numbness – (ns) – (ns) – (ns) Optimism – (ns) – (ns) – (ns) Pain 0.65 (p<0.01) 0.71 (p<0.01) 1.00 (p<0.002) Peacefulness – (ns) – (ns) 0 (ns) Physical vitality – (ns) 0 (ns) 0.14 (ns) Positivity – (ns) – (ns) – (ns) Pulsation – (ns) – (ns) – (ns) Receptivity – (ns) – (ns) – (ns) Relaxation 0.41 (p<0.05) 0.43 (ns) 0.71 (p<0.01) Relief 0.18 (ns) 0.29 (ns) 0.43 (ns) Restlessness – (ns) – (ns) – (ns) Satisfaction – (ns) – (ns) 0 (ns) Sensory acuteness – (ns) – (ns) – (ns) Sleepiness – (ns) – (ns) 0.14 (ns) Suppleness – (ns) – (ns) – (ns) Tension 0.06 (ns) 0 (ns) 0.29 (ns) Tingling – (ns) – (ns) – (ns) Vibration – (ns) – (ns) – (ns) Warmth – (ns) – (ns) – (ns) Worry – (ns) – (ns) – (ns) CVIL 0.53 (2 items) 0.71 (1 item) 0.86 (2 items) Table 10. Lawshe’s CVR and CVIL for items (with 2-tailed significance). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item CVR (all) CVRL (no res) CVRL (no

non-res) CVRL (no 0’s)

Appetite – (ns) – (ns) – (ns) – (ns) Being able to cope – (ns) – (ns) – (ns) 0 (ns) Being cleared out – (ns) – (ns) – (ns) – (ns) Being grounded – (ns) – (ns) – (ns) – (ns) Being stressed – (ns) – (ns) – (ns) – (ns) Bothersome heat – (ns) – (ns) – (ns) – (ns) Happiness – (ns) – (ns) – (ns) – (ns) Heat – (ns) – (ns) – (ns) – (ns) Heightened acuity – (ns) – (ns) – (ns) – (ns) Mobility – (ns) – (ns) – (ns) 0 (ns) Needing medication – (ns) – (ns) – (ns) – (ns) Self-awareness – (ns) – (ns) – (ns) – (ns) Self-confidence – (ns) – (ns) – (ns) – (ns) Self-esteem – (ns) – (ns) – (ns) – (ns) Sleep quality – (ns) 0.09 (ns) – (ns) 0.20 (ns) Something being lifted – (ns) – (ns) – (ns) – (ns) Tiredness 0.08 (ns) – (ns) 0.20 (ns) 0.4 (ns) Wellbeing 0.38 (ns) 0.27 (ns) 0.40 (ns) 0.8 (p=0.02) CVIL ns ns ns 0.8 (1 item)

Table 11. Lawshe’s CVR and CVIL for items (with 2-tailed significance) – for women only. ‘all’ = all women respondents; ‘no res’ = non-practitioner researchers excluded (Phase 1). Item CVR (all) CVRL (no res) Aliveness – (ns) – (ns) Being at ease 0.25 (ns) 0.33 (ns) Being blue / down in the dumps – (ns) – (ns) Being in control – (ns) – (ns) Being present 0 (ns) 0 (ns) Being spaced out 0.25 (ns) 0 (ns) Calmness 0.75 (ns) 0.67 (ns) Cheerfulness – (ns) – (ns) Clarity – (ns) – (ns) Comfort – (ns) – (ns) Connectedness with others – (ns) – (ns) Contentment – (ns) – (ns) Coolness – (ns) – (ns) Empowerment – (ns) – (ns) Excitement – (ns) – (ns) Expansion – (ns) – (ns) Floating – (ns) – (ns) Harmony – (ns) – (ns) Heaviness 0 (ns) 0 (ns) Hunger – (ns) – (ns) Inner bodily awareness – (ns) – (ns) Inner bodily flow – (ns) – (ns) Intestinal rumblings – (ns) – (ns) Looseness – (ns) – (ns) Melting – (ns) – (ns) Mental energy 0 (ns) 0 (ns) Mental focus 0 (ns) – (ns) Nervousness – (ns) – (ns) Numbness – (ns) 0 (ns) Optimism – (ns) – (ns) Pain 1.00 (p<0.01) 1.00 (p<0.002) Peacefulness 0 (ns) 0 (ns) Physical vitality 0.25 (ns) 0.33 (ns) Positivity – (ns) – (ns) Pulsation – (ns) – (ns) Receptivity – (ns) – (ns) Relaxation 1.00 (p<0.01) 1.00 (p<0.002) Relief 0.25 (ns) 0.33 (ns) Restlessness – (ns) – (ns) Satisfaction – (ns) – (ns) Sensory acuteness 0 (ns) 0 (ns) Sleepiness 0.50 (ns) 0.33 (ns) Suppleness – (ns) – (ns) Tension 0.50 (ns) 0.33 (ns)

Tingling – (ns) 0 (ns) Vibration – (ns) – (ns) Warmth – (ns) – (ns) Worry – (ns) – (ns) CVIL 1.00 (2 items) 1.00 (2 items) Table 12. Lawshe’s CVR and CVIL for items (with 2-tailed significance). For men only. ‘all’ = all men respondents; ‘no res’ = non-practitioner researchers excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 1). Item CVR (all) CVRL (no res) CVRL (no 0’s) Aliveness – (ns) – (ns) – (ns) Being at ease – (ns) – (ns) 0 (ns) Being blue / down in the dumps – (ns) – (ns) – (ns) Being in control – (ns) – (ns) – (ns) Being present – (ns) – (ns) – (ns) Being spaced out – (ns) – (ns) – (ns) Calmness – (ns) – (ns) 0 (ns) Cheerfulness – (ns) – (ns) – (ns) Clarity – (ns) – (ns) – (ns) Comfort – (ns) 0 (ns) 0.33 (ns) Connectedness with others – (ns) – (ns) – (ns) Contentment – (ns) – (ns) 0 (ns) Coolness – (ns) – (ns) – (ns) Empowerment – (ns) – (ns) 0 (ns) Excitement – (ns) – (ns) – (ns) Expansion – (ns) – (ns) – (ns) Floating – (ns) – (ns) – (ns) Harmony – (ns) – (ns) – (ns) Heaviness – (ns) – (ns) 0 (ns) Hunger – (ns) – (ns) – (ns) Inner bodily awareness – (ns) – (ns) 0 (ns) Inner bodily flow – (ns) – (ns) – (ns) Intestinal rumblings – (ns) – (ns) – (ns) Looseness – (ns) – (ns) – (ns) Melting – (ns) – (ns) – (ns) Mental energy – (ns) – (ns) – (ns) Mental focus – (ns) – (ns) – (ns) Nervousness – (ns) – (ns) – (ns) Numbness – (ns) – (ns) – (ns) Optimism – (ns) – (ns) 0 (ns) Pain 0.33 (ns) 0.5 (ns) 1.00 (p<0.01) Peacefulness – (ns) – (ns) 0 (ns) Physical vitality – (ns) 0 (ns) 0 (ns) Positivity – (ns) – (ns) 0 (ns) Pulsation – (ns) – (ns) – (ns) Receptivity – (ns) – (ns) – (ns) Relaxation – (ns) 0 (ns) 0.33 (ns)

Relief – (ns) 0.25 (ns) 0.67 (ns) Restlessness – (ns) – (ns) – (ns) Satisfaction – (ns) 0 (ns) 0.33 (ns) Sensory acuteness – (ns) – (ns) – (ns) Sleepiness – (ns) – (ns) – (ns) Suppleness – (ns) – (ns) – (ns) Tension – (ns) – (ns) 0 (ns) Tingling – (ns) – (ns) 0 (ns) Vibration – (ns) – (ns) – (ns) Warmth – (ns) – (ns) – (ns) Worry – (ns) – (ns) – (ns) CVIL ns ns 1.00 (1 item) Table 13. Lawshe’s CVR and CVIL for items (with 2-tailed significance) – for women only. ‘all’ = all women respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item CVR (all) CVRL (no

res) CVRL (no non-res)

CVRL (no 0’s)

Appetite – (ns) – (ns) – (ns) – (ns) Being able to cope – (ns) 0.2 (ns) – (ns) 0 Being cleared out – (ns) – (ns) – (ns) – (ns) Being grounded – (ns) – (ns) – (ns) – (ns) Being stressed – (ns) – (ns) 0 (ns) 0 Bothersome heat – (ns) – (ns) – (ns) – (ns) Happiness – (ns) – (ns) 0 (ns) 0 Heat – (ns) – (ns) – (ns) – (ns) Heightened acuity – (ns) – (ns) – (ns) – (ns) Mobility – (ns) – (ns) – (ns) – (ns) Needing medication – (ns) – (ns) – (ns) – (ns) Self-awareness – (ns) – (ns) – (ns) – (ns) Self-confidence – (ns) – (ns) – (ns) – (ns) Self-esteem – (ns) – (ns) – (ns) – (ns) Sleep quality 0.14 (ns) 0.6 (ns) 0 (ns) 0.33 (ns) Something being lifted – (ns) – (ns) – (ns) – (ns) Tiredness 0.43 (ns) 0.2 (ns) 0.67 (ns) 0.67 (ns) Wellbeing 0.71 (ns) 0.6 (ns) 0.67 (ns) 1.00 (p<0.01) CVIL ns ns ns 1.00 (1 item) Table 14. Lawshe’s CVR and CVIL for items (with 2-tailed significance) – for men only. ‘all’ = all men respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item CVR (all) CVRL (no

res) CVRL (no non-res)

CVRL (no 0’s)

Appetite 0 (ns) 0 (ns) 0 (ns) 0 (ns) Being able to cope – (ns) 0 (ns) 0 (ns) – (ns) Being cleared out – (ns) – (ns) – (ns) – (ns) Being grounded – (ns) – (ns) – (ns) – (ns)

Being stressed – (ns) – (ns) – (ns) – (ns) Bothersome heat – (ns) – (ns) – (ns) – (ns) Happiness – (ns) – (ns) – (ns) – (ns) Heat – (ns) 0 (ns) 0 (ns) – (ns) Heightened acuity – (ns) – (ns) – (ns) – (ns) Mobility 0 (ns) 0 (ns) 0 (ns) 0 (ns) Needing medication – (ns) – (ns) – (ns) – (ns) Self-awareness – (ns) – (ns) – (ns) – (ns) Self-confidence – (ns) – (ns) – (ns) – (ns) Self-esteem – (ns) – (ns) – (ns) – (ns) Sleep quality – (ns) – (ns) – (ns) – (ns) Something being lifted – (ns) – (ns) – (ns) – (ns) Tiredness – (ns) – (ns) – (ns) – (ns) Wellbeing 0 (ns) 0 (ns) 0 (ns) 0 (ns) CVIL ns ns ns ns The data was then analysed for I-CVI by allotting ‘useful’ either to ‘0’ (more stringently) (tables 15, 16) or to ‘1’ (less stringently) (tables 17-19 and tables 20-22), to see how this affected results. Table 15. I-CVI and S-CVI (scoring ‘useful’ as Inessential). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 1). Item I-CVI (all) I-CVI (no res) I-CVI (no 0’s) Aliveness <0.6 <0.6 <0.6 Being at ease <0.6 <0.6 <0.6 Being blue / down in the dumps <0.6 <0.6 <0.6 Being in control <0.6 <0.6 <0.6 Being present <0.6 <0.6 <0.6 Being spaced out <0.6 <0.6 <0.6 Calmness <0.6 <0.6 0.71 Cheerfulness <0.6 <0.6 <0.6 Clarity <0.6 <0.6 <0.6 Comfort <0.6 <0.6 <0.6 Connectedness with others <0.6 <0.6 <0.6 Contentment <0.6 <0.6 <0.6 Coolness <0.6 <0.6 <0.6 Empowerment <0.6 <0.6 <0.6 Excitement 0 0 0 Expansion 0 0 0 Floating <0.6 <0.6 <0.6 Harmony <0.6 <0.6 <0.6 Heaviness <0.6 <0.6 <0.6 Hunger <0.6 <0.6 <0.6 Inner bodily awareness <0.6 <0.6 <0.6 Inner bodily flow <0.6 <0.6 <0.6 Intestinal rumblings <0.6 <0.6 <0.6 Looseness <0.6 <0.6 <0.6

Melting <0.6 <0.6 <0.6 Mental energy <0.6 <0.6 <0.6 Mental focus <0.6 <0.6 <0.6 Nervousness <0.6 <0.6 <0.6 Numbness <0.6 <0.6 <0.6 Optimism <0.6 <0.6 <0.6 Pain 0.82 0.86 1.00 Peacefulness <0.6 <0.6 <0.6 Physical vitality <0.6 <0.6 <0.6 Positivity <0.6 <0.6 <0.6 Pulsation <0.6 <0.6 <0.6 Receptivity <0.6 <0.6 <0.6 Relaxation 0.71 0.71 0.86 Relief <0.6 0.64 0.71 Restlessness <0.6 <0.6 <0.6 Satisfaction <0.6 <0.6 <0.6 Sensory acuteness <0.6 <0.6 <0.6 Sleepiness <0.6 <0.6 <0.6 Suppleness <0.6 <0.6 <0.6 Tension <0.6 <0.6 0.64 Tingling <0.6 <0.6 <0.6 Vibration <0.6 <0.6 <0.6 Warmth <0.6 <0.6 <0.6 Worry <0.6 <0.6 <0.6 S-CVIUA (‘good’ or ‘excellent’) 0.042 0.042 0.083 S-CVIUA (‘excellent’) 0.021 0.021 0.042 S-CVIAV 0.294 0.292 0.357 S-CVIAV-UA (‘good’ or ‘excellent’) 0.765 0.785 0.820 S-CVIAV-UA (‘excellent’) 0.820 0.860 0.930 Table 16. I-CVI and S-CVI (scoring ‘useful’ as Inessential). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item I-CVI (all) I-CVI (no

res) I-CVI (no non-res)

I-CVI (no 0’s)

Appetite <0.6 <0.6 <0.6 <0.6 Being able to cope <0.6 <0.6 <0.6 <0.6 Being cleared out <0.6 <0.6 <0.6 <0.6 Being grounded <0.6 <0.6 <0.6 <0.6 Being stressed <0.6 <0.6 <0.6 <0.6 Bothersome heat <0.6 <0.6 0 <0.6 Happiness <0.6 <0.6 <0.6 <0.6 Heat <0.6 <0.6 <0.6 <0.6 Heightened acuity <0.6 <0.6 <0.6 <0.6 Mobility <0.6 <0.6 <0.6 <0.6 Needing medication <0.6 <0.6 <0.6 <0.6 Self-awareness <0.6 <0.6 <0.6 <0.6

Self-confidence <0.6 <0.6 <0.6 <0.6 Self-esteem <0.6 <0.6 <0.6 <0.6 Sleep quality <0.6 <0.6 <0.6 0.60 Something being lifted <0.6 <0.6 <0.6 <0.6 Tiredness <0.6 <0.6 0.60 0.70 Wellbeing 0.69 0.64 0.70 0.90 S-CVIUA (‘good’ or ‘excellent’)

n/a n/a n/a 0.06

S-CVIUA (‘excellent’) n/a n/a n/a 0.06 S-CVIAV 0.26 0.26 0.26 0.34 S-CVIAV-UA (‘good’ or ‘excellent’)

n/a n/a n/a 0.90

S-CVIAV-UA (‘excellent’) n/a n/a n/a 0.90 To test for the effect of respondent style on Lynn’s CVI, respondents were also analysed either (1) together (N=17 or N=13), or (2) by removing the non-practitioner researchers (leaving N=14 or N=11), or (3) in Phase 2, by removing the non-researcher practitioners (leaving N=10), or (4) by removing those who did not score any item as ‘essential’ (N=14 or N=10) (tables 17-22). In addition, to test for the effect of gender the group was split between women (N=8 or N=7) and men (N=9 or N=6) (tables 18, 19 and 21, 22). Polit et al. have proposed a modified kappa statistic which they then use to classify whether an I-CVI is fair, good or excellent [30]. To simplify, I-CVIs > 0.7 are good, and I-CVIs > 0.8 are excellent, regardless of the number of respondents involved. Tables 17-22 show the values for I-CVI and S-CVI found in the two Phases of this survey (‘excellent’ items indicated in bold). Table 17. I-CVI and S-CVI (scoring ‘useful’ as Essential). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded. Item I-CVI (all) I-CVI (no res) I-CVI (no 0’s) Aliveness 0.76 0.79 0.79 Being at ease 0.82 0.86 0.86 Being blue / down in the dumps 0.65 0.64 0.71 Being in control <0.6 <0.6 0.64 Being present 0.76 0.79 0.79 Being spaced out 0.65 0.64 0.79 Calmness 0.88 0.93 0.93 Cheerfulness 0.71 0.71 0.71 Clarity 0.71 0.71 0.86 Comfort 0.65 0.71 0.64 Connectedness with others <0.6 <0.6 <0.6 Contentment 0.71 0.71 0.79 Coolness 0.82 0.86 0.86 Empowerment 0.71 0.71 0.86 Excitement <0.6 <0.6 <0.6 Expansion <0.6 <0.6 <0.6 Floating <0.6 0.64 0.64

Harmony 0.65 0.64 0.71 Heaviness 0.82 0.86 0.86 Hunger <0.6 <0.6 <0.6 Inner bodily awareness 0.76 0.79 0.79 Inner bodily flow <0.6 0.64 0.64 Intestinal rumblings <0.6 <0.6 <0.6 Looseness <0.6 <0.6 <0.6 Melting <0.6 <0.6 <0.6 Mental energy 0.82 0.86 0.86 Mental focus 0.88 0.93 0.93 Nervousness 0.76 0.79 0.86 Numbness 0.76 0.79 0.79 Optimism <0.6 <0.6 0.64 Pain 0.88 0.93 1.00 Peacefulness 0.76 0.79 0.86 Physical vitality 0.76 0.79 0.86 Positivity 0.71 0.71 0.79 Pulsation <0.6 0.64 <0.6 Receptivity <0.6 0.64 <0.6 Relaxation 0.88 0.93 0.93 Relief 0.71 0.79 0.71 Restlessness <0.6 <0.6 0.64 Satisfaction <0.6 0.64 <0.6 Sensory acuteness <0.6 <0.6 0.71 Sleepiness 0.82 0.86 0.86 Suppleness <0.6 <0.6 <0.6 Tension 0.71 0.71 0.86 Tingling 0.71 0.71 0.71 Vibration <0.6 <0.6 <0.6 Warmth 0.76 0.79 0.79 Worry 0.71 0.71 0.79 S-CVIUA (‘good’ or ‘excellent’) 0.542 0.563 0.625 S-CVIUA (‘excellent’) 0.188 0.188 0.313 S-CVIAV 0.662 0.683 0.714 S-CVIAV-UA (‘good’ or ‘excellent’) 0.768 0.797 0.821 S-CVIAV-UA (‘excellent’) 0.847 0.891 0.883 Table 18. I-CVI and S-CVI (scoring ‘useful’ as Essential). Data for women only. ‘all’ = all women respondents; ‘no res’ = non-practitioner researchers excluded. Item I-CVI (all) I-CVI (no res) Aliveness 0.88 0.83 Being at ease 1.00 1.00 Being blue / down in the dumps 1.00 1.00 Being in control 0.63 <0.6 Being present 0.88 0.83 Being spaced out 1.00 1.00 Calmness 1.00 1.00

Cheerfulness 0.75 0.67 Clarity 0.88 0.83 Comfort <0.6 <0.6 Connectedness with others <0.6 <0.6 Contentment 0.88 0.83 Coolness 1.00 1.00 Empowerment 1.00 1.00 Excitement <0.6 <0.6 Expansion <0.6 <0.6 Floating 0.75 0.83 Harmony 0.88 0.83 Heaviness 1.00 1.00 Hunger 0.75 0.83 Inner bodily awareness 1.00 1.00 Inner bodily flow 0.63 0.67 Intestinal rumblings 0.63 0.67 Looseness <0.6 <0.6 Melting <0.6 <0.6 Mental energy 1.00 1.00 Mental focus 1.00 1.00 Nervousness 1.00 1.00 Numbness 1.00 1.00 Optimism 0.63 <0.6 Pain 1.00 1.00 Peacefulness 1.00 1.00 Physical vitality 1.00 1.00 Positivity 0.88 0.83 Pulsation 0.63 0.67 Receptivity 0.75 0.83 Relaxation 1.00 1.00 Relief 0.63 0.67 Restlessness 0.88 0.83 Satisfaction <0.6 <0.6 Sensory acuteness 1.00 1.00 Sleepiness 1.00 1.00 Suppleness 0.75 0.83 Tension 1.00 1.00 Tingling 0.88 0.83 Vibration <0.6 <0.6 Warmth 0.88 0.83 Worry 0.88 0.83 S-CVIUA (‘good’ or ‘excellent’) 0.708 0.688 S-CVIUA (‘excellent’) 0.604 0.688 S-CVIAV 0.798 0.784 S-CVIAV-UA (‘good’ or ‘excellent’) 0.926 0.929 S-CVIAV-UA (‘excellent’) 0.960 0.929

Table 19. I-CVI and S-CVI (scoring ‘useful’ as Essential). Data for men only. ‘all’ = all men respondents; ‘no res’ = non-practitioner researchers excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded. Item I-CVI (all) I-CVI (no res) I-CVI (no 0’s) Aliveness 0.67 0.75 0.67 Being at ease 0.67 0.75 0.67 Being blue / down in the dumps <0.6 <0.6 <0.6 Being in control <0.6 <0.6 0.67 Being present 0.67 0.75 0.67 Being spaced out <0.6 <0.6 <0.6 Calmness 0.78 0.88 0.84 Cheerfulness 0.67 0.75 0.67 Clarity <0.6 0.63 0.84 Comfort 0.78 0.88 0.84 Connectedness with others <0.6 <0.6 <0.6 Contentment <0.6 0.63 0.67 Coolness 0.67 0.75 0.67 Empowerment <0.6 <0.6 0.67 Excitement <0.6 <0.6 <0.6 Expansion 0.67 0.75 0.67 Floating <0.6 <0.6 <0.6 Harmony <0.6 <0.6 <0.6 Heaviness 0.67 0.75 0.67 Hunger <0.6 <0.6 0.67 Inner bodily awareness <0.6 0.63 <0.6 Inner bodily flow <0.6 0.63 0.67 Intestinal rumblings <0.6 <0.6 <0.6 Looseness <0.6 0.63 0.67 Melting <0.6 <0.6 <0.6 Mental energy 0.67 0.75 0.67 Mental focus 0.78 0.88 0.84 Nervousness <0.6 0.63 0.67 Numbness <0.6 0.63 <0.6 Optimism <0.6 0.63 0.67 Pain 0.78 0.88 1.00 Peacefulness <0.6 0.63 0.67 Physical vitality <0.6 0.63 0.67 Positivity <0.6 0.63 0.67 Pulsation <0.6 0.63 <0.6 Receptivity <0.6 <0.6 <0.6 Relaxation 0.78 0.88 0.84 Relief 0.78 0.88 0.84 Restlessness <0.6 <0.6 <0.6 Satisfaction 0.78 0.88 0.84 Sensory acuteness <0.6 <0.6 <0.6 Sleepiness 0.67 0.75 0.67 Suppleness <0.6 <0.6 <0.6

Tension <0.6 <0.6 0.67 Tingling <0.6 0.63 <0.6 Vibration <0.6 0.63 <0.6 Warmth 0.67 0.75 0.67 Worry <0.6 0.63 0.67 S-CVIUA (‘good’ or ‘excellent’) 0.146 0.354 0.167 S-CVIUA (‘excellent’) 0.146 0.146 0.167 S-CVIAV 0.537 0.604 0.599 S-CVIAV-UA (‘good’ or ‘excellent’) 0.778 0.801 0.833 S-CVIAV-UA (‘excellent’) n/a 0.875 0.833 Table 20. I-CVI and S-CVI (scoring ‘useful’ as Essential). ‘all’ = all respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item I-CVI (all) I-CVI (no

res) I-CVI (no non-res)

I-CVI (no 0’s)

Appetite 0.77 0.82 0.90 0.80 Being able to cope 0.92 0.91 1.00 1.00 Being cleared out <0.6 <0.6 <0.6 <0.6 Being grounded <0.6 <0.6 <0.6 <0.6 Being stressed 0.69 0.64 0.70 0.80 Bothersome heat <0.6 <0.6 <0.6 <0.6 Happiness 0.62 0.64 0.70 0.70 Heat 0.69 0.73 0.80 0.70 Heightened acuity 0.62 0.64 0.70 0.80 Mobility 0.69 0.73 0.80 0.70 Needing medication <0.6 <0.6 <0.6 <0.6 Self-awareness <0.6 <0.6 <0.6 0.60 Self-confidence 0.69 0.64 0.70 0.90 Self-esteem <0.6 <0.6 <0.6 0.70 Sleep quality 0.85 0.91 1.00 0.90 Something being lifted <0.6 0.64 0.70 <0.6 Tiredness 0.92 0.91 1.00 1.00 Wellbeing 0.92 0.91 1.00 1.00 S-CVIUA (‘good’ or ‘excellent’)

0.28 0.39 0.39 0.44

S-CVIUA (‘excellent’) 0.22 0.28 0.28 0.28 S-CVIAV 0.64 0.63 0.69 0.70 S-CVIAV-UA (‘good’ or ‘excellent’)

0.88 0.84 0.93 0.90

S-CVIAV-UA (‘excellent’) 0.90 0.89 0.98 0.96 Table 21. I-CVI and S-CVI (scoring ‘useful’ as Essential). Data for women only. ‘all’ = all women respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item I-CVI (all) I-CVI (no

res) I-CVI (no non-res)

I-CVI (no 0’s)

Appetite 0.71 0.80 0.67 0.83

Being able to cope 0.86 0.80 0.83 1.00 Being cleared out <0.6 <0.6 <0.6 <0.6 Being grounded <0.6 <0.6 <0.6 <0.6 Being stressed 0.71 0.60 0.67 0.83 Bothersome heat <0.6 <0.6 <0.6 <0.6 Happiness <0.6 0.60 <0.6 0.67 Heat <0.6 0.60 <0.6 0.67 Heightened acuity 0.71 0.80 0.67 0.83 Mobility <0.6 0.60 <0.6 0.67 Needing medication <0.6 <0.6 <0.6 0.67 Self-awareness 0.71 0.60 0.67 0.83 Self-confidence 0.86 0.80 0.83 1.00 Self-esteem 0.71 0.60 0.67 0.83 Sleep quality 0.71 0.80 0.67 0.83 Something being lifted <0.6 0.60 <0.6 <0.6 Tiredness 0.86 0.80 0.83 1.00 Wellbeing 0.86 0.80 0.83 1.00 S-CVIUA (‘good’ or ‘excellent’)

0.56 0.39 0.22 0.56

S-CVIUA (‘excellent’) 0.22 n/a 0.22 0.56 S-CVIAV 0.64 0.63 0.58 0.75 S-CVIAV-UA (‘good’ or ‘excellent’)

0.77 0.80 0.83 0.90

S-CVIAV-UA (‘excellent’) 0.86 n/a 0.83 0.90 Table 22. I-CVI and S-CVI (scoring ‘useful’ as Essential). Data for men only. ‘all’ = all men respondents; ‘no res’ = non-practitioner researchers excluded; ‘no non-res’ = non-researcher practitioners excluded; ‘no 0’s’ = those scoring no item as ‘essential’ excluded (Phase 2). Item I-CVI (all) I-CVI (no

non-res) I-CVI (no 0’s)

Appetite <0.6 <0.6 <0.6 Being able to cope <0.6 <0.6 <0.6 Being cleared out <0.6 <0.6 <0.6 Being grounded 0 0 0 Being stressed <0.6 0 <0.6 Bothersome heat <0.6 0 <0.6 Happiness <0.6 0 <0.6 Heat <0.6 <0.6 <0.6 Heightened acuity <0.6 <0.6 <0.6 Mobility <0.6 <0.6 <0.6 Needing medication 0 0 0 Self-awareness 0 0 0 Self-confidence 0 0 0 Self-esteem 0 0 0 Sleep quality <0.6 <0.6 <0.6 Something being lifted <0.6 <0.6 <0.6 Tiredness <0.6 <0.6 <0.6 Wellbeing <0.6 <0.6 <0.6

S-CVIUA (‘good’ or ‘excellent’)

n/a n/a n/a

S-CVIUA (‘excellent’) n/a n/a n/a S-CVIAV 0.22 0.21 0.22 S-CVIAV-UA (‘good’ or ‘excellent’)

n/a n/a n/a

S-CVIAV-UA (‘excellent’) n/a n/a n/a In Phase 1, Lawshe’s CVR was thus significant for only two items, ‘Pain’ and ‘Relaxation’, with significance greatest when those respondents scoring no items as ‘essential’ were removed from analysis, less significance when non-practitioner researchers were removed, and least good when all respondents were taken together. When results for women and men were segregated, CVR was significant for the same two items, but greater than when all respondents were taken together. For all men taken together and for men practitioners with researchers removed, CVR was not significant, and significant only for ‘Pain’ when those respondents scoring no items as ‘essential’ were removed from analysis. Lawshe’s CVIL showed good values for the listed items, but only if the EXPre and EXPost scales are reduced to these two items, which renders them useless. In Phase 2, Lawshe’s CVR was significant only for ‘Wellbeing’, but is again not really meaningful for a single item. Polit et al. [30] have recommended that for a scale to be judged as having excellent content validity using Lynn’s method, it would need to be composed of items that had I-CVIs of ≥ 0.78 and an S-CVIAV (i.e. for all items) of ≥ 0.90. If S-CVIUA is used rather than S-CVIAV, an 0.8 benchmark is usually considered acceptable [30]. When scoring ‘useful’ as Lynn’s Inessential, I-CVI was ‘good’ or ‘excellent’ only for the same three items for which Lawshe’s CVR was significant, and neither S-CVIUA nor S-CVIAV reached acceptable values. However, if those respondents scoring no items as ‘essential’ were removed from analysis, I-CVI became ‘good’ for ‘Calmness’ and ‘Relief’ in addition, but again neither S-CVIUA nor S-CVIAV were acceptable . If these two items were excluded, leaving only those rated as ‘excellent’ (‘Pain’ and ‘Relaxation’), then S-CVIAV-UA for this Phase 1 subgroup increased to 0.93, greater than the 0.9 benchmark (table 15). In Phase 2, the single-item ‘Wellbeing’ subgroup also reached the 0.9 benchmark (table 16). When scoring ‘useful’ as Lynn’s Essential, many more items showed ‘good’ or ‘excellent’ levels of I-CVI, as can be seen from Tables 17-22. Tables 23 and 24 list those items with I-CVI > 0.8 (I-ICVI for items in square brackets was < 0.8 but ≥ 0.78). S-CVIAV-UA was calculated from items for which I-CVI ≥ 0.78. Table 23. Phase 1 items with I-CVI > 0.8 (I-ICVI for items in square brackets was < 0.8 but ≥ 0.78). Cohort I-CVI (all) I-CVI (no res) I-CVI (no 0’s) All respondents

Being at ease Calmness

Being at ease Calmness

Being at ease Calmness

Coolness Heaviness Mental energy Mental focus Pain Relaxation Sleepiness (9 items)

Coolness Heaviness Mental energy Mental focus Pain Relaxation Sleepiness (9 items)

Clarity Coolness Empowerment Heaviness Mental energy Mental focus Nervousness Pain Peacefulness Physical vitality Relaxation Sleepiness Tension (15 items)

S-CVIAV-UA 0.85 0.89 0.88 Women only Aliveness

Being at ease Being blue / down in the dumps Being present Being spaced out Calmness Clarity Contentment Coolness Empowerment Harmony Heaviness Inner bodily awareness Mental energy Mental focus Nervousness Numbness Pain Peacefulness Physical vitality Positivity Relaxation Restlessness Sensory acuteness Sleepiness Tension Tingling Warmth Worry (30 items)

Aliveness Being at ease Being blue / down in the dumps Being present Being spaced out Calmness Clarity Contentment Coolness Empowerment Floating Harmony Heaviness Hunger Inner bodily awareness Mental energy Mental focus Nervousness Numbness Pain Peacefulness Physical vitality Positivity Receptivity Relaxation Restlessness Sensory acuteness Sleepiness Suppleness Tension Tingling Warmth Worry (34 items)

S-CVIAV-UA 0.96 0.93 Men only [Calmness]

[Comfort] [Mental focus] [Pain] [Relaxation]

Calmness Comfort Mental focus Pain Relaxation

Calmness Comfort Mental focus Pain Relaxation

[Relief] [Satisfaction] (7 items)

Relief Satisfaction (7 items)

Relief Satisfaction (7 items)

S-CVIAV-UA 0.78 0.88 0.83 Table 24. Phase 1 Items with I-CVI > 0.8 (I-ICVI for items in square brackets was < 0.8 but ≥ 0.78).

Cohort I-CVI (all) I-CVI (no res) I-CVI (no non-res)

I-CVI (no 0’s)

All respondents Being able to cope Sleep quality Tiredness Wellbeing (4 items)

Appetite Being able to cope Sleep quality Tiredness Wellbeing (5 items)

Appetite Being able to cope Sleep quality Tiredness Wellbeing (5 items)

[Appetite] Being able to cope [Being stressed] [Heightened acuity] Self-confidence Sleep quality Tiredness Wellbeing (8 items)

S-CVIAV-UA 0.90 0.89 0.98 0.90 Women only Being able to cope

Self-confidence Tiredness Wellbeing (4 items)

[Appetite] [Being able to cope] [Heightened acuity] [Self-confidence] [Sleep quality] [Tiredness] [Wellbeing] (7 items)

Being able to cope Self-confidence Tiredness Wellbeing (4 items)

Appetite Being able to cope Being stressed Heightened acuity Self-awareness Self-confidence Self-esteem Sleep quality Tiredness Wellbeing (10 items)

S-CVIAV-UA 0.86 0.80 0.83 0.90 Men only (0 items) (0 items) (0 items) S-CVIAV-UA n/a n/a n/a

Thus for women there were many more items with ‘excellent’ I-CVI than for men (for whom there were none in Phase 1), with fewer for men than for all respondents taken together. Considering just the retained items with I-CVI > 0.8, rather than all items, in Phase 1 excellent scale content validity (S-CVIAV-UA) eluded both the men and all respondents (though only just for all practitioners without the non-practitioner researchers). However, the women succeeded in agreeing excellent scale content validity (≥ 0.90) for a scale with a considerable number of items – more items (but lower S-CVIAV-UA) for the women practitioners alone. In Phase 2, excellent S-CVIAV-UA was attained by the whole group (men and women together), but not if non-practitioner researchers were excluded. For women on their own, excellent scale content validity was only attained if the respondent who scored all items as zero was excluded. Scale content validity again eluded the men. Inter-rater reliability Randolph’s inter-rater reliability (κfree) was very low, in Phase 1 being highest (0.56) for ‘Pain’, and in Phase 2 for ‘Wellbeing’ (0.33), although even these are considerably less than the 0.7 benchmark for kappa acceptability. However, if respondents who scored no items as ‘essential’ were excluded, κfree did exceed 0.7 for ‘Wellbeing’.

Tables 25 and 26 show those items with greatest and least κfree. Table 25. Phase 1 items with high and low multi-rater kappa (κfree) ‘top five’ items ‘bottom five’ items Pain Relaxation Melting Excitement Calmness

Optimism Being in control Harmony Contentment Being spaced out etc. (3 ties)

Table 26. Phase 2 items with high and low multi-rater kappa (κfree)

‘top three’ items ‘bottom three’ items Tiredness (all gps) Wellbeing (not for men) Self-confidence (not for men, or for no non-researchers)

Happiness (all gps) Being stressed (all gps) Self-awareness (not for all, or for men)

Note: In Phase 1, with 48 items, the ‘top/bottom five’ were selected, but in Phase 2, with only 18 items, only the ‘top/bottom three’. The first two high κfree items in tables 25 and 26 are those for which both Lawshe’s and Lynn’s methods showed good content validity. They also appear in tables 23 and 24, together with ‘Calmness’ and ‘Self-Confidence’, which also achieved high κfree (the latter for women only). There was less agreement between the κfree ratings and the respondent scores shown in tables 6 and 7 for the other items with highest and lowest κfree. Reducing questionnaire length from 32 to 20 items If the requirement for S-CVIAV-UA to be ≥ 0.90 is relaxed, but that for I-CVI to be ≥ 0.78 retained, then in Phase 1 subsets of at least seven items fulfilled these more lenient requirements for men and women alone, and for all respondents taken together (in Phase 2, subsets of at least four items fulfilled requirements for all subgroups of women alone and of all respondents taken together, but none for men). However, in each Phase there was agreement among these cohorts on only four items. Table 27 shows those items with I-CVI > 0.8 for which there was agreement among All respondent cohorts in both surveys. Table 27. Items with I-CVI > 0.8 for which there was agreement among All respondent cohorts in both surveys. Phase 1 survey (48 items) Phase 2 survey (18 items) Being at ease Being able to cope Calmnessa Sleep quality Coolness Tiredness Heaviness Wellbeing Mental energy (Self-confidence) Mental focusa Paina Relaxationa Sleepiness a. Agreement among all cohorts, whether for women or men, or All respondents (both together).

Tables 23, 24, 27, and 28 (Phase 1 respondent reservations about particular items – i.e. their ambiguity, synonymy or negativity), taken in conjunction with the results of the analysis of actual questionnaire usage [10], allow a reduction of the original 32-item questionnaires to more manageable 20-item instruments. The list finally arrived at, designated the EXP-20, is shown in table 29. A description of how it was compiled, and the actual questionnaires that result, is given below. A similar list was compiled to assess longer term changes after a series of treatments (table 30). Table 28. Phase 1 items considered for omission from the EXPre and EXPost questionnaires. Items in bold occur in two columns. Numbers indicate ‘essential’ scores in table 4. Too ambiguous Synonym available (in parentheses) Too negative/positive Being in control (4) Cheerfulness (3) Comfort (7) Connectedness with others (3) Contentment (6) Excitement (0) Expansion (0) Harmony (5) Inner bodily flow (2) Receptivity (3) Relief (10) Satisfaction (7)

Aliveness (4) (Physical vitality, 8) Being at ease (8) /Calmness (10) / Peacefulness (7) (Relaxation ,12) Being in control (Empowerment ,6) Being spaced out (6) (Floating, 3) Inner bodily flow (Vibration, 3) Looseness (1) (Suppleness, 4) Optimism (4) (Positivity)

Being blue (3) Melting (1) Mental energy (5) Nervousness (4) Positivity (6) Restlessness (3) Tension (9) Worry (4)

Description of compiling 20-item (EXP-20) questionnaires [a] High-scoring items When the original 32-item EXP questionnaires were used in Pilot studies, CPD training and student groups, the following items were the ‘top five’ for which change was most expected or experienced (Mayor/Steffert, 2013): EXPRe (expected): Aliveness, Calmness, Inner bodily awareness, Inner bodily flow, Relaxation, Tension, Tingling (items with high κfree being those in italics, together with Being at ease and Warmth or coolness). EXPost (experienced): Aliveness, Being at ease, Calmness, Mental focus, Relaxation, Tingling (the ‘most noticed’ items being those in bold, together with Heaviness, Pain, Tension and Warmth or coolness). Respondents in the Phase 1 validity survey commented on a number of these. For example, ‘Aliveness’ was considered by some respondents in the first survey as overlapping with Physical vitality (it was also considered as too abstract by one expert market researcher who was consulted). However, given that the I-CVI for Physical vitality only exceeded that for Aliveness when respondents rating no items as ‘essential’ were excluded, and that it was frequently rated as ‘Don’t know’ in the EXPost questionnaire, the somewhat vaguer (less specific) term (Aliveness) was considered preferable to the more restricted one (Physical vitality). An alternative suggested by one respondent to the first survey was Energetic, but again this is less ‘subtle’ in its connotations [33].

‘Being at ease’ relates to states such as Calmness, Comfort, Contentment and Relaxation (http://thesaurus.com/browse/at+ease), and so was considered as redundant by some respondents to the first survey. It could therefore be dropped. ‘Calmness’ relates to Peacefulness, but not so directly to Relaxation (http://thesaurus.com/browse/calmness), so that both terms can be retained without redundancy. ‘Inner bodily flow’ was considered as ambiguous and also (by one respondent) as synonymous with Vibration (see too below, under Low-scoring items). However, provided that the questionnaires continue to be used only with acupuncture (or other CAM) practitioners and students, for whom the 3-word term Inner bodily flow may well be meaningful, it could be retained. For other populations, it could perhaps be replaced with Inner bodily awareness. ‘Mental focus’ was thought to be probably meaningless to the average patient, but could be included for the same reasons as Inner bodily flow. ‘Tension’ was considered as overly negative or possibly indicative of an adverse effect if (an increase was) experienced. However, a change in tension could be a positive outcome. Furthermore, Tension is a possible antonym of Calmness, (Being at) ease, Looseness and Relaxation, and is related to (and often used together with) Stress (http://thesaurus.com/browse/tension). If might therefore be useful to retain Tension, or to replace it with Stress. ‘Warmth or coolness’ is clearly ambiguous, and should be replaced with a single term. One respondent suggested Temperature, but in normal usage this is rather more objective than subjective. An alternative – if the questionnaires are restructured to include questions about increase/decrease rather than just change – is to use Warmth alone. The other ‘top five’ items received no comments in the Phase 1 content validity survey. [b] Low-scoring items At the other end of the scale, the following were the ‘bottom five’ of the items for which change was most expected or experienced: EXPRe (expected): Being blue*, Being in control, Connectedness to others, Hunger* and Worry* (asterisked items were also those in our initial study [10] for which change had been least expected, as was Nervousness) EXPost (experienced): Being blue*, Hunger*, Intestinal rumblings*, Suppleness and Worry* (asterisked items were also those in the initial study [10] for which change was least experienced, as was Restlessness). In the first content validity survey, Hunger and Intestinal rumblings also received a low rating. In addition, respondents commented that Connectedness to others was ambiguous (as also Excitement).

Other items receiving a low rating in the two surveys were Expansion, Looseness, Melting, Pulsation, Self-confidence and Self-esteem. In the first survey, the ‘bottom five’ with low multi-rater κfree were Optimism, Being in Control, Harmony, Contentment, Being spaced out, Receptivity and Satisfaction, and from the Phase 2 survey, those items listed in table 26 above. Taking all these factors into consideration, two possible 20-item questionnaires emerge, one for use following a single treatment, to assess changes expected and then experienced during or immediately following the session, and one for use following a course of treatments, to assess longer term changes that may not be immediately apparent (tables 29 and 30). This distinction is somewhat arbitrary, but was considered potentially useful enough for further study. Table 29. EXP-20 questionnaire items, designed to assess changes during or immediately after a single treatment. Item number

Short term nonspecific feelings

1 Aliveness 2 Being spaced out 3 Being stressed 4 Calmness 5 Cheerfulness 6 Clarity 7 Heaviness 8 Inner bodily flow 9 Intestinal rumblings 10 Mental energy 11 Mental focus 12 Pain 13 Relaxation 14 Relief 15 Sensory acuity 16 Sleepiness 17 Tension 18 Tingling 19 Warmth 20 Wellbeing For the short term nonspecific feelings, two other possible items were considered but not included – Numbness and Optimism. Table 30. EXP-20 questionnaire items, designed to assess longer term changes after a series of treatments. Item number

Longer term nonspecific effects

1 Appetite 2 Being in control 3 Being stressed 4 Calmness 5 Happiness 6 Heat 7 Mental energy

8 Mobility 9 Needing medication 10 Pain 11 Physical vitality 12 Positivity 13 Relaxation 14 Relief 15 Self-awareness 16 Self-confidence 17 Sleep quality 18 Tension 19 Tiredness 20 Wellbeing For the longer term effects, Being in control was selected rather than Empowerment, and Self-confidence rather than Self-esteem, both because they were less reminiscent of therapeutic jargon. On the other hand, Positivity was chosen rather than the less general (but more familiar) term Optimism. DISCUSSION Achieving total consensus on content validity is difficult, particularly as the number of experts and items increases [30]. For 32-item questionnaires such as EXPre and EXPost, based on reporting highly subjective feelings that are in many cases difficult to define, it would be virtually impossible to obtain an acceptable S-CVI using standard methods such as Lawshe’s or Lynn’s with 17 respondents. Here, Lawshe’s method was found to be particularly insensitive. However, by adopting the modified outcome measure S-CVIAV-UA for Lynn’s method, it was possible to obtain acceptable content validity for a subset of items, although only for the women respondents. It seems that men had more difficulties in allotting ‘essential’ scores to the questionnaire items, rating more as ‘not necessary’ (table 8). Nonetheless, based on content validity analysis and the results of analysing actual questionnaire usage, a reduction of the original 32-item questionnaires to more manageable 20-item instruments has been achieved. These revised questionnaires were circulated to the survey respondents for further feedback, and are now being tested on EA training courses. It will be interesting to see whether results continue to differ for men and women, and also whether these questionnaires suggest that different feelings are elicited by different types of acupuncture (manual/electroacupuncture, verum/sham, traditional/medical), other CAM modalities (qigong, yoga, reiki, etc.) or even other, unrelated activities (walking, choral singing, listening to poetry). Limitations In this survey respondents were simply asked to rate items for their inclusion in ‘a list of the nonspecific effects of acupuncture’, but without any definition being offered of the word ‘nonspecific’ in this context. A discussion of this problem and how different stakeholders may understand the term will be published elsewhere.

CONCLUSIONS A modified version of Lynn’s content validity assessment method was found to give more usable results than either of the conventional methods (Lawshe’s or Lynn’s). With this new approach, 32-item questionnaires on expectation and experience of nonspecific feelings were shortened to more manageable 20-item instruments. In general, CV was greater among women than among men. Acknowledgments To Tony Steffert for creating web-based versions of these questionnaires; to all respondents for their willingness to take part in a survey at very short notice, and to Stephen Birch in particular for engaging in discussion on the meaning of ‘nonspecific effects’; to Louise Percival, market research consultant, for her considerable input on questionnaire design; and of course to the acupuncturists and other health practitioners who gave up considerable amounts of their time to take part in our study on the effects of electroacupuncture on the EEG and heart rate variability [32], and those acupuncture students and practitioners who completed questionnaires as part of their learning about electroacupuncture. Authors All research by the author Financial support None Competing interests/Funding None REFERENCES 1. Peters D, ed. Understanding the Placebo Effect in Complementary Medicine: Theory, practice and research. London: Churchill Livingstone, 2001 2. Caspi O, Bootzin RR. Evaluating how placebos produce change. Logical and causal traps and understanding cognitive explanatory mechanisms. Eval Health Prof. 2002;25,4:436-464 3. Walach H, Jonas WB. Placebo research: the evidence base for harnessing self-healing capacities. J Altern Complement Med. 2004;10 Suppl 1:S103-S112 4. Pacheco-López G, Engler H, Niemi MB et al. Expectations and associations that heal: immunomodulatory placebo effects and its neurobiology. Brain Behav Immun. 2006;20,5:430-446 5. Geers AL, Wellman JA, Fowler SL et al. Placebo expectations and the detection of somatic information. J Behav Med. 2011;34,3:208-217 6. Salih N, Bäumler PI, Simang M et al. Deqi sensations without cutaneous sensory input: results of an RCT. BMC Complement Altern Med. 2010;10:81 7. Kerr CE, Shaw JR, Conboy LA et al. Placebo acupuncture as a form of ritual touch healing: a neurophenomenological model. Conscious Cogn. 2011;20,3:784-791

8. Warber SL, Cornelio D, Straughn J et al. Biofield energy healing from the inside. J Altern Complement Med. 2004;10,6:1107-1113 9. Mayor D. Elemental souls and vernacular qi: some atributes of what moves us. In: Mayor D, Micozzi MS, eds. Energy Medicine East and West: A natural history of qi. Edinburgh: Churchill Livingstone, 2011:24-47 10. Mayor D, Steffert T. Expectation and experience of ‘nonspecific’ feelings elicited by acupuncture: Developing and piloting a set of questionnaires. http://www.qeeg.co.uk/electroacupuncture/eaq1.htm, 2013 11. Dennehy EB, Webb A, Suppes T. Assessment of beliefs in the effectiveness of acupuncture for treatment of psychiatric symptoms. J Altern Complement Med. 2002;8,4:421-425 12. Chae Y, Kim SY, Park HS. Experimentally manipulating perceptions regarding acupuncture elicits different responses to the identical acupuncture stimulation. Physiol Behav. 2008;95,3:515-520 13. Mao JJ, Armstrong K, Farrar JT et al. Acupuncture expectancy scale: development and preliminary validation in China. Explore. 2007;3,4:372-377 14. Mao JJ, Xie SX, Bowman MA. Uncovering the expectancy effect: the validation of the acupuncture expectancy scale. Altern Ther Health Med. 2010;16,6:22-27 15. Kim YJ, Lee IS, Kim HS et al. Validation of the Korean version of the Acupuncture Expectancy Scale. Acupunct Med. 2013 Oct 8. doi: 10.1136/acupmed-2013-010412 16. Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000;31,2:73-86 17. Vincent CA, Richardson PH, Black JJ et al. The significance of needle placement site in acupuncture. J Psychosom Res. 1989;33,4:489-496 18. Park H, Park J, Lee H et al. Does Deqi (needle sensation) exist? Am J Chin Med. 2002;30,1:45-50 19. Park J, Park H, Lee H et al. Deqi sensation between the acupuncture-experienced and the naïve: a Korean study II. Am J Chin Med. 2005;33,2:329-337 20. Kim Y, Park J, Lee H et al. Content validity of an acupuncture sensation questionnaire. J Altern Complement Med. 2008;14,8:957-963 21. White P, Bishop F, Hardy H et al. Southampton needle sensation questionnaire: development and validation of a measure to gauge acupuncture needle sensation. J Altern Complement Med. 2008;14,4:373-379 22. Pach D, Hohmann C, Lüdtke R et al. German translation of the Southampton Needle Sensation Questionnaire: use in an experimental acupuncture study. Forsch Komplementmed. 2011;18,6:321-326 23. Kong J, Fufa DT, Gerber AJ et al. Psychophysical outcomes from a randomized pilot study of manual, electro, and sham acupuncture treatment on experimentally induced thermal pain. J Pain. 2005;6,1:55-64 24. Kong J, Gollub R, Huang T et al. Acupuncture de qi, from qualitative history to quantitative measurement. J Altern Complement Med. 2007;13,10:1059-1070 25. Yu DT, Jones AY, Pang MY. Development and validation of the Chinese version of the Massachusetts General Hospital Acupuncture Sensation Scale: an exploratory and methodological study. Acupunct Med. 2012;30,3:214-221 26. Blasche G, Marktl W, Eisenwort B et al. The treatment experience questionnaire: development and validation of a questionnaire assessing the individual's emotional, perceptual, and cognitive reactions to alternative, physical, and dental treatments. Forsch Komplementmed. 2013;20,3:205-212 27. Lawshe CH. A quantitative approach to content validity. Personnel Psychology. 1975;28,4:563-575 28. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35,6:382-385

29. Wilson FR, Pan W, Schumsky DA. Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development. 2012;45,3:197-210 30. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30,4:459-467 31. Randolph JJ. 2008. Online Kappa Calculator. http://justus.randolph.name/kappa, 2008 [accessed 16 Dec 2011] 32. Mayor DF, Steffert TC. A Study on the effects of electroacupuncture and/or transcutaneous electrical nerve stimulation on the EEG and Heart Rate Variability. http://www.qeeg.co.uk/electroacupuncture, 2013 33. Ots T. 1994 The silenced body – the expressive Leib: on the dialectic of mind and life in Chinese cathartic healing. In: Csordas TJ, ed. Embodiment and Experience: The existential ground of culture and self. Cambridge: Cambridge University Press, Cambridge, 1994:116-136 Wording of the EXP-20 (feelings) questionnaires

If you would like to use these questionnaires, please contact the author before doing so. EXP-20 © David Mayor and Louise Percival 2013

EA-EEG Study Expectation of feelings questionnaire (pilot)

This questionnaire should only take you about 5 minutes to complete. Please complete it in full, without taking a break. Your personal ID is Your date of birth .................................. Please sign below to confirm that you understand that this survey is for research purposes only and that your responses and identity will be kept in the strictest confidence.

A00

Signature ........................................................... Today’s date ..................................

Instructions Yes No

There are twenty FEELINGS listed on the following page which we would like you to consider carefully. We would like to know if you expect any of these feelings to increase or decrease for you personally in response to the standardised EA or TEAS treatment that you will receive today. This will be at points such as LI-4 (Hegu) and ST-36 (Zusanli). Please remember there are no right or wrong answers. Consider each feeling in isolation and answer as honestly and accurately as possible according to your own personal expectations.

For each feeling, circle the word or abbreviation that best represents your expectation of change in the feeling in your own case.

Do make sure that you put your circle in the correct row and column.

The Feelings follow on the next page.

Relative to how you feel NOW, during or immediately following EA/TEAS do you expect to experience any change AT ALL in the feeling of …

Feeling Expect to experience a change

Not expect to experience a change

Don’t know / can’t say

If Yes, expect feeling to INCREASE

If Yes, expect feeling to DECREASE

Aliveness Yes No DK/CS Inc Dec Being spaced out Yes No DK/CS Inc Dec Being stressed Yes No DK/CS Inc Dec Calmness Yes No DK/CS Inc Dec Cheerfulness Yes No DK/CS Inc Dec Clarity Yes No DK/CS Inc Dec Heaviness Yes No DK/CS Inc Dec

First, please tell us if you have received EA or TEAS before

Inner bodily flow Yes No DK/CS Inc Dec Intestinal rumblings

Yes No DK/CS Inc Dec

Mental energy Yes No DK/CS Inc Dec Mental focus Yes No DK/CS Inc Dec Pain Yes No DK/CS Inc Dec Relaxation Yes No DK/CS Inc Dec Relief Yes No DK/CS Inc Dec Sensory acuity Yes No DK/CS Inc Dec Sleepiness Yes No DK/CS Inc Dec Tension Yes No DK/CS Inc Dec Tingling Yes No DK/CS Inc Dec Warmth Yes No DK/CS Inc Dec Wellbeing Yes No DK/CS Inc Dec

Do you have anything else you would like to add?

EA-EEG Study Feelings follow-up questionnaire (pilot)

Completing this questionnaire should take you about 7 minutes. Please complete it in full, without taking a break. Your personal ID is 1. Please confirm that you are willing for the data from this questionnaire to be used for our research purposes by ticking the box below

(tick box)

2. Now record here the acupuncture points and electrical parameters used when you received treatment. If possible, include:

A00

Points stimulated ............................... Frequency ........................................... Amplitude ........................................... Pulse duration ..................................... Duration of electrical stimulation .... Name of device used ......................... and any problems you found ...............................................................

3. There are twenty FEELINGS listed on the following page which we would like you to consider carefully. We would like to know if you experienced any changes in these feelings in response to the standardised EA or TEAS treatment that you received today.

Please remember there are no right or wrong answers. Consider each feeling in isolation and answer as honestly and accurately as possible according to your own personal experience. For each feeling, circle the word or abbreviation that best represents what you experienced in your own case. Do make sure that you put your circle in the correct row and column.

Please also asterisk (*) those ‘Yes’ changes you noticed most!

The Feelings follow on the next page. Relative to how you felt when you completed the earlier questionnaire, during or immediately following EA/TEAS did you experience a change in the feeling of …

Feeling Experienced a change

Not experienced a change

Don’t know / can’t say

If Yes, experienced INCREASE in feeling

If Yes, experienced DECREASE in feeling

Aliveness Yes No DK/CS Inc Dec Being spaced out Yes No DK/CS Inc Dec Being stressed Yes No DK/CS Inc Dec Calmness Yes No DK/CS Inc Dec Cheerfulness Yes No DK/CS Inc Dec Clarity Yes No DK/CS Inc Dec Heaviness Yes No DK/CS Inc Dec

Inner bodily flow Yes No DK/CS Inc Dec Intestinal rumblings

Yes No DK/CS Inc Dec

Mental energy Yes No DK/CS Inc Dec Mental focus Yes No DK/CS Inc Dec Pain Yes No DK/CS Inc Dec Relaxation Yes No DK/CS Inc Dec Relief Yes No DK/CS Inc Dec Sensory acuity Yes No DK/CS Inc Dec Sleepiness Yes No DK/CS Inc Dec Tension Yes No DK/CS Inc Dec Tingling Yes No DK/CS Inc Dec Warmth Yes No DK/CS Inc Dec Wellbeing Yes No DK/CS Inc Dec

Don’t forget to asterisk (*) those ‘Yes’ changes you noticed most!

If you have any comments you would like to make, you can include them here: