Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma

Orlaith Cormican; Maura Dowling

doi:10.12688/hrbopenres.12863.3

Home Browse Using nominal group technique to compare patients’ and clinicians’...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma

[version 3; peer review: 1 approved, 2 approved with reservations]

Previously titled: Using nominal group technique to inform the development of a self-management app for patients with relapsed myeloma

Orlaith Cormican^1,2, Maura Dowling²

PUBLISHED 21 Oct 2024

Author details Author details

¹ Cancer Clinical Trials Unit, HRB Clinical Research Facility, University of Galway, Galway, Ireland
² School of Nursing and Midwifery, University of Galway, Galway, Ireland

Orlaith Cormican
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Investigation, Methodology, Project Administration, Resources, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Maura Dowling
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background

The nominal group technique (NGT) allows stakeholders to directly generate items for needs assessment. The objective was to demonstrate the use of NGT to inform the development of a healthcare app in patients with relapsed myeloma. Healthcare professionals with experience in the care of patients with relapsed/refractory myeloma were invited to participate.

Methods

One NGT group was conducted. In the group, health care professionals working in haematology were asked to vote anonymously in order of highest priority, on symptoms previously highlighted by relapsed/refractory myeloma patients in four focus groups.

Results

A total of 18 healthcare professionals working in the area of haematology participated in the NGT discussion; consultants (n=6), haematology registrars (n=2), specialist nurses [Advanced Nurse Practitioner/Clinical Nurse Specialist] (haematology) (n=3), staff nurse (n=1), and “other” health care professionals (n=6). Participants ranged in experience of working with myeloma patients from 2 years to over 27 years. The symptoms voted in highest priority were: Pain, Fatigue, Peripheral Neuropathy, Infection Risk and Steroid Induced Side Effects.

Conclusions

The NGT was an efficient method for obtaining information to inform a healthcare app.

Keywords

Nominal Group Technique, Relapsed Myeloma, Self Management, Patient Priorities, Healthcare app, Chronic illness management

Corresponding author: Orlaith Cormican

Competing interests: No competing interests were disclosed.

Grant information: Health Research Board Ireland [PDG-2016-2].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2024 Cormican O and Dowling M. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Cormican O and Dowling M. Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma [version 3; peer review: 1 approved, 2 approved with reservations]. HRB Open Res 2024, 1:21 (https://doi.org/10.12688/hrbopenres.12863.3) First published: 31 Aug 2018, 1:21 (https://doi.org/10.12688/hrbopenres.12863.1) Latest published: 21 Oct 2024, 1:21 (https://doi.org/10.12688/hrbopenres.12863.3)

Revised Amendments from Version 2

The changes to the manuscript relate to the feedback received from the review Debra Howell.
Clarity has been added in relation to the purpose of the assessment tool and how the symptoms were to be prioritised - ie in terms of what had the most impact on a patients quality of life and their ability to manage them
In the discussion section - we have included a recommendation for future work and a limitation.

See the authors' detailed response to the review by Debra Howell
See the authors' detailed response to the review by Peter Cudd
See the authors' detailed response to the review by Angela Stover

Introduction

Patient participation and engagement in research for the development of health research priorities has increased in importance more recently. The Irish health care system has supported this through the development of patient and public involvement in research. Involving patients is particularly important as their views may often differ from health care professionals and current evidence based practice, and their involvement can provide valuable insights. In addition, patient involvement provides opportunities for increased awareness of research trends, and communicating issues of importance to patients and their families in relation to their needs¹. Moreover, patient and health care professional partnerships in research can improve the patient’s health and can benefit the healthcare system generally.

In the study discussed here, the population were patients with relapsed multiple myeloma (MM). MM accounts for 2% of all cancers and is the 2^nd most common hematological malignancy². MM is a B cell malignancy resulting in uncontrolled production of plasma cells. These plasma cells overproduce immunoglobulins (heavy and light-chain M-proteins) known as a paraprotein, which build up in the bone marrow and often enter adjacent bone. The cause of MM is unknown; however it is known that the production of paraprotein in MM is associated with specific chromosomal abnormalities (deletions or translations). Almost all patients with MM will relapse, with each remission duration reducing as the number of treatment regimens increase³. Patients with relapsed or refractory disease are defined as those who, having achieved minor response or better, relapse and then progress while on salvage therapy or experience progression within 60 days of their last therapy⁴.

Patients with MM are living longer resulting in its treatment as a chronic disease rather than a terminal illness. Chronic diseases are life changing and increasingly patients with MM are living with long term symptoms as a result of treatment toxicities and side effects. Symptoms have been acknowledged as a hindrance to quality of life as well as having significant effects on emotional status, activity and participation in life⁵.

In order for MM to be managed as a chronic condition patients need to be treated holistically, addressing not only their diagnosis but the symptoms and toxicities associated with treatment⁶. MM patients also need to be empowered to take control of their condition enabling them to take control of their health⁷. Evidence suggests that by helping patients to self-manage, the risk of adverse events and hospital admissions are reduced and patients’ quality of life is improved⁸.

Treatment-specific clinically focused assessment tools can be useful for optimising therapy and implementing supportive care strategies (e.g., growth factors, transfusion support, intravenous hydration, bisphosphonates, antiviral therapies) to manage treatment-related symptoms. Patient education opportunities also arise when patients are assessed resulting in a heightened awareness and encouraging prompt reporting of symptoms and side effects. When education is approached efficiently it can significantly reduce the severity of symptoms and improve overall quality of life⁹.

We report here on phase 2 (nominal group technique (NGT)) of a study aimed to develop a symptom management tool for patients with relapsed myeloma to help people manage their own symptoms and to give patients the opportunity to highlight and discuss them with clinical staff. The NGT approach was used to determine a consensus on the top 5 symptoms to include in the development of a symptom management tool. The top 5 symptoms were to be ranked in terms of the impact they had on the patient’s quality of life and their ability to manage them. In the context of consensus methods, experts are those individuals who have knowledge about the topic of concern and it is a useful method to explore stakeholder views¹⁰. In this case the experts were the healthcare professionals involved in the care of these patients.

Modified NGT is a consensus planning tool that helps to prioritise issues¹¹. This technique was devised by Delbecq and Vandeven (1971)¹² and was chosen as the basis of our consensus as it preserved anonymity as well as fostering an exchange of opinions among multiple stakeholders in a non-threatening environment where balanced individual opinions are prioritised through a democratic method¹². In addition, the nominal group technique allows a comparison to be made between the healthcare consumer (i.e. the patient) and healthcare professionals but also ensures patient centred healthcare. This approach has proven popular in seeking patients’ involvement^13,14.

While NGT is essentially a qualitative technique, its product is usually presented quantitatively by ranking the items produced by that group¹⁵.

Advantages of the NGT include; ensuring dominant participants avoid control in a group setting, avoiding “quick” decision making, enabling the measurement of the importance of ideas/items to individuals and ensuring group cohesiveness¹⁵. While NGT is capable of advancing knowledge in areas where conventional research evidence is lacking, it should not be viewed as a substitute for scientific methods of proven validity¹⁵.

In recent years, the lack of definitive studies on the effectiveness and appropriateness of health care interventions and policies has led to the increasing use of consensus methods¹⁵. By including healthcare professionals in this exercise, a multi stakeholder approach is gained and this leads to the co production of health services research between patients and healthcare professionals, addressing the gap that often exists. It is also known that medical tools developed with clinician input have greater clinical application and relevance¹⁶.

Methods

Ethical considerations

Ethical approval was sought and granted by the Research Ethics Committee in National University of Ireland Galway (Approval Number 16-May-05). All health care professionals agreed to partake by consenting to complete the voting card anonymously (Supplementary File 1).

The nominal group technique employed in this paper was part of a 3 phase project for the development of a symptom management tool for patients with relapsed myeloma. Phase 1 involved focus groups with patients and their carers, phase 2 is discussed in this paper and phase 3 led to the development and pilot of a symptom management tool for patients with relapsed/refractory myeloma¹⁷.

Previous focus group – phase 1

We conducted four focus groups with relapsed multiple myeloma patients in various locations across Ireland (Table 1). Some carers also attended the focus groups with their relatives. The findings of these focus groups are discussed elsewhere¹⁸. We chose this method for phase 1 as we wanted to gain a description of the lived experience of symptom burden for this population and what self care strategies patients were already using.

Table 1. Patient demographic characteristics of focus groups.

Population of relapsed multiple myeloma patients who participated in Phase 1 of the Research Project.

Participant (Myeloma patients) Number: 15
Age
18–45 years	0
46–55 years	1
56–65 years	8
66–80 years	6
80 + years	0
Gender
Male	9
Female	6
Length of Diagnosis
< 2 years	0
2–5 years	7
5–10 years	2
>10 years	5
Unknown	1
Marital Status:
Single (never married)	1
Married	11
Separated/Divorced	0
Widowed	2
Unknown	1
Highest Educational Level
Completed Primary School	0
Completed Secondary School	6
Attended Some Secondary School	5
Completed Third Level Education	4
Employment Status
Employed	2
Unemployed	2
Retired	11
Supportive Measures Used
Emotional/Psychological Support	1
Financial Support	2
Transportation Services	1
Support Groups	4
Internet Forums	2
Complementary Therapies	1
Charity Funded Organisations	2
Cancer Nurse Line	1
Nothing	9
Number of patients currently on treatment: 9
Number of patients who know what treatment they are on: 6
Number of patients who know how many lines of treatment they have received: 8

At the focus groups, participants were asked to discuss their experience of common symptoms and adverse events and how these impacted on their quality of life. Patients’ experience of symptom management and what improvements could be made to symptom management were also explored as well as asking patients for their opinions on a self-management tool¹⁸. All interviews were audio-recorded and transcribed and then analysed using thematic analysis, revealing 12 common symptoms within the patient experience. We did not explore what the top 5 symptoms of priority were for patients at this time.

Modified NGT – phase 2

Modified NGT was subsequently employed, with a panel of health care professionals attending a local haematology seminar. The seminar takes place every quarter and health care professionals working within the area of haematology and with experience of caring for MM patients attend. Using NGT, we wished to identify the top 5 priority symptoms to be included in a symptom management tool. Therefore participants at the seminar were asked to vote (using anonymous voting cards) in order of priority^1–5, what they deemed to be of highest priority symptoms for attention in the care of patients with relapsed myeloma (Supplementary File 1). Due to time and the feasibility of the project we chose to include only the 5 most prioritised symptoms that were considered burdensome from a patient and healthcare professional viewpoint.

Recruitment

Recruitment of health care professionals was via communication channels of a regional haematology group who meet at set times over a calendar year. The chair of the haematology seminar informs health care professionals about the upcoming meetings and agenda via a group email. Following discussion with the chair, an email was sent to all members describing the research being undertaken, its aims and expected outcomes. Members were also informed about the intention to present phase 1 of this research project at the upcoming haematology seminar and the plan to conduct the modified nominal group technique following same. It was emphasised that partaking in the exercise would be voluntary and consent would be obtained by agreeing to fill out the voting card. See Supplementary File 1.

Initially one researcher presented the findings at the haematology seminar from phase 1 of the research project “Symptom management through self-management: Improving the outcomes of patients with relapsed myeloma.” This presentation included the background, progress and findings of phase 1 of the project as well as an explanation of the overall aim of the project, i.e. to develop a symptom management tool for patients. In addition the symptoms that were ranked as burdensome and explored in detail by patients were presented to the healthcare professionals (12 in total, Table 2).

Table 2. Votes from the modified group technique.

Symptoms/Side Effects	Number of Votes	Top 5 Highest Priority
Steroid Induced Side Effects	11	5
Anxiety	5
Depression	4
Fatigue	15	2
Peripheral Neuropathy	13	3
Infection Risk	12	4
Cytopenia	5
Pain	16	1
Dental Issues	1
Muscle Cramps	0
Decreased Mobility	5
GI Issues	3

Health care professionals consented to participate in the NGT by way of agreeing to complete the questionnaire as well as the inclusion of a consent statement, to be completed before proceeding with the voting. A total of 18 health care professionals agreed to partake in the study.

Voting cards were distributed by the 2^nd author post-presentation and participants were asked to vote anonymously on what they perceived to be the five most burdensome symptoms for patients (Supplementary File 1).

Generally the Nominal Group technique involves a process of: “(i) silent generation of ideas by each individual, (ii) round-robin recording of ideas, (iii) structured and time-limited discussion of ideas, and (iv) selection and ranking of ideas (voting)” In this case the generation of ideas had already been obtained from the results of the focus group with patients and we did not seek engagement from the participants in a round robin recording of ideas¹⁹. Following the presentation on the findings from phase 1 of the project, a group discussion took place on the meaning of the findings and then health care professionals were asked to vote about what they believed to be of most importance. Our end goal was to prioritise the symptoms based on data which had already been completed therefore we did not conduct the silent generation of ideas by individuals or the round robin recording of ideas.

Results

In total, 18 health care professionals participated in the modified NGT; 6 Consultants, 2 haematology registrars, 3 specialist nurses [Advanced Nurse Practitioner/Clinical Nurse Specialist] (haematology), 1 staff nurse and 6 “other” health care professionals. Participants ranged in experience of working with myeloma patients from 2 years to greater than 27 years.

Our aim was to include the top 5 priorities for inclusion in the app; therefore the results of the voting cards were reviewed to determine which symptoms received the most votes and the ranking of the votes. Of the 12 symptoms identified by patients in the focus groups, pain was the top symptom voted by the HC professionals as essential for inclusion in a patient app. The symptom voted second was fatigue with peripheral neuropathy, infection risk and steroid induced side effects following as the symptoms that were of highest priority in the management of patients with relapsed or refractory multiple myeloma (Table 2).

Discussion

This modified nominal group process provided useful information from health care professionals for the purposes of gaining consensus on what they perceived to be symptom priority for relapsed myeloma patients. Interestingly, there were similar findings between patients and health care professionals in relation to the most challenging symptoms experienced.

Fatigue and steroid induced side effects were considered two of the symptoms of highest priority for inclusion in the symptom management app. Fatigue is described in the literature as one of the most common distressing symptoms in MM. Associated causes include anaemia, pain, mood disturbance, diminished strength, diminished endurance, decreased sleep efficiency at night and advanced disease. It has also been reported that fatigue has the greatest negative effect on physical functioning in those with a diagnosis of MM. Despite this ongoing symptom burden, treatment remains episodic and self-management programmes have been limited for those on lifelong treatment²⁰. Patients have already described some self-care strategies used such as resting times, distraction techniques “mind over matter”, exercise and “walking off the fatigue.”¹⁸

Steroid induced side effects are often acknowledged in the development of fatigue as a long term side effect in patients with relapsed myeloma. Despite the significant impact steroids have on neuropsychiatric function and health related quality of life, it remains a useful therapy in the control of MM through improving overall patient response as well as progression free survival. Their mechanism is complex; however they are thought to modify the body’s immune system and produce powerful anti-inflammatory properties. In contrast to their ability to help in the control of MM they are known to cause intense adverse effects including elevated blood sugars, increased risk of infection, mood swings and insomnia, which are often long term and not widely described or documented despite the common use of steroids²¹. Phase 1 of our study iterated these adverse events with patients describing hyperactivity, the “let down” effect and excessive diaphoresis as having a significant impact on their health related quality of life¹⁸. Patients were very aware of the associated side effects of steroids and reported the side effects as out of control at times. However it has been noted that patients appreciate the impact steroids can have on their disease response so they often fail to report side effects affecting their health related quality of life¹⁸. As such, it is of paramount importance that health care professionals educate patients and their caregivers on the importance of reporting adverse effects and thus improve patient outcomes. Education and patient involvement could improve the longer term outcomes of relapsed MM patients on steroids¹⁸.

The technique that we used involved a modification of the NGT. Generally there are at least 5–6 steps involved in the process; however we chose a two-step process which included two different groups of participants (patients and health care professionals). Using this technique ensured that individuals did not feel pressure to conform based on perceived status within the group, with all health care professionals voting anonymously. In addition it allowed the stakeholders to prioritise the symptoms through a democratic process²².

This approach has however a number of limitations. Focus group discussions with health care professionals would have provided a more comprehensive picture of symptom management for relapsed MM patients. Using just the modified NGT was a very blunt approach which provided conclusions without context. Further data would most likely have been generated if we had not modified the NGT process. The sample included within this study was small (12 participants in total) which introduces limited generalisability and caution is advised when interpreting results.

Nominal groups often request participants to record ideas independently and in private, then sharing, listing and discussing the ideas, and finally judging or voting on the ideas independently. It is usual for participants to be asked to identify issues before attending the NGT session, however in this case the issues were already identified¹³.

Conclusion

NGT offers opportunity to involve both patients and health care professionals to be involved in healthcare improvement and design. Research prioritisation now needs to focus on both patient and health care professional viewpoints to ensure a comprehensive approach to care. NGT offers potential in gaining an insight to health care professionals’ priorities for patients and addressing the gap that is often described between health care professionals and patient values. The findings in this work provide opportunity to address the gap that often is described between patient and health care professional values. Future work could focus on identifying the top five symptoms priorities for patients to see if they correlated with the healthcare professionals findings.

Data availability

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Figshare: Supplementary_File_1 Voting Card

https://doi.org/10.6084/m9.figshare.13954283.v1²³

This project contains the following underlying data:

- Supplementary_File_1 Voting Card.docx (Using NGT, we wished to identify the top 5 priority symptoms as voted by health care professionals to be included in a symptom management tool for patients with relapsed/refractory myeloma. Participants at the seminar were asked to vote (using anonymous voting cards) in order of 1–5 , what they deemed to be of highest priority symptoms for attention in the care of patients with relapsed myeloma (Supplementary File 1).)

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

Faculty Opinions recommended

References

1. The Medical Research Charities Group: Patient and Public Involvement in Health research in Ireland? A Review of progressto date. Dublin, Ireland. 2016. Reference Source
2. Bianchi G, Anderson KC: Understanding biology to tackle the disease: Multiple myeloma from bench to bedside, and back. CA Cancer J Clin. 2014; 64(6): 422–444. PubMed Abstract | Publisher Full Text
3. Rajkumar SV: Multiple myeloma: 2016 update on diagnosis, risk-stratification, and management. Am J Hematol. 2016; 91(7): 719–34. PubMed Abstract | Publisher Full Text | Free Full Text
4. Richardson PG, Sonneveld P, Schuster MW, et al.: Safety and efficacy of bortezomib in high-risk and elderly patients with relapsed multiple myeloma. Br J Haematol. 2007; 137(5): 429–35. PubMed Abstract | Publisher Full Text
5. Osborne TR, Ramsenthaler C, de Wolf-Linder S, et al.: Understanding what matters most to people with multiple myeloma: a qualitative study of views on quality of life. BMC Cancer. 2014; 14: 496. PubMed Abstract | Publisher Full Text | Free Full Text
6. Molassiotis A, Wilson B, Blair S, et al.: Unmet supportive care needs, psychological well-being and quality of life in patients living with multiple myeloma and their partners. Psychooncology. 2011; 20(1): 88–97. PubMed Abstract | Publisher Full Text
7. Ryhänen AM, Rankinen S, Siekkinen M, et al.: The impact of an empowering Internet-based Breast Cancer Patient Pathway program on breast cancer patients’ clinical outcomes: a randomised controlled trial. J Clin Nurs. 2013; 22(7–8): 1016–25. PubMed Abstract | Publisher Full Text
8. Koch T, Jenkin P, Kralik D: Chronic illness self-management: Locating the ’self’. J Adv Nurs. 2004; 48(5): 484–92. 2004. PubMed Abstract | Publisher Full Text
9. Colson K: Treatment-related symptom management in patients with multiple myeloma: a review. Support Care Cancer. 2015; 23(5): 1431–45. PubMed Abstract | Publisher Full Text
10. McMillan SS, King M, Tully MP: How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3): 655–62. PubMed Abstract | Publisher Full Text | Free Full Text
11. Rankin NM, McGregor D, Butow PN, et al.: Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science. BMC Med Res Methodol. 2016; 16(1): 110. PubMed Abstract | Publisher Full Text | Free Full Text
12. Delbecq AL, Van De Ven AH: A Group Process Model for Problem Identification and Program Planning. J Appl Behav Sci. 1971; 7(4): 466–92. Publisher Full Text
13. Corner J, Wright D, Hopkinson J, et al.: The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer. 2007; 96(6): 875–81. PubMed Abstract | Publisher Full Text | Free Full Text
14. Howell M, Tong A, Wong G, et al.: Important outcomes for kidney transplant recipients: a nominal group and qualitative study. Am J Kidney Dis. 2012; 60(2): 186–96. PubMed Abstract | Publisher Full Text
15. Cantrill AJ, Sibbald B, Buetow S: The Delphi and nominal group techniques in health services research. Int J Pharm Pract. 1996; 4: 67–74. Publisher Full Text
16. Lehoux P, Miller FA, Daudelin G, et al.: Providing value to new health technology: the early contribution of entrepreneurs, investors, and regulatory agencies. Int J Health Policy Manag. 2017; 6(9): 509–18. PubMed Abstract | Publisher Full Text | Free Full Text
17. Cormican O, Dowling M: Evaluating a digital self-management tool for people living with multiple myeloma. Cancer Nursing Practice. 2020. Reference Source
18. Cormican O, Dowling M: Living with relapsed myeloma: Symptoms and self-care strategies. J Clin Nurs. 2018; 27(7–8): 1713–21. PubMed Abstract | Publisher Full Text | Free Full Text
19. Olsen J: The Nominal Group Technique (NGT) as a Tool for Facilitating Pan-Disability Focus Groups and as a New Method for Quantifying Changes in Qualitative Data. Int J Qual Methods 2019. Publisher Full Text
20. Kurtin S: Living with Multiple Myeloma: A Continuum-Based Approach to Cancer Survivorship. Semin Oncol Nurs. 2017; 33(3): 348–61. PubMed Abstract | Publisher Full Text
21. King T, Faiman B: Steroid-Associated Side Effects: A Symptom Management Update on Multiple Myeloma Treatment. Clin J Oncol Nurs. 2017; 21(2): 240–9. PubMed Abstract | Publisher Full Text
22. Dobbie A, Rhodes M, Tysinger JW, et al.: Using a modified nominal group technique as a curriculum evaluation tool. Fam Med. 2004; 36(6): 402–6. PubMed Abstract
23. Cormican O, Dowling M: Supplementary_File_1 Voting Card.docx. figshare. Figure. 2021. http://www.doi.org/10.6084/m9.figshare.13954283.v1

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 31 Aug 2018

Author details Author details

¹ Cancer Clinical Trials Unit, HRB Clinical Research Facility, University of Galway, Galway, Ireland
² School of Nursing and Midwifery, University of Galway, Galway, Ireland

Maura Dowling
Roles: Conceptualization, Formal Analysis, Funding Acquisition, Methodology, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Health Research Board Ireland [PDG-2016-2].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (3)

version 3

Revised

Published: 21 Oct 2024, 1:21

https://doi.org/10.12688/hrbopenres.12863.3

version 2

Revised

Published: 16 Feb 2021, 1:21

https://doi.org/10.12688/hrbopenres.12863.2

version 1

Published: 31 Aug 2018, 1:21

https://doi.org/10.12688/hrbopenres.12863.1

© 2024 Cormican O and Dowling M. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

VIEWS

981

downloads

Citations

SEE MORE DETAILS

CITE

how to cite this article

Cormican O and Dowling M. Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma [version 3; peer review: 1 approved, 2 approved with reservations]. HRB Open Res 2024, 1:21 (https://doi.org/10.12688/hrbopenres.12863.3)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 16 Feb 2021

Revised

Views

Reviewer Report 03 Sep 2024

Debra Howell, University of York, York, UK

Approved with Reservations

https://doi.org/10.21956/hrbopenres.14348.r41786

Thank you for asking me to review this revised paper. The subject is of interest. Comments are as follows:
It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.

I wasn't sure when data were collected as there is no date.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: 25 years examining blood cancer patient experiences, including symptoms in the context of myeloma.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 17 Oct 2024

Orlaith Cormican, National University of Ireland, Galway, Ireland

17 Oct 2024

Author Response

Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool ... Continue reading Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.
The purpose of the assessment tool was to help people manage their own symptoms and to highlight or discuss them with staff. I will make the necessary changes to the article so there is clarity.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.
The symptoms were to be prioritised in terms of what had the most impact on patients quality of life and their ability to manage them.

I wasn't sure when data were collected as there is no date.
The research was conducted in 2018.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.
The participants in the NGT were clinicians - their characteristics are described under the results section.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.
The 12 symptoms initially included were described by the patients in the focus groups. We did not explore what was of the highest priority for the patients after they were identified by the clinicians but this could be considered in future work going forward.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.
This is correct - I will include this as a limitation.

Many thanks,
Orlaith Cormican.
Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.
The purpose of the assessment tool was to help people manage their own symptoms and to highlight or discuss them with staff. I will make the necessary changes to the article so there is clarity.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.
The symptoms were to be prioritised in terms of what had the most impact on patients quality of life and their ability to manage them.

I wasn't sure when data were collected as there is no date.
The research was conducted in 2018.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.
The participants in the NGT were clinicians - their characteristics are described under the results section.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.
The 12 symptoms initially included were described by the patients in the focus groups. We did not explore what was of the highest priority for the patients after they were identified by the clinicians but this could be considered in future work going forward.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.
This is correct - I will include this as a limitation.

Many thanks,
Orlaith Cormican.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 17 Oct 2024

Orlaith Cormican, National University of Ireland, Galway, Ireland

17 Oct 2024

Author Response

Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool ... Continue reading Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.
The purpose of the assessment tool was to help people manage their own symptoms and to highlight or discuss them with staff. I will make the necessary changes to the article so there is clarity.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.
The symptoms were to be prioritised in terms of what had the most impact on patients quality of life and their ability to manage them.

I wasn't sure when data were collected as there is no date.
The research was conducted in 2018.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.
The participants in the NGT were clinicians - their characteristics are described under the results section.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.
The 12 symptoms initially included were described by the patients in the focus groups. We did not explore what was of the highest priority for the patients after they were identified by the clinicians but this could be considered in future work going forward.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.
This is correct - I will include this as a limitation.

Many thanks,
Orlaith Cormican.
Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.
The purpose of the assessment tool was to help people manage their own symptoms and to highlight or discuss them with staff. I will make the necessary changes to the article so there is clarity.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.
The symptoms were to be prioritised in terms of what had the most impact on patients quality of life and their ability to manage them.

I wasn't sure when data were collected as there is no date.
The research was conducted in 2018.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.
The participants in the NGT were clinicians - their characteristics are described under the results section.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.
The 12 symptoms initially included were described by the patients in the focus groups. We did not explore what was of the highest priority for the patients after they were identified by the clinicians but this could be considered in future work going forward.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.
This is correct - I will include this as a limitation.

Many thanks,
Orlaith Cormican.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 15 Mar 2021

Angela Stover, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, USA; Department of Health Policy and Management, University of North Carolina at Chapel Hill, Chapel Hill, USA

Approved

https://doi.org/10.21956/hrbopenres.14348.r28915

Thank you for revising your manuscript. It is clear ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 31 Aug 2018

Views

Reviewer Report 16 Nov 2020

Not Approved

https://doi.org/10.21956/hrbopenres.13930.r28231

This is an observational study where 18 healthcare professionals anonymously voted once on their top 5 MM symptoms. MM is an area ripe for interventions to detect and manage symptoms given the high symptom burden.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process.

Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important.

The paper could be tightened and strengthened throughout.

The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded.

Previous focus group: what were the 12 common symptoms from the patient experience?

Modified NGT. Please explain what the modification is and why it is needed.

Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study.

Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms.

Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper.

Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

No

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Patient reported outcome (PRO) development, symptom science, mixed methods, oncology, implementation science.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Author Response 16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

16 Feb 2021

Author Response
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.
- Title is misleading. Paper is not about development
... Continue reading
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process. We have changed the title and are hopeful that this reflects what the article contains.

Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important. More information has been added to the introduction which will hopefully clarify this.

The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded. We have added to the introduction to include more information about the Nominal Group technique and have taken your suggestion about the paragraph on page 4 and moved it.

Previous focus group: what were the 12 common symptoms from the patient experience? These are listed in Table 2.

Modified NGT. Please explain what the modification is and why it is needed. Same has been added.

Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study. Justification now added.

Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms. The methods are not called a focus group. The focus group was phase 1 of a larger project which led to phase 2 of this project. I have included further clarification in the methods section.

Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper. We thought it was important to include table 1 to give background to the project and the patients that partook in phase 1 that led on to phase 2; that is the modified NGT.

Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion. We have included a header which should make this table clearer.
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process. We have changed the title and are hopeful that this reflects what the article contains.

Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important. More information has been added to the introduction which will hopefully clarify this.

The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded. We have added to the introduction to include more information about the Nominal Group technique and have taken your suggestion about the paragraph on page 4 and moved it.

Previous focus group: what were the 12 common symptoms from the patient experience? These are listed in Table 2.

Modified NGT. Please explain what the modification is and why it is needed. Same has been added.

Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study. Justification now added.

Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms. The methods are not called a focus group. The focus group was phase 1 of a larger project which led to phase 2 of this project. I have included further clarification in the methods section.

Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper. We thought it was important to include table 1 to give background to the project and the patients that partook in phase 1 that led on to phase 2; that is the modified NGT.

Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion. We have included a header which should make this table clearer.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

16 Feb 2021

Author Response
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.
- Title is misleading. Paper is not about development
... Continue reading
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process. We have changed the title and are hopeful that this reflects what the article contains.

Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important. More information has been added to the introduction which will hopefully clarify this.

The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded. We have added to the introduction to include more information about the Nominal Group technique and have taken your suggestion about the paragraph on page 4 and moved it.

Previous focus group: what were the 12 common symptoms from the patient experience? These are listed in Table 2.

Modified NGT. Please explain what the modification is and why it is needed. Same has been added.

Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study. Justification now added.

Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms. The methods are not called a focus group. The focus group was phase 1 of a larger project which led to phase 2 of this project. I have included further clarification in the methods section.

Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper. We thought it was important to include table 1 to give background to the project and the patients that partook in phase 1 that led on to phase 2; that is the modified NGT.

Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion. We have included a header which should make this table clearer.
Thank you for your comments and taking the time to review our article. Please see below our comments in bold.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process. We have changed the title and are hopeful that this reflects what the article contains.

Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important. More information has been added to the introduction which will hopefully clarify this.

The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded. We have added to the introduction to include more information about the Nominal Group technique and have taken your suggestion about the paragraph on page 4 and moved it.

Previous focus group: what were the 12 common symptoms from the patient experience? These are listed in Table 2.

Modified NGT. Please explain what the modification is and why it is needed. Same has been added.

Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study. Justification now added.

Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms. The methods are not called a focus group. The focus group was phase 1 of a larger project which led to phase 2 of this project. I have included further clarification in the methods section.

Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper. We thought it was important to include table 1 to give background to the project and the patients that partook in phase 1 that led on to phase 2; that is the modified NGT.

Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion. We have included a header which should make this table clearer.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 16 May 2019

Peter Cudd, CATCH, University of Sheffield, Sheffield, UK

Approved with Reservations

https://doi.org/10.21956/hrbopenres.13930.r26415

The article addresses an important need to involve users – patients and professionals of proposed technology in its specification – in this case a proposed app for better management of symptoms of relapsed myeloma. There are a number of issues that would improve the article, those that are essential are asterisked.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].*

The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).*

The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*

In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*

I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers).

It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Digital technology and service innovation through user participation, Adult persistent fatigue (across diseases).

CITE

Report a concern

Author Response 16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

16 Feb 2021

Author Response
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.
- The Nominal Group technique appears to be extensively
... Continue reading
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].* We have added further clarification to same which better reflects this.

The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).* We have included detail about recruitment. We believe that the sample is homogenous given that the health care professionals all work in the same area, i.e haematology. Regarding your comment “is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views” The results in Table 2 reflect how health care professionals voted, therefore we don’t consider this an assumption. Regarding your comment on bias: We have included further information regarding bias in terms of decision making, however the sample was chosen at random and we did not know how many doctors would be recruited over nursing staff.

The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? We have included the explanation for this at the end of the introduction. It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*More clarification has been added about why the top 5 symptoms were to be included in the tool. I don’t think it is necessary to include further detail about phase 3 of the project.

In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). We have removed some information from the conclusion so that the focus remains on the NGT, If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*More information added to the conclusion

I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers). All care was taken to reduce bias in phase 1 of the study and this is discussed elsewhere. I don’t see how bias is an issue here.

It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it. This is discussed elsewhere.
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].* We have added further clarification to same which better reflects this.

The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).* We have included detail about recruitment. We believe that the sample is homogenous given that the health care professionals all work in the same area, i.e haematology. Regarding your comment “is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views” The results in Table 2 reflect how health care professionals voted, therefore we don’t consider this an assumption. Regarding your comment on bias: We have included further information regarding bias in terms of decision making, however the sample was chosen at random and we did not know how many doctors would be recruited over nursing staff.

The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? We have included the explanation for this at the end of the introduction. It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*More clarification has been added about why the top 5 symptoms were to be included in the tool. I don’t think it is necessary to include further detail about phase 3 of the project.

In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). We have removed some information from the conclusion so that the focus remains on the NGT, If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*More information added to the conclusion

I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers). All care was taken to reduce bias in phase 1 of the study and this is discussed elsewhere. I don’t see how bias is an issue here.

It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it. This is discussed elsewhere.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

16 Feb 2021

Author Response
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.
- The Nominal Group technique appears to be extensively
... Continue reading
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].* We have added further clarification to same which better reflects this.

The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).* We have included detail about recruitment. We believe that the sample is homogenous given that the health care professionals all work in the same area, i.e haematology. Regarding your comment “is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views” The results in Table 2 reflect how health care professionals voted, therefore we don’t consider this an assumption. Regarding your comment on bias: We have included further information regarding bias in terms of decision making, however the sample was chosen at random and we did not know how many doctors would be recruited over nursing staff.

The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? We have included the explanation for this at the end of the introduction. It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*More clarification has been added about why the top 5 symptoms were to be included in the tool. I don’t think it is necessary to include further detail about phase 3 of the project.

In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). We have removed some information from the conclusion so that the focus remains on the NGT, If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*More information added to the conclusion

I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers). All care was taken to reduce bias in phase 1 of the study and this is discussed elsewhere. I don’t see how bias is an issue here.

It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it. This is discussed elsewhere.
Thank you for your comments and taking the time to review our article. Please see our responses below in bold.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].* We have added further clarification to same which better reflects this.

The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).* We have included detail about recruitment. We believe that the sample is homogenous given that the health care professionals all work in the same area, i.e haematology. Regarding your comment “is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views” The results in Table 2 reflect how health care professionals voted, therefore we don’t consider this an assumption. Regarding your comment on bias: We have included further information regarding bias in terms of decision making, however the sample was chosen at random and we did not know how many doctors would be recruited over nursing staff.

The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? We have included the explanation for this at the end of the introduction. It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*More clarification has been added about why the top 5 symptoms were to be included in the tool. I don’t think it is necessary to include further detail about phase 3 of the project.

In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). We have removed some information from the conclusion so that the focus remains on the NGT, If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*More information added to the conclusion

I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers). All care was taken to reduce bias in phase 1 of the study and this is discussed elsewhere. I don’t see how bias is an issue here.

It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it. This is discussed elsewhere.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 31 Aug 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 3 (revision) 21 Oct 24
Version 2 (revision) 16 Feb 21		read	read
Version 1 31 Aug 18	read	read

Peter Cudd, University of Sheffield, Sheffield, UK
Angela Stover, Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, USA; University of North Carolina at Chapel Hill, Chapel Hill, USA
Debra Howell, University of York, York, UK

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

7 Views

03 Sep 2024 | for Version 2

Debra Howell, University of York, York, UK

7 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

25 years examining blood cancer patient experiences, including symptoms in the context of myeloma.

Respond to this report

Responses (1)

Author Response

17 Oct 2024

Orlaith Cormican, National University of Ireland, Galway, Ireland

Hi Debra,
Thank you for completing this review.
Please see comments in relation to your review (included in italics)

It was unclear if the purpose of the assessment tool was to help people manage their own symptoms (would this be possible for some of the issues), or so they can highlight/discuss them with clinical staff.
The purpose of the assessment tool was to help people manage their own symptoms and to highlight or discuss them with staff. I will make the necessary changes to the article so there is clarity.

Regarding 'symptom priority', was this in terms of the issues that were most serious, or that had most impact on quality of life for patients, or were most inconvenient? This was unclear and needs to be explained.
The symptoms were to be prioritised in terms of what had the most impact on patients quality of life and their ability to manage them.

I wasn't sure when data were collected as there is no date.
The research was conducted in 2018.

The characteristics of patients from the focus groups are shown in Table 1, not participants in the NGT. For this paper, I would have liked to know if they were representative of the population of myeloma patients.
The participants in the NGT were clinicians - their characteristics are described under the results section.

I would have like to know if the 5 priorities given by the clinicians matched those of patients? Surely that information is relevant and would need further exploration for the app. This is linked to the definition of 'symptom priority' - as noted above.
The 12 symptoms initially included were described by the patients in the focus groups. We did not explore what was of the highest priority for the patients after they were identified by the clinicians but this could be considered in future work going forward.

The sample size for what is essentially a ranking exercise is very small, which raises issues of generalisability.
This is correct - I will include this as a limitation.

Many thanks,
Orlaith Cormican.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

20 Views

15 Mar 2021 | for Version 2

20 Views Cite this report Responses(0)

Approved

Thank you for revising your manuscript. It is clear now and flows well. Reviewer concerns have been addressed.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Patient reported outcome (PRO) development, symptom science, mixed methods, oncology, implementation science.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

28 Views

16 Nov 2020 | for Version 1

28 Views Cite this report Responses(1)

Not Approved

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

No

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Patient reported outcome (PRO) development, symptom science, mixed methods, oncology, implementation science.

Respond to this report

Responses (1)

Author Response

16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

Thank you for your comments and taking the time to review our article. Please see below our comments in bold.

Title is misleading. Paper is not about development of a self-management app, but asking healthcare professionals for their perception of what the top 5 MM symptoms are. It sounds like it is step one of a long development process. We have changed the title and are hopeful that this reflects what the article contains.
Introduction starts with the importance of involving patients in generating symptom domains, but the patient focus group was published elsewhere. The current paper is about the clinician perspective so the intro would make more sense with that focus. Another option is to reframe the paper as using NGT to compare patient and clinician perspectives on MM symptoms to include in an app. The intro would still need to include why the clinician perspective is important. More information has been added to the introduction which will hopefully clarify this.
The introduction would be strengthened by explaining what the Nominal Group Technique is and why it is relevant here. The conclusion in the abstract is that NGT was useful so setting up the paper with a discussion of NGT is warranted. The NGT paragraph on pg. 4 could be moved to the intro and expanded. We have added to the introduction to include more information about the Nominal Group technique and have taken your suggestion about the paragraph on page 4 and moved it.
Previous focus group: what were the 12 common symptoms from the patient experience? These are listed in Table 2.
Modified NGT. Please explain what the modification is and why it is needed. Same has been added.
Please provide a justification for why clinicians were asked to generate only 5 symptoms. MM has many possible symptoms. Patients generated 12 symptoms in a previous study. Justification now added.
Calling the methods a focus group is misleading throughout. Clinicians individually voted (anonymously) on their top 5 symptoms. The methods are not called a focus group. The focus group was phase 1 of a larger project which led to phase 2 of this project. I have included further clarification in the methods section.
Table 1 includes the patient characteristics... but this study is not about the patient perspective unless you reframe the paper. We thought it was important to include table 1 to give background to the project and the patients that partook in phase 1 that led on to phase 2; that is the modified NGT.
Is Table 2 from the patient or clinician perspective? It is labeled as both in the text. Labeling the column headers better would help reduce confusion. We have included a header which should make this table clearer.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

29 Views

16 May 2019 | for Version 1

Peter Cudd, CATCH, University of Sheffield, Sheffield, UK

29 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Digital technology and service innovation through user participation, Adult persistent fatigue (across diseases).

Respond to this report

Responses (1)

Author Response

16 Feb 2021

Orlaith Cormican, National University of Ireland, Galway, Ireland

Thank you for your comments and taking the time to review our article. Please see our responses below in bold.

The Nominal Group technique appears to be extensively used in myeloma research, the cited references and introduction could better reflect this. For example the closing paragraph of the Introduction could state that ‘the approach was to determine a consensus using the widely used NGT [refs here].* We have added further clarification to same which better reflects this.
The study design is one amongst many possibilities and some justification is given for its selection. However the authors failed to describe the where and how of the recruitment of the professionals for replication and also no consideration is provided for how homogeneous the context is, i.e. is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views. The latter becomes important for interpretation of findings given the bias in recruits of doctors over nursing staff in their sample (which also isn’t discussed).* We have included detail about recruitment. We believe that the sample is homogenous given that the health care professionals all work in the same area, i.e haematology. Regarding your comment “is it reasonable to assume there is widely shared symptoms/issues of concern, that staff in one service is going to have similar profiles of views, that different types of professionals will have similar views” The results in Table 2 reflect how health care professionals voted, therefore we don’t consider this an assumption. Regarding your comment on bias: We have included further information regarding bias in terms of decision making, however the sample was chosen at random and we did not know how many doctors would be recruited over nursing staff.
The authors have found that the method was useful but several questions arise that it would have explained. Why patients and then service staff in the two phases, why not the other way round, or if able to do both have some involved in both phases (possibly separated) and possibly something that would strengthen the findings to different recruits in the implied 4 groups in the latter case? We have included the explanation for this at the end of the introduction. It may be convenience and/or feasibility with project resources or indeed the intended use of the list of the top 5.* Just what value can the reader place on the particular findings they have – so that some judgement can be made about applying the technique elsewhere. The authors have given a vague description of what is next – but they should make clear how what they have done here fits in with that.*More clarification has been added about why the top 5 symptoms were to be included in the tool. I don’t think it is necessary to include further detail about phase 3 of the project.
In the Methods section we are led to believe that the ‘Modified NGT – phase 2’ is the focus of the article. But when you read the conclusions you find that you could make the inference that Phase 1 is part of the NGT. The comments about patients and professionals are able to take part using this technique is generally true and the opening sentences are statements that could be made from other articles from the field (and yet they are not cited to support this conclusion). We have removed some information from the conclusion so that the focus remains on the NGT, If these were added then a more appropriate positioning of the findings of the authors work could be made by starting a sentence to follow which states the findings in this study confirm user participation.*More information added to the conclusion
I suggest the final sentence of the Conclusions is moderated somewhat concerning the gap, e.g. it is possible for NGT approach to make a particular contribution to the gap between professional and patient values, that contribution very much depends on how it is implemented and biases of those involved (including the researchers). All care was taken to reduce bias in phase 1 of the study and this is discussed elsewhere. I don’t see how bias is an issue here.
It may be useful for the authors to consider that the patients will be the ones who will be asked to routinely adhere to using the app and yet the professionals have to believe in it to prescribe/recommend it. This is discussed elsewhere.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. The Medical Research Charities Group: Patient and Public Involvement in Health research in Ireland? A Review of progressto date. Dublin, Ireland. 2016. Reference Source

[2] 2. Bianchi G, Anderson KC: Understanding biology to tackle the disease: Multiple myeloma from bench to bedside, and back. CA Cancer J Clin. 2014; 64(6): 422–444. PubMed Abstract | Publisher Full Text

[3] 3. Rajkumar SV: Multiple myeloma: 2016 update on diagnosis, risk-stratification, and management. Am J Hematol. 2016; 91(7): 719–34. PubMed Abstract | Publisher Full Text | Free Full Text

[4] 4. Richardson PG, Sonneveld P, Schuster MW, et al.: Safety and efficacy of bortezomib in high-risk and elderly patients with relapsed multiple myeloma. Br J Haematol. 2007; 137(5): 429–35. PubMed Abstract | Publisher Full Text

[5] 5. Osborne TR, Ramsenthaler C, de Wolf-Linder S, et al.: Understanding what matters most to people with multiple myeloma: a qualitative study of views on quality of life. BMC Cancer. 2014; 14: 496. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Molassiotis A, Wilson B, Blair S, et al.: Unmet supportive care needs, psychological well-being and quality of life in patients living with multiple myeloma and their partners. Psychooncology. 2011; 20(1): 88–97. PubMed Abstract | Publisher Full Text

[7] 7. Ryhänen AM, Rankinen S, Siekkinen M, et al.: The impact of an empowering Internet-based Breast Cancer Patient Pathway program on breast cancer patients’ clinical outcomes: a randomised controlled trial. J Clin Nurs. 2013; 22(7–8): 1016–25. PubMed Abstract | Publisher Full Text

[8] 8. Koch T, Jenkin P, Kralik D: Chronic illness self-management: Locating the ’self’. J Adv Nurs. 2004; 48(5): 484–92. 2004. PubMed Abstract | Publisher Full Text

[9] 9. Colson K: Treatment-related symptom management in patients with multiple myeloma: a review. Support Care Cancer. 2015; 23(5): 1431–45. PubMed Abstract | Publisher Full Text

[10] 10. McMillan SS, King M, Tully MP: How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016; 38(3): 655–62. PubMed Abstract | Publisher Full Text | Free Full Text

[11] 11. Rankin NM, McGregor D, Butow PN, et al.: Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science. BMC Med Res Methodol. 2016; 16(1): 110. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Delbecq AL, Van De Ven AH: A Group Process Model for Problem Identification and Program Planning. J Appl Behav Sci. 1971; 7(4): 466–92. Publisher Full Text

[13] 13. Corner J, Wright D, Hopkinson J, et al.: The research priorities of patients attending UK cancer treatment centres: findings from a modified nominal group study. Br J Cancer. 2007; 96(6): 875–81. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Howell M, Tong A, Wong G, et al.: Important outcomes for kidney transplant recipients: a nominal group and qualitative study. Am J Kidney Dis. 2012; 60(2): 186–96. PubMed Abstract | Publisher Full Text

[15] 15. Cantrill AJ, Sibbald B, Buetow S: The Delphi and nominal group techniques in health services research. Int J Pharm Pract. 1996; 4: 67–74. Publisher Full Text

[16] 16. Lehoux P, Miller FA, Daudelin G, et al.: Providing value to new health technology: the early contribution of entrepreneurs, investors, and regulatory agencies. Int J Health Policy Manag. 2017; 6(9): 509–18. PubMed Abstract | Publisher Full Text | Free Full Text

[17] 17. Cormican O, Dowling M: Evaluating a digital self-management tool for people living with multiple myeloma. Cancer Nursing Practice. 2020. Reference Source

[18] 18. Cormican O, Dowling M: Living with relapsed myeloma: Symptoms and self-care strategies. J Clin Nurs. 2018; 27(7–8): 1713–21. PubMed Abstract | Publisher Full Text | Free Full Text

[19] 19. Olsen J: The Nominal Group Technique (NGT) as a Tool for Facilitating Pan-Disability Focus Groups and as a New Method for Quantifying Changes in Qualitative Data. Int J Qual Methods 2019. Publisher Full Text

[20] 20. Kurtin S: Living with Multiple Myeloma: A Continuum-Based Approach to Cancer Survivorship. Semin Oncol Nurs. 2017; 33(3): 348–61. PubMed Abstract | Publisher Full Text

[21] 21. King T, Faiman B: Steroid-Associated Side Effects: A Symptom Management Update on Multiple Myeloma Treatment. Clin J Oncol Nurs. 2017; 21(2): 240–9. PubMed Abstract | Publisher Full Text

[22] 22. Dobbie A, Rhodes M, Tysinger JW, et al.: Using a modified nominal group technique as a curriculum evaluation tool. Fam Med. 2004; 36(6): 402–6. PubMed Abstract

[23] 23. Cormican O, Dowling M: Supplementary_File_1 Voting Card.docx. figshare. Figure. 2021. http://www.doi.org/10.6084/m9.figshare.13954283.v1

Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma

Abstract

Background

Methods

Results

Conclusions

Keywords

Revised Amendments from Version 2

Introduction

Methods

Ethical considerations

Previous focus group – phase 1

Table 1. Patient demographic characteristics of focus groups.

Modified NGT – phase 2

Recruitment

Table 2. Votes from the modified group technique.

Results

Discussion

Conclusion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a HRB-funded researcher?

Thank you!

Using nominal group technique to compare patients’ and clinicians’ perspectives on symptoms in multiple myeloma to inform the development of a self management tool for patients with relapsed myeloma

Abstract

Background

Methods

Results

Conclusions

Keywords

Revised Amendments from Version 2

Introduction

Methods

Ethical considerations

Previous focus group – phase 1

Table 1. Patient demographic characteristics of focus groups.

Modified NGT – phase 2

Recruitment

Table 2. Votes from the modified group technique.

Results

Discussion

Conclusion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a HRB-funded researcher?

Thank you!