Palliative and End-of-Life Care Initiative: Impact Assessment – Report

A report prepared for the CIHR Institute of Cancer Research

Ottawa, October 2009

Highlights and Conclusions

Highlights of the Palliative and End-of-Life Care Initiative

What did the Initiative accomplish?

Engaged 18 partners and focused considerable resources – and national attention – on a critical but previously neglected health need.
Developed teams which have proved to be excellent models of effective integrated knowledge translation in action.
Built major clinical research capacity, creating a community extremely young in research experience but wise in health care practice and decision-making.
Increased both the quantity and quality of PELC research many-fold.
Developed strong and effective partnerships with user communities, including decision-makers and patients.
Is producing results that are being integrated into practice guidelines, health professional training, and policy discussions.

What did we learn?

PELC research is highly applied, practice-oriented, and mostly undertaken by care providers, not academic investigators.
Most PELC researchers are thus outside CIHR's normal sphere of funding or influence, but the NET structure is particularly effective for integrating key users, communities and collaborators from beyond mainstream academia.
PELC trainees are mostly well-established health professionals - not young students - seeking to improve their own clinical, management and policy decision-making.
Teams are a hugely rewarding and effective way to do PELC research, but take enormous time to build trust and make them work – time not valued by employers.
Funders and grantees both need to plan their exit strategy from day one, and start working to build a sustainable community which outlasts the strategic funding cycle.

What are the future opportunities?

Capture and synthesize the research outcomes into a transferable, useable format.
Hold a post-initiative meeting with the PELC community to launch processes to move those outcomes into practice and start planning for its own future.
Create research release-time support and encourage the creation of jobs which support practice-based research clinicians, like PELC trainees, doing integrated KT.
Capture and share lessons-learned from the rich experiences of the NETs in building partnerships, integrating KT and developing community-based research approaches.
Define the desired outcomes of translational research and develop new metrics which assess achievement against such outcomes.

Conclusions about the Palliative and End-of-Life Care Initiative

The Palliative and End-of-Life care (PELC) initiative was developed by the Canadian Institutes of Health Research's Institute of Cancer Research in collaboration with the 18 partners listed below. The objectives of the Initiative were to support infrastructure development, enhance interdisciplinary research collaboration, encourage the development of early career researchers and attract trainees to this emerging area. Since 2004, the PELC initiative has supported nineteen Pilot Projects, ten New Emerging Team Grants (NETs), one Career Transition Award and a Strategic Training Program (STIHR). With a total investment of $16.5 million over six years, the Initiative is the largest research investment in PELC research in the world.

Alberta Cancer Board
British Columbia Cancer Agency
Canadian Breast Cancer Research Alliance
CancerCare Manitoba
CIHR Institute of Aboriginal Peoples' Health
CIHR Institute of Aging
CIHR Institute of Cancer Research
CIHR Institute of Circulatory and Respiratory Health
CIHR Institute of Gender and Health
CIHR Institute of Health Services and Policy Research
CIHR Institute of Human Development, Child & Youth Health
CIHR Institute of Neurosciences, Mental Health & Addiction
CIHR Knowledge Translation Branch
Health Canada
Heart and Stroke Foundation of Canada
National Cancer Institute of Canada
National Ovarian Cancer Association

Impacts on the research agenda

Palliative and end-of-life care (PELC) was receiving limited attention in the research and health care communities when the new CIHR Institute for Cancer Research (ICR) identified it in 2003 as its top priority. Respondents commended the Institute for its courage and leadership in championing PELC research, and making a compelling case that here was an ideal juxtaposition of great health need with significant research opportunity.
The Institute's budget for strategic research is a very small piece of Canada's large and complex cancer research funding; its Board felt that "we needed something that was not just incremental but Big and Bold – as compared to investing in genomics or imaging, where ICR would have little added-value." By investing a substantial portion of its strategic funds into palliative and end-of-life care, the Institute was able to have an enormous impact in this area.
PELC research presents significant unique methodological, logistical and ethical challenges. PELC research involves extremely vulnerable populations and thus needs highly-trained personnel, increasing its cost and complexity. The PELC practice community is itself nascent, and few health care practitioners in PELC have research training . For these reasons – and many more – Canada had only a very small PELC research community. Respondents identified the decision to focus the majority of ICR's resources in this one initiative as both brave and risky. However, the Institute undertook a range of activities and events to maximize the impact of its investment in PELC research: its comprehensive approach to consultation, planning and execution interactions drew high praise from all we interviewed.

Impact on research productivity and quality

Since the launch of the Institute, CIHR's support for PELC research has increased sixty-fold, from less than $100,000 to almost $6 Million, invested in all four theme areas, with a preponderance of clinical research. Some 60% of CIHR funding comes from outside the Initiative. Pilot Projects attracted new researchers to PELC, tripled their productivity, and greatly exceeded expectations when 2/3 obtained follow-on CIHR grants. The Institute has clearly spurred considerable activity beyond its own budgetary limits, and PELC research is growing and competing in CIHR competitions.
As a result, Canada's almost doubled its world share of PELC publications between 2004 and 2009: at 8% it is almost twice Canada's overall world share of health research publications. NET funding has already enormously increased the productivity of the NET PIs: while doubling their productivity, they also moved from authoring 29% to authoring 37% of Canada's rapidly growing body of PELC publications. Overall, Initiative investigators (not just NET PIs) accounted for half of all Canadian PELC publications between 2006 and 2008, and 70% of Canada's increased research productivity.
Citation measures show Canadian PELC papers have significant impact: for example, Canadians are consistently over-represented among the world's top 40 most-highly cited PELC papers.
NET PIs collaborated with 50% more investigators than Canada's average, and twice as many as the general PELC average. The PELC research community, previously isolated, is now well-connected internationally, with international co-authors on almost 40% of Canada's papers.
In Canada, PELC is strongly practice-based and implementation-oriented: "there's virtually no such thing as a PELC researcher who is not also a care provider." A UK study backs up this belief, identifying Canada's PELC research as the most "clinically-oriented" in the world.

Impact on research capacity

The primary objective of this initiative was to create research capacity. Peer reviewers, investigators and decision-maker partners all agree there have been substantive improvements in both the quantity and quality of PELC research in Canada. For example, the number of unique Canadian authors has doubled since the Initiative's launch (from ~540 to ~1090).
The "youth" of this community, in terms of research experience, is remarkable: although the Initiative, as intended, drew some experienced researchers to focus their expertise on PELC issues, the majority of the added PELC capacity comes from new researchers. For example, half of today's top ten most productive Canadian PELC authors weren't yet publishing in any field in 2001-03. The vast majority of researchers today receiving CIHR funding for PELC research were not yet receiving any CIHR operating grants in 2001-03.
Another unanticipated finding is the extent to which the PELC research community existed – and is growing – outside of CIHR's traditional sphere of activity and influence. For example, despite its massive increase in PELC spending, and increase in absolute number of funded PELC researchers, the proportion of Canadian PELC authors receiving CIHR funding has actually declined since 2001 from 40% to 34% (from 218 /540 to 367/1090).
Respondents consistently depict most PELC research as implementation-oriented, undertaken primarily by active health professionals in the course of their clinical duties, and funded largely by internal or local resources. This description is consistent with our finding that the majority of publishing PELC researchers do so without benefit of CIHR support. However, we believe that through the NETs, CIHR is indirectly reaching many more people than official numbers show, and developing the research interests and skills of a wide range of health professionals - collaborators, partners, and trainees - who could not access CIHR funding as investigators but who are a substantial portion of that other 66% of authors. The cancellation of the open team grants competition eliminates a major support of mentoring, pilot funding and collaboration, and may significantly hinder the future productivity of these supposedly "non-CIHR funded" authors.

Impact of NETs and STIHR on training

NETs have proven to be fruitful training ground for students and young investigators. They provide a wide range of contacts and experiences, as well as mentors and research infrastructure, which can leap-frog training or new investigator development and provide added credibility and a competitive edge to job-seeking and grant and award proposals. Trainees all described their NET and STIHR training as "a much bigger experience": "there were enormous differences between my training and what was available to the others [in my cohort], huge benefits for me." However, team work can be detrimental to career development: "I can get further faster by working in this group, but my Chair likes to see my name by itself on papers."
The most striking aspects of the PELC trainees interviewed are their maturity and experience. A significant proportion of the trainees attracted to the NETs and STIHR were practicing health professionals - nurses, social workers, psychologists and others - many with decades of experience and leadership roles within the health system. These trainees seek a very different career path from typical new PhDs and post-docs.
Rather than seeking the first steps of the tenure track, most wish to continue leading change in the health system, ideally splitting time between research and care so as to identify problems and be able to fix them. The PELC trainees have the credibility to engage health care providers and access populations, and the skills and reputation to get the results implemented: integrated Knowledge Translation (KT) in action. However, such positions are almost non-existent in the health care system, nor do salary awards provide protected time for this kind of part-time researcher. As a result, few PELC trainees were taking on posts in which they could put their training to use, that is, where they could both do research, and then implement it in practice.

Impact of NETworking

Investigators lauded the NET approach, agreeing their work was "enhanced by the many perspectives brought to the table. The richness of research ideas and wide range of approaches to problem solving were largely due to this interdisciplinary culture." NET participants agreed that individual operating grants would not have achieved a fraction of the same impact. Achieving true interdisciplinary team work requires trust, which can only be built through an enormous (the NETs would repeat, enormous) time commitment to communicating, especially face-to-face, which teams are uniquely able to fund. The NET structure is particularly well-suited to integrating knowledge users, and supports many important collaborators and user communities who on their own could not access CIHR funds or projects. Sadly, investigators still find these leadership activities count against tenure and promotion, and urge CIHR to do more to influence Universities to value the kinds of research activity that CIHR wants them to pursue.

Impact on KT

Despite a Request for Applications (RFA) which included no KT requirements, all the NETs have substantively engaged user communities in their work, making the NET itself into the primary structure for integrated knowledge transfer. And though some PELC investigators still tend to think of the partner's primary role as end-of-grant dissemination, every partner we spoke with saw their time as best invested at the front-end, strategic phase of the research program. We are confident that the NETs have developed close linkages among investigators, health professionals, managers and policy makers, and user communities, and are investing in on-going knowledge translation to maximize the potential benefits of these innovations. NETs have already been prolific users of CIHR KT support, with over 30 funded KT grants among them.

Impacts on research-supportive infrastructure

CIHR created a dedicated PELC peer review panel, and although researchers agree this panel is extremely important, they are not yet using it in large numbers. Of ongoing concern is the widespread belief that certain kinds of research did not get appropriate review in this panel. Our extensive analysis suggests the panel is functioning well and free of systematic bias. Nonetheless, the persistent concern among both applicants and panel members themselves suggests on-going attention should be paid to panel composition and expertise.
A variety of formal and informal attempts have been made by the Institute of Cancer Research, Health Canada, and the research community to network the PELC community. These efforts seem characterized by a wealth of good intentions but little follow-through by either sponsors or participants. Such efforts require committed leaders, and the nascent NETs had few resources to spare for anything not yielding immediate results. And while Initiative partners encouraged these networking efforts, ultimately there was no institutional support for the necessary resources. There is now a significant appetite for maintaining and enhancing connections across the new PELC community, but some fear the opportunity is already lost.

Maximizing the return-on-investment from the PELC Initiative

The PELC initiative has many impressive achievements to date, but its gains are fragile. The capacity built could be lost to health care practice or better-funded team opportunities and research areas – or just never quite make it through the next step to achieving CIHR competitiveness. The teams and partnerships so painstakingly built can disintegrate without ongoing nurturing. The community is still small and dispersed across Canada, and needs to be better connected to maximize synergy in research and knowledge exchange if potential health outcomes are to be realized. Finally, to have real impact on practice nationally, the outputs of the Initiative as a whole need to be collected and packaged for knowledge users.
A key next step could be an end-of-initiative forum to showcase synthesized results from across all the funded research and plan next steps for the PELC community. Knowledge users and researchers should work together to identify: 1) what was learned; 2) what should be done with it; 3) specific needs for a community-wide knowledge translation and exchange network; and 4) next directions for a PELC research agenda.
The Initiative created a cadre of "research-clinicians" seeking to improve health care in real-time by researching change and implementing what works. Mechanisms are needed to encourage and support integrated researcher/decision-makers (in all applied health areas), while open grants competitions should continue to adapt policies, procedures and review criteria to reward the behaviors CIHR seeks to encourage through interdisciplinary teams and KT. Mentoring and developmental grant approaches could help new researchers in nascent fields make the huge leap from strategic support to open competition. Reinstating the open team grants will be crucial to obtaining the benefits from CIHR's many strategic investments in new emerging teams.

Best practices in strategic initiatives

A clear lesson from many initiatives, and emphasized by many funders with whom we spoke, is that it's unrealistic to expect any field of research to move from small and fragmented to world-class in just five years. Nor is a single 5-year infusion of funding to a single cadre of investigators likely to result in a sustainable, self-renewing community. A successful initiative, therefore, needs to plan for the long term at the outset, and include continuous capacity strengthening and community building activities in throughout and beyond the primary funding cycle. Comparatively small investments in research-enabling activities add huge value.
Academic definitions of worthiness, excellence and success continue to dominate program design and peer review, and these are often incompatible with effective KT and knowledge implementation. Research whose goal is to achieve health impacts must be judged by its ability to produce health impacts – not by its ability to produce academic outputs.
Given the large investments teams entail, it's in the funders' interest to help build better-functioning teams. The NETs' lessons learned, in creating and sustaining multidisciplinary, integrated KT teams, could be captured in a workshop, casebook and/ or training module bringing together recent research findings with the practical experiences of team participants.
We believe large strategic initiatives need an explicit up-front KT strategy and dedicated KT resources. A knowledge broker assigned to a large initiative could expand the reach and enhance the outcomes of an initiative by working across individual teams, partners and KT staff.
Finally, CIHR has been innovative in trying out new tools or models of funding, in new areas of research, with new kinds of investigators and partners, and it is recognized internationally for its novel approaches, a number of which have been emulated by other agencies. There is enormous scope to share these experiences and best practices in more systematic ways.

Integrated KT In Action

VP-NET found misperception and distrust kept disabled patients from accessing effective palliative care. A NET post-doc with an English background developed a theatrical play to highlight issues and opportunities for collaborative problem solving to reduce suffering and untimely death.
VPRN-NET turned their research findings about how skilled physicians effectively and compassionately communicate prognostication information to patients and their families into a series of DVDs called "Breaking Bad News," and are using them to train BC medical students, and physicians around the world.
The Difficult Pain NET has widely shared lessons learned in PELC clinical trials, including: novel approaches to assess study feasibility; low-cost data transfer to a central repository (replacing $100k software); dealing with multiple Research Ethic Boards (REBs); validating methodologies and improving trial reporting.
The Family Caregiving NET is creating a book of advice based on letters written by bereaved caregivers to help other caregivers in similar circumstances.
The Winnipeg Regional Health Authority has provided VP-NET with $5 million to operationalize Dignity Conserving Care, transforming the culture of health care throughout the authority.
A key partner, the Canadian Hospice Palliative Care Association (CHPCA), "takes the research findings and makes them more ready to use." CHPCA incorporates Initiative research findings into its conferences and public events, and synthesizes results into factsheets, websites, press releases and other materials.
The Difficult Pain NET developed the world's first on-line palliative care research methods course, now mandatory or strongly recommended in most palliative medicine residency programs across Canada.
To better talk to patients about where they want to die, a STIHR trainee developed decision-support tools and training which proved so successful in testing with nurses, pharmacists and social workers that the Registered Nurses' Association of Ontario asked the trainee to co-lead the development of decision-support Evidence-Based Best Practice Guidelines, and to sit on their panels to develop end-of-life practice guidelines; develop new criteria for the hospice palliative care exam; and the gerontology exam.
A Cross-Cultural NET investigator developed KT products based on needs expressed by research participants: a booklet to share support-group experiences with new patients (distributed through the cancer agency); an article about prostate cancer misdiagnosis (in GP Review); recommendations to support group funders to improve their services.
The New Interventions NET validated a simple instrument to assess pain in people with limited ability to communicate: participants experienced 95% relief from common catastrophic cancer complications.
The New Interventions NET has established a national collaboration with CPAC and the Quebec Health agency to provide cancer navigators with working tools and training curriculum as part of a Canadian Navigator manual. They are also improving patients' continuity of care with a pilot intervention to increase interprofessional collaboration, particularly among family physicians and nurse navigators.
VPRN NET has developed several web-based tools for physicians, including a web-based risk calculator and set of web-based prognostication tools, based on a database of over 10,000 anonymized palliative care patient records from Canada and the US.
Policy planners and program/service managers are using a NET's survival estimates to inform policy changes around eligibility for palliative benefits plan enrolment and admission criteria to hospices and acute/tertiary palliative care units.
Advance care planning (ACP) recently emerged as a major issue; regions are testing models, provinces are enacting legislation, and the federal government is developing national policy. The Cross-Cultural NET found that the relationships it had fostered with regional policy makers, managers, and clinicians allowed it to quickly develop a collaborative approach to researching and implementing ACP in its partners' organizations

Chapter 1 Rationale for the Palliative and End-of-life Care Initiative

The National Cancer Institute of Canada, the Canadian Association of Provincial Cancer Agencies, Health Canada and the CIHR Institute of Cancer Research undertook a collective priority setting exercise in 2001, which identified a number of key challenges for Canadian health care and research. The Institute of Cancer Research built on this process to finalize its own priorities; the result was, as one respondent said: "a huge shock for the research community when CIHR identified palliative care as its top priority and then invested so much in it"¹. Respondents lauded the Institute for championing PELC research, and making a compelling case that here was an ideal juxtaposition of great health need with significant research opportunity.

“In palliative care, patients almost defy you to do your research.”
Researcher

The Institute identified a number of key areas where change was needed to improve the care and management of patients approaching the end-of-life, to reduce suffering throughout the course of illness and, for families, into bereavement. Both quality and quantity of care were major problems: too little care for many who could benefit from palliative care but did not receive it, and too much care in the form of heroic treatment for those who preferred a less aggressive course. Poor communication and decision-making among patients, families and care providers were strongly associated with poor experiences. Exacerbating these challenges, "the statistics keep telling us that in 10-15 years, we're going to have 20% more deaths with 20% fewer health care providers" (NET decision-maker).
In the Institute's Advisory Board discussions, it became clear that members "were not just interested in funding research excellence – they wanted research that makes a real difference." The Institute's budget was, and still is, a small fraction of Canada's large and complex cancer research funding, so "we needed something that was not just incremental but 'Big and Bold' – as compared to investing in genomics or imaging, where ICR would have little added-value."
Palliative and end-of-life care research clearly fit the bill. In addition to being a major health care need, it was seen as underfunded in Canada, and applications for funding to CIHR and Medical Research Council (MRC) had low success rates in traditional peer reviewed competitions. We heard that young investigators were discouraged from entering this area, while funders' priorities fell elsewhere.

“People usually stop studying patients when they enter the palliative phase: ‘there’s nothing more we can do’”
Researcher

PELC research presents some special challenges. Many considered it unethical to undertake research on this especially-vulnerable dying population. Caregivers are reluctant to "give up" on patients, and may view palliative care as a failure to be avoided at all costs. Patients are often referred too late to provide appropriate care, let alone engage them in research. Clinicians' and investigators' time with the patient is very short, patients are extremely sick and often unable to actively contribute to or complete a study. PELC research requires study designs that address the reality and needs of this population, as well as unusual analysis strategies and extended periods to collect data. One respondent noted "You can't do a 'perfect' randomized controlled trial with a dying population, you can only do an 'as good as I can get' trial," which may not be good enough for reviewers.
Reviewers and CIHR staff agree that demonstrating credibility in CIHR's highly competitive competitions requires that a project already be 40-60% complete at the time of application². Gathering sufficient data for this can take several years in PELC research, and interventions developed in other fields require extensive adaptation and pilot testing before being safe and appropriate to use in this vulnerable population: a particular challenge is incorporating the very different physical, psychosocial and spiritual needs of PELC patients. Many respondents noted the risk that less well-trained research staff could actually cause harm, and thus they employ only trained PELC clinicians in their research, adding a significant cost in time and training. These challenges are primarily associated with patient-oriented PELC research: it's worth noting that some well-established investigators in areas such as sociology or health economics, attracted to PELC by the Initiative, did not find PELC research substantively different.
We have briefly surveyed other national strategies and efforts in PELC (Appendix E) to place ICR's effort in an international context. Although Canada is just one of many countries to identify palliative care as an urgent problem in recent years, it is unique in the scope and scale of its research response: as both the largest and probably the broadest national palliative care research initiative to date, ICR's effort is the envy of PELC researchers worldwide. Common themes emerged from all the national strategies. As many have noted, "Research questions in the areas of supportive, palliative, and end-of-life care usually are complex and require interdisciplinary approaches, yet the research community is relatively small and often is fragmented along professional or other lines³," and all co-ordinated national strategies have therefore advocated the need to build research capacity, particularly among clinicians, and to establish PELC as a respectable new inter-disciplinary, or better, trans-disciplinary research field. The common challenges identified by these national strategies underline the potential benefits to be gained by increasing international collaboration.
This initiative was launched in CIHR's early days, and respondents described the decision to focus the majority of ICR's resources on this one initiative as both brave and risky. Even in hindsight, however, Institute Advisory Board members are still in remarkable agreement that "the rationale for this investment was, and is, compelling."

Chapter 2 The Design and Execution of the Initiative

Research funding

Table 1.1: Initiative Objectives²:
support infrastructure development
enhance interdisciplinary research collaboration
encourage the development of early career investigators
attract trainees to this emerging area

The Institute engaged eight other CIHR Institutes and Branches, Health Canada, three provincial cancer agencies, and four charities⁴ to invest $16.5M over six years in the strategic initiative and related PELC research, ultimately supporting:

10 New Emerging Teams (NETs)
19 Pilot Project grants
1 Career Transition Award (CTA)
1 Interdisciplinary Capacity Enhancement grant (ICE)
1 Strategic Training Initiative in Health Research (STIHR)

The Pilots and the Career Transition were one-year, individual awards; the others were 5-year team awards finishing in 2009⁵.
Partners and knowledge users are extremely supportive of the Institute's processes to select and address PELC as its priority. With the research funding for the Initiative committed, ICR could have adopted the conventional "fund and forget" approach, but instead it sought ways to enhance the impact of the Initiative by providing ongoing assistance to the growing PELC community.

New peer review committee

To enhance support to PELC research outside of Initiative-funded projects, ICR successfully made the case that CIHR create a new, dedicated and permanent operating grant peer review committee (called PLC): no small feat, according to Advisory Board members. Stakeholders ranging from the research community, to Health Canada, to Senator Carstairs⁶, believed such a panel would be critical to sustaining palliative care research. Because of the percentile system CIHR uses to allocate grants in the open competition, the panel provides guaranteed minimum funding for a proportion of PELC applications, so long as they exceed a minimum quality rating⁷, and should give investigators more confidence to apply to CIHR. By securing their "own" committee, PELC applications are not competing directly against more established areas, though applicants are free to select any review committee they wish.
Much to the surprise of those who worked so hard to get the committee created, application numbers to the panel have been consistently low (Fig 1.1), while many applications are being sent to other committees. Of the total 51 open competition PELC operating grants funded by CIHR, only 25 were reviewed by PLC. The remainder were reviewed by 14 other committees, with health services and behavioural science committees predominant. Fourteen funded grants were reviewed by other committees after the PLC was established: this would correspond to about 60-70 applications, compared to 119 reviewed by PLC over the same period. Clearly, many PELC investigators, particularly in health services research, do not regard PLC as the first choice for review of their applications.

“Look at the palliative care committee: they’ve been crying for it for 20 years, and now they’re not using it!”
NET researcher

With respect to overall application pressure, the major NET funding was sufficient to keep a small, nascent community well-occupied, especially as much PELC research is still in the early (and therefore cheaper) phase of pilot and descriptive studies. Many respondents expect application pressure to increase as investigators complete their NET obligations: "NETs aren't done yet, and they are a heck of a lot of work," and move on to more expensive, intervention-oriented phases of their work. Nonetheless, applications to the March 2009 competition were the lowest ever. However, many NETs have been focusing their renewal energies on team, not project funding, first to the cancelled Open Team Grants, and now wherever they can.
Both panel members and investigators expressed concern about the PLC committee's ability to review the full breadth of research submissions it had to review. Areas where both investigators and reviewers felt that inappropriate reviews had occurred were in qualitative research, particularly in less mainstream areas, as well as in complex multi-approach programs of research. Several experienced investigators described applications scoring very poorly with PLC, but ranking at the top when re-submitted to their customary committee. Conversely, others are overjoyed at no longer being bounced from one committee to another, and having a committee which truly understands the challenges of PELC research.

Fig 1.1 Number of applications to PLC: "additional funded" means through priority announcements, etc.

Many concerns relate simply to the current state of CIHR funding and its associated low success rates: if only ~15 applications are submitted, it is unlikely that more than 2 will be funded. Peer review becomes more conservative when funding is tight: one PLC member said "the panel went for shorter, smaller projects with surer outcomes." While many agree that PELC studies need not only preliminary data but in fact formal feasibility studies before being ready to compete for long-term funding, NETs were the major source of such funding and other, institutional, sources are becoming scarce.
Much of the PELC community is young in research experience, and needs to work its way up to CIHR competitiveness. The huge time required to develop a CIHR application, combined with the feeling that in this funding environment it's just "too darned difficult," exacerbated by a fear that applications may not get appropriate review anyway, leads many to think applying to CIHR isn't worth it, and to seek any alternative to CIHR funding. We note that PLC has a high rate of withdrawals (Fig 2.1), i.e. where the applicant signals intent to submit, but never does.
Because of the mixed views about PLC, we conducted an analysis of its review record (Appendix C). We concluded that there is no evidence from the quantitative data that PLC is other than a well-functioning committee with an appropriate membership for its workload, free of any flagrant bias, and providing good advice to applicants. This is not to dismiss the angst of both applicants and reviewers about the committee, which seems to be based on the inherent difficulty of reviewing such a complex multidisciplinary field. There is a "chicken and egg" problem here that may not apply so strongly to traditional disciplinary review committees: the small number of applications to PLC does not justify recruiting a large committee, making it more likely that there will be gaps in the expertise represented round the table, which then further deters applications. The small size of the PELC community and its success at collaborating exacerbates the problem of providing expert review while avoiding conflict of interest.
In summary, the PELC research community is grateful that this panel was created, but many avoid using it. Application pressure over the next 12 months, following the end of the NETs, should show whether it is worth continuing the panel.

Canadian Strategy on Palliative and End-of-Life Care Research Working Group

ICR and Health Canada combined their efforts to create a joint National Working Group on Palliative Care Research, co-chaired by the Institute and Health Canada. This Working Group focused initially on the creation of PLC, and had begun to develop a long-term research strategy. The Health Canada Secretariat provided encouragement and support to the PELC community, including funding CIHR's PLC review panel (still ongoing) and promoting efforts to create a pan-Canadian PELC network. Unfortunately, the Strategy's demise in 2006-7 left the Secretariat unable to provide follow-through support to a network.

Birmingham UK Workshop 2005

The Birmingham International Workshop on Supportive, Palliative, and End-of-Life Care Research brought together investigators and funders from Canada, the UK and the USA. The workshop report⁸ points out the similar challenges facing investigators in the three nations, and provides a series of recommendations for funding to facilitate international collaboration, for focus on a series of high-priority research issues, and for standardized documentation of symptoms and their outcomes. The report ends with a plea for "the agencies that sponsored this meeting, along with other agencies and foundations, to help address these urgent research needs."

“This meeting showed we were really serious about PELC, not just to invest $ but to use institutional influence to take it to an international level.”
IAB member

Participants agreed that the joint Birmingham workshop was a good idea, but didn't result in concrete outcomes: it generated reasonable goals, but there was little organized follow-through, and no infrastructure for starting to collaborate. At the time of the meeting, the two UK teams had just begun, and were not ready to think beyond establishing their national collaborations. The call for the participating funding agencies to facilitate international work remains unanswered, though a follow-up meeting is now being discussed.

Canadian workshop 2007

The second networking event was a meeting of the NET leaders held in November 2007, focussed on documenting research outcomes. There is no formal record of this meeting.

Infrastructure and networking

A joint workshop on infrastructure (meaning a structure for national networking of the dispersed and formerly isolated PELC research community) was held in February 2003, jointly with the Health Canada Secretariat. Participants proposed creating a national network⁹. Subsequently, a business plan was developed but never realised, due to failure to identify a source for the significant funding required to operate such a network. Some of the internet-based facilities envisioned at the workshop do now exist in the Canadian Virtual Hospice¹⁰.
There have been several other attempts to convene the PELC community for network planning, using meetings held in conjunction with other events. For example, the Cross-Cultural NET persuaded the Health Canada Secretariat to support a 2005 workshop to identify common NET KT needs and a network structure to support them¹¹. The Canadian Hospice Palliative Care Association held a meeting for NET PIs at their annual conference, in a similarly unsuccessful attempt.
Overall, these various formal and informal meetings were appreciated, but made surprisingly little impression on the participants: we were a bit taken aback by how many respondents couldn't remember for certain if they'd participated. Several respondents expressed frustration with the inability of this research community to collectively take action to secure its own future. However, such efforts require committed leaders, and NETs had few resources to spare for anything not yielding immediate results. While partners encouraged these networking efforts, ultimately there was no institutional support for the necessary resources.

Evaluation

In addition to this current assessment, ICR has undertaken an interim survey of NETs, and required end-of-grant reports from pilot project grants and NETs; the latter are detailed reports focused on the outcomes and impacts of the funding, KT activities, and plans for sustainability. All this evaluation material has been incorporated into this current assessment, together with new data from interviews, database analysis and literature review. Each of the program elements in the RFA included a commitment "to assess performance of this initiative through ongoing monitoring and periodic evaluation," though CIHR has not yet set up routine processes to do so. RFA-specific evaluation requirements and the available evidence on outcomes are summarized in Appendix D.
In summary, while it is easy, with hindsight, to point out some of the unrealised good intentions of this strategic initiative launched during CIHR's early days, ICR's overall approach to consultation, planning and execution drew high praise from all we interviewed.

Chapter 3 Results and Outcomes

Enhanced funding for PELC research

To place the Initiative in context, it is important to note that CIHR's support for PELC research is by no means restricted to the Initiative funding. MRC spending on PELC¹² in its last year of existence (FY 1999-2000) was $96,034 or 0.03% of the MRC budget, supporting two career awards and one grant (Fig 3.1). By 2002-03, immediately prior to the launch of the Initiative, CIHR's support for PELC had increased to $571,769 or 0.1% of CIHR's budget, funding eight grants in a variety of programs, and five training and salary awards. The other primary source of funding for PELC research prior to the Initiative was the National Cancer Institute of Canada (now the Canadian Cancer Society Research Institute) and Health Canada's now-defunct National Health Research & Development Program (NHRDP).

Fig 3.1 CIHR spend on PELC¹³ by fiscal year (columns, left axis) and % of CIHR total research spend (line, right axis) * 2009-10 funding is to date, not a full fiscal year

CIHR spending on PELC by fiscal year (columns, left axis) and % of CIHR total research spending (line, right axis). 2009-10 funding is to date, not a full fiscal year

Training	#	$	%
Table 3.1 CIHR spending on PELC research, 2000-2009
Graduate awards	8	$ 477,000	9%
Fellowships	7	$ 573,625
STIHRs	3	$ 1,970,375
Sub-total	18	$ 3,021,000
Career Awards
New Investigator	7	$ 1,257,269	6%
Other (includes CTA)	6	$ 594,459
Sub-total	13	$ 1,851,728
Grants
Collaborative (includes NETs)	14	$ 11,773,250	85%
Capacity-building (includes Pilots)	18	$ 1,088,041
Development (includes planning and LOI preparation grants)	19	$ 173,549
Open Competition	51	$ 11,769,652
Knowledge translation	10	$ 296,422
Other Strategic Grants	19	$ 3,427,746
Sub-total	131	$ 28,528,660
Total	162	$ 33,401,388	100%

By 2008, CIHR's proportional support for PELC research had increased six-fold to almost $6M, with some 60% of this funding coming from outside the Initiative. Despite concerns about the low uptake of the PLC peer review panel, it is clear CIHR-wide activity in PELC has rapidly expanded, spurred by both the increased awareness and the growing capacity developed as a result of the Initiative.
Almost half of the funding has been attributed by its recipients to ICR, with significant amounts also attributed to IHSPR (consistent with the Canadian specialization in health services research as shown in Fig 3.6), IA and IHDCYH. All other Institutes have been minor players (Fig 3.2, right). CIHR funding has been primarily (85%) to operating grants of various types (Table 3.1), and the rest to personnel support.

Fig 3.2 CIHR funding for PELC (2000-2009), by theme (left) and Institute (right)

CIHR funding for PELC (2000-2009), by theme and Institute

Strategic grants

Pilot Project Grants

For some newly-launched investigators, a PELC pilot grant jump-started a career, making it possible to get feasibility data, community partners, career awards, and subsequent grants for follow-on projects. For others, the projects were small self-contained studies from which the PI has moved on. It was originally hoped that 20-25% of the Pilot Project Grants would lead to an application for a CIHR open operating grant., however, nine of the 18 PIs received subsequent operating grant funding for projects that were clearly related to the topic of the Pilot, and three others received funding for other PELC-related studies. In the five years prior to the start of the Pilot, collectively the PIs received $1.68M in CIHR grant funding, while in the five years following the Pilot their CIHR grant support increased almost four-fold to $6.46M. The expectation for a 20-25% "yield" in terms of subsequent operating grant applications was thus greatly exceeded, with 66% receiving funding.
We also compared the number of PELC publications¹⁴ from each of the Pilot PIs. In 2006-08, collectively the Pilot PIs published 36 papers, compared to ten in 2001-03. By 2006-08, 13 of the PIs had published in the PELC literature, compared to six before the Pilots. Clearly the award of the Pilot was associated with an improvement in publication in the PELC literature¹⁵.
We were initially concerned that six of the 18 PIs never received CIHR funding as a PI subsequent to their Pilot, and we examined closely the publication and funding record of these individuals. Only one of the six had vanished without a trace from the literature, five were still active in PELC research, and two of these have had highly productive research careers, without benefit of CIHR support. Overall, therefore, the Pilots have proven highly beneficial to their recipients, and this scheme clearly over-achieved its objectives.

New Emerging Teams (NETs)

Funding ended in September 2009, and teams report many publications still in development, but it is already clear that participation in a NET has been highly beneficial for the productivity of the PIs (Appendix B-B). The future NET PIs, as a group, were authors on 29% of Canadian PELC publications from 2001-03. In 2006-08, after receiving NET funding, those same investigators had better than doubled their productivity, and were authors on 37% of the now much larger Canadian oeuvre. In addition, and as we would hope for NET leaders, they became increasingly collaborative, moving from the Canadian average of 3.1 co-authors per paper in 2001-03 to an average 4.6 co-authors per paper by 2006-08: meanwhile, other PELC researchers averaged 2.7 co-authors. The stimulatory association between NET funding and collaboration is shown graphically in Fig 3.3. Although some NETs have not yet contributed co-publications, NET funding was associated with an increasingly rich collaborative publication record among the team members in both PELC and related research, satisfying one of the objectives of the Initiative. The publication record of the NETs should be re-examined in 2012 to obtain a complete picture of the true impact of this funding scheme.

Career Transition Award (CTA)

There was limited interest in this program, and only one award was made: one respondent suggested "I think people are more likely to make that transition within a team-supporting environment." Unfortunately, we were unable to arrange an interview with the one recipient. However, she reported elsewhere that "the career transition award has allowed me to develop an area of expertise that is unique and personally rewarding, and one that I anticipate will enhance the food-related quality of life for those in end-of-life care"¹⁶. Examination of the recipient's publication record reveals that a burst of PELC publications followed closely on the award period, thus achieving an objective of the CTA award.

Advancing knowledge – research outputs

Research productivity

We discovered that at the time the Initiative was launched, PELC research in Canada was by no means trivial or endangered. Canada was among the top 3 nations publishing in PELC research, particularly strong in clinical research, and some Canadian research was world-leading in terms of impact. The UK National Cancer Research Institute kindly provided us with an internal bibliometric study of 18 leading nations that it conducted in support of its own PELC initiative. This study covers the period 1994-2002, and contributes to the baseline data, though of course the focus was only on PELC relevant to cancer care. Canada ranked third in the number of papers, and even higher in the proportion of its overall research output having a PELC focus¹⁷. Thus even prior to the Initiative, research in this area was relatively strong in Canada compared to other nations.

Fig 3.3 Co-publication among members of PELC NETs¹⁸: each line represents a co-publication; each colour a different team.

Fig 3.4 PELC publications per year, World and Canada

The Initiative did not start from scratch, therefore, but built on a small but solid nucleus of Canadian strength. However, the most productive PELC researcher in the world (E. Bruera) had left Canada in 1999, perhaps contributing to a feeling of desperation among the PELC community, who were also disillusioned by the low level of funding invested by the federal funding agency in this area. Many investigators, recalling the PELC research community in the early 2000s, commented on how fragmented and isolated it was, compared to today.
In order to establish baseline conditions, from which we could assess the impact of the Initiative, we conducted a bibliometric analysis of Canadian research in PELC from 1997 onwards¹⁹. For the years prior to the Initiative, Canadian publications/ year were constant or slowly increasing²⁰, with significant growth subsequently, coincident with the timing of the Initiative (Fig 3.4). World publications, on the other hand, began to increase around the millennium, with the result that Canada's share of world publications declined through to about 2004.

Fig 3.5 Canadian PELC productivity (publications/year) relative to that of other leading nations

The Initiative was therefore launched against a background of increasing worldwide research interest and publication in palliative care, and declining Canadian share of that global research production. Although Canadian share of PELC publications was small, Canada has always been one of the top 3 nations in PELC research in terms of number of publications (Fig 3.5). The current Canadian 8% world share of publications in this field (Fig 3.4) is much higher than Canada's overall world share of health research publications (4-5%), and Canada's almost doubling of world share between 2004 and 2009 is notable²¹.
Initiative-funded researchers were the major contributors to Canada's increased research productivity from 2006-08. Relevant publications authored and co-authored by Initiative investigators²² accounted for half of the Canadian PELC publications between 2006 and 2008, and 70% of the increased productivity.

Research quality

By a range of citation measures, Canadian PELC research has a relatively high impact. In fact, ten years ago, Canadian articles were cited almost twice as often as the world average²³ (Appendix B-C); however, this lead is dwindling, suggesting that other nations are also publishing recent papers with higher impact.

Table 3.2: Rank of Canadian Publications among 40 most-cited from each 3-year period
1996-08	2001-03	2006-08
3 8 21 31 35 39	21 25 30	3 4 21

Looking at citations for individual papers, we consistently find Canadians represented among the world's top 40 most-highly cited papers (Table 3.2). Overall, in the three time periods we examined, Canada has about 10% of the top 40 publications, which is almost twice the percentage of Canadian publications in the world PELC literature over the same time period, again suggesting that Canadian publications include more than their proportional share of highly-cited papers. Similarly, using the WoS database, we found that during the period 2000-09, Canadian publications represented 5.5% of the world pool, but 16% of the 25 most-highly cited. The UK study referred to above found Canada tied with USA for the highest proportion of publications in the top-tier journals.
Although PELC publications are few, quality, as determined by citations/publication has been and continues to be comparable to or better than Canadian publications in other clinical disciplines, even the enormous discipline of medicine. (Appendix B-C).

Canadian research specializations

“Important factors in measuring impact of basic research don’t work or have meaning here: if I publish in clinical journals and clinicians use my results – who knows? Clinicians don’t publish a paper to say they’ve changed their practice patterns!”
Researcher

In the UK study, Canada was the first-ranked nation in terms of the "clinicity" of its published papers, with 85% of the papers falling in the most clinically-oriented category, as opposed to their "basicity," i.e. dealing with basic science. In Canada, PELC has proven to be strongly practice-based and implementation-oriented: as one respondent noted "there's virtually no such thing as a PELC researcher who is not also a care provider." As such, we must be cautious in using academic impact factors as measures of true influence: "we have published our manuscripts in journals that are widely read in our community, though not "high impact" on the general scale. This has ensured that the most receptive knowledge users (i.e. palliative care providers) will have the opportunity to benefit from our research and apply our findings to their daily practice" (NET PI).
Canadian PELC research is particularly specialized²⁴ in the areas of health services and public health, with a smaller contribution in the fields of nursing, geriatrics and gerontology, as related to PELC (Fig. 3.6).

Fig 3.6 Index of Specialization of Canadian PELC research, 1997-2008: an index of >1 shows Canada is relatively specialised in that field.

International collaboration

Figure 3.7 Percentage of Canadian PELC Publications with International Co-Authors

Another measure of national reputation is international collaboration²⁵: it's difficult to join a club if you have nothing to offer the other members. Fig. 3.7 shows that international collaboration in this field was very low pre-CIHR, and increased only slightly by 2001-03, but has risen sharply coincident with the Initiative so that today almost 40% of publications have at least one international co-author, comparable to Canadian clinical research as a whole (see also Appendix B-C). Thus the PELC research community has transformed itself from one that engaged in very little international collaboration to one that is well-connected internationally. The greatest numbers of collaborations are with the USA, followed, since 2001-03, with the UK and Australia, presumably as a result of those countries launching their own PELC research initiatives

Advancing knowledge – research capacity

How big is the new capacity?

We heard that the Initiative "galvanized the palliative care community, created huge excitement and hope," and as a result, "the community has demonstrated capacity beyond what we would have thought possible: publications, protocols, groups have all come together." The primary objective of this initiative was to create research capacity. Peer reviewers, investigators and their decision-maker partners all agree that there have been substantive improvements in both the quantity and quality of PELC research in Canada.

“We started with just three of us. Now there are at least 25 team members. If that’s a reflection of the rest of the community, it speaks volumes about the interest and the possibilities from this Initiative.”
NET investigator

“Early in the game of palliative care research, I could keep all the palliative care investigators on my speed dial. Now I don’t even know how many there are – that’s terrific. The next generation of investigators has been created.”
Initiative Partner

The number of active PELC investigators in Canada, (i.e. those who publish research findings) has increased in parallel with CIHR funding. The number of authors whose names appear on Canadian publications is easy to determine from the Scopus database (Table 3.3, row A). From these names we can identify the number of unique authors (row D). This number includes, in addition to independent investigators working at Canadian institutions, their trainees, research assistants, collaborators from the health-care sector, and foreign collaborators: it is the broadest definition of the Canadian PELC research community. We estimated the number of non-Canadians among these authors by sampling 50 names from among the authors in 2001-03, and 100 names for the 2006-08 list: the resulting estimate for the number of Canadian-affiliated authors is shown in row F. Despite the approximations involved, it is clear there has been a dramatic growth in the Canadian research community since 2003, coincident with the Initiative.

Table 3.3: Canadian authors publishing palliative care research	1996-8	2001-3	2006-8
A. total authors	911	1199	2226
B. number of publications	258	294	507
C. authors/publication	3.53	4.07	4.39
D. unique authors	699	843	1532
E. % non-Canadian authors	N/A	~36%	~29%
F. Approximate size of Canadian PELC research community	N/A	540	1090

We have already noted that Canadian authors are now much more involved in international collaboration²⁶. The same applies to domestic collaboration, as measured by the number of authors associated with each research publication (Table 3.3, row C).
Two remarkable features of the Canadian PELC research community have been the turnover of the most-productive authors, and their increased productivity. Of the ten most-productive authors in 1996-98, only one remained on this list by 2006-08²⁷. If we examine the publication history of the 2006-08 top ten authors we find the converse: that is, only three were among the top ten of 2001-03 and five of the ten weren't even publishing then. In other words, the current most productive authors have arrived recently, replacing earlier leaders who have retired from active research in Canada. Furthermore, to be included in the top ten today requires considerably greater productivity: in 1996-98, five publications/three years earned a spot in the top ten: ten years later this productivity would put an author only in 33^rd position, and eight publications/three years was the minimum necessary to earn a top ten position.
We find similar evidence for an extraordinary turnover in capacity when we examine CIHR-funded investigators. Table 3.4 shows that in 2006-08, CIHR funded a total of 367 unique investigators, including co-investigators. Surprisingly, given the ten-fold increase in funding between 2001 and 2008, the number of investigators supported in 2006-08 was not even twice the number supported in 2001-03. This means that the average funding per investigator must have increased many-fold between these two time periods. In other words, the large growth in funding represents a much larger quality increase than quantity increase in CIHR-funded research capacity.
The arrival of both new and more productive PELC investigators bodes well for both the continuing quantity and quality of Canadian research.

Table 3.4 Number of unique investigators supported by CIHR	As PIs	As Co-investigators	Total
Number of investigators funded in 2001-3 by CIHR	64	165	218*
Number of investigators funded in 2006-8 by CIHR	112	291	367*

* some individuals were both PIs and Co-Investigators, so column 3 is less than columns 1+2
Many respondents spoke of the importance of recognizing how young the PELC community is in terms of research experience. Again, CIHR's funding substantiates the evidence from publications: the change in numbers of PIs between the two time periods is the balance of attrition and addition. Of the 218 investigators funded in 2001-03, 89 were no longer funded by 2006-08, and conversely, 238 of the 367 investigators funded in 2006-08 had not been funded by CIHR in 2001-03 for PELC research. There has clearly been a very significant entry of "new blood" into the CIHR-funded cohort of PELC investigators, over a very short period of time. Out of a sample of 50 of those "new blood" investigators, 13 of them had been grant-funded by CIHR in 2001-03 for work not classified as palliative care, though usually related (e.g. cancer therapy, pain control). By extrapolation, as many as 60 established investigators may have been attracted to PELC research by the availability of Initiative funding, consistent with its goals.
As noted (table 3.3), the number of unique Canadian authors has doubled since the Initiative's launch (from ~540 to ~1090), but despite its massive increase in spending, CIHR's funding "penetration" into the PELC research community has not increased, but declined slightly, from 40% (218 funded/540 authors) in 2001-3 to 34% (367/1090) in 2006-08.
The finding that the majority of publishing PELC researchers do so without benefit of CIHR grants is consistent with respondents' depiction of PELC research as mostly highly-applied and implementation-oriented, undertaken primarily by active health professionals in the course of their clinical duties, and funded largely by internal or local resources (see Appendix B-A). It is also consistent with the situation that existed in the late 1990s where negligible MRC funding supported PELC research, but Canadian authors were publishing 90 papers/year. Although acknowledgment data also suggest that a significant number of investigators are able to publish in this field without CIHR support²⁸, we believe that through the NETs, CIHR is supporting many more people than its official numbers show, and developing the research interests and skills of a wide range of health professionals - collaborators, partners, and trainees²⁹ - who would not normally be CIHR-funded investigators but who are a substantial portion of that other 66% of authors. This conclusion is supported by the finding that Initiative-funded investigators are included as authors on half of all Canadian papers, even though all CIHR-funded grantees are only 34% of the total author community. The lack of a follow-on open team grants competition eliminates a major source of mentoring, small pilot funding and collaborative opportunity, and may thus hinder the future productivity of these supposedly "non-CIHR funded" authors.

Where did the new capacity come from?

Our interviews revealed that the NETs tended to attract two different types of trainees and new investigators: those emerging from the PELC practice community, and those attracted from other areas of health research. Of course, all the teams carry elements of both groups, but nonetheless, at the risk of caricature, we think it is useful to distinguish the two, as they brought significantly different expertise to the Initiative, make different kinds of contributions to PELC research, and have equally different needs for sustainability of the capacity built during the Initiative.
The teams with strong clinical roots were accustomed to working in an interdisciplinary manner, often had highly experienced leaders and managers in charge, and engaged knowledge users as a matter of course. These teams have high credibility in the health care system, but may have lower formal research productivity and poor records for training leading to tenure track positions. On the other hand, such teams are attractive to experienced health professionals who seek research expertise to help them solve the problems they encounter in their work. With their concentration on clinical practice and implementation, many of the investigators and trainees in these groups will have trouble competing in the open grants competition, where these activities are less valued than traditional academic achievements.

“The NET is a perfect program for new people - involvement in a NET grant can just propel their research and career in ways you couldn’t do otherwise. People who are really senior tend not to want to stray too much.”
NET PI

Of course, the Initiative also attracted investigators well-established in PELC, plus those from other fields who focused their existing expertise on PELC-related questions. Such teams had the academic expertise to provide strong research mentoring for their members, but were less likely to have experience in interdisciplinary team working and interacting with knowledge users. Most of these established investigators recruited into the PELC field wanted to continue their PELC studies after the end of NET funding – though many would not now categorise themselves as "PELC investigators" – and especially to maintain the teams developed through the NETs. However, since the cancellation of the open team grants competition, these investigators are likely to migrate to team funding opportunities in other areas. As well-established investigators in their own fields, they will generally continue to be equally competitive in the future, and most will apply to their traditional open operating grant peer review committees, where their work is well-known, rather than the PLC committee.

Key capacity gap – clinician researchers

A number of respondents identified the need for more PELC clinicians with research training, so they might be more effective research collaborators. In particular, the lack of a palliative care specialization for physicians is a major barrier to developing this field of research: only physicians who specialize get research training. One decision maker noted: "We need to move palliative care forward in a more academic and deliberate sense. It's still a voluntary movement in many ways: these people are valuable, but the academic side lacks strength or credibility."
To help address this gap, NET investigators participated in a wide range of informal mentoring, much of it dedicated to helping clinical colleagues become more able to engage in research. One noted: "I also have industry-sponsored trials. I am trying to build a group to teach other physicians how to get involved in the trial. I choose only trials that will help with teaching, provide co-workers with learning experiences." Another commented: "I build research capacity in clinicians, helping them write papers and get involved in research."

Mentoring: NETs as incubators for New Investigators

“One of our trainees has moved from new Investigator to Program Leader – now she’s nationally recognized as a leader in the area, not only in research but in implications to health authorities, who now seek her help. This is what we really want to see from the NET.”
NET PI

“It’s been remarkable - the maturation of the research groups, teams working together to further develop individuals with research interests.”
Working Group member

For a new investigator, a NET salary is an enormous benefit, providing completely protected research time to establish oneself as an independent investigator. New investigators also valued the wide range of contacts and connections made through the NET. The slow progress of collaborative research can, however, be a challenge for a new investigator trying to start up a research program on a tight timeline. On the other hand, another noted that being in the NET hugely sped up her work, because she could draw on the NET infrastructure and on baseline work already available.
A risk for new investigators working in a team setting is "the tendency for one's own projects to get lost when working with senior investigators, who don't always keep the best interests of the junior investigator in mind. It's not deliberate – it's just that such people are so busy and focused on their own work, and they see your best opportunity as working with them." Another new investigator succinctly summarized the Catch-22 described by many respondents: "I can get further faster by working in this group, but as a new person, it's important to separate myself from the PI. My Chair likes to see my name by itself on papers." The institutional doublethink which encourages team work but only rewards "independent investigators" is still a major impediment to collaborative health research.
Overall, new investigators found the NETs gave them a competitive edge for grants and salary awards; their proposals were strengthened both by NET-funded feasibility work and the credibility the team's presence brought to the individual applicant. Thus, while there are challenges to working in a team, new investigators found the NETs to be supportive and fertile environments in which they could develop their research skills and accelerate their achievements, thus meeting another Initiative objective.

Training – who was attracted to the Initiative?

“I was an advanced practice nurse in palliative care. One piece that’s really important for me is trying to create knowledge and better understand what we’re doing. Is it making a difference? But I realized I didn’t have the skill set, I wanted to make a stronger contribution. As a clinician, I’m really interested in doing things relevant to the real world. I saw a lot of gaps. I was always asking ‘can we do something about that?’”
Doctoral trainee

Several respondents remarked that whereas department chairs used to warn students away, there was now interest in PELC among students in their department, as a result of the NETs. Others were being approached, sometimes unsolicited, by recruits of unprecedented quality and qualifications.
The most striking characteristics of the trainees we interviewed are their maturity and experience. A significant proportion of the trainees attracted to the NETs and the STIHR were practicing health professionals, many with decades of experience and leadership roles within the health system. These trainees described a shared passion to improve the care they provide and the decisions they make, but "I was very aware that I was missing the academic skills I needed to better understand what was happening on the clinical side."
Few of these trainees are interested in a tenure track academic future; most wish to continue leading change in the health system, ideally splitting their time between research and care so as to identify problems and be able to fix them. They do not seek to be full-time investigators, and are unlikely to develop the level of research productivity that would allow them to be competitive in the world of CIHR open operating grants.
Some respondents were concerned that many trainees seemed to be "failing," that is, not going on to tenure-track positions and independent grants and awards, the traditional measures of training success. This Initiative highlights the need to broaden those measures, because what these trainees are doing instead is changing palliative care, rapidly and substantively, which is what research in this field is all about. As well as being clinicians and health system leaders, they are members of regional, provincial and national bodies which set policy and standards for PELC: their research and expertise feeds directly into the development of best practice guidelines, provincial funding policy, and professional standards. They have the credibility to engage health care providers and access populations, and the skills and reputation to get the results implemented. This is integrated KT in action.

Portrait of a doctoral student
Nurse and health system leader, one NET trainee holds academic posts in two universities, is part of The Canadian Partnership Against Canacer’s (CPAC) Clinical Practice Guidelines Action Group, and past-president of a provincial Hospice Palliative Care Association, not to mention recipient of their Award of Excellence. Or, as she so aptly put it, “I am my own decision maker”.

Training: Benefits and challenges of NETs and STIHRs

Stipends are particularly important to attract trainees who are already well-paid, mature professionals, with significant personal financial obligations and family responsibilities: "The support from the CIHR was fabulous, but I was making a great salary as an advanced practice nurse, and it was still a loss in income." These trainees face a number of other challenges not typical for grad students, including the need to convince their employer, and sometimes their staff, of the value of investing in research training, and to negotiate new contracts with leave-time, job-sharing, or sabbaticals. These trainees are driven by organizational needs rather than academic or personal ambitions. For example, one noted: "I've been offered two post-docs. I will take the one my organization says will benefit it the most."

“Collaboration is fantastically improved. The NET has enabled the hiring of staff and the travel for individuals to come together in ways never before possible.” NET PI

Trainees all described their NET and STIHR training as "a much bigger experience" than a conventional graduate program; and found "there were enormous differences between my training and what was available to the other five (in my cohort), huge benefits for me." While their cohort wrote their thesis proposals, NET trainees also wrote funding applications; undertook needs/gaps analyses and participated in strategic planning; developed collaborations and interacted with partners; presented results and wrote articles for both academic journals and user forums; and designed and implemented KT strategies. Like new investigators, trainees found that being positioned within a NET made them more competitive for postdoctoral and faculty positions, as well as grants and awards.
STIHR trainees agreed that most useful STIHR activity was protocol development training: students presented their drafts to a broad group of mentors and received wide-ranging, constructive criticism. Students often had their thesis protocols completed a year before others in their peer group, and felt that theirs were more substantial and better-developed. STIHR trainees felt they had a much deeper and broader grounding in both qualitative and quantitative methods, and PELC's unique feasibility and ethics issues. They developed a strong support network of contacts and advisors across Canada, particularly helpful to establishing the next steps of their research career. "There were continuous opportunities to talk about the challenges we were experiencing, and provide feedback to others based on my experiences so far."

Training: Career paths of trainees

“I don’t want the academic path at the expense of my clinical work – I need to be able to integrate these in some sustainable way.”
Trainee

For trainees coming out of the practice environment, the question is not whether they will stay in PELC, but rather whether they can stay in research: "How will I continue to utilize my research skills while working in a health care organization? Or am I just going to be the most over-qualified social worker around?" Most of the trainees are nurses, social workers, psychologists and other health professionals, who have found that "there are whole systems set up to allow physicians to manage this dual path, but not for any other clinicians." Health care institutions have few posts for non-physician clinician scientists, and their ability to fund non-core activities such as research and education is rapidly dwindling. The few such posts which do exist usually require the investigator to be a general organizational research resource, which precludes establishing an independent research program.

“I got clinician release time (6 months, but enough!); I’m the only nurse who ever got one of these CIHR awards, which led to my current grant.”
For this new investigator, a $25,000 CIHR investment in release time led to over $300,000 in funding from open grants.

For almost all the trainee respondents, their "ideal-world" scenario was a clinical post with about 2 days per week of protected research time; call it, perhaps, a research-clinician rather than a clinician-researcher. Most respondents believe their health care employers would be quite willing to provide infrastructure, access, and other resources to a research-clinician who could find that release-time funding. However, given the non-existence of part-time research salary awards of this nature, few trainees were taking on posts in which they thought they would be able to put their training to use, that is, where they could do research, and then implement it.

Collaboration and Teamwork

Investigators lauded the NET approach, agreeing their work was "enhanced by the many perspectives brought to the table. The richness of research ideas and wide range of approaches to problem solving were largely due to this interdisciplinary culture." NET participants agreed that individual operating grants would not have achieved a fraction of the same impact. Achieving true interdisciplinary team work requires trust, which can only be built through an enormous (the NETs would repeat, enormous) time commitment to communicating, especially face-to-face, which teams are uniquely able to fund. The NET structure is particularly well-suited to integrating knowledge users, and supports many important collaborators and user communities who, on their own, would not be able to access CIHR funds or projects.

“In my communications with UK and US investigators, I get the sense of how unusual these collaborations are. I call myself a CIHR-outsider, being a SSHRC-funded researcher. But I find the NETs thoughtful and forward-looking, so I feel comfortable in this team.”

NETs provide a crucial source of funding for developmental work – a huge challenge in an emerging field - supporting data collection, tool validation, feasibility and other work which CIHR rarely funds, but requires it to be done in a competitive proposal. The block funding is flexible: teams brought on new projects, partners, and investigators to respond to emerging challenges. NETs are a fertile environment for learning from each other, whether new research techniques, how to access policy makers, or team administration; even the most senior team members found that the learning opportunities invariably ran both ways. For established researchers, the NET allowed the productivity, and perhaps the cachet, to "move up the ladder."

“As a multidisciplinary group, it was challenging at times to include all members on all projects and to meet everyone’s individual goals on each project due to the varied interests of the team. It was therefore essential to engage with one another’s research interests and work collaboratively in support of all projects. Our team established a charter, detailing decision-making processes and levels of consensus.”

Most NETs have devoted extraordinary effort and resources to building and sustaining interdisciplinary teams: leaders agree that "teams have more profile, productivity, diligence – but they are a huge load to organize and administer"; "If you take the task of teams seriously – it's very time-consuming, but very rich." Most NETs agree that it took about two years to establish the team before true collaborative research could even begin, and urged CIHR to ensure funding and evaluation allows for this. Several teams shared the load among multiple leaders; some found teaching release time, or highly-skilled administrative staff who could reduce the burden. While they can receive administrative support, rarely does a NET PI have protected time for their leadership activities, which one estimated at 40% of his time "if I do it right." Investigators claim these leadership activities count against tenure and promotion, and urge CIHR to do more to influence universities to value the kinds of research activity that CIHR wants them to pursue.
Most participants lauded the communications efforts within their NETs, while a few wished there had been more communications or transparency. Most interviewees wish to sustain the collaborations forged in the NET through other mechanisms, though not every PI was prepared to tackle such a large workload again.

Networking

Respondents all agreed that a Canadian PELC research community had been built, thanks to the Initiative. Institutions have multiple PELC investigators instead of isolated individuals, reducing fragmentation and attracting students. We found a gratifying amount of cross-NET pollination, including: shared members, trainees, tools and outcomes measures; validation of each other's tools in new regions or populations; joint meetings, and other collaborative projects. Although previous efforts to develop a "Net of NETs" didn't get very far (see Chapter 2), there is now a significant appetite for maintaining or enhancing connections across the new PELC community. Some, however, fear the opportunity is already lost.

Informing decision making

Facilitating research use and useful research

“Interdisciplinary teams are highly translational, and include people who work day-to-day with patients. It keeps you grounded in what you’re trying to achieve, in what’s really important to this population. When I work with this group, I know it will be highly relevant and able to be applied.”
NET New Investigator
“Real partnerships are embodied in the NET: they’ve created good engagement for us with the medical community. The NET was able to connect to many organizational structures, an unusual thing to be able to say about a research program. What has emerged is really new and could not have been predicted.”
NET community partner

Despite an RFA which included no KT requirements³⁰, all the NETs have substantively engaged user communities in their work, an extraordinary commitment which speaks volumes about the nature of Canada's PELC research and researchers. NETs have engaged knowledge users into their activities in a variety of ways, making the NET itself into the primary structure for integrated knowledge transfer. For example, NETs have:

Interviewed federal and provincial policy leaders
Held focus groups with key user communities to identify issues
Created steering committees or advisory boards
Engaged knowledge users as collaborators, even co-investigators
Met one-on-one with knowledge users to review results, next steps
Created community forums to share research outcomes with clinicians, family members, patients and policy makers, and suggest possible public policy and advocacy strategies
Sought user input in protocol development and implementation
Engaged research participants – both patient and health care provider – in reviewing and interpreting the data and results
Incorporated investigators/ trainees involved with agencies to improve health care and health professional education

The NETs found it time-consuming and challenging to build effective trust relationships with knowledge users – but often far more rewarding than they'd ever anticipated: "our investigators are saying that the value of what they're seeing is enhanced, and it's changed their whole approach." Researchers have found they have better access to populations, care teams, data, and policy leaders; more feasible protocols and easier recruitment; and more implementable results and more effective dissemination strategies³¹.

Examples of NET KT in action

VP-NET found misperception and distrust kept disabled patients from accessing effective palliative care. A NET post-doc with an English background developed a play to highlight and challenge the issues, while pointing out opportunities for reconciliation and collaborative problem solving to reduce suffering and untimely death.
VPRN-NET turned their research findings about how skilled physicians effectively and compassionately communicate prognostication information to patients and their families into a series of DVDs called "Breaking Bad News," and are using them to train BC medical students, as well as physicians around the world.
Access ICE developed a project Advisory Board with a focus on dissemination: partners help formulate recommendations, co-write policy-briefs, provide feedback on reports, enable access to key decision makers. Knowledge users are so engaged, the ICE expects this year's public "mini-conference" to draw over 200 people.
The Difficult Pain NET focused on sharing lessons it learned in clinical trials, including: novel approaches to assess study feasibility; low-cost data transfer to the central repository (instead of $100k software); dealing with multiple Research Ethic Boards (REB); validating the Cochrane Risk of Bias tool and improving reporting of educational intervention trials.
The Family Caregiving NET is collecting letters written by bereaved caregivers into a book of advice they would give to themselves if they were just starting out, to help other caregivers in similar circumstances.
The Access ICE seeks to improve the Compassionate Care Benefit in real time: it is identifying who most needs to know what about the benefit, and where they live; the next step will "mobilize the messages and the model." All the while they are working with policy makers to identify ways of increasing the Benefit's uptake and effectiveness.
CARENET studied organizational safety and culture factors which relate to better quality of care from families' perspective, and is testing interventions to improve domains such as leadership, communication, and conflict strategy. The team is reviewing individual results and potential strategies with each participating organization.
Sometimes it's most important to find out what we already know. PedPal NET found that as life-threatening diseases progressed, children become increasingly unable to tolerate enteral feeding. They gathered researchers, clinicians and stakeholders from around the world to reach consensus about how to best help these children.
The Difficult Pain NET engaged the experts in the development of a new tool: patients with long histories of managing breakthrough cancer pain were taught to use a qualitative research methodology called the "think aloud" process. The revised tool should be patient-friendly and more successful in practice.
PedPal NET partners with families to design meaningful pediatric palliative care studies that not only address critical areas for developing new knowledge, but are planned and conducted in ways that honour families' experiences and minimize their burden.
Research from VPRN-NET has been incorporated into three new guidelines to be piloted, tested and ultimately sent to every GP in BC, while another tool is incorporated into KGH's practice guidelines. VPRN-NET and the ELCS NET are developing delirium guidelines in palliative care, with the Canadian Coalition for Seniors' Mental Health.
The Cross-Cultural NET has helped develop culturally competent palliative care guidelines in BC health agencies, such as to implement Advance Care Planning, or in using the NET's Palliative Outcome Scale (POS) screening.
The Winnipeg Regional Health Authority believes that Dignity Conserving Care is so important that it has provided VP-NET with $5 million for the first two years to operationalize it, transforming the culture of health care throughout the authority. In the meantime, the New Interventions NET is further testing Dignity Therapy in French, in Quebec.
A key partner, the Canadian Hospice Palliative Care Association (CHPCA), "takes the research findings and makes them more ready to use." CHPCA incorporates research findings into its conferences and public events, and synthesizes results into factsheets, websites, press releases and other materials.

“I can really help with the feasibility: whose buy-in do you need? For example, a lot of research requires asking nurses to double-chart – this is a lot of extra work. So we had to invest a lot of time in getting the nurses on board. We engaged a nursing educator. We documented their current practices and showed how those were putting people at risk. We engaged them in the problem, and then in creating the solution.”
Decision-maker partner
“I ask, what can I do to get us, as national organizations, to collaborate better to support this work?”
Decision maker partner

Improving KT

One mantra arose from all knowledge users and partners: the earlier our engagement with the research, the better. Some PELC investigators still tend to think of the partner's primary role as end-of-grant dissemination, but every partner we spoke with saw their time as best invested at the front-end, strategic phase of the research program: "I've not been involved early enough to inform the questions. But it would be great if we first talked about what I needed to know, how would I use the results, what products would be useful. Also, how would you go about this kind of research? I know a lot which would help them there, too."
Although CIHR was praised for being far ahead of universities in valuing and supporting KT, knowledge users and researchers alike complained that standard CIHR grant mechanisms do not support integrated KT activities. As one NET PI noted: "The decision-maker role doesn't usually fall neatly into one of the allowed CIHR categories, nor is it supported or enabled by CIHR funding." As a result: "collaborating with community partners is almost impossible under traditional funding – they can only participate at their own expense."
In addition, community partners' willingness to become "entangled in CIHR red tape" is exceedingly low. A couple of teams noted they had partners who remained "unofficial" because, for example, "our (First Nations) elders will NOT fill in the Common CV – not a chance!" Other collaborators were removed from grant applications because they lacked the administrative resources to complete the Common CV on time.

Changes in health practice, policy, services and education

NETs have developed a creative array of courses, seminars and Summer Institutes for their trainees. The New Interventions NET is particularly noteworthy for its energy: it developed six training seminars for graduate students; three scientific days attended by more than one hundred clinicians, researchers, decision-makers, and students; three training workshops presented at national and international conferences; and 48 research conferences in Quebec City for researchers, students, and clinicians. The team also co-leads a national education initiative to implement an interdisciplinary web-based course in psychosocial oncology for Canadian graduate students. NET researchers have also disseminated results through a wide range of courses and workshops to students, residents, local and international practitioners.
As a result of their involvement in the PELC initiative, decision makers were more likely to describe changes in the way they work rather than specific changes in practice: as one said: "It still feels like early days." Nonetheless, the Initiative has supported the development and testing of a rapidly expanding portfolio of health care innovations which are based on high quality research and have excellent potential to improve palliative and end-of-life care in Canada. Policy does not change quickly, and rarely on the basis of research alone, let alone the results of a single study. Even so, NETs also identified a number of examples where policy change is already being debated as a direct result of NET research.
We are confident that the NETs have developed close linkages among investigators, health professionals, managers and policy makers, and user communities, and are investing in on-going knowledge translation to maximize the potential benefits of these innovations. NETs have been prolific users of CIHR KT support, with over 30 funded grants with a KT component between them. However, the NET PIs are not KT experts, nor can they alone undertake the initiative-wide KT activities which are essential next steps. As discussed in Chapter 4, we suggest there are a number of things ICR and CIHR should do enhance the effectiveness and impact of these KT efforts.

Changing health professional education

The Difficult Pain NET developed the world’s first on-line palliative care research methods course; most palliative medicine residency programs across Canada made the 12-week course mandatory or strongly recommended. The program is being adapted for students.
To better talk to patients about where they want to die, a STIHR trainee developed tools, an educational intervention and workshop. The decision-support skills proved remarkably successful in testing with nurses, pharmacists and social workers. As a result, the Registered Nurses' Association of Ontario asked the trainee to co-lead the development of decision-support Evidence-Based Best Practice Guidelines, and to sit on their panels to develop end-of-life practice guidelines; develop new criteria for the hospice palliative care exam; and the gerontology exam.
VP-NET is identifying curriculum changes which will help medical students practice palliative care in a way which is positive rather than threatening for disabled patients.
CARENET discovered medical trainees have significant exposure to dying patients, and are often responsible for determining their level of care. As a result, they are suffering significant sadness and fear, or complex emotions such as distress, grief, and guilt. The NET is exploring training and healing tools, working with key leaders in medical curriculum development.

Health, social and economic impacts

Commercialization of research and its downstream economic impact is an important outcome of CIHR-funded research. Given the health-services orientation of much of Canadian research in PELC, we would expect most economic impacts would be indirect, such as reduced health care costs. Determining these impacts was beyond our resources, and in any event would be premature. However, we did examine patent applications³² and found Canadians PELC-related filings rose from 3 in 2000 to 24 in 2006, and covered the range of medical devices, pharmaceuticals, biologics and diagnostics. Several were filed by CIHR, but not Initiative-funded, investigators. We speculate, however, that the eight-fold increase in identifying PELC as an application of the patent candidate discovery results from the Initiative's successful efforts to increase the profile and importance of PELC in Canada.

Supporting practice innovation

End-of-life sedation is a controversial and ill-defined clinical practice. 200 palliative care physicians from Quebec and Switzerland are working with the New Interventions NET to identify and improve sedation use.
A Cross-cultural NET investigator worked with support group facilitators to "choose together which groups might benefit the most from this kind of research." KT products evolved from needs expressed by the patients: a booklet documenting support group experiences to share with other new patients (distributed through the cancer agency); an article about prostate cancer misdiagnosis for GP Review, recommendations to funders for improving the groups.
Seriously-ill patients avoid life support to improve their quality of care, but does it? Right now, no one knows, but a CARENET study will provide them with the information they need to make this choice: 12-month survival and family caregivers' perspective on quality of care following critical illness.
The New Interventions NET has validated a simple instrument to assess pain in people with limited ability to communicate, and is now CIHR-funded to test a pain management program which incorporates this tool. The study has shown that participants experienced a 95% rate of relief from common catastrophic cancer complications without decreasing survival. At the same time, PedPal NET is testing a possible bio-marker of pain, to assess pain in children.
Family Caregivers NET seeks to reduce caregivers' stress and improve their sleep and appetite through tools and interventions to address the physical, emotional, and spiritual well-being of family caregivers. They are testing mindfulness-based stress reduction, meaning-making and exercise interventions.
A CARENET member is applying to an "innovation fund" in Halifax, to support an entirely new position to orchestrate a new model of care.
The Difficult Pain NET has two tools which are central components of a computerized patient assessment package; over one thousand patients internationally are being approached to participate in what will be one of the largest and most comprehensive cancer pain assessment and classification research initiatives ever undertaken, under the European Palliative Care Research Collaborative.
The New Interventions NET has established a national collaboration with CPAC and the Quebec Health agency to, among other things, provide cancer navigators with working tools and training curriculum as part of a Canadian Navigator manual. Data from this work also feeds into a pilot study to design a multi-faceted intervention to improve interprofessional collaboration, particularly among family physicians and nurse navigators, and ultimately improve continuity of care for patients with cancer.
VPRN NET has developed several web-based tools for physicians, including a web-based risk calculator and set of web-based prognostication tools, based on a database of over 10,000 anonymized palliative care patient records from Canada and the US.
CARENET is testing a home-based intervention to reduce symptom and caregiver burden for patients living at home with advanced COPD.
The Difficult Pain NET has developed a tool that classifies cancer pain, and will help allocate resources more quickly and efficiently to those in greatest need.
Other tools developed across the NETs include: a tool to assess the primary caregiver's burden in the context of palliative care, which is informing the development of specific interventions to help caregivers; a tool to self-report symptom intensity by advanced cancer patients, which has been widely adopted in palliative care programs nationally and internationally; and a new delirium screening tool now being implemented in Canada, Germany, USA, Australia, China and France among others.

Supporting policy change and resource allocation

One health authority is reviewing their ER deaths to look for potential policy changes needed in response to a NET study on deaths following ER admissions.
A NET has obtained a CIHR Knowledge to Action grant to support participatory action research which will engage front-line practitioners in a change process to transform the ways in which end-of-life care is provided on acute medical wards and extended care units.
Policy planners and program/service managers are using a NET's survival estimates to inform policy changes around eligibility for palliative benefits plan enrolment and admission criteria to hospices and acute/tertiary palliative care units.
Advance care planning (ACP) recently emerged as a major issue; regions are testing models, provinces are enacting legislation, and the federal government is developing national policy. The Cross-Cultural NET found that the relationships it had fostered with regional policy makers, managers, and clinicians allowed it to quickly develop a collaborative approach to researching and implementing ACP in its partners' organizations.
Several investigators are challenging Canada's "gold standard" of home death, and providing policy maker with critical information about patient and family preference and quality of care which is causing many organizations to re-consider their policies to better meet patient needs.
New Interventions NET provided the first comprehensive picture of service use and costs incurred by palliative care patients and their families, government and NGOs, while PedPal NET has analyzed 20 years of provincial data to determine causes, location and trends in death. These studies are informing ongoing provincial and regional service planning and resource allocation, to improve "what is often described by families as a 'disjointed and dysfunctional health care system'."
While it's no surprise that admitting a family member to the ICU is stressful, a CARENET investigator found 26% of family members of patients admitted to an ICU were struggling to cope and had high-risk post traumatic stress disorder symptoms two months later. CARENET is developing tools to identify who most needs help, and in what ways.
PedPal Net is undertaking the first prospective, longitudinal study of children with non-treatable, life-threatening conditions and their families. The NET aims to identify the changing needs of children over time, and the bio-psychosocial-spiritual impact on their families, to develop better systems to support both patients and families over the course of illness.
Reducing breakthrough cancer pain is a huge challenge. Pharmaceutical companies are seeking out PainNET's expertise to help identify and prove which drugs make a difference.
PedPal NET has discovered that the vast majority of the literature on family functioning is actually about maternal functioning, and seeks to inform policy about key areas where fathers have different needs.

“Funds have been well spent, but the investment won’t have the legacy that’s expected, unless it is sustained”
Knowledge user

Chapter 4 The Future for PELC Research

Respondents unanimously agree that the Initiative transformed the landscape for palliative and end-of-life care in Canada; many echoed the Advisory Board member who said "We now have a real research community, one of the best in the world. We should take pride in having done this, and in having the guts to do it." Respondents are less sure, however, whether in the face of such success, a responsible funder consolidates the investment... or moves on. Can we say that these successes mean that the Initiative has achieved its goals? Created in CIHR's early days, the Initiative, like all initiatives of its time, brings to these questions neither an evaluation framework nor an exit strategy.

What is successful capacity building?

The PELC experience highlights the challenge of making the leap from the old MRC to a new CIHR definition of success. The Initiative sought to build capacity, and did indeed significantly increase productivity and people. Yet many feel capacity-building will ultimately fail, as insufficient trainees are going on to tenure track positions and independent CIHR grants. They feel that, as an academic investment, success is transient and equivocal.
However, much progress has been made towards improving palliative and end-of-life care. The new capacity has strengths and goals resonant with CIHR's strategic objectives: health care and policy leaders are engaged in research, and a new cadre of research-trained health professionals is finding and implementing research-based solutions to health challenges. However, many of the newly trained investigators are not, and will never be, academically competitive by CIHR's world-class standards, because their priority is practice and policy change rather than the academic outputs valued by traditional peer review.

What are funders' roles and responsibilities to newly-built capacity?

ICR created the PELC initiative with an explicit understanding that this funding would be one-time only. Should we therefore accept, as one Advisory Board member put it, that "the pump is primed. This community must stand on its own two feet"? Certainly PELC has received an extraordinary infusion of funding, and the PELC community is well aware and appreciative of its good fortune.
However, many, and not just PELC grantees, believe that on principle, "it's wrong to set up this capacity then leave it to fail." The Canadian PELC research community is extraordinarily young in research experience, and PELC itself is considered an "immature" field worldwide. Funders internationally recognize that building capacity in a nascent area requires long-term commitment, not just a single shot of funds. UK funders, for example, told us they expect to continue PELC support for at least ten years, perhaps longer. Without a long-term sustainability plan, ICR risks losing much of the return on the investment already made. We, therefore, believe there are specific actions that ICR should take, which avoid creating perpetual dependency on strategic funding, and yet provide mentoring and developmental support to the PELC community as it matures.
The Initiative's funding partners are satisfied with its conduct and outcome, but unengaged with its future. They all see palliative and end-of-life care as relevant, but not their priority: "there could be an opportunity to partner, but not for us to lead." If ICR wishes to maximize the return on investment in palliative and end-of-life care research, it will need to engage partners in ways which make sense to those partners and forward their own agendas. A future coalition should look beyond the original Initiative partners³³, and well beyond cancer, to find those organizations whose needs can be met through engaging with this developing research and practice community. Palliative care is no longer congruent with cancer care, and is merging into long-term care for chronic disease. We also suggest that potential partners focus explicitly on the resources and expertise each could bring to next steps: when it comes to facilitating knowledge exchange and collaborating, partners have much to contribute beyond money.

What still needs to be done?

Expecting immature fields of research to "compete for open operating grants – just like the rest of us!' is equitable, but not overly productive. Capacity has been created, but it needs time and nurturing to move up to the internationally competitive levels needed for success in CIHR's increasingly competitive grants competitions. Otherwise, researchers and teams have huge incentives to seek funds elsewhere, and shift to other strategic areas of research with lower entry barriers and better funding. The completion of the Initiative will leave some major gaps in the resources needed to sustain PELC:
Researchers attracted from other areas: PELC attracted well-established researchers from other fields; they have been and should continue to be competitive for regular CIHR funds, but may revert to their original interests, where their reputation was built, after the end of the Initiative.
New investigators: NETs provided research infrastructure, contacts, mentors and significant funds for pilot work, all essential to develop sufficiently competitive grant and award proposals. This resource will disappear.
New trainees: a new cadre of "research-clinicians" needs part-time release which allows them to continue to be health professionals, managers and policy makers while also undertaking and implementing research related to their practice and decision-making.
Teams: NETs support the coordination and extensive communications essential for building trust among team members and creating an environment where creativity can flourish.
Partners: NETs provide resources and flexibility to support partners and the activities they value, including time for relationship-building, research planning and knowledge exchange.
PELC research community: The community is still small and dispersed across Canada, and needs to be better connected to maximize synergy in research and knowledge exchange.

Gaps in maximizing health care outcomes

NETs largely spent two years building relationships, and three years on foundational work: descriptive studies, problem identification, framework development, tool and instrument design and validation. An immature field like PELC needs assistance to transition into the next phase of feasible, fundable and ultimately successful intervention studies.

“For PELC, we need to work hand in hand with people offering care and services and evaluate what’s actually being done, what works in real time, and continuously work together and continuously evaluate what works.”
IAB member

Highly translational areas like PELC need funding for small, practical studies which, by academic standards, are neither rigorous enough (in the conventional hypothesis-testing manner) nor conceptually novel enough to pass muster at peer review. In addition, PELC researchers are engaging in participatory action research, recommended by CIHR's KT training modules, but not appreciated by its peer review committees because the objectives and research program cannot be fully predicted at outset.
Finally, the outputs of the Initiative as a whole need to be collected and packaged for use.

Recommended next steps

Building sustainable research capacity

CIHR looks to the Institutes to build capacity in critical gaps and emerging areas which meet the need of its translational mandate. However, there are major deficiencies in CIHR's open grants "tool box" when it comes to sustaining the new capacity it needs to deliver its new strategic plan. We recommend that CIHR make its existing open grants programs more open to translational research, through the use of explicit review criteria and changes in application forms, eligibility rules, and allowable costs. Proposal competitiveness should be enhanced, not discredited, by the inclusion of: complex and long-term programs of research; interdisciplinary teams and knowledge users, and the activities and associated costs to make them work; integrated KT approaches such as participatory action research; and a focus on dissemination and implementation activities far beyond conventional academic channels. New fields and investigators should be judged against appropriate "peers" rather than established academic disciplines, and, in collaboration with the research institutions, processes should be established and supported which help investigators "move up to the next level of competitiveness" (for example, mentoring and training systems, a voluntary developmental letter of intent process, etc.).

“I think PELC is about integrating research, evaluation, policy and practice – and this isn’t what open grants do”
IAB member

The most urgent missing item in CIHR's tool box is the open team grants program, whose cancellation seriously undermines CIHR's ability to deliver its new strategic plan. There is no point in funding NETs if there is no subsequent team grant program for them to emerge into.
CIHR also needs ongoing open grant mechanisms for supporting:

Part-time release for health-system based research clinicians
Networks, especially in emerging/strategic priority areas
Small feasibility studies (needed for operating grant proposals, emphasis on new researchers/areas); and small practice/system-oriented evaluation research. The Catalyst Grants are a useful addition to the tool box, but are available only in a few strategic areas.
KT support for informal synthesis and packaging of team-wide results

Maximizing research outcomes from the Initiative

We suggest the minimum necessary step is that Initiative partners hold an end-of-initiative forum to showcase synthesized results from across all the funded research and plan next steps and the future research agenda for PELC. Knowledge users and researchers should work together to identify: 1) what we have learned; 2) what we should do with it; 3) our specific needs for a community-wide knowledge translation and exchange network³⁴; and 4) the next directions, and how to support them.

The cost of doing nothing

If we extrapolate from current CIHR funding³⁵, we might expect CIHR to keep funding about $4M of PELC research per year through open operating grants. However, there is a high risk that a significant portion of the capacity created through this initiative will be lost if no further action is taken: NET PIs are not applying to the PLC panel as we would expect, nor are they submitting CIHR project grants with team members³⁶. This result is disturbing, but not surprising given the importance researchers placed on interdisciplinarity and research approaches, which are a poor fit with open operating grants. Trends suggest instead that teams are searching for other types of team funding, and researchers will be drawn to previous or new areas with more funding, or prestige.
Much of the newly developed capacity, as well as the next cadre of PELC researchers, could be lost to CIHR through lack of support to develop the young research community to the next level of CIHR competitiveness, or through drift to other, currently-funded strategic areas, where the barriers to entry are not so overwhelming. Without protected research time, the clinically-based NET and STIHR trainees will drift back to full-time health care roles and the return on investment in their training will be minimized. While a core of PELC researchers will certainly continue to work in this area, the fragmentation and loss of momentum will hinder research development and the translation of PELC research into practice.

“We always say these programs are not permanent, that at some point they sunset. If you don’t, you create a sense of entitlement. But you don’t want to be premature, to turn the tap off and kill what you’ve created. For PEOLC, I suspect it’s probably too early to cut the flow off”
A partner agency
“It would be wonderful to sustain the NETs, but that’s not realistic. But at least some funding to sustain the collaborations. I can’t over-emphasize the value of bringing collaborators together. It’s out of this that creative work happens.”
NET PI

References and notes

All quotations are drawn from interviews between the authors and key respondants from the Canadian and international cancer research and care community, unless otherwise noted.
Bisby Mark and Michelle Campbell. “Impacts of the CIHR Institute of Infection and Immunity 2000-2008”
Neil A. Hagen et al. “The Birmingham International Workshop on Supportive, Palliative, and End-of-Life Care Research” Cancer 107: 874-881.
ICR “A New Era in Canadian Palliative and End-of-Life Care Research”
The ICE, STIHR, and one NET were funded opportunistically: they were under review in other competitions at the time the Initiative was launched, and were deemed relevant to the Initiative.
Senator Sharon Carstairs has long been an advocate for palliative care, and was instrumental in persuading the government to develop the National Palliative Care Strategy, with a secretariat in Health Canada. She has led the development of numerous Senate reports addressing palliative care and most recently, aging:
- Of Death and Dying (1995)
- Quality End-of-Life Care: The Right of Every Canadian (2000)
- Still Not There: Quality End-of-Life Care - A Progress Report (2005)
- Canada’s Aging Population: Seizing the Opportunity (2009)
Although the minimum rating for funding is 3.5, funded applications in PELC, like in other open grants panels, have usually scored over 4.0, and in recent competitions, at least 4.5.
Neil A. Hagen et al “The Birmingham International Workshop on Supportive, Palliative, and End-of-Life Care Research” Cancer 107: 874-881 (2006)
ICR/Health Canada “Report of Palliative and End-of-Life Care Research Networking Infrastructure Workshop”
Canadian Virtual Hospice
For example, the Palliative Care Cross-Cultural NET and the Health Canada Secretariat saw the opportunity to piggy-back a PELC event on an existing KT forum, and were able to attract researcher and partner participants from 8 of the NETs to a Knowledge Translation Symposium and Workshop for the PELC NETs in Ottawa September, 2005.
Obtained by searching CIHR’s public database
Funding allocations were determined by searching the CIHR public funding database. Identification of a grant or award as “palliative care” is done by the applicant at the time of application for funding, and is not verified by CIHR.
Using the Scopus database, with search terms [palliative care] and [Canada]
One reason for this was that for 11/18 PIs, the Pilot Project Grant was their first CIHR grant and so they would not be expected to be highly productive earlier, as they were likely still in graduate training. Of the other PIs, most had only received a CIHR grant a year or so earlier, and were also at an early stage of their career.
ICR “Palliative and End-of-Life Care”
Canada had the had the highest ratio of PELC to oncology papers, and was second only to Norway in the ratio of PELC papers to biomedical papers: for example, the latter ratio was 0.006 for Canada, compared to 0.003 for the USA
Each circle represents one PI/co-PI in each NET. (All NETs except one had six or fewer members, and for the exception, fewer than six members have co-published). The upper circle represents the PI. Each coloured line represents the shortest loop linking team members who co-published. (For example, if members 1,2, 5 and 6 published a paper together, a link runs between 1 and 2, 2 and 5, 5 and 6, and 6 and 1, but not between 1 and 5 or between 2 and 6.) Publications of each NET are indicated in different colours. Not all NETs are represented as some have not yet co-published in the searched literature set.
The thin black lines represent co-publications not classified as “palliative care.” For clarity, they are not colour-coded.
The left figure represents publications of future NET participants during the period 2001-3, that is, from work performed shortly before the NET was funded. The right figure is from 2006-8, while the NET was in progress. Searches were conducted on the set of health research publication classified in the Scopus database as subject: “palliative care,” affiliation: “Canada.”
All bibliometric data reported are obtained from the Scopus database, unless otherwise indicated, using the simple subject search string “palliative care” and the affiliation string “Canada,” or with the affiliation string blank (for world publications).
There was a decline in share between 1997 and 2002, which is evident in other fields of health research, and then a subsequent recovery beginning in 2004-5, which may be attributable to the increased investment in health research of the early 21st century, including the Canadian Foundation for Innovation (CFI), CIHR, and the Canada Research Chairs program. A time lag between reinvestment and increased productivity in publication of three to five years would be expected. There has been a big increase in share for 2009 to date (end of April), though by year-end this may not be so pronounced.
There was a decline in share between 1997 and 2002, which is evident in other fields of health research, and then a subsequent recovery beginning in 2004-5, which may be attributable to the increased investment in health research of the early 21st century, including CFI, CIHR, and the Canada Research Chairs program. A time lag between reinvestment and increased productivity in publication of 3-5 years would be expected. There has been a big increase in share for 2009 to date (end of April), though by year-end this may not be so pronounced.
PIs and Co-Investigators of the NETs, Pilot Project Grants, ICE, STIHR and Career Transition Award funded under the auspices of the Initiative accounted for 230/458 or 50% of the total Canadian PELC publications between 2006 and 2008. The increase in number of publications between the two three-year periods 2001-03 and 2006-08 was 193, and the increase in publications authored and co-authored by the Initiative-funded PIs was 137, or 71% of the increase.
Table 4 shows the number of citations received per publication, again with data pooled over three-year periods, because the small number of Canadian publications resulted in considerable annual fluctuations in citations/publication. The number of citations/publication decreases in more recent periods, not because of declining quality of research, but because newer publications haven’t had sufficient opportunity to be cited by other authors.
An index of specialization (IS) of one means that Canadian publications in a particular sub-field are in the same proportion to all PELC publications as in the world literature: an IS>1 means that this is a particularly well-published research area in Canada, relative to the world literature
These data were more easily obtained from WoS, where they are available as a standard report.
At first glance, this seems incompatible with the declining percentage of foreign co-authors on Canadian publications shown in Table 3.3. However, a plausible interpretation is that there is a shift from foreign-led studies on which Canadians are collaborators to Canadian-led studies on which foreign authors are collaborators: this would be consistent with the development of the PELC research community in Canada.
The loss in position of the most productive 1996-08 authors is not simply due to their maintaining their productivity while newer authors leapfrog past them: collectively, the top ten authors had 109 publications in 1996-8, but only 47 in 1006-08. Two of the 1996-08 top ten were not represented among the 2006-08 authors.
When we examined the funding acknowledgements in a sample of 75 publications from 2005-09, we found that, while CIHR was the most frequently acknowledged funding source, 34 did not acknowledge any source of funding.
Trainees are ineligible to be listed as investigators on CIHR grants, but trainees who have CIHR awards (e.g. Canada Graduate Scholarships, Postdoctoral Fellowships) are included in the CIHR-funded list of names.
At the time the RFA was developed, KT was still in an embryonic state at in CIHR, and the NET RFA included no specific requirement for KT activities, though support for “research exchange” was an allowable cost. The striking emphasis on KT developed in the NETs reflects the attitudes of the funded investigators, rather than a deliberate aim of the Initiative.
NETs have been creative in developing dissemination approaches; for example:
- Special dedicated journal editions and text-book chapters in new areas
- DVDs, “how-to” pamphlets, research-based factsheets and policy briefs distributed through user communities; focus on user-oriented journals and outlets
- Audio CDs for rural patients with long drives to appointments
- Presentations at user-oriented events and conferences; e-mail updates to key knowledge users
- Part-time research librarian on NET staff CIHR-funded Café Scientifique, newspaper, radio and television interviews
- CVH and other knowledge portals; funding a communications consultant
- Discussions with parliamentarians at a “Day on the Hill”
As listed in the Scopus database, Canadian filings identified by the applicants as relevant to PELC.
Respondents identified some key potential partners in palliative and end-of-life care, including the Canadian Hospice Palliative Care Association, the Canadian Association of Provincial Cancer Agencies, the Canadian Cancer Society, Canadian Partnerships Against Cancer, and the Quality End-of-Life-Care Coalition, which brings together many charities with a wide range of (non-cancer) interests.
We emphasize that any plan for a PELC research/ KT network should originate from the research community, and be driven by demonstrable need, and tangible goals, rather than a generic belief that networking is good; the purpose of such an entity might be knowledge synthesis, mentoring and training, and international collaboration, for example. It must be driven by a coordinating communicating, and partnering network, similar to the recently-formed Canadian Dementia Knowledge Translation Network. Finally, before beginning any such discussion, ICR and the PELC partners must agree to commit support to a resulting entity, if the community can come together around an economic, feasible, and compelling proposal.
The more seasoned investigators funded through the Initiative will compete for funding in CIHR’s open grants and awards competitions. If we extrapolate current trends in PELC applications, success rates and funding , we would expect CIHR to be supporting at steady-state ~24 grants for an investment of ~$2.4M/year. Of course, there will likely be other relevant strategic funding opportunities from time to time, and PELC investigators will continue to be successful in CIHR’s open training and salary award competitions. Collectively, these other sources of CIHR support for PELC amount to just under $1M a year, so CIHR’s future support for PELC could be in the range of $3M to $4M, compared to less than $600k when the Initiative was launched.
We are concerned by what has already happened... or not happened. Of the 23 principal investigators (PIs) supported by the NETs, only 15 have applied for CIHR operating grants through PLC. Only two have been funded as PIs, four as co-investigators, and four applications are pending. The NET collaborations appear to be disintegrating: only 4 of the NET PIs have applied for operating grants in collaboration with two or more members of their teams, and only one of these applications has been funded: one is pending

Translate

Sunday, November 16, 2014

Palliative and End-of-Life Care Initiative: Impact Assessment – Report

A report prepared for the CIHR Institute of Cancer Research

Table of Contents

Highlights and Conclusions

Highlights of the Palliative and End-of-Life Care Initiative

What did the Initiative accomplish?

What did we learn?

What are the future opportunities?

Conclusions about the Palliative and End-of-Life Care Initiative

Impacts on the research agenda

Impact on research productivity and quality

Impact on research capacity

Impact of NETs and STIHR on training

Impact of NETworking

Impact on KT

Impacts on research-supportive infrastructure

Maximizing the return-on-investment from the PELC Initiative

Best practices in strategic initiatives

Integrated KT In Action

Chapter 1 Rationale for the Palliative and End-of-life Care Initiative

Chapter 2 The Design and Execution of the Initiative

Research funding

New peer review committee

Canadian Strategy on Palliative and End-of-Life Care Research Working Group

Birmingham UK Workshop 2005

Canadian workshop 2007

Infrastructure and networking

Evaluation

Chapter 3 Results and Outcomes

Enhanced funding for PELC research

Strategic grants

Pilot Project Grants

New Emerging Teams (NETs)

Career Transition Award (CTA)

Advancing knowledge – research outputs

Research productivity

Research quality

Canadian research specializations

International collaboration

Advancing knowledge – research capacity

How big is the new capacity?

Where did the new capacity come from?

Key capacity gap – clinician researchers

Mentoring: NETs as incubators for New Investigators

Training – who was attracted to the Initiative?

Training: Benefits and challenges of NETs and STIHRs

Training: Career paths of trainees

Collaboration and Teamwork

Networking

Informing decision making

Facilitating research use and useful research

Examples of NET KT in action

Improving KT

Changes in health practice, policy, services and education

Changing health professional education

Health, social and economic impacts

Supporting practice innovation

Supporting policy change and resource allocation

Chapter 4 The Future for PELC Research

What is successful capacity building?

What are funders' roles and responsibilities to newly-built capacity?

What still needs to be done?

Gaps in maximizing health care outcomes

Recommended next steps

Building sustainable research capacity

Maximizing research outcomes from the Initiative

The cost of doing nothing

References and notes

No comments:

Post a Comment