Published: 22-03-2018

Targeted reviews have always been the inferior offering of the literature synthesis portfolio. Although relatively quick and inexpensive to produce, their methodology is usually more opaque than a full systematic review, leaving them vulnerable to bias in the studies selected for inclusion and therefore in the conclusions of the report. 

At Crystallise, we believe that quick and affordable reviews do not have to be incompatible with systematic and transparent. 

We have two unique offers that let us transform the targeted literature review without increasing timelines and costs: the heoro.com database and the Evidence Map deliverable.

Targeted reviews powered by heoro.com

We recently completed a targeted review for a pharmaceutical client who had a product undergoing phase 3 trials and who was gearing up for a NICE submission in 18 months' time. The review had two main aims: to determine the PRO tools that were most likely to be responsive to change in the specific patient population of interest to guide future clinical trial design, and to be a foundation for a future full systematic review for the NICE submission. 

Expanding the breadth of the search without substantially increasing the workload

We designed and ran a systematic search of PubMed, Embase, the Cochrane library and heoro.com for the humanistic and economic burden, efficacy and epidemiology of the disease in question. The search of the larger databases, PubMed and Embase, had to be specific to keep the number of identified abstracts to a manageable size. Our search identified 2,887 abstracts from PubMed, 4,794 from Embase and 163 from the Cochrane library on the specific subtype of the disease that reflected the patient population of the phase 3 trial. This was a large number of abstracts for a targeted review but, despite this, we were concerned that we might have missed a number of relevant studies where our target population was a subset of all study participants, or that could fill evidence gaps from the specific disease searches.

Our solution was simple. We could search heoro.com much more broadly - going back to 1960 for the publication date, and searching for the broader disease type, rather than the specific subtype alone. Because the studies in the heoro.com database are pre-indexed, our search for PRO studies, costs and resource use studies, economic evaluations and mortality studies in the broader disease yielded "just" 767 abstracts - not insubstantial, but far fewer than such a broad boolean search would have found from PubMed and Embase. The approach is still transparent, as the studies in heoro.com are selected via a systematic search of the PubMed database, and the search strategy is published here. Indexing of the studies is done by our custom-built, automated software ahead of time, so there is no bias in how the studies are labelled, and the automated tagging is subsequently checked by our team of human experts to boost the accuracy of the indexing. 

We used this heoro.com yield to supplement the more focused searches of the larger databases, and the final number of abstracts we had to screen was 6,939. 

Targeted reviews delivered with Evidence Maps

The second way in which we transformed the targeted review was to generate an Evidence Map after abstract screening. 

Crystallise Evidence Maps are built in an interactive Excel spreadsheet that allows users to select one or two fields of interest at a time, and see the number of studies for each subcategory within those fields. Clicking on a specific cell in the Mapper allows users to see the citations indexed to that subcategory. 

An example of an Evidence Map for all studies on Pneumocystis pneumonia can be downloaded here. Please note that, to use the Map, you need to open it in Excel and enable macros.

We developed an Evidence Map that categorised 1,924 relevant abstracts according to the disease subtypes, interventions, geographical location, publication date, study type (Costs and resource use, Economic evaluations, PRO studies, Mortality studies, Epidemiology, Prognosis, and Diagnosis) and study methodology (e.g. RCT, systematic reviews, cohort studies). 

We then demonstrated the Map to the client and, between us, we selected the subcategories that were of most interest for the targeted review. Because we could see how many studies were in each subcategory, we could easily keep our selection within the agreed scope of 50 included studies for the targeted review. The client however had a copy of the Evidence Map, and could therefore have immediate access to the citations for the other categories. The targeted review was therefore completely transparent and the client had the Evidence Map deliverable just 4 weeks after agreeing the search strategy. Any gaps in the evidence were very easy to spot on the Evidence Map, which was very helpful for planning evidence generation activities. 

We then retrieved the full text of the selected papers and proceeded to data extraction and report writing as usual, delivering the report after a further 6 weeks, which included the Christmas and New Year break. The client was delighted with the Map, and we will use it as the foundation for a full systematic review later this year. This will be quicker and cheaper to produce than normal, as the abstract screening has been done already and so we only need to update the search and screen the new abstracts.  

To find out more about our transformational approach to targeted reviews please contact me: alison.martin@crystallise.com, or call on +44 (0)1375 488020.

  Alison Martin, Director and head of Reviews team