Connect with us
  • Elysium


Escaping rehab’s ‘black box’ problem

How rehabilitation patients and research projects are being hamstrung by the field’s inability to properly characterise treatments and interventions.



Deep-rooted in the rehabilitation field is the scourge of the “black box”.

It prevents research progress, hinders patient outcomes and makes it harder to justify spending on rehab services.

Its origins as a popular phrase among healthcare professionals can be traced back to the early 90s.

Its relevance has endured to present day, however; much to the dissatisfaction of a band of clinicians and researchers in the US.

As one paper they produced earlier this year puts it, the black box of rehabilitation is: “Our inability to characterise treatments in a systematic fashion across diagnoses, settings, and disciplines, so as to identify and disseminate the active ingredients of those treatments.”

Another paper in the same series notes: “Most often, (rehabilitation) treatments are defined by either discipline (“X hours of occupational therapy”) or the problem being treated (“gait training”), neither of which describes what the clinician actually does to affect functioning.

“Research reports that include detailed protocols often lack information about how a treatment was administered; for example, instead of reporting what quantities of active treatment ingredients were provided, treatment dose descriptions simply state the duration or number of sessions.

“Even published treatment manuals frequently lack sufficient details to enable other researchers to replicate findings or build on previous results or for clinicians to confidently implement published treatments in everyday care.”

Dr John Whyte (pictured) is among those who have been proactively rallying against the black box problem for over a decade.

Whyte is founding director of the Moss Rehabilitation Research Institute in Philadelphia and a specialist in traumatic brain injury and disorders of consciousness.

He has also spent over 10 years on a journey to solve the limitations of classifications in rehabilitation.

“The vast majority of our treatments are not drugs, but experiences that patients have in collaboration with a clinician such as a physical or occupational therapist,” he says.

“We’ve never had a good way of defining and labelling them. One of the most common ways of talking about the treatments people get in rehabilitation is by discipline – ‘he got six weeks of physical therapy’ – as though either everything a physical therapist does is effective, or nothing they do is effective, and all you need to know is how many times they met.

“Alternatively, we name treatments by the problem that they’re treating, like ‘gait training’, ‘memory remediation’, ‘social skills training’, but again the classification gives us no idea what actually transpired in those treatment sessions.”

The American rehab community, with international support, has been leading the way in unravelling the black box puzzle. In 2005, an American Congress of Rehabilitation Medicine taskforce became a hub for discussion of the black box problem.

The (then) National Institute on Disability and Rehabilitation Research organised a funding announcement that invited development of a new approach to treatment classification and measurement in rehabilitation.

A consortium of researchers and clinicians was then awarded a five-year grant and embarked on what has ultimately been a journey spanning over a decade.

Whyte has been a key participant in developing the concepts in that time and is now chair of the ACRM task force that continues to promote their implementation.

Thanks to a research contract from the Patient Centered Outcomes Research Institute, these procedures have this year been distilled into the Manual for Rehabilitation Treatment Specification (which can be downloaded here

Whyte says: “Essentially, we need to get to the point where the label of the treatment allows clinicians and researchers to say ‘this works’ or ‘that doesn’t work’, because everything with that label works or doesn’t. And, everything with that label can be studied and compared to something with a different label.

“The Rehabilitation Treatment Specification System (RTSS) gives a set of rules and procedures by which we would argue every rehabilitation treatment can be specified with respect to its known or hypothesised active ingredients.

“From here we can begin to use it as a taxonomy for grouping treatments. We can use it to aggregate treatments and show how one active ingredient is effective versus another active ingredient.”

The RTSS could be pivotal to improving rehabilitation research as well as patient outcomes.

Its development comes amidheightened pressure on the rehabilitation field to establish an evidence base for its interventions.

“Increasingly, this means the need for comparative effectiveness research on treatments with established efficacy, as well as sophisticated methods for synthesising evidence across studies,” says a paper published by those behind the RTSS initiative.

The same report also highlights the complications of proving the efficacy of rehabilitation treatments. There is the field’s complex, multidisciplinary team structure and its emphasis on behavioural treatments, “which depend heavily on the training of new skills and on the interpersonal dynamics between patient and provider”.

Also, “rehabilitation applies to a great diversity of populations, settings, and clinical problems, from impairments through activities to participation”.

The paper also notes that rehabilitation is patient-centred in a way that fosters “a nearly infinite variety” of treatment goals.

RTSS remains a work in progress for Whyte and his fellow black box crusaders.

There is an ongoing forum for discussion with rehabilitation professionals with diverse backgrounds who are interested in issues related to treatment specification (see more here).

Various projects are also underway to inform further refinement and implementation of the RTSS.

Educational efforts to increase awareness of the RTSS and enable its use by rehabilitation professionals are also planned.

Click on the links below to read more about the black box problem and the RTSS that could help to solve it.

A Theory-Driven System for the Specification of Rehabilitation Treatments (2019, Jan.)

The Rehabilitation Treatment Specification System: Implications for Improvements in Research Design, Reporting, Replication, and Synthesis (2019, Jan.)

Advancing Rehabilitation Practice Through Improved Specification of Interventions (2019, Jan.)

The Importance of Voluntary Behavior in Rehabilitation Treatment and Outcomes (2019, Jan.)

Editorial: An End to the Black Box of Rehabilitation? (2019, Jan.)

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *


Making the shift from victim to survivor



After having a stroke two years ago at the age of 39, former international swimmer Craig Pankhurst founded the charity Stroke of Luck to support stroke survivors through activity and exercise. Jessica Brown reports.

“Stroke survivors are in one of two places – they either see themselves as a victim, with a not very positive outlook,” Pankhurst says.

“Or they see their stroke as a bump in the road, but that no one will stop them from having a fulfilled life, just one that’s different to the one they were leading before.

“We put in a halfway line to move people from the victim to survivor mentality.”

Pankhurst wanted to build the charity’s website to enable interaction between stroke survivors and experts in neuropsychology and personal trainers trained to work with special population groups.

When the coronavirus pandemic hit the UK in March, Pankhurst decided to do live sessions on social media, where he brought in guests and spoke about his experience and the charity he’s set up – and says he got good interaction with viewers.

His efforts caught the attention of the World Stroke Organisation, which partnered with Pankhurst to create videos for stroke survivors for what they can do to aid their recovery safely at home, with the help of specialist physiotherapists.

The videos are uploaded by both Stroke of Luck and the World Stroke Organisation.

“I spoke to the World Stroke Organisation over a number of weeks and we agreed to collaborate to create story-specific exercise and activity videos for stroke survivors, to start releasing over 12 weeks, to see if they get good engagement.

“Then we’ll carry on, and do some more,” he says.

The videos are now organised into a library, colour-coordinated into red, amber and green, depending on the viewer’s ability. The library also includes specific videos for carers.

Continue Reading


The man who couldn’t see numbers

The unusual case of a man who can’t see numbers has led researchers to argue that the brain can process things without a person being aware of what they’re looking at.



Researchers from Johns Hopkins University studied a 60-year-old man known as RFS, who has a rare degenerative brain disease that prevents him from seeing numbers two to nine.

He would describe seeing one of these numbers as a tangle of black lines that changed every time he looked at it. He had otherwise normal vision, and had no problem identifying letters and other symbols.

The problem would happen before he knew which number he was looking at, which meant his brain had to at least know that numbers were in the same category before something could then go wrong; study author Mike McCloskey tells NR Times.

“It didn’t matter how we presented digits to him, they were always distorted,” says McCloskey, a researcher in the Cognitive Science Department at Johns Hopkins University.

The researchers didn’t know what they were looking for when they started working with RFS, as this specific pattern has never been recorded before. The closest recorded cases are of patients who see distorted faces.

The researchers, whose findings were published in the journal, ‘Proceedings of the National Academy of Sciences,’ also found that RFS couldn’t recognise anything placed near or on top of a number.

They recorded RFS’s brainwaves while he looked at a number with a face embedded on it, and found that his brain detected the face, even though he was unaware of it.

“In one experiment, we showed him a big digit with a face on top of it and recorded EEG signals to see how his brain responded to the face.

“Even though he couldn’t see the face at all, we could pick up a response in the brain 170 milliseconds after the face was presented.

“We saw a perfectly normal brain response to the face, which told us his brain unconsciously identified the face as a face, even though he wasn’t aware of it at all.”

In another experiment, they put words next to the numbers and told him a target word. When he saw the target word, his brain had a bigger response even though he said he couldn’t see the word. They also did tests where they placed a number in front of RFS and asked him to guess what a number was, to test implicit knowledge.

“Sometimes, blind people say they can’t see a light, but can often point to it accurately when forced to make a guess,” McCloskey says.

“We did that with him and saw absolutely no indication he had any implicit knowledge. He couldn’t tell us if numbers were the same or different, odd or even – yet the EEG showed his brain was responding.

The reason it could just be numbers that are affected, he says, is because evidence suggests the brain treats categories of things differently.

“Furniture, fruit and vegetables, for example, may be treated separately, so it’s possible for some areas to be affected and some not.”

The findings demonstrated that the complex processing needed to detect words, numbers and other visual stimuli isn’t enough to make a person aware of what they’re seeing.

“We can draw conclusions about what’s necessary for you to be aware of what you’re seeing. You’d think that, if the brain has done enough work on something to know it’s a face or a particular word, you’d be aware of it.

“These results tell us the brain can do an awful lot of processing on something you’re looking at without you being aware of it at all,” McCloskey says.

“Something else needs to happen after the brain has identified what it’s looking at before you become aware of it at all. ”

And the reason these findings apply to everyone else is because the researchers assumed RFS’s brain was the same as anyone else’s, except for this one thing that went wrong.

“In order to become aware of something, you have to do more than processing to allow you to identify what you see – we think this is true for everyone.”

As for RFS, who was a geological engineer, the story has a happy ending.

“Because RFS couldn’t see regular digits, this was a real problem for him. We created a new set of symbols for him for digits, to see if he could use those,” McCloskey says, as well as a calculator on his phone using the digits.

“He learned them very easily – we wondered if they’d get distorted for him but fortunately, they didn’t. He says he’s been using the symbols ever since – he uses them in his daily life and stayed in his job two years longer than he would’ve otherwise, because of them.”

Continue Reading


“Assume brain injury” after domestic violence, researcher urges

Domestic violence survivors in hospital should automatically be tested for traumatic brain injury (TBI) because they, and doctors, may not be aware of the symptoms.



That is according to researcher Jonathan Lifshitz, director of the Translational Neurotrauma Research Programme at the University of Arizona’s college of medicine.

When a patient goes to the doctor with a cough, they’re tested for numerous diseases to rule them out, but with intimate partner violence (IPV), Lifshitz says, we should “flip the script”.

He tells NR Times: “If the individual doesn’t have encyclopaedic knowledge of what TBI is, they may not offer all the symptoms up to their healthcare provider.”

Similarly, the brain injury itself may prevent the patient from being able to detect their symptoms.

Instead, practitioners should suspect that victims of IPV have a head injury, so they can be tested.

“If we tested all people experiencing intimate partner violence for TBI, and are able to screen them using objective tests, we’re going to have far fewer people who experienced intimate partner violence and go untreated,” he says.

In one study, Lifshitz found that 62 per cent of people subject to IPV and diagnosed with TBI were unaware of their TBI when they sought treatment.

While it’s a challenge to determine that someone has TBI, the risk of missing something, Lifshitz says, is much greater. And while increased testing would incur more cost, due to additional testing, Lifshitz says it would save money.

“An individual may be able to hold down a job better, be less dependent on services and won’t need healthcare services as much in the long run,” he says.

Lifshitz is involved in the Maricopa County Collaboration on Concussion from Domestic Violence (MC3DV), a county-wide collaboration in Arizona. It aims to increase the suspicion of head injury by analysing health data for patterns and problems that can be targeted with a county-wide approach.

It educates police officers to recognise symptoms, social workers to better identify abusive relationships, emergency services to profile forensic evidence and clinical partners to assess and treat symptoms of TBI and concussion.

Also, prosecutors through the Maricopa Country attorney’s office are supported in being able to build their case against the assailant; while scientists and process developers also help to bring everything together.

Meanwhile, social workers and nurses are educated on the signs and symptoms of TBI, proposing an objective measure where head injury is implied.

Hospitals are a key area of focus for MC3DV, where one challenge is rebuilding trust between medical practitioners and patients who have previously suffered discrimination, and as a result have a lack of trust.

“It would be easier to implement this change in one crisis shelter or emergency department, when we have the opportunity to regulate and control the organisation we’re working with and we can put in new policies and procedures,” Lifshitz says.

“When trying to coordinate multiple systems in multiple organisations, it’s much more challenging.

“While everyone is receptive to the topic, the problem is having enough resources to do it.”

MC3DV is also hoping to replicate state-wide efforts made in 2012 to better detect evidence of strangulation.

As a result, Maricopa County prosecutors attributed the rise in domestic violence prosecution and decrease in domestic violence deaths to this change.

“Arizona recently changed the way the legal system deals with strangulation, in terms of how it sees evidence,” Lifshitz says.

“Prosecution is much more rapid and severe; it’s unburdened the legal system because many more cases are starting as guilty.”

Lifshitz hopes better testing and evidence gathering will act as a deterrent for abusers, and provide additional motivation for victims to step forward, although, he concedes, psychological, emotional and financial controls an intimate partner has over their victim complicates this scenario.

“A patient wanting to seek treatment is very different from the ability to seek treatment,” he says.

Lifshitz hopes there will be some real changes to come out of the research programme.

“I’ve always needed a bigger social driving force to keep me motivated. It’s unconscionable to know about this and not do anything about it.

“This programme helps to bridge the gap between social work, police work and biomedical research, to attack this problem from multiple angles.

“The majority of the work we do is stepping back, looking at what healthcare data we might have, and asking very specific questions.

“We sit around the table not necessarily with the smartest people, but with the most passionate people. It’s not any one person doing the work, but relying on a community of providers to support the victims and warn abusers.”

Continue Reading
Softer Foods