Deep-rooted in the rehabilitation field is the scourge of the “black box”.
It prevents research progress, hinders patient outcomes and makes it harder to justify spending on rehab services.
Its origins as a popular phrase among healthcare professionals can be traced back to the early 90s.
Its relevance has endured to present day, however; much to the dissatisfaction of a band of clinicians and researchers in the US.
As one paper they produced earlier this year puts it, the black box of rehabilitation is: “Our inability to characterise treatments in a systematic fashion across diagnoses, settings, and disciplines, so as to identify and disseminate the active ingredients of those treatments.”
Another paper in the same series notes: “Most often, (rehabilitation) treatments are defined by either discipline (“X hours of occupational therapy”) or the problem being treated (“gait training”), neither of which describes what the clinician actually does to affect functioning.
“Research reports that include detailed protocols often lack information about how a treatment was administered; for example, instead of reporting what quantities of active treatment ingredients were provided, treatment dose descriptions simply state the duration or number of sessions.
“Even published treatment manuals frequently lack sufficient details to enable other researchers to replicate findings or build on previous results or for clinicians to confidently implement published treatments in everyday care.”
Dr John Whyte (pictured) is among those who have been proactively rallying against the black box problem for over a decade.
Whyte is founding director of the Moss Rehabilitation Research Institute in Philadelphia and a specialist in traumatic brain injury and disorders of consciousness.
He has also spent over 10 years on a journey to solve the limitations of classifications in rehabilitation.
“The vast majority of our treatments are not drugs, but experiences that patients have in collaboration with a clinician such as a physical or occupational therapist,” he says.
“We’ve never had a good way of defining and labelling them. One of the most common ways of talking about the treatments people get in rehabilitation is by discipline – ‘he got six weeks of physical therapy’ – as though either everything a physical therapist does is effective, or nothing they do is effective, and all you need to know is how many times they met.
“Alternatively, we name treatments by the problem that they’re treating, like ‘gait training’, ‘memory remediation’, ‘social skills training’, but again the classification gives us no idea what actually transpired in those treatment sessions.”
The American rehab community, with international support, has been leading the way in unravelling the black box puzzle. In 2005, an American Congress of Rehabilitation Medicine taskforce became a hub for discussion of the black box problem.
The (then) National Institute on Disability and Rehabilitation Research organised a funding announcement that invited development of a new approach to treatment classification and measurement in rehabilitation.
A consortium of researchers and clinicians was then awarded a five-year grant and embarked on what has ultimately been a journey spanning over a decade.
Whyte has been a key participant in developing the concepts in that time and is now chair of the ACRM task force that continues to promote their implementation.
Thanks to a research contract from the Patient Centered Outcomes Research Institute, these procedures have this year been distilled into the Manual for Rehabilitation Treatment Specification (which can be downloaded here http://mrri.org/innovations/manual-for-rehabilitation-treatment-specification/).
Whyte says: “Essentially, we need to get to the point where the label of the treatment allows clinicians and researchers to say ‘this works’ or ‘that doesn’t work’, because everything with that label works or doesn’t. And, everything with that label can be studied and compared to something with a different label.
“The Rehabilitation Treatment Specification System (RTSS) gives a set of rules and procedures by which we would argue every rehabilitation treatment can be specified with respect to its known or hypothesised active ingredients.
“From here we can begin to use it as a taxonomy for grouping treatments. We can use it to aggregate treatments and show how one active ingredient is effective versus another active ingredient.”
The RTSS could be pivotal to improving rehabilitation research as well as patient outcomes.
Its development comes amidheightened pressure on the rehabilitation field to establish an evidence base for its interventions.
“Increasingly, this means the need for comparative effectiveness research on treatments with established efficacy, as well as sophisticated methods for synthesising evidence across studies,” says a paper published by those behind the RTSS initiative.
The same report also highlights the complications of proving the efficacy of rehabilitation treatments. There is the field’s complex, multidisciplinary team structure and its emphasis on behavioural treatments, “which depend heavily on the training of new skills and on the interpersonal dynamics between patient and provider”.
Also, “rehabilitation applies to a great diversity of populations, settings, and clinical problems, from impairments through activities to participation”.
The paper also notes that rehabilitation is patient-centred in a way that fosters “a nearly infinite variety” of treatment goals.
RTSS remains a work in progress for Whyte and his fellow black box crusaders.
There is an ongoing forum for discussion with rehabilitation professionals with diverse backgrounds who are interested in issues related to treatment specification (see more here).
Various projects are also underway to inform further refinement and implementation of the RTSS.
Educational efforts to increase awareness of the RTSS and enable its use by rehabilitation professionals are also planned.
Click on the links below to read more about the black box problem and the RTSS that could help to solve it.
The psychiatrist fighting for domestic violence victims
Australia’s New South Wales government has promised to improve brain injury testing for domestic abuse victims after a psychiatrist drew attention to inconsistent care for vulnerable women. Psychiatrist Karen Williams urged the government to adopt a concussion protocol for family and domestic violence victims after doing her own research and being shocked at what she found.
It started when Williams noticed the disparity in how her patients were diagnosed and treated.Williams specialises in the treatment of post-traumatic stress disorders (PTSD), often with military, police, emergency personnel and other first responders.
But she also treats the general population, the vast majority of whom are women with histories of child and domestic abuse.
“I was getting two populations,” she tells NR Times. “The military patients, who are clearly identified as having PTSD, and the female population, who are mostly identified as having depression, anxiety and personality disorders, but had incredibly high rates of abuse in their histories.But Williams saw that whilst both groups had similar symptoms, and similar levels of trauma, they had very different treatment options.
“There’s a lot more funding put into supporting traumatised soldiers and first responders than there is for women who have experienced trauma within their home.
“In Australia, we don’t have much at all for women and children victims of abuse.”
This was a particular concern because of the amount of times Williams had heard about multiple head injuries and concussion among women who were victims of domestic abuse, which is similar to boxers and those player high contact sports.
“Women who’ve been unconscious several times or strangled have symptoms such as memory deficits, insomnia, migraine and mood swings, which all could be put down to PTSD and depression, but also brain injuries.”
But if Williams wanted to find out if a patient had a history of brain injuries, she would have to refer them for neuropsychiatric testing, which costs up to AUS$1000.
“This is completely unaffordable for many abuse victims so it just doesn’t happen, so we don’t investigate women who’ve had brain injuries.
“One brain injury unit told me they would consider taking on a patient if they could provide evidence that an assault happened – such as hospital records.
“This completely fails to take into account that the vast majority of domestic violence survivors will not report any assault to anyone and will not have so-called evidence.”
Then, Williams was speaking to a colleague whose son had had a head injury in a sporting field.
While they were together, a nurse rang to follow up the treatment he’d received in the emergency department.
“The nurse asked how her son’s personality and memory was, and gave a fantastic run-down of the symptoms that can happen after a concussion,” Williams says.
Williams was shocked – she’d never heard of someone ringing up women after a head injury in a domestic violence case.
She rang the local emergency department and asked about their protocol following a head injury obtained during sport.
She was given a detailed outline of the observations they take, their plan over the weeks following the patient’s injury and the advice they give the patient.
Williams called several emergency departments in other Australian states, and whilst all had a protocol for sports players following a concussion, none said they had a protocol for women who had been the victim of domestic abuse.
“There wasn’t one place that said they had a particular protocol.
“If they knew the woman had had a head injury they’d give them the basic head injury protocol, but nothing specific that took into account the very individual needs that a woman with a head injury in a domestic situation might have,” she says.
Williams says research indicates health care professionals correctly identify family violence victims about one per cent of the time.
“In sporting players’ protocol, there’s a recognition that says that your patient may not know what they’ve experienced in the past was a head injury, so the advice is to be really explicit. They’re given a list of questions to break it down with that player to make sure they understand what could be a head injury.
“There is opportunity for scanning, and neuropsychological testing if there is evidence of persistent symptoms.”
Williams says doctors should be going through the history of women, too, to see if they’ve lost consciousness in the past.
“There are a variety of mechanisms in which a woman experiences brain injuries in a domestic situation, many more than sporting probably, and the more head injuries a woman has, the greater her chance of long-term problems,” Williams says.
This includes a higher risk dementia, PTSD, migraines, learning problems and memory problems.
“But women aren’t told this, so many don’t know that they’re at risk of these things.”
Williams says there is a ’hidden epidemic’ of women in the community with brain injuries no one knows about, who could have been diagnosed with mental health issues instead.
In 2018, Brain Injury Australia released its findings after looking at the prevalence of brain injury in victims of domestic violence.
It found that 40 per cent of victims who attended hospitals in Victoria, Australia, for domestic violence had a brain injury and the majority were women.
But there’s no specific treatment for these women, Williams says, and many won’t even know they have a brain injury.
“Abused women are a very neglected population, and when you think about the money being spent on sports, and sports players, there’s no reason we can’t look after woman as well,” Williams says.
But despite these findings, Williams says it didn’t lead to any change.
“When I found all this out, I was angry and upset,” Williams says.
She arranged to meet New South Wales’s Labour MP Anna Watson in August, and when Williams told her what she’d found, she says Watson was ‘mortified’.
“She immediately got on the phone with the office of the minister for the prevention of domestic violence, and requested a meeting as soon as possible.
In the Zoom meeting a month later, Williams went over what she had found with Mark Speakman, Attorney General and Minister for the Prevention of Domestic Violence, and outlined the obstacles facing women.
But he made no promises, and Williams said she felt he didn’t understand the urgency or gravity of the situation.
Watson then went to the media, and the story was reported on. Within a week, the government produced a statement promising to investigate the issue.
“We’ll all be watching and holding them accountable,” Williams says.
“I will be trying to follow up, I won’t let it go.”
Williams is disheartened that it took media coverage to get the government to respond, but says she’s learnt a valuable lesson.
“Part of the reason I’ve spoken to the media and been vocal about it, is my experience is that when we do things quietly and ask for things politely, the government says there’s no money, despite being one of the wealthiest countries in the world. But when the voting population starts getting angry and asking what’s going on, that’s when we see an answer.
“It’s been a sad realisation for me to recognise that people don’t respond to do the right thing, they respond to winning the vote, so I will keep being as loud as possible in the media.”
Leaving brain injuries undiagnosed has significant consequences, Williams says.
“You’ve got women feeling like they’re a bit crazy, women wondering, ‘Why don’t I remember things, why have I got headaches all the time, why can’t I sleep?’
“It makes women feel worse, like something is wrong with them rather than identifying the underlying cause that we’re completely missing.
“If women are unable to work due to the physical and psychological side effects of a recurrent head injury, they need to be able to apply for NDIS funding (National Disability Insurance Agency). f they don’t know they have a brain injury they will be left to flounder – which is what is happening now.
“The vast majority of doctors don’t know about this. The medical system failing these women.”
There will be a lot to work out as support becomes available, Williams says, as some women could fear that having brain injury diagnosis could interfere with them getting custody of their children.
But, ultimately, change will benefit these women.
“All women deserve to know the truth about what’s happening to them,” she says.
“In some cases, their brain injury will be the final straw. They might think an act of violence isn’t a big deal, but if a doctor says, ‘Look how many times this has happened to you, you could end up long term brain damage’, that might be the final push that makes her take steps to leave. There’s no excuses to justify why these conversations aren’t had.”
Interview: Inside one of the world’s biggest concussion studies
Concussion is a huge concern across the US military and in sports. In 2018, 19,000 military personnel were diagnosed with a traumatic brain injury, while college athletes had an average of 10,500 concussions for past five years.
Despite the numbers, many say there’s a lack of research to inform ways that government and industry can best tackle this problem.
In response, the largest prospective concussion study was formed to fill the gaps in understanding, to see what recovery from a concussion looks like in athletes and cadets.
More than 44,000 people have since enrolled in the CARE (concussion assessment, research and education) consortium since its inception in 2014, across 30 universities and four military service academies across the US.
It has so far captured data on more than 4,300 concussions. The study is funded by the National Collegiate Athletic Association (NCAA) and the Department of Defence.
It’s believed that NCAA athletes represent the best model for what happens with concussion in the military.
Researchers involved in the study hope their findings will allow them to predict what happens to people after a concussion; information which can then help inform protocols that could become the standard for universities and the military.
Steve Broglio, associate professor at the University of Michigan’s School of Kinesiology and departments of neurology and physical medicine and rehabilitation, and one of the project’s leaders, says the initial aim was to be able to define the acute history of concussion, and see what happens to people after they have a concussion, establishing both a clinical arm and a research arm.
“In the first days of the project, we enrolled 35,000 civilian athletes and military service cadets to try to understand what was going on,” he says.
“We captured this by understanding their clinical natural history,” such as if they went to the doctor about their symptoms.
“The second arm of the project was to understand what’s going on a biological level, using genetics and biomarkers and advanced imaging, to see if recovery on a biological level reflects the medical level,” Broglio says.
In 2018, the team moved on to the second phase of the project, which was to understand the persistent and long-term effects of concussion.
“We continued to enrol people and we now have 55,000 participants. Each one receives a baseline exam when they enter institutions,” Broglio says.
“The second phase is now starting to get exit data as gradates do another evaluation to see if their concussion has had any effect on their brain functioning.
“In parallel, we’re also reaching out to people who graduated from intuitions, so they can do online evaluations to see if the long-term reflects just after they graduated, and within the first five years of gradation. The goal now is to start tracking people for their whole life to see the trajectory, and to see what percentage of people have issues,” he says.
Thanks to its findings so far, the consortium has participated in setting the concussion policy for the NCA, which outlines how concussions are managed, Broglio says.
So far, CARE has published around 60 papers relating to various findings, and Broglio says some of the consortium’s findings have had more impact than others.
In general, he says, findings that chime with a wider body of research that came to the same conclusions are more likely to help enact changes in policy because they will carry more weight.
“Some of what we’ve found doesn’t match what other people have found, some things have been consistent with other studies. When it matches, we can say, ‘Right, we need to change something’,” he says. This research is unlike any other, he says, partly because of how far-reaching it is.
“We were interested in getting a broad understanding of what’s going on across all cohorts. The very first goal is to understand the natural history of concussions, and the recovery rate of athletes and cadets participating in multiple levels and across different sports and different sexes.
“Prior to the project, most of the literature focused on male contact collision sport athletes, such as American football, maybe ice hockey and lacrosse. We have close to 50 per cent women in the study, across every NCA sport.”
These sports include basketball, baseball, ice hockey, water polo and cross country. The areas with the most reported concussions, according to the NCAA, are women’s soccer, football, ice hockey and wrestling.CARE’s most recent research, which is yet unpublished, shows that there are different recovery rates based on the sport.
“No one has ever shown this,” Broglio says.“There’s almost an identical recovery rate between men and women that hasn’t been found before, and which we didn’t anticipate. It’s largely been recorded that women take longer to recover, but when matched with equivalent sports, men and women’s recovery rates are virtually identical, which is a pretty significant finding.
“We’ve also had a series of papers looking at the cognitive performance of contact athletes relative to non-contact. They perform the same, if not better. This, he says, runs counter to the school of thought that repeated blows to the head causes chronic traumatic encephalopathy, a neurodegenerative disease which causes severe and irreparable brain damage.”
Some of those things are different to what’s been previously reported, which also opens the door for more research and conversation,” he says.
As well as research, the project is also focused on education for athletes, trainers, coaches and families.
Funding for CARE expires in one yea, and the consortium is in the process of submitting for the next five-year cycle.
“We could be around for as short as 12 months, or it could be another five years. Ideally, it would be another 50 years, so we can track participants,” he says.
“You can’t wallow – you’ve got to get on with it”
Having experienced a brain haemorrhage five years ago, 52-year-old Jane Hallard from Gloucester has had to rebuild her life. Here, she details her struggle and how she has learned to look to the future with positivity.
In the five years since my brain injury, I’ve had to come to know a whole new me.
While I look the same as I did, I’m far from being the same person.
On that day, back in 2015, when I was helping my son to clean his car, little could I have imagined what lay ahead.
There was nothing unusual about that day, I felt fine, nothing was amiss at all.
Then, without warning, I can only describe the pain as if my head was being hit by a sledge hammer. I had experienced, I later learned, a subarachnoid haemorrhage.
I was rushed to hospital, where I underwent life-saving surgery and then spent the next eight weeks recovering from the ordeal which came out of nowhere, but was to change my life more than I could ever have predicted.
The main issues I experienced, and continue to have to overcome on a daily basis, are the impact on my mental health and the brain fatigue.
Both completely unseen by others, they are hugely difficult for me and impact on my life in so many ways.
They are very hard to explain to people – I couldn’t understand what was happening to myself at first, so couldn’t really expect family and friends to.
My children, then aged 22, 21 and 15, were fantastic, and my mum was hugely supportive too – although my husband, who I had only married six weeks before my haemorrhage, decided he couldn’t cope and left.
The emotional impact of that was another big barrier to overcome.
My mental health is up and down and finding ways to manage that has been hard, particularly over the past few weeks during the COVID-19 pandemic.
That did set me back greatly, but I feel like I have picked myself up again now.
Emotions are very difficult to deal with, and I often react to things very differently to how I used to. Furthermore, the brain fatigue has a huge effect.
Often I describe this to people, and they’ll say ‘I get really tired too’, which I appreciate is them trying to understand, but they really don’t.
Whereas a person without a brain injury will go to bed and sleep, during which time their brain recovers, for me, my brain will only recover to perhaps 70 per cent of what it used to.
I often feel I’m starting a day six steps behind everyone else. I generally go to bed at 8pm, as I only have a certain amount of energy to use to get through the day, and while that gives me the best chance of feeling able to face the following day, having such an early bedtime means I have no social life.
I returned to work around eight months after my brain injury, and because I looked fine, people presumed I was back to ‘normal’, little realising I was far from the person they used to work with.
People just don’t understand. I used to print off leaflets about brain injury and its impact and leave them around the office, hoping colleagues would read them and learn a bit more about what I was living with on a daily basis, but people just don’t find time for things like that in their busy lives.
One of the most distressing parts about my whole ordeal was finding out upon discharge from hospital how little support and guidance there is out there.
When I was discharged, I was given a follow-up appointment a few weeks down the line, a couple of leaflets, and that was it.
I was left with a life-changing injury and just released back into my old life and expected to get on with it.
I had no idea what to do or where to go. My children looked online to find information, online forums and local groups for me to attend, so I could find somewhere where people genuinely understood.
My Headway branch in Gloucester was a lifeline for me. In fact, if it wasn’t for them, I don’t know where I’d be. I don’t know whether I’d be here at all.
I still upsets me thinking of people who are in the position now that I was five years ago, not being directed towards Headway or any other kind of support.
I’ve been there and now how that feels, and I strongly believe that needs to change.
Now, generally, I feel in a much better place. I’m all set to start a new job, which is a new challenge for me doing something totally different, and I can’t wait to get started.
I’ve found that you can’t wait for things to get better, and it might not get easier, but you can find ways to deal with it. I do a lot of volunteering work at a local food bank, and that puts things into perspective for me.
You can choose the way you go in life – you can sit around wallowing, or you can get on and make the best of it. That’s the attitude I have learnt to adopt and the one I’m going to live my life by from now on.
- Opinion11 months ago
Biting back against a common threat
- News1 year ago
The next gen-air-ation of sensory spaces
- Interviews2 months ago
The neuropsychologist teaching tai chi
- Opinion1 year ago
How to find the perfect powerchair
- News1 month ago
Achieving independent bed access with neurological conditions
- News3 months ago
‘Excluded’, ‘stigmatised’ and ‘feeling inadequate’
- More headlines11 months ago
2020: The year of the exoskeleton?
- Insight2 years ago
A ticking timebomb in stroke care