AI can tell if a patient battling cancer needs mental health support

Psychiatrists and computer scientists at UBC and BC Cancer have developed a new artificial intelligence (AI) model that can accurately predict if a person receiving cancer care will require mental health services during treatment.

The AI uses natural language processing and advanced neural networks to analyze an oncologist’s notes from their first consultation appointment with a patient. While these appointments are typically highly focused on a patient’s medical history and treatment options, the AI picks up on subtle clues hidden within the medical language that suggest a patient may benefit from early psychiatric or counselling interventions.

“Battling cancer can be a harrowing experience, affecting not only our bodies, but our minds and emotions,” said principal investigator Dr. John-Jose Nunez, a psychiatrist and clinical research fellow with the UBC Mood Disorders Centre and BC Cancer. “These findings demonstrate the tremendous potential of AI to essentially act as a personal assistant to oncologists, enhancing patient care by helping identify mental health needs sooner and ensuring more patients receive the support they need.”

According to findings recently published in Communications Medicine, AI can predict with greater than 70 per cent accuracy whether a patient will see a psychiatrist or counsellor within a year. It is the first time AI has been used to predict mental health outcomes from non-psychiatric medical documents.

“Neural models have been commonly called black boxes because it’s difficult to interpret how they arrive at an answer,” said study co-author Dr. Raymond Ng, a professor in UBC’s department of computer science and Data Science Institute. “These cutting-edge techniques allow us to unravel these mysteries for multiple documents at a time, which could further empower researchers and clinicians to understand the complex intersections of oncology and mental health.”

Mental health has been shown to have a profound impact on treatment outcomes and quality of life for people with cancer. On average, patients with significant depression and anxiety have poorer survival outcomes. Experts believe it’s because these patients having more difficulties following through with treatment recommendations and tolerating side effects.

The Canadian Association of Psychosocial Oncology estimates that approximately 15 per cent of cancer patients require psychiatric services, while another 45 per cent would benefit from seeing a counsellor. However, barriers such as stigma, lack of awareness of services, and difficulties diagnosing mental health conditions can prevent people from accessing psychosocial care.

As a next step, Dr. Nunez hopes to work with oncologists and patients to explore how such an AI tool could be implemented to improve early access to these important services.

“Perhaps the AI prompts the oncologist to talk with their patient about psychiatric services, or it could trigger an email to patients with a tailored list of available services,” said Dr. Nunez. “There are a lot of possibilities, but it’s important that we work with health professionals and patients to ensure the AI is meeting people where they need it.”

The AI was developed by an interdisciplinary team with expertise in computer science, medical oncology and psychiatry. The researchers trained and tested the model using data from 59,800 patients across all six BC Cancer sites located across geographically diverse regions of British Columbia.

To protect privacy, patient data remained stored securely at BC Cancer and was presented anonymously. Unlike patient chart reviews by human research assistants, the new AI approach has the added benefit of maintaining complete confidentiality of patient records.

The researchers also made significant advances in AI interpretability, developing a new technique that allowed them to peer inside the model to see how it was making predictions. Common themes arose, with certain key words and topics suggesting a greater need for mental health services: family history of cancer, patterns of alcohol or substance use, and certain types of aggressive cancers and treatment strategies. 

The team envisions expanding the tool’s use beyond oncology to other fields of medicine where psychosocial factors significantly impact patient outcomes. This could pave the way for a broader application of AI in healthcare, targeting early intervention across various medical disciplines.