Q&A

Question on machine learning- who is actually implementing the unsupervised machine learning model? Are these data scientists? From IT? Also how do you extract data from flow sheet in EPIC? Again is it through the IT?

JP: Definitely a team-based approach with these projects. Key players include: clinician/clinician investigator, data science team (statistics department), AI health organization (embeds data scientists, clinicians and programmers together), data programmers/analysts who pull the data out of Epic. However, Epic is now creating more self-service tools to pull de-identified data (e.g. SlicerDicer).

Does Epic currently support LLM/NLP features? 

JP: Nothing off the shelf to do this in real time. More tools like this being are being built in Epic for retrospective analyses. 

This is great to see that AI is planned to be integrated with Epic. Do you know what are the timelines and participating Epic areas/regions?

JP: Multiple AI tools for EPIC currently being rolled out, including tools related to ambient listening, notes summarization, in-basket messaging, and billing. Additionally, 4Ms Epic AI tools being rolled out to centralize 4Ms features around the clinical note into one place in the EHR. Deploying these tools at a larger scale does take time because of the robust evaluation prior to widespread implementation. 

How can we collaborate with GWEPs who are implementing AI? This will help us in learning and adapting in our care settings.

SL: Another great question- we are working on developing a clinical partner registry exactly to help connect folks like you with other partners

Additional links to materials/resources mentioned

• VA Innovation Ecosystem: https://www.innovation.va.gov/ecosystem/views/national-centers/ncchi/  
• Outcomes library: https://afhsresearchnetwork.ucsf.edu/4ms-evaluation-metrics-resource-library
• AI adoption by industry in 2024, Isakova: https://indatalabs.com/blog/ai-adoption-by-industry
• Age-Friendly Learning Health Systems: Opportunities for Model Synergy and Care Improvement: https://pmc.ncbi.nlm.nih.gov/articles/PMC9378562/
• Machine learning functional impairment classification with electronic health record data:https://pmc.ncbi.nlm.nih.gov/articles/PMC10524844/

Links/Resources

• Link to the slides used:  Age Friendly Research Network Equity Final - Read-Only 
• Connected Care for Older Adults: https://connectedcareforolderadults.org 
• An editorial: Disparities in High-Risk Medication Use—Implications for Health Equity and Deprescribing Practices: https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2832761 
• Rush Epic caregiver/care recipient template (an internal IT build at Rush that gives the ability to add caregivers to the care recipient’s chart): Age Friendly Research Network Equity Final - Read-Only  (see slide 47). 

Q&A

Q: for Elizabeth- are the CHW's using the same EMR as PCPs?
A: Yes! The CHWs spend a little time in clinic, where they do their charting and talk directly with the PCP/care teams

Q for Emily - can you tell us what the medication quality measure is that you’re using in the run charts? Is it med rec?
A: I see you answered it! It’s high risk meds - I’m assuming this is from the Beers criteria.
A: We pull data on all the medication on the IHI list, so if the patient is on 1 or more of the medications, then they are considered to have not met the measure. I know that is more strict that the goal of the M, which is medication review, but it was the best way for us to capture the data.

Learn more about the project discussed in this session

• You can visit the PA Teaching Nursing Home Collaborative website to learn more about the project: https://www.patnhc.org/

Q&A

• “For those of you involved with Geriatric Workforce Enhancement Programs (GWEPs), one of the requirements (of the 12 tasks) is to implement age-friendly health systems and training for age-friendly care across settings. One of those settings is nursing homes”
• At HIMSS last week I also had a chance to educate the EHR systems (Epci and PCC for example) to embed 4Ms.
  • Are the 4Ms still an add-on for EHR providers or are they getting better about integrating them into the standard system?
    • We’ve (IHI) had conversations with Point Click Care and Cerner too - hoping that they will integrate into their EHRs too…some progress, but very slow..
    • We just had a call with Epic yesterday. Epic has embedded the 4Ms for acute care and now the Epic team is working embedding 4Ms in post-acute care
    • From our experience, still an add on... still a work in progress
• We have traditional BSN/MSN degree programs. What is the best way to introduce 4Ms to the faculty and create a comprehensive curriculum around it?
  • Tools to get started: https://www.patnhc.org/get-started
  • Identify who your champions are (either faculty or students) annd support them and connect them to relevant resources

Useful tools & Links 

• Free and downloadable guides for Age-friendly care (IHI): https://www.ihi.org/age-friendly-health-systems-resources-and-news
  • Scroll down to find the guide specific for your setting.
  • You can also find a graphic on the 4Ms that we use widely and any organization can download and print for their teams.
• Dr. Lionel Lim, Geriatrician-Educator, from the VA Bedford and team developed an educational resource for teaching medical residents about Age-Friendly care in the nursing Home. The VA selected the rotation in 2023 as the agency’s exemplar of a team-based clinical learning environment CLE for the National Collaborative for Improving the Clinical Learning Environment. They have made this resource, A GEM of a GME Geriatric Medicine Experience, available to all https://hdl.handle.net/2144/49571.
• AACN Competences and 4Ms Crosswalk here: https://www.aacnnursing.org/essentials/database/kit/i/nursing-home-4m
• Survey: minimum data set https://www.cms.gov/medicare/quality/nursing-home-improvement/minimum-data-sets-swing-bed-providers
• This website is increasingly adding resources to support AFHS research e.g., a 4M evaluation metrics resource library https://afhsresearchnetwork.ucsf.edu/4ms-evaluation-metrics-resource-library
• Contact Aaron Guest for AFU https://www.afugn.org
  • AFU is part of JAHF’s Age-Friendly ecosystem team
• New textbook for NH leadership: https://www.amazon.com/Practice-Leadership-Nursing-Homes-Academic-Practice/dp/1646481259

Calls for Collaboration and Ongoing Work

• “I am very interested in developing a NH Medical Director curriculum - to lead to true engagement (not just awareness)”. If anyone has resources to share, please reach out for Cheryl Phillips from JAHF’s contact information.
• We're in the process of developing a dashboard using an implementation science approach to collect 4M data that will be accessible, usable, interactive, and public-facing to support the proof of concept. We also plan to do interviews and focus groups to learn about facilitators and barriers to implementation of the 4Ms in participating sites. We'd enjoy sharing this work when we're further along in our process. (AFHS RN admin working to create network content around this) 

Do you have any recommendations for screening caregiver stress?/How are you assessing for caregiver stress in the acute care setting?

• Caregiver stress assessments - Zarit Burden Interview (ZBI), Perceived Stress Scale (PSS), and Revised Caregiving Appraisal Scale (CAS)
• I believe ZBI is required under GUIDE but it is long. We would like to align caregiver stress screenings between outpatient and inpatient settings (CMS Age-Friendly Measure) but having inpatient staff do the ZBI seems daunting.
• RUSH uses the Burden Scale for Family Caregivers - short version - and for our GUIDE model caregivers we do use the Zarit due to the requirement.
• Tebb, S. S., Berg-Weger, M., & Rubio, D. M. (2013). The Caregiver Well-Being Scale: Developing a Short-Form Rapid Assessment Instrument. Health & Social Work, 38(4), 222–230. https://doi.org/10.1093/hsw/hlt019
• What are the approved screening tools for dementia staging? Can GUIDE Participants use other tools to evaluate beneficiaries?
  • The approved screening tools include two tools to report dementia stage – the Clinical Dementia Rating (CDR) or the Functional Assessment Screening Tool (FAST) – and one tool to report caregiver strain, the Zarit Burden Interview (ZBI). CMS may add screening tools throughout the course of the model.
  • GUIDE Participants have the option to seek CMS approval to use an alternative screening tool by submitting the proposed tool, along with published evidence that it is valid and reliable and a crosswalk for how it corresponds to the model’s tiering thresholds. CMS has full discretion on whether it will accept the proposed alternative tool. Details on the process for submitting proposed tools and tiering information will be shared with GUIDE Participants after July 1, 2024

Status of Age-Friendly Health Systems site recognition: 

• There are currently 4874 sites of care that have achieved Age-Friendly Health Systems recognition
• Do all of these sites include all 4 Ms? I’ve seen some use or focus on one or 2 but not always 4.
• The recognition status is the commitment to address all 4 Ms. The Committed to Care Excellence are those sites that report on all 4 Ms for at least 3 months (there are 2366 CCE sites!)

Is the consideration moving to the care partner language vs. “caregiver”?

• I've worked on projects where the caregivers have indicated they don't like "care partner" but open to hear what others think about language
  • Same for us - they prefer caregiver
• We use care partner or support person now
• Care partner includes a broader array of support persons and the continued engagement of the care recipients

Who is doing the burden scales at your orgs?

• Outpatient LCSW or GUIDE RN Care Navigator within the Geriatrics Clinic

Additional resources/recommendations

• "If you don't know, get them to someone that does!" (Alzheimer's Association hotline, Area Agency on Aging, etc.)
• FYI Some information about the CAP-CT program: carepartners.ucsf.edu
  • https://ucsf.box.com/s/wlroc21sqf44rgik566j9xfdhqr30ymf
• 2025 SLU Caregiver Virtual Conference- Theme on "Addressing Resilience Through Types of Loss and Grief" March 28th 9-3 central: https://slu.cloud-cme.com/course/courseoverview?EID=16254
• Other research on Daily Care Checklists to support caregivers: Dr. Atul Grande (Checklist Manifesto)

Time and Cost 

MW: Important work. Have you been able to build in the time cost and also whether there are improved outcomes? 
KH: Margaret - important questions - the time cost and change in outcomes / impact is a major goal of our Wake Forest academic group as we spread AFHS work to the rest of our Advocate MW hospitals and into the 30+ Advocate southeast hospitals 

Nursing/Provider Documentation 

CG: Are the 4M requirements being documented by nursing or providers? 
LS: In the VA we have different people from the IDT filling out different Ms. But the nurses and the clinicians do most of the documentation. 
AP: In our ambulatory care setting - the nurse may introduce the 4Ms and start the assessment process. The provider completes the assessment and “act ons” and documents those in the Epic 4Ms workflow. 
AG: The 4Ms are being assessed/documented for the most part by Nursing with the exception of Medication. But as we all know, it takes a disciplinary team to care for the patient so there is a collaborative effort on the 4Ms with Rehab, Pharmacy etc. 

Dashboard tools 

SH: What tools do you use for the dashboard (slicerdicer, reporting workbench, tableau etc)? 
AP: We use primarily reporting workbench for a provider-facing dashboard in Epic (with 6 months of AFHS 4Ms performance rates) and Tableau for system reporting. 
SH: Anybody try slicerdicer? A bit more naive but reporting workbench loads too slowly for us 
AP: We use slicer dicer for some data analytics - but we built formal dashboards for reporting workbench that could populate for the provider team. The primary approach is data extraction to a Tableau dashboard for tracking reliable use of the 4Ms at the national, regional, clinic and provider levels - along with race, ethnicity, primary diagnosis, and other data points. 
SH: Thanks! Yea I wish slicerdicer has more capabilities, but might need a combo of slicerdicer and reporting workbench. Trying to avoid tableau because I want providers to use it and opening another app is not going to happen. 

AP: Our providers don’t typically have access to slicer dicer - it’s more of a high-level data analytic and clinical leadership tool. Also, can spend hours in slicer dicer (as you likely know) and the providers are seeing patients and don’t have time. 

AG: For our audit tool we use Reporting Workbench. Our Data Dashboard pulls data from EPIC and we use BI Launch Pad. 

AP: Also, Epic has been working to integrate the 4Ms into their various platforms and in “snapshot” for example - so connecting with your EHR provider on what they already have available is key. 
 

Free text 

SH: For those that use free texting for what matters, any tips on how to quickly collate the info? 

AP: There is always a balance between free text notes (which aren’t typically reportable unless there is a full AI process in place) and what is “reportable” - allowing for both free text and reportable elements is helpful. Our system decided to go with reportable elements for standardization and evaluation. We can pull in certain free text items, but it requires much more work to review and analyze at scale. 
SH: My struggle too. Trying to find sophisticated ways of grouping these messages. 

Deciding on measures 

GK: Ann, you may have already mentioned this, but how did you decide on which measures to utilize for each of the 4Ms. What we’re finding is that even for what matters most, this is documented in different places, by different staff/providers in Epic 
SL: This is a great question. We’ve created an evaluation measure library here that might help. We are always looking for feedback on what might be helpful, so please let us know if you have any thoughts or ideas of other resources that might be helpful! https://afhsresearchnetwork.ucsf.edu/4ms-evaluation-metrics-resource-library 

AG: We worked with our System Nursing Research Manager to identify our system policies and also with a pilot of 6 hospitals to understand the current state and what is feasible. Based on what we learned and how we could optimize that, we then set standards for each M across our system. 

RT: For our evaluation work, we’ve chosen to align our measurement with the “minimal” set of care practices required for IHI level 1 recognition. 

 
Potential alert fatigue 

SH: How successful have the 4M bpa’s been - utility, change in behavior, etc. have they led to more alert fatigue? 

LS: We have not seen alert fatigue reported but have seen decreased falls and increased deprescribing in the VA. 
 

Medication outcomes 

GE: For medication outcomes, are groups focusing on all potentially inappropriate medications or more targeted, such as medications that impact mobility and mentation? 

LS: It can vary. The high-risk meds listed by IHI are a great starting point. 
SH: I’ve been using an arbitrary list: opioids, benzos, antipsychotics, etc. but it would be nice to have a ‘standard’ list. 
TL: We will be defining PIMs as defined by HEDIS. 

Cerner documentation 

TK: We are working toward our Level 2 Age Friendly accreditation in a clinic setting. Currently, I have a printed document that we are addressing the 4Ms on that is being scanned in. This month will be my first submission. My plan is to send those papers with my submission and those are scanned into the individual patient charts. We currently use Cerner and, as said, they are a little behind in getting in getting the EMR set up for documentation. For our clinic setting, is this sufficient or will we need to be documenting in the EMR more for data pulls? 
SL: Good question- do you mean for accreditation? Someone from IHI might have an answer here! 
TK Yes. I am just wanting to make sure I am doing these correctly. 

LS: You only have to submit the number of people you did the 4M’s on to IHI. Just keep count for each month. 

Video Anne mentioned 

LS: That video is a great idea for education on how to use the 4M’s. 
AM: Agreed- can that video be shared? 
AP: It’s specific to our documentation workflow of course, but I can check to see if we can share :). 

Suggestion – Microsoft form 

EG: Maybe we could send out a quarterly Microsoft forms to seek input on the documentation?