This article was written by Toniann Richard, Executive Director, Health Care Collaborative of Rural Missouri, for the “Networking News” monthly newsletter. The Network Technical Assistance Project is funded by the Federal Office of Rural Health Policy, Health Resources and Services Administration, U.S. Department of Health and Human Services through a contract to Rural Health Innovations, LLC, a subsidiary of the National Rural Health Resource Center.
At the Health Care Collaborative of Rural Missouri (HCC), having encountered both challenges and successes, we unequivocally say Friend!
The HCC is a rural health network, committed to improving the health status of underserved populations in a three county area. HCC was formally established in 2006 following a three year period of working informally together to address area health care needs. The purpose was to ensure that the health care needs of all citizens in our service area were met, particularly the needs of the under- and uninsured. Since forming, our focus has been to develop and implement programs that are responsive to the documented health needs of county residents, with specific health status indicators as benchmarks for progress on addressing those needs. Of course, this was how we embarked on our telehealth journey in 2011.
HCC applied for and received a USDA Distance Learning and Technology grant with network partner Lafayette Regional Health Center (a Critical Access Hospital). We purchased several telehealth units for the rural health clinics as well as one for the emergency department and outpatient clinic. The initial project was focused on integration of mental health services with HCC’s network partner Pathways Community Health, a Community Mental Health Center.
Initially we had several bumps in the road. Let’s start with the obvious…high speed internet. Our network is located in a very rural part of Missouri - think dial-up connectivity. So, step one was finding a connection that was both fast enough and affordable enough for all of our partners. We began by partnering with an information technology vendor who understood the value of purchasing connectivity in rural Missouri. We then applied for Universal Service Administrative Company funding and were granted reduced cost connectivity.
With the connectivity problem overcome, we developed another problem…firewalls. The point to point connection from A to B was a much bigger hurdle than we could imagine. There were two firewalls, along with the bridge firewall at the HCC office. It took three information technology consultants, a technology vendor and multiple staff from the network and network members to get the problem started and solved, multiple times. Each time the firewall changed for the members, we had to bring the consultant team back together to troubleshoot the problem yet again. It is an ongoing challenge for us, but we have learned how to work together to meet the requirements for connectivity.
The next phase of our project was even more interesting. HCC had made a decision in 2012 to apply for a new access point (Community Health Center) funding for our network. HCC was awarded this designation in late 2013 and one of our first orders of business was to launch our own telehealth project, which focused on behavioral health. We were so excited to launch this program because there is a major lack of mental health providers in our area and psychiatry is definitely a major shortage. We partnered with long-term friend of HCC, Pathways Community Mental Health, for a behavioral health consultant (face to face) and a psychiatrist (telehealth). As a result, Pathways is paving the way for telehealth statewide. On average, they currently do 3000 telehealth visits per month with over 30 of those visits being for HCC.
You must be asking yourself, “But what do the patients think?” Well, I am glad you asked! Pathways conducts annual patient satisfaction surveys with their telehealth patients. We were proud to learn that their overall satisfaction was over 90% and over 20% would choose telehealth as their form of treatment. As healthcare leadership, this tells us that telehealth is being accepted more and more across rural America.
Last but never least, during implementation we were also working on policy. Senate Bill 262 was introduced in 2012 and passed in 2013. Senate Bill 262 prohibits health carriers from denying coverage for a health care service on the basis that the service was provided through telemedicine if the same service would be covered when delivered in person. This was a major hurdle for rural Missouri and it is one of the areas where Missouri set the tone for the rest of the Midwest.
We look forward to our next step in Telehealth, which will be the addition of a new unit at our second health center and plans to develop a comprehensive, vertical telehealth network with our partners.
The Health Care Collaborative of Rural Missouri (HCC) has a mission to "Cultivate partnerships and deliver quality health care to strengthen rural communities." Since its inception, the HCC has developed into a comprehensive rural health network, with a wide variety of health, social services and community partners that provide health and wellness prevention and treatment programs for all citizens in our service area, and focused on the health care needs of low-income, under- and uninsured residents. The HCC's strength lies in developing collaborative relationships, utilizing the strengths of individual organizations to develop programs and services that are larger than any one organization. HCC is the first rural health network to receive HRSA's Bureau of Primary Health Care 330 funding to operate a community health center. HCC has been an active member of NCHN (National Cooperative of Health Networks) since 2010
This article was written by Brendan L. Ashby, MBA, MPH, MCHES, FACHE, Dean of Health Sciences and Service Programs, Saint Paul College for the “Networking News” monthly newsletter. The Network Technical Assistance Project is funded by the Federal Office of Rural Health Policy, Health Resources and Services Administration, U.S. Department of Health and Human Services through a contract to Rural Health Innovations, LLC, a subsidiary of the National Rural Health Resource Center.
Having been involved in the development and leadership of rural based healthcare networks in Minnesota and health workforce training in post-secondary academic institutions has shown me the importance of strategic planning. As network leaders, our charge is to assess the viability of the current or emerging network, gauge if the network is tactically positioned to meet its goals and objectives, and identify which strategic concerns and challenges warrant immediate leadership attention (Ashby, 2014). However, as important as strategic planning is, I have found it useful to adopt a mindset of strategic process that involves strategic thinking, acting, and learning that are just as important if not more important than any approach to strategic planning (Ashby, 2014). To help foster that mindset of strategic process, I want to share two of the tools that have helped my stakeholders and me-the Business Model Canvas and Strategy Change Cycle.
Business Model Canvas
When I was preparing for strategic planning sessions with my network, I wanted a novel approach and a colleague of mine suggested I try to develop business model canvases that she had effectively incorporated into her strategy sessions. The Business Model Canvas is a strategic tool developed by Alexander Osterwalder and Yves Pigneur as a way to visually capture and describe a network’s business model. I have had great success when using the business model canvas, especially when engaging reticent stakeholders that might have limited experience in any type of strategy planning or experts who appreciate the pragmatic framework. This unpretentious but powerful tool can demonstrate what is happening within a network and its value proposition in nine key areas:
Strategy Change Cycle
The Strategy Change Cycle developed by John M. Bryson can assist network leaders to figure out what the challenges are and provides ten steps to work through the strategic planning process. The strategy cycle will help network leaders think about your stakeholders and who needs to be part of the discussion; what details does the network leadership need and if you are missing any information; how you are going to implement this strategy process; if this is realistic; and lastly how can we create the highest enduring value for the people that your network serves (Ashby 2014).
The ten steps are as follows:
The final steps of using both the Business Model Canvas and Strategy Change Cycle will occur when your network reassess your identified strategies and remember to be agile, change when necessary, and make corrections as needed. You need to constantly be thinking strategically. Remember, this is a process and not a one-time project. If you keep that in mind then you will be successful.
Ashby, B.L. (2014). Topic Based Essay. Creighton University, Omaha, NE.
Bryson, J. M. (2011). Strategic planning for public and nonprofit organizations. San Francisco, CA, USA: Jossey-Bass.
Osterwalder, A., & Pigneur, Y. (2010). Business model generation. Hoboken, NJ, USA: John Wiley & Sons.
Thompson, A. A., Peteraf, M. A., Gamble, J. E., & Strickland III, A. J. (2014). Crafting and executing strategy: Concepts and readings (Vol. 19th ed.). New York, NY, USA: McGraw-Hill Irwin.
This article was written by Chirsty Sullenberger, MS, Director of Member Services, and Rebecca J. Davis, Ph. D., Executive Director for NCHN, for the “Networking News” monthly newsletter. The Network Technical Assistance Project is funded by the Federal Office of Rural Health Policy, Health Resources and Services Administration, U.S. Department of Health and Human Services through a contract to Rural Health Innovations, LLC, a subsidiary of the National Rural Health Resource Center.
Evaluating the network organization or a specific network program is an essential element of performance and process improvement, as well as overall assessment of effectiveness. Evaluations are used to improve programs, build organizational capacity, demonstrate value, and provide a basis for decision-making. While there is no set method for network evaluation, an evaluation typically addresses inputs, processes, outputs, and outcomes. Each of these elements provides insight into a different aspect of the network organization. Taken together, evaluation is a useful tool for the network leader, Board, stakeholders, and present and future funders.
A well-executed evaluation will also provide a statement of value, which is notoriously difficult for many health networks. Historically, health networks can illustrate positive value and promising programs, but quality quantitative data is lacking. In order to provide a good illustration of a network program’s impact, and ultimately the value of the network an evaluation plan should be developed and executed. The process should begin by determining clearly stated and measurable goals for the program, define metrics (how will you measure the impact of project), collect data, analyze the data against program goals and objectives, and then illustrate the connection of the program’s outcomes to the network’s value.
An evaluation should show a direct link between the program activities and outcomes and should address a number of questions, including:
-What do I need to know to make program decisions and adaptations?
-What is working well and what is not?
-How well does the program deliver value to members and stakeholders?
Potential funders, including private foundations and governmental agencies, are seeking justification for the investments they make in rural health networks. A good evaluation plan, along with previous outcomes, assists them in achieving this goal. A good example of the importance of evaluation to HRSA Rural Health Network Development Programs, is stated in a recent RFP Guidance,
“Evaluation is a very important component of the RHND Program. The collection of performance measures from past RHND cohorts and numerous rural health network case studies demonstrated positive outcomes. But, due to the lack of evidence and challenges using traditional quantitative methodologies to measure network outcomes, it is difficult to ascertain the significance and uniqueness of rural health networks that support positive health outcomes in rural communities. Project-level evaluations of RHND grantees will assist in determining and validating the reasons why rural health networks are an important strategy in the improvement of rural healthcare. A comprehensive evaluation approach should contain contextual, implementation and outcome evaluative components. And the process and result of evaluation should not only assist in the understanding of the benefits of rural health networks but be utilized in a manner that enhances and improves the functions and activities of the network.” (p.5, Rural Health Network Development Program Funding Opportunity Announcement, FY 2014, Health Resources and Services Administration, Office of Rural Health Policy (ORHP))
Rural Health Network Evaluation: When and How?
As part of your rural health network development grant proposal, you have already outlined program activities and expected outcomes. Prior to developing the proposal, you likely completed a needs assessment and know exactly what your vision is for the implemented project. You will now need to execute your evaluation plan. The evaluation approach you choose guides you in the collection and organization of data, so it is important to develop an evaluation plan early in the process of implementation. In addition, as you begin collecting data, you can conduct an ongoing evaluation, which determines if implementation is going as expected. This will lead to the final evaluation of the project and will determine if the stated objectives were met.
An evaluation may be goals-based, process-based (formative), outcomes-based (summative), or a combination of these. A goals-based evaluation determines whether you are meeting your overall objectives. A process-based evaluation addresses how your program works and highlights strengths and weaknesses. An outcomes-based evaluation addresses the benefits of your program to network members and/or the community. A final evaluation will often include pieces of all of these approaches.
Before developing your evaluation plan, it is essential to sit down and re-summarize your proposed program’s activities and objectives. One common guideline for developing goals and objectives is the SMART acronym. All objectives should be Specific, Measurable, Attainable, Realistic, and Time bound. Ensure that you have a strong list of realistic goals, achievable objectives, and appropriate activities that link directly to the desired results. Consider the purpose of your evaluation, outline the specific questions that the evaluation will answer, and decide how and how often data will be collected. You may want to consider using a logic model to assist in planning (see more at The Logic Model). Once this template is developed, you can use it to identify specific metrics, determine the kind of data that you want to collect, and choose an evaluation approach that is appropriate to the project.
There are a variety of evaluation methods and models. Each method has strengthens and weaknesses. The important question to answer, is which method will best provide actual data that can be used to determine the effectiveness of the proposed project. An evaluation plan for the proposed project may need to incorporate different approaches. Some basic methods of program evaluation include the following:
-Questionnaires and Surveys: can be analyzed and presented numerically/quantitatively
-Interviews: provide primarily qualitative outcomes and can be conducted in person or on the phone and should be targeted and clear
-Documentation review: can be inexpensive, but may not provide a complete picture
-Focus groups: can provide a range of feedback, but may be slightly difficult to present analytically
-Case studies: can provide an in-depth look at a program and many variables
-Others(see Overview of Methods to Collect Information for a list of primary methods and the pros and cons of each)
The guiding principle in the selection of an evaluation method/s is to collect and present the most useful information about a program. Throughout the process, keep in mind that the method you choose will determine how the results are collected. And, in order to avoid introducing bias, you will need to develop a process that ensures that data is collected in the same way each and every time.
Common Components of an Evaluation Report
When you have identified your method/s and collected data, you will then need to communicate your findings. There are many ways to structure the evaluation, and organization and content will depend on your process and methodology, but the following is a common structure of an evaluation:
1. Title page
2. Table of contents
3. Executive summary
4. Purpose of the evaluation
5. Organization and program background
6. Overall evaluation goals
7. Methods used
8. Interpretations and conclusions
or more detail on the above, see Contents of an Evaluation Plan. As you embark on your program evaluation and tie it into your network assessment, keep in mind that the most important element in the evaluation process is that you start early in the process of implementation and that you are consistent in your collection methods.
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources (Free Management Library)
Basic Guide to Program Evaluation (Including Outcomes Evaluation) (Free Management Library)
Critical Components of Evaluation by Alana Knudson, Ph.D., National Rural Health Resource Center Evaluation Workshop (August 6, 2014)
Designing Evaluations (GAO/PEMD-10.1.4, United States General Accounting Office - Program Evaluation and Methodology Division, 1991)
Evaluation for Nonprofits (Nonprofit Answer Guide)
Tools and Resources for Assessing Social Impact (TRASI) (Foundation Center)
Tools and Strategies for Managing Health Networks: Network Evaluation (NCHN)
The National Rural Health Resource Center (The Center) is a nonprofit organization dedicated to sustaining and improving health care in rural communities. Rural Health Innovations, LLC is a subsidiary of the National Rural Health Resource Center.