Aligning and streamlining your planning efforts for long-term success
This article was written by Bonnie Noble, PhD, RN, Founder of The Ondina Group, for the April 2017 edition of “Networking News.” The Network Technical Assistance Project is funded by the Federal Office of Rural Health Policy, Health Resources and Services Administration, U.S. Department of Health and Human Services through a contract to Rural Health Innovations, LLC, a subsidiary of the National Rural Health Resource Center.
We’ve all heard that familiar quote, “Failing to plan is planning to fail." This is likely a contemporary paraphrase of one of Benjamin Franklin's quotes: "By failing to prepare, you are preparing to fail." And then, of course, Winston Churchill said, "He who fails to plan is planning to fail."
OK. You get it. You know planning is important. But, sometimes it feels as if we can spend so much time planning that we don’t have time to get anything done. And, what about all those plans required when we’re seeking funding? Just the other day, a client commented on how the funding agencies “require an odd collection of similar-looking documents—Strategic Plan, Logic Model, and Action Plan.” She groaned when I replied, “Don’t forget about the Evaluation Plan and the Sustainability Plan.”
This “odd collection of similar-looking documents” each have a specific purpose and make an important contribution to program and organizational success. Moreover, it is helpful to understand how these various plans fit together in a sort of “less is more” approach that provides simplicity, clarity, and good design while streamlining your planning and writing efforts.
First, let’s briefly examine the key purpose for each one of these plans.
So, how do these plans overlap with and link to one another? The following Planning Crosswalk describes, visually, how these various plans are related.
It is important that these plans are aligned and integrated. For example, your three- to five-year program goals and strategies identified in your Strategic Plan align with the program-specific impact and outcomes in your Logic Model(s). Likewise, your Work Plan is a more detailed description of the initiatives outlined in your Strategic Plan and the activities described in your Logic Model.
The usefulness of each of these planning tools is enhanced by regularly consulting and comparing them. Developing, linking, and using these planning tools will help to ensure that your programs, and your organization, remain focused on its core mission and reaches its goals and vision.
Two important Baldridge program concepts are especially useful here—alignment and integration.
Examples of alignment and integration include linking key goals and objectives in your overall organizational Strategic Plan and your program Logic Model(s). Then, the Work Plan provides more detail on how your stated objectives will be achieved and who will be responsible for doing the actual day-to-day work. Likewise, the Evaluation Plan is a drill-down on how you will collect, analyze, and report data to ensure you remain on target towards reaching stated goals. Finally, the Sustainability Plan describes what actions you will take to ensure long-term viability of your program.
There is great value in aligning and integrating this “odd collection of similar-looking documents,” and doing so will enhance the effectiveness of your organization and its various programs. And, of course, you will more efficiently utilize the most precious resource—your time.
Bonnie Noble, PhD, RN, has an extensive background in the healthcare industry, with more than 30 years of experience working in a variety of healthcare organizations. She has expertise in many quality and performance improvement methodologies, is certified in patient safety, and is a certified professional in healthcare quality. Bonnie has served a National Examiner for the Baldrige National Quality Award and also has managed large federal contracts with the Centers for Medicare & Medicaid Services (CMS) and the Agency for Healthcare Research and Quality (AHRQ). She currently serves as the project director for the Mendonoma Health Alliance, a grantee of the Rural Health Network Development Planning Grant Program through the Health Resources and Services Administration (HRSA).
[i] Balanced Scorecard Institute. Retrieved March 1, 2017 at http://www.balancedscorecard.org/Resources/Strategic-Planning-Basics
[ii] W.K. Kellogg Foundation. East Battle Creek, Michigan. 2004. https://ag.purdue.edu/extension/pdehs/Documents/Pub3669.pdf
[iii] Developing an Effective Evaluation Plan. Atlanta, Georgia: Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; Division of Nutrition, Physical Activity, and Obesity, 2011.
[iv] The Grant Helpers.com. Five Key Elements of an Effective Sustainability Plan. 2014. Retrieved on March 2, 2017. http://www.thegranthelpers.com/blog/bid/204687/The-Five-Key-Elements-of-an-Effective-Sustainability-Plan-for-Grants
[v] Baldrige Performance Excellence Program. 2017. 2017–2018 Criteria for Performance Excellence.
Gaithersburg, MD: U.S. Department of Commerce, National Institute of Standards and Technology. http://www.nist.gov/baldrige
This article was written by Chirsty Sullenberger, MS, Director of Member Services, and Rebecca J. Davis, Ph. D., Executive Director for NCHN, for the “Networking News” monthly newsletter. The Network Technical Assistance Project is funded by the Federal Office of Rural Health Policy, Health Resources and Services Administration, U.S. Department of Health and Human Services through a contract to Rural Health Innovations, LLC, a subsidiary of the National Rural Health Resource Center.
Evaluating the network organization or a specific network program is an essential element of performance and process improvement, as well as overall assessment of effectiveness. Evaluations are used to improve programs, build organizational capacity, demonstrate value, and provide a basis for decision-making. While there is no set method for network evaluation, an evaluation typically addresses inputs, processes, outputs, and outcomes. Each of these elements provides insight into a different aspect of the network organization. Taken together, evaluation is a useful tool for the network leader, Board, stakeholders, and present and future funders.
A well-executed evaluation will also provide a statement of value, which is notoriously difficult for many health networks. Historically, health networks can illustrate positive value and promising programs, but quality quantitative data is lacking. In order to provide a good illustration of a network program’s impact, and ultimately the value of the network an evaluation plan should be developed and executed. The process should begin by determining clearly stated and measurable goals for the program, define metrics (how will you measure the impact of project), collect data, analyze the data against program goals and objectives, and then illustrate the connection of the program’s outcomes to the network’s value.
An evaluation should show a direct link between the program activities and outcomes and should address a number of questions, including:
-What do I need to know to make program decisions and adaptations?
-What is working well and what is not?
-How well does the program deliver value to members and stakeholders?
Potential funders, including private foundations and governmental agencies, are seeking justification for the investments they make in rural health networks. A good evaluation plan, along with previous outcomes, assists them in achieving this goal. A good example of the importance of evaluation to HRSA Rural Health Network Development Programs, is stated in a recent RFP Guidance,
“Evaluation is a very important component of the RHND Program. The collection of performance measures from past RHND cohorts and numerous rural health network case studies demonstrated positive outcomes. But, due to the lack of evidence and challenges using traditional quantitative methodologies to measure network outcomes, it is difficult to ascertain the significance and uniqueness of rural health networks that support positive health outcomes in rural communities. Project-level evaluations of RHND grantees will assist in determining and validating the reasons why rural health networks are an important strategy in the improvement of rural healthcare. A comprehensive evaluation approach should contain contextual, implementation and outcome evaluative components. And the process and result of evaluation should not only assist in the understanding of the benefits of rural health networks but be utilized in a manner that enhances and improves the functions and activities of the network.” (p.5, Rural Health Network Development Program Funding Opportunity Announcement, FY 2014, Health Resources and Services Administration, Office of Rural Health Policy (ORHP))
Rural Health Network Evaluation: When and How?
As part of your rural health network development grant proposal, you have already outlined program activities and expected outcomes. Prior to developing the proposal, you likely completed a needs assessment and know exactly what your vision is for the implemented project. You will now need to execute your evaluation plan. The evaluation approach you choose guides you in the collection and organization of data, so it is important to develop an evaluation plan early in the process of implementation. In addition, as you begin collecting data, you can conduct an ongoing evaluation, which determines if implementation is going as expected. This will lead to the final evaluation of the project and will determine if the stated objectives were met.
An evaluation may be goals-based, process-based (formative), outcomes-based (summative), or a combination of these. A goals-based evaluation determines whether you are meeting your overall objectives. A process-based evaluation addresses how your program works and highlights strengths and weaknesses. An outcomes-based evaluation addresses the benefits of your program to network members and/or the community. A final evaluation will often include pieces of all of these approaches.
Before developing your evaluation plan, it is essential to sit down and re-summarize your proposed program’s activities and objectives. One common guideline for developing goals and objectives is the SMART acronym. All objectives should be Specific, Measurable, Attainable, Realistic, and Time bound. Ensure that you have a strong list of realistic goals, achievable objectives, and appropriate activities that link directly to the desired results. Consider the purpose of your evaluation, outline the specific questions that the evaluation will answer, and decide how and how often data will be collected. You may want to consider using a logic model to assist in planning (see more at The Logic Model). Once this template is developed, you can use it to identify specific metrics, determine the kind of data that you want to collect, and choose an evaluation approach that is appropriate to the project.
There are a variety of evaluation methods and models. Each method has strengthens and weaknesses. The important question to answer, is which method will best provide actual data that can be used to determine the effectiveness of the proposed project. An evaluation plan for the proposed project may need to incorporate different approaches. Some basic methods of program evaluation include the following:
-Questionnaires and Surveys: can be analyzed and presented numerically/quantitatively
-Interviews: provide primarily qualitative outcomes and can be conducted in person or on the phone and should be targeted and clear
-Documentation review: can be inexpensive, but may not provide a complete picture
-Focus groups: can provide a range of feedback, but may be slightly difficult to present analytically
-Case studies: can provide an in-depth look at a program and many variables
-Others(see Overview of Methods to Collect Information for a list of primary methods and the pros and cons of each)
The guiding principle in the selection of an evaluation method/s is to collect and present the most useful information about a program. Throughout the process, keep in mind that the method you choose will determine how the results are collected. And, in order to avoid introducing bias, you will need to develop a process that ensures that data is collected in the same way each and every time.
Common Components of an Evaluation Report
When you have identified your method/s and collected data, you will then need to communicate your findings. There are many ways to structure the evaluation, and organization and content will depend on your process and methodology, but the following is a common structure of an evaluation:
1. Title page
2. Table of contents
3. Executive summary
4. Purpose of the evaluation
5. Organization and program background
6. Overall evaluation goals
7. Methods used
8. Interpretations and conclusions
or more detail on the above, see Contents of an Evaluation Plan. As you embark on your program evaluation and tie it into your network assessment, keep in mind that the most important element in the evaluation process is that you start early in the process of implementation and that you are consistent in your collection methods.
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources (Free Management Library)
Basic Guide to Program Evaluation (Including Outcomes Evaluation) (Free Management Library)
Critical Components of Evaluation by Alana Knudson, Ph.D., National Rural Health Resource Center Evaluation Workshop (August 6, 2014)
Designing Evaluations (GAO/PEMD-10.1.4, United States General Accounting Office - Program Evaluation and Methodology Division, 1991)
Evaluation for Nonprofits (Nonprofit Answer Guide)
Tools and Resources for Assessing Social Impact (TRASI) (Foundation Center)
Tools and Strategies for Managing Health Networks: Network Evaluation (NCHN)
The National Rural Health Resource Center (The Center) is a nonprofit organization dedicated to sustaining and improving health care in rural communities. Rural Health Innovations, LLC is a subsidiary of the National Rural Health Resource Center.