Challenge
In 2012, shortly before it launched, Onward Israel engaged Rosov Consulting as an evaluation partner for its new offering of immersive, eight-week-long resume-building experiences in Israel for college students and young professionals. Onward’s leadership team wanted to know the impact on participants’ relationships to Israel and Jewish peoplehood, personal development, and professional aspirations. Digging deeper, Onward wanted to lay the evaluation groundwork to then learn which of these impacts remained over time, and what exactly about the Onward experience contributed to changes in participants’ attitudes, knowledge, and behaviors.
Approach
Rosov Consulting’s work with Onward Israel has evolved as the organization’s needs, scale, and learnings have developed. In the early stages, through quantitative and qualitative data collection, Rosov Consulting’s evaluation primarily focused on Onward’s immediate outcomes for participants. During this time, Rosov and Onward together learned about the program’s influence regarding participants’ connection to and knowledge of Israel, Jewish identity, and Jewish peoplehood. Because the outcomes data were limited due to small sample sizes (in those years fewer than 500 people participated in the program), Rosov Consulting focused on understanding people’s motivations for coming on the program. These learnings helped to inform Onward as it designed and marketed the program.
As sample sizes grew, Rosov Consulting’s evaluation went deeper into pre- and post-trip analysis to understand the program’s outcomes. This continues to be at the heart of evaluation efforts today.
Now, with a larger pool of Onward alumni, Rosov Consulting’s evaluation explores Onward’s longer-term impacts on participants. In the summer of 2017, for the first time, Rosov Consulting surveyed and spoke with alumni three years following their participation in Onward (Cohort III alumni, from the summer of 2014). Additionally, for the first time, in this past year Rosov Consulting gathered satisfaction data from participants during their last days on the program and plans to integrate these with their outcomes data.
Results
Rosov Consulting’s work has yielded important insights and learnings—not just about Onward’s impact, but about specific features of the program in ways that help Onward’s leadership continue to refine and advance the program. Some of the most important findings include the following:
- Participants’ connection to and knowledge of Israel, Jewish identity, and Jewish peoplehood increased following their participation in the program.
- The immediate outcomes produced by participating in Onward largely remain over time. For example, the percentage of alumni who perceive themselves to have high levels of knowledge of Israeli life and a strong connection to Israel three years after Onward is quite similar to that of alumni shortly after their return from their Onward program. While one might expect to see these elements fade over time since alumni are far-removed from their Onward experience, that is not the case.
- Three years after participating in Onward, alumni seem much more aware of a clear, strong relationship between Israel and their Jewish identity, compared to what Rosov Consulting hears in conversations with participants at the onset of their program: that Israel and Jewish identity are discrete.
- Many alumni say that the sense of community and the friendships they made during the trip were some of the most meaningful experiences and takeaways that they still value today. In our conversations with alumni, most described their time on Onward as one of the most significant life experiences they’ve had.
- Given that most Onward participants are college students, three years later most have already graduated college. Many alumni credit Onward with offering them their first “real” professional experience, providing them with skills that can be transferable to their future workplace.
- Three years after taking part in Onward, nearly two-thirds of alumni indicated that they are involved in the Jewish community. Overall, approximately one in four alumni indicated that their current involvement in the Jewish community is “a lot” or “very much” tied to their Onward experience.
