ÓÐÁϺÐ×ÓAPP

The Department of Veterans Affairs (VA) works diligently to provide services to millions of our nation’s veterans, their families, caregivers, and survivors. To give veterans a voice and to understand their perspectives, VA created the Veterans Experience Office (VEO), charged with collecting qualitative and quantitative customer experience (CX) data and continuously working to improve and enhance VA services based on that data. ÓÐÁϺÐ×ÓAPP helped VA pioneer a repeatable, scalable CX measurement capability that uses human-centered design (HCD) and advanced analytics to understand the moments that matter to veterans—and helps save lives.

Historically, VA had not deployed near-real time digital feedback listening channels to capture the voice of the veteran, family members, caregivers, and survivors. To help inform for immediate to short-term service recovery in addition to long-term systems improvements, VA began deploying digital feedback listening channels meant to collect and analyze insights about the specific experience encounters the veteran community had at the point of service.

Within the digital space, VA began collecting free-text responses to capture concerns, compliments, and recommendations in near-real time (within a matter of days after receiving the service). For example, veterans were sent a link via email to provide feedback about their experiences with VA’s services—from applying for a loan through the Veterans Benefits Administration, to receiving a dental appointment with the Veterans Health Administration, to a survivor working with the National Cemetery Administration.

As VA deployed additional surveys with program offices around the agency, some surveys allowed veterans to leave free-text responses. Within these free-text responses, veterans offered specific recommendations of how VA could improve their services, shared concerns that needed to be addressed, and offered compliments about specific experiences. 

To evaluate the free-text responses, our team supported the development of a new CX measurement methodology based on a new artificial intelligence (AI) systems capability deployed by VA in partnership with ÓÐÁϺÐ×ÓAPP. Our experts in HCD, statistics, and advanced analytics worked side by side, amplifying each other’s capabilities to better measure the veteran’s experience.

By synthesizing HCD research with the power of advanced analytics, statistics, and AI, our team took on VA’s mission as our own, providing real-time, systematic, data-driven feedback to VA leadership and employees to understand and improve VA service delivery from the perspective of the veteran.

Beginning with empathy, the team used HCD research techniques to conduct face-to-face generative research with veterans to identify the moments that matter. Our human-centered designers then synthesized these insights into visual artifacts while our statisticians developed custom sampling instruments for VA’s diverse services.

We analyzed large volumes of data—more than 4 million survey responses—and used machine learning and natural language processing AI techniques to derive actionable insights from the data. With our prediction models, VA leaders can quickly identify emerging issues, predict which topics have the greatest impact on veteran trust, detect sentiment in responses, and highlight the challenges and successes veterans may encounter at specific VA medical facilities.

The prediction models we developed are saving lives—identifying free-text responses that may suggest a veteran is at risk of suicide, homelessness, or other risk factors. Using Veterans Signals, free-text responses are automatically flagged for review by VA staff who forward actionable cases to the Veterans Crisis Line and National Call Center for Homeless Veterans.

Since 2017, this process has identified more than 1,200 suicide and 450 homelessness alerts where at-risk veterans received emergency support services, demonstrating how CX management powered by advanced analytics and AI technologies can connect resources in new ways to save lives.