Illinois Statistics Datathon 2024
Beyond Numbers: Synchrony’s IVR Quest for Insight
The Illinois Department of Statistics partnerted with Synchrony Financial to host the 8th annual Illinois Statistics Datathon in March 2024. Synchrony’ challenge was to identify improvement opportunities in their Interactive Voice Response (IVR) system based on a extensive dataset of customer interactions. The IVR system is a technology that allows customers to interact with a computer-operated phone system to access information and perform tasks without human assistance. Synchrony estimates that over 20 million customers call into the IVR each month with a 78% resolution rate. Even a 1% improvement in the resolution rate would save 200,000 agent/floor calls per month. The goal of the Datathon was to uncover patterns within the IVR system that could lead to a better customer experience and operational efficiency. They requested to put emphasis on calls that the IVR could not handle.
Fig 1. Exploratory Data Analysis
Sankey diagrams provided a clear visualization of users interactions.
Exploratory Data Analysis
Upon initial exploration of the data, it was clear for the team that we needed to allocate most of the time to establish a solid correlation between the attributes to find out what were the root causes behind floord calls. Synchrony claimed their prediction model was highly reliable, and therefore exploring machine learning (ML) approaches to improve their model was not a priority. Instead of focusing on the prediction model, the team decided to focus on the data itself. The team used Sankey diagrams to visualize the flow of calls through the IVR system, and to identify the most common paths that led to a floor call.
Fig 2. In-Depth Analysis
A linear regression model provided insights on the relationship of every attribute with the flooring/resolution of calls.
It was identified that 20% of all calls going through the IVR system were being transferred. The team evidenced an issue with the call labeling system, which would assign by default that of the last interaction with the customer. This was causing the system to misclassify the nature of the call, leading to a high number of floor calls. To address this issue, the top reasons for calling (those that covered 90% of the total calls) were identified and a new labeling system was proposed. It was also noted that the time a customer would spend on the IVR system was directly related to the number of transfers. The dataset contained almost 2 million calls, with columns containing information through different dates. Using a correlation matrix and a contingency table, the team was able to identify the most related attributes to the number of transfers. The team also used a linear regression model to understand the influence of each attribute on the number of transfers.
Recommendations
Because more than 8% of the resolved queries were customers calling multiple times, the team recommended to implement a system that would recognize the customer's phone number to bypass initial autentication steps. However, Synchrony's team raised concerns about privacy issues. Instead, it was proposed to add features to enable more operations through the IVR system, such as auto-pay enrollment, e-bill enrollment, and use cases regarding card activation status checks. A new labeling system was recommended to improve the classification of calls, and remove bias from the prediction model. The system needs to be able to identify inmediately if it can actually provide a solution to the customer's query. If not, it should avoid navigating the IVR and floor the call. As long as the reason behind the flooring is identified, Synchrony will have data to keep upgrading the IVR system.
Having started on Friday night, we started wrapping up the results on Sunday afternoon. Thank God Seon Mi was on our team. Her ease to deliver the results in a clear and concise manner was key to our success. It was my first experience on a Datathon and I was thrilled to have been part of it. Thirty-six hours of intense work, but it was worth it. We finished our submission close to the deadline. Kudos to Farid for assembling the crew, and to Divyansh for introducing us to visualizations I never used before. We ranked 4th out of 345 teams! I am looking forward to the next Datathon. I truly find amazing how much you can learn from top students in such a short period of time, and I'm glad I was surrounded by them.
Last updated on April 1st, 2024
