3 minute read
Topics: FCR, AHT, CSAT
What's on this page?
This article explores how Microsoft used AI from Elephants Don’t Forget to improve employee in-role competence and correlate this with direct improvements in KPIs including First Contact Resolution (FCR), Customer Satisfaction (CSAT), Average Handling Time (AHT) and Help Resolved (HR).
Using AI-powered continual assessment, Microsoft wanted to acquire an objective view of employee subject knowledge as dynamic data points for critical evaluation. It was their intention that this would empower their leadership teams and outsourced delivery partners to hold targeted training interventions that would support individual employees in specific areas where knowledge and competency gaps / fade could be detrimental to improving core KPIs.
Microsoft had three key objectives:
Prior to implementation, Microsoft and Elephants Don’t Forget took a collaborative approach to determine the specific measures that would be central to gauging the impact of Clever Nelly, opting for the Kirkpatrick Model of assessment.
Out of 2,500 employees within their Xbox, Surface and Devices teams, half of the control group used Clever Nelly alongside their existing training platforms and content, whilst the other half only had access to the existing training available. This was done to help their teams evaluate performance improvement from using Clever Nelly. From Microsoft’s existing training content, the expert authoring team at Elephants Don’t Forget crafted specific assessment questions that would assess employee knowledge on core subject categories.
Individual employee knowledge baselines were established during the first 45 days of deployment, where users responded to two knowledge questions (interventions) per day – this enabled their L&D and management teams to specifically identify what each employee knew – and did not know – about their products, services, processes and policies they had been trained on.
Once established, they applied the principles of spaced learning, repetition, and self-testing – the core principles of Clever Nelly’s AI – and concentrated their efforts on improving and plugging transient knowledge gaps by re-enforcing key subject matter, within the flow of work, following initial training.
Once fully activated, the AI began addressing subject areas where each employee was demonstrating competency or knowledge fade, adjusting the quantity and content of interventions in real-time based on the needs of each user to automatically repair gaps. Questions were delivered via email and could be completed at the convenience of the individual recipient, taking just 1 minute 47 seconds of an employee’s day – on average.
By focusing the AI on the knowledge and skills that most influenced these KPIs, Microsoft were able to gently and continually improve operational performance from those teams, which has since been sustained. They are able to clearly demonstrate cause and effect of their L&D interventions on bottom-line performance – with independent ROI evidence.
We continually reviewed employee adoption of Clever Nelly at Microsoft, asking them to provide feedback via an employee survey to gauge sentiment in relation to user experience, content relevancy and application of knowledge acquired. These were conducted at specific intervals aligned with the program objectives and measures set by Microsoft and Elephants Don’t Forget.
Of employees agreed or strongly agreed that Clever Nelly helps to highlight the areas of their role they did not know as much about as other areas.
Of employees agreed or strongly agreed that Clever Nelly had the potential to improve their workplace knowledge.
Of employees agreed or strongly agreed that they felt their knowledge had improved since they started using Clever Nelly.
Of employees agreed or strongly agreed that knowledge reinforcement through Clever Nelly would help them feel more confident when speaking to customers.