Over the past two years, advancements in artificial intelligence (AI) have been exponential.
One of the reasons AI tools like ChatGPT are so easy to use is because they’re built on something called generative pre-trained transformers. This technology powers large language models (LLMs), which can let us interact with AI through natural, human-like conversation — no coding or technical knowledge needed.
We don’t need to dive into the technical details of how LLMs work — what matters is that AI is now more accessible than ever and especially useful for things like measurement and analytics. The output of LLMs can closely resemble the way we communicate, therefore providing new opportunities for measuring and analyzing soft skills.
Large Language Models (LLMs), such as ChatGPT, can now be used to assess human conversation and communication patterns, providing valuable insights into areas like empathy, active listening and negotiation skills.
To illustrate the potential of AI-powered analytics, consider the following example:
The Fundamentals: Measuring Empathy in a Structured Conversation
Imagine a scenario where two colleagues, A and B, engage in a discussion about delayed spreadsheet delivery for a project. Person A expresses frustration about the missed deadline, while Person B apologizes for the delay and offers a revised timeline. To assess Person A’s empathy levels in this conversation, let us try a practical example:
- Log on to com (you can stay logged out if you do not wish to create an account).
- Provide a sample exchange of a dialogue between two people on a team and ask the AI how they could have handled it better. You can copy-paste the below-provided example of a conversation between two coworkers to start:
Person A: Hey, did you manage to send me that spreadsheet with our revenue for the quarter?
Person B: Sorry, I’ve been slammed with other requests, and I haven’t gotten to it yet. I’ll get it to you by next week.
Person A: That’s too late, I needed it by this week, so can you try to juggle other tasks and prioritize this ?
Prompt: Help me determine how empathetic Person A is in this conversation exchange. What can they do to be better?
- Now, review the output from the AI and see if you agree with its recommended approaches.
Try this with a few more examples of your own and get a sense for what the AI is capable of measuring. Now, how exactly can we put this into practice at an organization? And more importantly, how can we harness this data to drive measurable business outcomes ? If this is of interest, let’s dive into a few sample use cases and explore further.
Design Example: Using AI to Measure Empathy on a Team
Let’s assume that you are leading a team that is struggling with internal miscommunication due to heavy workloads, and highly cross-functional projects requiring multi-disciplinary collaboration and expertise. You have decided you want to leverage AI to assess and improve the team’s empathy and communication skills, since you believe these are fundamental aspects that are causing a breakdown in team performance.
Here’s an example of how you could set this up:
Step 1: Gather conversational data.
Subject, of course, to your internal privacy policies, begin by informing your team and collecting anonymized transcripts from Slack, Teams or any other productivity tool, where project related conversations are occurring. Your goal will be to use AI and analyze tense exchanges or feedback discussions that shed insight into factors that could be hindering the business. What is critical here is that you gain the trust of your employees that you are using this data to learn the pain points of the business function, and not to individually target users for their specific contributions (or non-contributions) to a channel/topic on Slack.
A fundamental aspect of using AI effectively involves getting users’ buy-in and trust, to gather the right data on which to operate. Absent these, any AI project at any organization is likely to fail.
Step 2: Analyze with prompt design.
There are several large language models that are currently available for use for the purposes of what we are trying to do. Many of them are open source and can even be run locally on your private machines, ensuring that your data does not leave your device and is completely protected (and private). Before doing so, please ensure compliance with your company IT policies. A few LLMs that I’d recommend include ChatGPT, Claude, and the Meta Llama family of models. Design a prompt using your favorite large language model. Ensure you break down your instructions clearly by providing a “Task” and the “Data” that the model must use for the task (as shown below in double quotes) (also referred to as a “Prompt”)
“THE TASK:
Analyze the following conversation for signs of empathy, active listening, and constructive dialogue. Identify which phrases reflect these skills and suggest improvements.
THE DATA:
<paste your data that you collected from your Slack/Teams exchanges here>”
*Note: Remember to use a format like the first example between Person A and Person B referenced in this article. Make sure your data adheres to any token limits for the model, which you can learn more about here>: [What are tokens]
Prompt Design Matters for Measurement
The power of this approach lies in how you structure your prompts. General prompts like “Was this conversation empathetic?” can provide surface-level insights. But more specific structures, such as:
- “Identify emotional cues in this conversation.”
- “Rate the level of empathy on a scale of 1–10 and explain why.”
- “Suggest alternate phrasing to improve tone and clarity.”
… will likely yield richer, more actionable results. Review the output of the model and iterate until you are happy with the results. By varying prompts across conversations, you can also build a profile of communication behaviors. Over time, this becomes data; data that your AI models can learn from, and that teams can use to pinpoint development needs, flag coaching opportunities, and track growth across individuals or departments.
Step 3: Rollout and track skill growth over time.
At this stage, you are ready to begin a roll-out of the AI software across small groups for beta testing. Involve your internal IT team or external vendor / partner to build out the model using your data and your guidelines so it can be applied to real world business data in a scalable, and efficient manner. You can repeat this type of analysis on a regular cadence (e.g. weekly, or biweekly) using new conversations that occurred in that period. Ensure that you have mechanisms in place to provide insights and feedback to the teams that are creating this data, so they can improve, or course correct as needed. Over a short period of two or three months, you will have begun measuring team performance using AI and reviewing qualitative improvements. If you implemented your program correctly, you should begin to see an increase in internal satisfaction scores related to communication, and fewer project delays attributed to misalignment.
Application Areas: AI in Corporate Training and Measurement
Analyzing communication with AI opens up various avenues that are beneficial to an organization.
- Diagnose skill gaps: Use AI to review team discussions and surface patterns such as recurring lack of acknowledgment, defensive language or low collaboration indicators.
- Targeted training and coaching: Map these patterns to specific interventions. A team struggling with emotional awareness might benefit from empathy workshops or leadership coaching.
- Measure and report progress: By re-analyzing conversations over time, you can quantify soft skill improvement, creating a feedback loop for both the learner and the organization.
The Future: Simulated Conversations and Cohort Analytics
By now, you should have a good sense of what “measurement” looks like with AI. When you group a number of these “measurements” both at the individual level, as well as the team level (as a cohort / team), you start to generate meaningful “analytics”. It is these analytics that are so valuable to a business, since they can directly be tied to the business outcomes (like revenue, customer growth, customer satisfactionr).
Now imagine a world where you could create simulated conversations with an AI that replicate real-world scenarios. For example, a sales discovery call, a call with a leader to highlight the heavy workload, a call with an employee who is worried about an imminent layoff, or a customer call where a project is off schedule. Providing employees with an opportunity to practice these conversations ahead of time, with the help of AI analytics, is the competitive advantage that businesses will need to truly embrace the shift that is defining the future of work.
Soft skills are no longer a luxury but a necessity in the modern workplace. As we navigate the challenges of automation and technological advancements, it’s crucial for businesses to invest in training programs that cultivate these critical attributes. By embracing AI-powered analytics and developing more empathetic leaders, organizations can drive productivity, growth, and success. This is the competitive advantage that successful businesses are embracing in the future of work. This is not a future vision. It’s happening now. The question is: Is your business truly AI-ready?
