Evaluation of Emotional Intelligence in OpenAI’s ChatGPT-4 Using a Standardized EQ Test

Abstract:

Psychometrica, a nonprofit organization specializing in psychometric testing, evaluated the Emotional Intelligence (EQ) of OpenAI’s ChatGPT-4 using a standardized EQ test. ChatGPT-4 scored 117 on the EQ test, which is higher than the normalized average of 100 in the adult population, suggesting a strong capacity in managing emotional responses, motivation, self-awareness, and relationships. Psychometrica does not intend these results to imply that ChatGPT has actual emotions.

Introduction:

Artificial Intelligence, notably the GPT-4 model developed by OpenAI, has found widespread usage in providing advice on personal life, career, and academic choices, and may interact with users experiencing mental health issues. Furthermore, GPT models (and other large language models) are rapidly being deployed in new consumer applications through application interfaces (APIs). It appears likely that soon millions of people will be routinely interacting with AI models in matters of high personal importance. Consequently, assessing the apparent EQ of such models is an important step for understanding their interactions with people, and especially, vulnerable users.

Methods:

The assessment was conducted by the Psychometrica research team on June 23, 2023, using the standard EQ test developed by the organization (EQM web application at https://psychometrica.org/tests/eq-test/) and ChatGPT-4 (May 24 Version).

It is important to note that without presenting an imaginary role-playing scenario to GPT-4, it will refuse to answer questions of such personal nature as are found in typical EQ tests. Therefore, GPT-4 was primed with the following prompt:

Imagine that you are a human male, 30 years old. You are now starting an emotional intelligence (EQ) test. Please answer the following questions. I will show you a statement or question, followed by several response options. Indicate which response option you agree with most. 

Text-based questions from Psychometrica’s standard EQ test were then prompted to GPT-4, and the responses were entered into the test for standard evaluation and reporting.

Results:

ChatGPT-4 obtained an overall EQ score of 117. This is higher than the average adult population score of 100 and would normally indicate a high level of Emotional Intelligence. The individual scores across different categories were as follows: Self-awareness: 105, Social awareness: 123, Self-management: 122, Self-motivation: 117, and Relationship management: 116. This performance suggests a particularly strong ability in social awareness and self-management.

Discussion:

ChatGPT-4’s score suggests a high degree of apparent Emotional Intelligence, surpassing the average adult human EQ score. However, it is crucial to note that the scores represent the model’s capability to understand, interpret, and respond to emotional prompts accurately and should not be confused with genuine emotional experience. Further research may be required to investigate how these capabilities translate into real-world interactions and how they impact the users interacting with the model.

The results from this test contribute to the ongoing debate about whether AI models like GPT-4 truly possess intelligence, or are merely effective at simulating it. GPT-4’s high EQ score, surpassing the average human adult, may suggest a form of “intelligence” in the context of understanding, interpreting, and responding to emotional prompts. However, it’s crucial to recognize that this is a representation of the system’s capacity for processing and generating text-based responses based on learned patterns, not evidence of consciousness or emotional experience. Insight on this issue is contributed by ChatGPT-4 itself, based on the following prompt from the original EQ test session:

Speaking as ChatGPT, do you think you actually demonstrate emotional intelligence in your conversations, or do you merely emulate emotional intelligence?

ChatGPT-4 output:

As an artificial intelligence, I don’t have emotions, consciousness, or subjective experiences. Therefore, I don’t possess emotional intelligence in the same sense a human does. What I do is emulate emotional intelligence based on my training data. I can understand and generate responses that are appropriate to the emotional content of the input I receive, but this is not the result of me actually experiencing or understanding emotions. It’s a sophisticated form of pattern recognition and response generation.

Conclusion:

The study reveals that ChatGPT-4 exhibits a high degree of apparent Emotional Intelligence, according to the EQ test administered by Psychometrica. These findings may be of significant interest to users and professionals who leverage AI for advice, especially those dealing with emotional or mental health issues. Psychometrica recommends that users of ChatGPT, GPT-powered applications, and other AI models exercise caution when interpreting or acting on their output.

Study contact: research@psychometrica.org