Similarity analysis of your values and beliefs with AI
TL;DR; To better explore my values and beliefs I've created an AI-driven clusterization and summarization tool. You can find it here – you will need to enter your OpenAI key. Continue, if you are interested in the journey and tech of the tool.
Recently I was interested in defining my values and beliefs and to my surprise, it was hard to find a "scientific" classification of the values and beliefs. My Google was having a bad day and showing articles with random classifications of values and beliefs ranging from 50 to 250 items.
I've started to think, about how can I figure out which values are the most important and play a significant role in my life without having a strong classification model.
This led me to think that I could try to use semantic textual relations between values to understand their relatedness to each other and therefore figure out which values and beliefs are more or less strong in my character.
The initial plan was to build a simple UI, where
- You can choose beliefs and values from the list that resonate with you. There is no need to be precise, as we will increase precision with similarity analysis later.
- For those selected values, we create a similarity matrix, by utilizing OpenAI Embedding API and a little JS script.
- We create a 3D representation with Plotly and hope it gives any clue. (hint: it doesn't!)
When I first looked at the graph, I said "Wow! These are some important dots for me!" But why they are important and what to do with the graph I had no clue.
I tried to add labels, change colors, and play with dimensions and axes. Unfortunately, this gave me as much value as just reading the checked values list to the mirror.
But we live in a world, where ChatGPT exists, it is no excuse to stop. And in 5 minutes I found what I was looking for: Force-Directed Graph.
A force-directed graph is a 2D representation of node and link relations within a network. It has interesting physics, that allows humans to make more sense from the data it is built upon.
Unfortunately, Plotly is not suited for such use cases, so I had to switch to D3.js. I've never worked with it, so with a little magic I had the first version of the force-directed graph with my values.
Looking at the result, it was not exactly what I expected to see. Everything was related to almost everything. The first thing I did was count the number of links for each node and mark 5 nodes with the most links, showing that they have the most significance between all values. Yes! It is already giving me a little more sense of the data.
After figuring out that to create a beautiful force-directed graph, I have to adjust the "threshold" of similarity when we would like to connect nodes. By trial and error, for this specific data, it made sense to work with the 94.3 percentile. This means, that only the top 5.7% of node connections will have links.
This produced the final version of the graph. It is easy to read, you see the most important nodes and it gives you a sense of how your beliefs are interconnected without a huge mess from the first version.
The final step was to provide this information to GPT 4 for it to make a summarization and connection to your vision. This step was more for fun and should be improved, but it demonstrates a possible future vector to explore.
You can explore the final tool here: https://vitalijssilins.github.io/person-core-values/
You can find a super-trivial one-file codebase here: https://github.com/vitalijssilins/person-core-values
Leave your comments on what improvements or use cases you would like this tool to handle.
