Currently, I am working on a research project wherein I am developing a metric that could determine the usability of a website based on a word embedding approach by making a dictionary of frequently used words in UX testing. Now, the tool can identify words that signify a UX problem and can also identify the overall experience of the participant (positive or negative). Based on all this data, the tool assigns a score to the interface in question.
Here is a small example of what the tool can do (In the example above I tested my tool on the sentence: I am thinking, I can’t figure out how to do this. I am lost and confused). I have been testing my tool on some usability transcripts and the ratings seem to be consistent. However, a large DB of transcripts could really help me in training my tool.
This research is in a preliminary stage right now.