Открытие чёрного ящика: Извлечение семантических факторов Осгуда из языковой модели word2vec
Ключевые слова:
аффект, семантика, пространство, Осгуд, смысл, язык, word2vec, чёрный ящик, интерпретацияАннотация
Современные модели искусственного интеллекта развиваются в парадигме чёрного ящика, когда значима только информация на входе и выходе системы, тогда как внутренние представления интерпретации не имеют. Такие модели не обладают качествами объяснимости и прозрачности, необходимыми во многих задачах. Статья направлена на решение данной проблемы путём нахождения семантических факторов Ч. Осгуда в базовой модели машинного обученния word2vec, представляющей слова естественного языка в виде 300-мерных неинтерпретируемых векторов. Искомые факторы определяются на основе восьми семантических прототипов, составленных из отдельных слов. Ось оценки в пространстве word2vec находится как разность между положительным и отрицательным прототипами. Оси силы и активности находятся на основе шести процессно-семантических прототипов (восприятие, анализ, планирование, действие, прогресс, оценка), представляющих фазы обобщённого кругового процесса в данной плоскости. Направления всех трёх осей в пространстве word2vec найдены в простой аналитической форме, не требующей дополнительного обучения. Как и ожидается для независимых семантических факторов, полученные направления близки к попарной ортогональности. Значения семантических факторов для любого объекта word2vec находятся с помощью простой проективной операции на найденные направления. В соответствии с требованиями к объяснимому ИИ, представленный результат открывает возможность для интерпретации содержимого алгоритмов типа ``чёрный ящик'' в естественных эмоционально-смысловых категориях. В обратную сторону, разработанный подход позволяет использовать модели машинного обучения в качестве источника данных для когнитивно-поведенческого моделирования.
Литература
1. Mikolov T., Yih W., Zweig G. Linguistic Regularities in Continuous Space Word Representations. Proceedings of NAACL-HLT. 2013. pp. 746–751.
2. Pennington J., Socher R., Manning C.D. Glove: Global vectors for word representation. Proceedings of the 2014 conference on empirical methods in natural language processing. 2014. pp. 1532–1543.
3. Radford A., Narasimhan K., Salimans T., Sutskever I. Improving Language Understanding by Generative Pre-Training. 2018.
4. Yang Z., Dai Z., Yang Y., Carbonell J., Salakhutdinov R., Le Q.V. XLNet: Generalized autoregressive pretraining for language understanding. Proceedings of 33rd Conference on Neural Information Processing Systems. 2019.
5. Mikolov T., Joulin A., Baroni M. A Roadmap Towards Machine Intelligence. Computational Linguistics and Intelligent Text Processing. Cham: Springer, 2018. pp. 29–61.
6. Al-Saqqa S., Awajan A. The Use of Word2vec Model in Sentiment Analysis: A Survey. ACM International Conference Proceeding Series. 2019. pp. 39–43.
7. Dhar A., Mukherjee H., Dash N.S., Roy K. Text categorization: past and present. Artificial Intelligence Review. 2021. vol. 54. no. 4. pp. 3007–3054.
8. Konstantinov A., Moshkin V., Yarushkina N. Approach to the Use of Language Models BERT and Word2vec in Sentiment Analysis of Social Network Texts. Recent Research in Control Engineering and Decision Making. Cham: Springer, 2021. pp. 462–473.
9. Gunning D., Stefik M., Choi J., Miller T., Stumpf S., Yang G.Z. XAI—Explainable artificial intelligence. Science Robotics. 2019. vol. 4. no. 37.
10. Suvorova A. Interpretable Machine Learning in Social Sciences: Use Cases and Limitations. Proceedings of Digital Transformation and Global Society 2021. Communications in Computer and Information Science, vol. 1503. Cham: Springer, 2022. pp. 319–331.
11. Osgood C.E. The nature and measurement of meaning. Psychological Bulletin. 1952. vol. 49. no. 3. pp. 197–237.
12. Osgood C.E. Studies on the generality of affective meaning systems. American Psychologist. 1962. vol.17. no. 1. pp. 10–28.
13. Hollis G., Westbury C. The principals of meaning: Extracting semantic dimensions from co-occurrence models of semantics. Psychonomic Bulletin and Review. 2016. vol. 23. no. 6. pp. 1744–1756.
14. Lenci A. Distributional Models of Word Meaning. Annual Review of Linguistics. 2018. vol. 4. no. 1. pp. 151–171.
15. Gunther F., Rinaldi L., Marelli M. Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions. Perspectives on Psychological Science. 2019. vol. 14. no. 6. pp. 1006–1033.
16. Samsonovich A.V., Ascoli G.A. Principal Semantic Components of Language and the Measurement of Meaning. PLoS ONE. 2010. vol. 5. no. 6.
17. Eidlin A.A., Eidlina M.A., Samsonovich A.V. Analyzing weak semantic map of word senses. Procedia Computer Science. 2018. vol. 123. pp. 140–148.
18. Samsonovich A.V. On semantic map as a key component in socially-emotional BICA. Biologically Inspired Cognitive Architectures. 2018. vol. 23. pp. 1–6.
19. Pretrained word2vec model “GoogleNews-vectors-negative300.bin.gz”. Google Code Archive. https://code.google.com/archive/p/word2vec/. 2013.
20. Mikolov T., Sutskever I., Chen K., Corrado G., Dean J. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems. 2013.
21. Surov I.A. Natural Code of Subjective Experience. Biosemiotics. 2022. vol. 15. no. 1. pp. 109–139.
22. Siegel M. The sense-think-act paradigm revisited. Proceedings of the 1st International Workshop on Robotic Sensing. 2003.
23. Gastaldi J.L. Why Can Computers Understand Natural Language? Philosophy & Technology. 2021. vol. 34. no. 1. pp. 149–214.
24. Jensen A.R. The relationship between learning and intelligence. Learning and Individual Differences. 1989. vol. 1. no. 1. pp. 37–62.
25. Sowa J.F. The Cognitive Cycle. Proceedings of the 2015 Federated Conference on Computer Science and Information Systems. 2015. vol. 5, pp. 11–16.
26. Wang Y., Yao Q., Kwok J.T., Ni L.M.: Generalizing from a Few Examples: A Survey on Few-shot Learning. ACM Computing Surveys. 2021. vol. 53. no. 3. pp. 1–34.
27. Hoenkamp E. Why Information Retrieval Needs Cognitive Science: A Call to Arms. 2005.
28. Turney P.D., Pantel P. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research. 2010. vol. 37. pp. 141–188.
29. Wang B., Buccio E.D., Melucci M. Representing Words in Vector Space and Beyond. Quantum-Like Models for Information Retrieval and Decision-Making (eds: Aerts D., Khrennikov A., Melucci M., Toni B.). Cham, Springer. pp. 83–113. 2019.
30. beim Graben P., Huber M., Meyer W., R ̈omer R., Wolff M. Vector Symbolic Architectures for Context-Free Grammars. Cognitive Computation. 2022. vol. 14. no. 2. pp. 733–748.
31. Coenen A., Reif E., Kim A.Y.B., Pearce A., Vi ́egas F., Wattenberg M. Visualizing and measuring the geometry of BERT. Proceedings of the Advances in Neural Information Processing Systems. 2019.
32. Tanaka Y., Oyama T., Osgood C.E. A cross-culture and cross-concept study of the generality of semantic spaces. Journal of Verbal Learning and Verbal Behavior. 1963. vol. 2. no. 5-6. pp. 392–405.
33. Tanaka Y., Osgood C.E. Cross-culture, cross-concept, and cross-subject generality of affective meaning systems. Journal of Personality and Social Psychology. 1965. vol. 2. no. 2. pp. 143–153.
34. Osgood C.E., May W.H., Miron M.S. Cross-cultural universals of affective meaning. Champaign, University of Illinois Press. 1975.
35. Zajonc R.B. Feeling and thinking: Preferences need no inferences. American Psychologist. 1980. vol. 35. no. 2. pp. 151–175.
36. Duncan S., Barrett L.F. Affect is a form of cognition: A neurobiological analysis. Cognition and Emotion. 2007. vol. 21, no. 6. pp. 1184–1211.
37. Lipton Z.C. The Mythos of Model Interpretability. Queue. 2018. vol. 3. pp. 31–57.
38. Molnar C. Interpretable Machine Learning. A Guide for Making Black Box Models Explainable. 2019.
39. Guidotti R., Monreale A., Ruggieri S., Turini F., Giannotti F., Pedreschi D. A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys. 2019. vol. 51. no. 5.
40. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence. 2019. vol. 1. no. 5. pp. 206–215.
41. Vassiliades A., Bassiliades N., Patkos T. Argumentation and explainable artificial intelligence: A survey // Knowledge Engineering Review. 2021. vol. 36, pp. 1-35.
42. Borrego-D ́iaz J., Gal ́an-P ́aez J. Explainable Artificial Intelligence in Data Science. Minds and Machines. 2022.
43. Chou Y.L., Moreira C., Bruza P., Ouyang C., Jorge J. Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Information Fusion. 2022. vol. 81. pp. 59–83.
44. Tian L., Oviatt S., Muszunski M., Chamberlain B.C., Healey J., Sano, A. Applied Affective Computing. ACM Books. 2022.
45. Michelucci P. (ed.) Handbook of Human Computation. New York, Springer. 2013.
46. Samsonovich A.V. (ed.) Biologically Inspired Cognitive Architectures. Advances in Intelligent Systems and Computing vol. 948. Cham, Springer. 2020.
47. Adadi A., Berrada M. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access. 2018. vol. 6. no. 52. pp. 138–152.
48. Dennett D.C. The intentional stance. Cambridge, MIT Press. 1998.
49. Caporael L.R. Anthropomorphism and mechanomorphism: Two faces of the human machine. Computers in Human Behavior. 1986. vol. 2. no. 3. pp. 215–234.
50. Guthrie S.E. Anthropomorphism: A definition and a theory. Anthropomorphism, anecdotes, and animals. (eds. Mitchell R.W., Thomson N.S., Miles H.L.), chap. 5, pp. 50-58. State University of New York Press, New York. 1997.
51. Watson D. The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence. Minds and Machines. 2019. vol. 29, no. 3. pp. 417–440.
52. Salles A., Evers K., Farisco M. Anthropomorphism in AI. AJOB Neurosci. 2020. vol. 11. no. 2. pp. 88–95.
53. Maclure J. AI, Explainability and Public Reason: The Argument from the Limitations of the Human Mind. Minds and Machines. 2021.
54. Arnulf J.K., Larsen K.R., Martinsen Ø.L., Bong C.H. Predicting survey responses: How and why semantics shape survey statistics on Organizational Behaviour. PLoS ONE. 2014. vol. 9. no. 9.
55. Jones M.N., Gruenenfelder T.M., Recchia G. In defense of spatial models of semantic representation. New Ideas in Psychology. 2018. vol. 50. pp. 54–60.
56. Arnulf J.K. Wittgenstein’s Revenge: How Semantic Algorithms Can Help Survey Research Escape Smedslund’s Labyrinth. Respect for Thought (eds. Lindstad T.G., Stanicke E., Valsiner J.), chap. 17, pp. 285–307. Springer, Cham. 2020.
2. Pennington J., Socher R., Manning C.D. Glove: Global vectors for word representation. Proceedings of the 2014 conference on empirical methods in natural language processing. 2014. pp. 1532–1543.
3. Radford A., Narasimhan K., Salimans T., Sutskever I. Improving Language Understanding by Generative Pre-Training. 2018.
4. Yang Z., Dai Z., Yang Y., Carbonell J., Salakhutdinov R., Le Q.V. XLNet: Generalized autoregressive pretraining for language understanding. Proceedings of 33rd Conference on Neural Information Processing Systems. 2019.
5. Mikolov T., Joulin A., Baroni M. A Roadmap Towards Machine Intelligence. Computational Linguistics and Intelligent Text Processing. Cham: Springer, 2018. pp. 29–61.
6. Al-Saqqa S., Awajan A. The Use of Word2vec Model in Sentiment Analysis: A Survey. ACM International Conference Proceeding Series. 2019. pp. 39–43.
7. Dhar A., Mukherjee H., Dash N.S., Roy K. Text categorization: past and present. Artificial Intelligence Review. 2021. vol. 54. no. 4. pp. 3007–3054.
8. Konstantinov A., Moshkin V., Yarushkina N. Approach to the Use of Language Models BERT and Word2vec in Sentiment Analysis of Social Network Texts. Recent Research in Control Engineering and Decision Making. Cham: Springer, 2021. pp. 462–473.
9. Gunning D., Stefik M., Choi J., Miller T., Stumpf S., Yang G.Z. XAI—Explainable artificial intelligence. Science Robotics. 2019. vol. 4. no. 37.
10. Suvorova A. Interpretable Machine Learning in Social Sciences: Use Cases and Limitations. Proceedings of Digital Transformation and Global Society 2021. Communications in Computer and Information Science, vol. 1503. Cham: Springer, 2022. pp. 319–331.
11. Osgood C.E. The nature and measurement of meaning. Psychological Bulletin. 1952. vol. 49. no. 3. pp. 197–237.
12. Osgood C.E. Studies on the generality of affective meaning systems. American Psychologist. 1962. vol.17. no. 1. pp. 10–28.
13. Hollis G., Westbury C. The principals of meaning: Extracting semantic dimensions from co-occurrence models of semantics. Psychonomic Bulletin and Review. 2016. vol. 23. no. 6. pp. 1744–1756.
14. Lenci A. Distributional Models of Word Meaning. Annual Review of Linguistics. 2018. vol. 4. no. 1. pp. 151–171.
15. Gunther F., Rinaldi L., Marelli M. Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions. Perspectives on Psychological Science. 2019. vol. 14. no. 6. pp. 1006–1033.
16. Samsonovich A.V., Ascoli G.A. Principal Semantic Components of Language and the Measurement of Meaning. PLoS ONE. 2010. vol. 5. no. 6.
17. Eidlin A.A., Eidlina M.A., Samsonovich A.V. Analyzing weak semantic map of word senses. Procedia Computer Science. 2018. vol. 123. pp. 140–148.
18. Samsonovich A.V. On semantic map as a key component in socially-emotional BICA. Biologically Inspired Cognitive Architectures. 2018. vol. 23. pp. 1–6.
19. Pretrained word2vec model “GoogleNews-vectors-negative300.bin.gz”. Google Code Archive. https://code.google.com/archive/p/word2vec/. 2013.
20. Mikolov T., Sutskever I., Chen K., Corrado G., Dean J. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems. 2013.
21. Surov I.A. Natural Code of Subjective Experience. Biosemiotics. 2022. vol. 15. no. 1. pp. 109–139.
22. Siegel M. The sense-think-act paradigm revisited. Proceedings of the 1st International Workshop on Robotic Sensing. 2003.
23. Gastaldi J.L. Why Can Computers Understand Natural Language? Philosophy & Technology. 2021. vol. 34. no. 1. pp. 149–214.
24. Jensen A.R. The relationship between learning and intelligence. Learning and Individual Differences. 1989. vol. 1. no. 1. pp. 37–62.
25. Sowa J.F. The Cognitive Cycle. Proceedings of the 2015 Federated Conference on Computer Science and Information Systems. 2015. vol. 5, pp. 11–16.
26. Wang Y., Yao Q., Kwok J.T., Ni L.M.: Generalizing from a Few Examples: A Survey on Few-shot Learning. ACM Computing Surveys. 2021. vol. 53. no. 3. pp. 1–34.
27. Hoenkamp E. Why Information Retrieval Needs Cognitive Science: A Call to Arms. 2005.
28. Turney P.D., Pantel P. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research. 2010. vol. 37. pp. 141–188.
29. Wang B., Buccio E.D., Melucci M. Representing Words in Vector Space and Beyond. Quantum-Like Models for Information Retrieval and Decision-Making (eds: Aerts D., Khrennikov A., Melucci M., Toni B.). Cham, Springer. pp. 83–113. 2019.
30. beim Graben P., Huber M., Meyer W., R ̈omer R., Wolff M. Vector Symbolic Architectures for Context-Free Grammars. Cognitive Computation. 2022. vol. 14. no. 2. pp. 733–748.
31. Coenen A., Reif E., Kim A.Y.B., Pearce A., Vi ́egas F., Wattenberg M. Visualizing and measuring the geometry of BERT. Proceedings of the Advances in Neural Information Processing Systems. 2019.
32. Tanaka Y., Oyama T., Osgood C.E. A cross-culture and cross-concept study of the generality of semantic spaces. Journal of Verbal Learning and Verbal Behavior. 1963. vol. 2. no. 5-6. pp. 392–405.
33. Tanaka Y., Osgood C.E. Cross-culture, cross-concept, and cross-subject generality of affective meaning systems. Journal of Personality and Social Psychology. 1965. vol. 2. no. 2. pp. 143–153.
34. Osgood C.E., May W.H., Miron M.S. Cross-cultural universals of affective meaning. Champaign, University of Illinois Press. 1975.
35. Zajonc R.B. Feeling and thinking: Preferences need no inferences. American Psychologist. 1980. vol. 35. no. 2. pp. 151–175.
36. Duncan S., Barrett L.F. Affect is a form of cognition: A neurobiological analysis. Cognition and Emotion. 2007. vol. 21, no. 6. pp. 1184–1211.
37. Lipton Z.C. The Mythos of Model Interpretability. Queue. 2018. vol. 3. pp. 31–57.
38. Molnar C. Interpretable Machine Learning. A Guide for Making Black Box Models Explainable. 2019.
39. Guidotti R., Monreale A., Ruggieri S., Turini F., Giannotti F., Pedreschi D. A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys. 2019. vol. 51. no. 5.
40. Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence. 2019. vol. 1. no. 5. pp. 206–215.
41. Vassiliades A., Bassiliades N., Patkos T. Argumentation and explainable artificial intelligence: A survey // Knowledge Engineering Review. 2021. vol. 36, pp. 1-35.
42. Borrego-D ́iaz J., Gal ́an-P ́aez J. Explainable Artificial Intelligence in Data Science. Minds and Machines. 2022.
43. Chou Y.L., Moreira C., Bruza P., Ouyang C., Jorge J. Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Information Fusion. 2022. vol. 81. pp. 59–83.
44. Tian L., Oviatt S., Muszunski M., Chamberlain B.C., Healey J., Sano, A. Applied Affective Computing. ACM Books. 2022.
45. Michelucci P. (ed.) Handbook of Human Computation. New York, Springer. 2013.
46. Samsonovich A.V. (ed.) Biologically Inspired Cognitive Architectures. Advances in Intelligent Systems and Computing vol. 948. Cham, Springer. 2020.
47. Adadi A., Berrada M. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access. 2018. vol. 6. no. 52. pp. 138–152.
48. Dennett D.C. The intentional stance. Cambridge, MIT Press. 1998.
49. Caporael L.R. Anthropomorphism and mechanomorphism: Two faces of the human machine. Computers in Human Behavior. 1986. vol. 2. no. 3. pp. 215–234.
50. Guthrie S.E. Anthropomorphism: A definition and a theory. Anthropomorphism, anecdotes, and animals. (eds. Mitchell R.W., Thomson N.S., Miles H.L.), chap. 5, pp. 50-58. State University of New York Press, New York. 1997.
51. Watson D. The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence. Minds and Machines. 2019. vol. 29, no. 3. pp. 417–440.
52. Salles A., Evers K., Farisco M. Anthropomorphism in AI. AJOB Neurosci. 2020. vol. 11. no. 2. pp. 88–95.
53. Maclure J. AI, Explainability and Public Reason: The Argument from the Limitations of the Human Mind. Minds and Machines. 2021.
54. Arnulf J.K., Larsen K.R., Martinsen Ø.L., Bong C.H. Predicting survey responses: How and why semantics shape survey statistics on Organizational Behaviour. PLoS ONE. 2014. vol. 9. no. 9.
55. Jones M.N., Gruenenfelder T.M., Recchia G. In defense of spatial models of semantic representation. New Ideas in Psychology. 2018. vol. 50. pp. 54–60.
56. Arnulf J.K. Wittgenstein’s Revenge: How Semantic Algorithms Can Help Survey Research Escape Smedslund’s Labyrinth. Respect for Thought (eds. Lindstad T.G., Stanicke E., Valsiner J.), chap. 17, pp. 285–307. Springer, Cham. 2020.
Опубликован
2022-09-28
Как цитировать
Суров, И. А. (2022). Открытие чёрного ящика: Извлечение семантических факторов Осгуда из языковой модели word2vec. Информатика и автоматизация, 21(5), 916-936. https://doi.org/10.15622/ia.21.5.3
Раздел
Искусственный интеллект, инженерия данных и знаний
Copyright (c) Илья Алексеевич Суров
Это произведение доступно по лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.
Авторы, которые публикуются в данном журнале, соглашаются со следующими условиями: Авторы сохраняют за собой авторские права на работу и передают журналу право первой публикации вместе с работой, одновременно лицензируя ее на условиях Creative Commons Attribution License, которая позволяет другим распространять данную работу с обязательным указанием авторства данной работы и ссылкой на оригинальную публикацию в этом журнале. Авторы сохраняют право заключать отдельные, дополнительные контрактные соглашения на неэксклюзивное распространение версии работы, опубликованной этим журналом (например, разместить ее в университетском хранилище или опубликовать ее в книге), со ссылкой на оригинальную публикацию в этом журнале. Авторам разрешается размещать их работу в сети Интернет (например, в университетском хранилище или на их персональном веб-сайте) до и во время процесса рассмотрения ее данным журналом, так как это может привести к продуктивному обсуждению, а также к большему количеству ссылок на данную опубликованную работу (Смотри The Effect of Open Access).