Román Pásaro, JavierCarrillo Casado, ÁlvaroMata Vázquez, JacintoPachón Álvarez, Victoria2024-11-272024-11-272024Román-Pásaro, J., Carrillo-Casado, A., Mata-Vázquez, J., & Pachón-Álvarez, V. (2024). I2C-UHU at HOMO-MEX 2024: Leveraging Large Language Models and Ensembling Transformers to Identify and Classify Hate Messages Towards the LGBTQ+ Community. CEUR Workshop Proceedings, 3756. IberLEF 2024 - Proceedings of the Iberian Languages Evaluation Forum, co-located with the Conference of the Spanish Society for Natural Language Processing, SEPLN 2024https://hdl.handle.net/10272/24525This paper was presented at the I International Workshop on Conspiracy theories and hate speech online: Comparison of patterns in narratives and social media about Covid 19, immigrants, refugees and LGTBIQ+ people. Universidad de Huelva, July 12 14, 2023 (https://eventos.uhu.es/99642/detail/i-international-workshop-nonconspirahate-project.html). Este estudio presenta las estrategias avanzadas por el Grupo I2C para abordar la Tarea HOMO-MEX de IberLEF-2024: Detección de discursos de odio en mensajes en línea dirigidos a la población LGBTQ+ hispanohablante mexicana. La principal contribución ha sido la integración de Modelos de Lenguaje Amplios (LLMs) para la clasificación mediante prompting, junto con un ensemble de Transformadores. Al aprovechar las capacidades avanzadas de los LLM para tareas de clasificación directa, se lograron mejoras significativas en el rendimiento. El enfoque de conjunto, que combina varios modelos, mejoró aún más los resultados al aprovechar los puntos fuertes individuales de cada modelo. Los experimentos pusieron de manifiesto la importancia de seleccionar los hiperparámetros adecuados durante el proceso de entrenamiento de los modelos. Mediante una experimentación meticulosa y la evaluación de diferentes combinaciones de hiperparámetros, se identificaron los ajustes óptimos para lograr el mejor rendimiento. En los experimentos de la Tarea 1, se probaron varios modelos y se crearon varios ensambladores. El primer ensamblador combinó Transformers, y su resultado se ensambló además con dos LLM, obteniendo la mejor puntuación F1 para este conjunto de datos. El modelo presentado para la tarea 1 obtuvo una puntuación F1 del 87,64%, lo que le situó en el tercer puesto de la competición. The paper is part of the I+D+i Project titled "Conspiracy Theories and Hate Speech Online: Comparison of Patterns in Narratives and social networks about COVID-19, immigrants, refugees, and LGBTI people [NON-CONSPIRA-HATE!]", PID2021-123983OB-I00, funded by MCIN/AEI/10.13039/501100011033/ and by "ERDF/EU." (https://eseis.es/investigacion/discursos-de-odio/discursos-odio-tc). We are also grateful for the support of our research group: "Estudios Sociales E Intervención Social" (GrupoESEIS), and the research center "Pensamiento Contemporáneo e Innovación para el Desarrollo Social" (COIDESO), and the Applied Computational Social Science Lab, CISCOA-Lab, at the University of Huelva.This paper was presented at the I International Workshop on Conspiracy theories and hate speech online: Comparison of patterns in narratives and social media about Covid 19, immigrants, refugees and LGTBIQ+ people. Universidad de Huelva, July 12 14, 2023 (https://eventos.uhu.es/99642/detail/i-international-workshop-nonconspirahate-project.html). This study presents the strategies advanced by the I2C Group to address the IberLEF-2024 Task HOMO-MEX: Hate speech detection in Online Messages directed towards the Mexican Spanish-speaking LGBTQ+ population. The major contribution has been the integration of Large Language Models (LLMs) for classification through prompting, alongside an ensemble of Transformers. By leveraging the advanced capabilities of LLMs for direct classification tasks, significant improvements in performance were achieved. The ensemble approach, which combines multiple models, further enhanced the results by leveraging the individual strengths of each model. The experiments highlighted the importance of selecting appropriate hyperparameters during the model training process. Through meticulous experimentation and evaluation of different hyperparameter combinations, the optimal settings for achieving the best performance were identified. In the experiments for Task 1, several models were tested, and multiple ensemblers were created. The first ensembler combined Transformers, and its result was further ensembled with two LLMs, obtaining the best F1-Score for this dataset. The model submitted for Task 1 achieved an F1-Score of 87.64%, ranking in 3rd place in the competition. The paper is part of the I+D+i Project titled "Conspiracy Theories and Hate Speech Online: Comparison of Patterns in Narratives and social networks about COVID-19, immigrants, refugees, and LGBTI people [NON-CONSPIRA-HATE!]", PID2021-123983OB-I00, funded by MCIN/AEI/10.13039/501100011033/ and by "ERDF/EU." (https://eseis.es/investigacion/discursos-de-odio/discursos-odio-tc). We are also grateful for the support of our research group: "Estudios Sociales E Intervención Social" (GrupoESEIS), and the research center "Pensamiento Contemporáneo e Innovación para el Desarrollo Social" (COIDESO), and the Applied Computational Social Science Lab, CISCOA-Lab, at the University of Huelva.engAtribución-NoComercial-SinDerivadas 3.0 Españahttp://creativecommons.org/licenses/by-nc-nd/3.0/es/Hate speech detectionLarge Language ModelsTransformersPromptingLGBTQ+ communityNatural Language ProcessingAprendizaje profundoDeep Learningconference outputopen access33 Ciencias Tecnológicas