Cambio dataset de ARG y agrego preprocesado #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Notas de los aportes más importantes:
👉 Soy de Argentina y algunas palabras del dataset no eran consideradas insultos acá. Las que dejé considero que son bastante útiles porque representan la mayoría de casos.
👉 Agregué la clase RemoveDuplicates: Remueve caracteres duplicados en las palabras. Si bien no es bueno para palabras inocuas que contienen letras duplicadas, es excelente para no dejar pasar casos donde los usuarios repiten letras en insultos para saltearse filtros.
🗂️ En cuanto a los otros archivos, agregue algunos que permiten ahorrar un poco de tiempo a la hora de crear mas datasets o actualizarlos, ya que la mayoría de insultos tienen dos géneros (con la implementación se escriben una sola vez en el archivo).
Espero que sirva, y ante cualquier novedad me encantaría permanecer en contacto!