Construcción gobernada de conocimiento funcional desde la interfaz de un ERP para asistencia conversacional

Cesar Andres Mendieta Espinoza; Jonathan Joseph Chalco Berrezueta; Joffre Jeorwin Cartuche Calva; Edison Luis Lojan Cueva

doi:10.59169/pentaciencias.v8i4.1921

Authors

Cesar Andres Mendieta Espinoza Universidad Técnica de Machala https://orcid.org/0000-0001-6335-3637
Jonathan Joseph Chalco Berrezueta Universidad Técnica de Machala https://orcid.org/0009-0002-8687-523X
Joffre Jeorwin Cartuche Calva Universidad Técnica de Machala https://orcid.org/0000-0002-1633-2291
Edison Luis Lojan Cueva Universidad Técnica de Machala https://orcid.org/0000-0002-7092-1281

DOI:

https://doi.org/10.59169/pentaciencias.v8i4.1921

Keywords:

knowledge acquisition; enterprise resource planning; hybrid retrieval; knowledge graphs; human-in-the-loop review

Abstract

Enterprise resource planning (ERP) systems hold a great deal of operational knowledge that is rarely documented. We propose Chat-CBMM, a method that learns from the system’s interface, sends its findings through human review, and answers questions using only approved knowledge. We tested it at the Cuerpo de Bomberos Municipal de Machala with a campaign defined in advance: a language model first scored the responses, and experts confirmed the scores. The crawler found all 57 eligible routes and 14 modules. The generated semantics matched the reference only in part (F1 between 0.30 and 0.43); the change between rounds came mainly from how the scores were adjudicated, not from human review. Across 54 queries, approved semantics raised behavioral accuracy from 35.2% to 44.4%, although not significantly. The writer and the graph added small differences, and in queries that modify data, all conditions stayed safe. Our reading is that the system works as supervised support, but not yet as an autonomous assistant.

Downloads

Download data is not yet available.

References

Al-Hawari, F., & Barham, H. (2021). A machine learning based help desk system for IT service management. Journal of King Saud University - Computer and Information Sciences, 33(6), 702–718. https://doi.org/10.1016/j.jksuci.2019.04.001

Amado, A., & Belfo, F. P. (2021). Maintenance and support model within the ERP systems lifecycle: Action research in an implementer company. Procedia Computer Science, 181, 580–588. https://doi.org/10.1016/j.procs.2021.01.205

Arslan, M., Ghanem, H., Munawar, S., & Cruz, C. (2024). A survey on RAG with LLMs. Procedia Computer Science, 246, 3781–3790. https://doi.org/10.1016/j.procs.2024.09.178

Chou, H.-W., Chang, H.-H., Lin, Y.-H., & Chou, S.-B. (2014). Drivers and effects of post-implementation learning on ERP usage. Computers in Human Behavior, 35, 267–277. https://doi.org/10.1016/j.chb.2014.03.012

Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '09) (pp. 758–759). Association for Computing Machinery. https://doi.org/10.1145/1571941.1572114

Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., & Larson, J. (2024). From local to global: A graph RAG approach to query-focused summarization. arXiv. https://doi.org/10.48550/arXiv.2404.16130

Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024). A survey on RAG meeting LLMs: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '24) (pp. 6491–6501). Association for Computing Machinery. https://doi.org/10.1145/3637528.3671470

Han, H., Shomer, H., Wang, Y., Lei, Y., Guo, K., Hua, Z., Long, B., Liu, H., & Tang, J. (2025). RAG vs. GraphRAG: A systematic evaluation and key insights. arXiv. https://doi.org/10.48550/arXiv.2502.11371

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design science in information systems research. MIS Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625

Hilel, M., Karmim, Y., De Bodinat, J., Sarehane, R., & Gillon, A. (2025). Building specialized software-assistant chatbot with graph-based retrieval-augmented generation. arXiv. https://doi.org/10.48550/arXiv.2511.05297

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248, 1–38. https://doi.org/10.1145/3571730

Landeta-López, P., García, J. M., Guevara-Vega, C., & Ruiz-Cortés, A. (2026). LLMs applied to web scraping and web crawling: A systematic review. Computing, 108, 78. https://doi.org/10.1007/s00607-026-01666-5

Lehnen, N. C., Kürsch, J., Wichtmann, B. D., Wolter, M., Bendella, Z., Bode, F. J., Zimmermann, H., Radbruch, A., Vollmuth, P., & Dorn, F. (2025). Llama 3.1 405B is comparable to GPT-4 for extraction of data from thrombectomy reports - a step towards secure data extraction. Clinical Neuroradiology, 35, 495–510. https://doi.org/10.1007/s00062-025-01500-z

Maiya, A. S. (2025). OnPrem.LLM: A privacy-conscious document intelligence toolkit. arXiv. https://doi.org/10.48550/arXiv.2505.07672

Mesbah, A., van Deursen, A., & Lenselink, S. (2012). Crawling Ajax-based web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web, 6(1), 1–30. https://doi.org/10.1145/2109205.2109208

Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023). Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review, 56(4), 3005–3054. https://doi.org/10.1007/s10462-022-10246-w

Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77. https://doi.org/10.2753/MIS0742-1222240302

Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., & Tang, S. (2025). Graph retrieval-augmented generation: A survey. ACM Transactions on Information Systems. https://doi.org/10.1145/3777378

Tian, F., Wang, T., Liang, P., Wang, C., Khan, A. A., & Babar, M. A. (2021). The impact of traceability on software maintenance and evolution: A mapping study. arXiv. https://doi.org/10.48550/arXiv.2108.02133

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Benchmarks Track. https://doi.org/10.48550/arXiv.2306.05685