info@ijrretas.com
E-ISSN: 2455-4723 | Impact Factor: 7.95
+91 77710 84928 Support
IJRRETAS logo
  • About
    • About The Journal
    • Aim & Scope
    • Privacy Statement
    • Journal Policies
    • Disclaimer
    • Abstracting and Indexing
    • FAQ
  • Current
  • Archive
  • For Author
    • Submit Paper Online
    • Track Paper Status
    • Pay Publication Fee
    • Verify Certificate
    • Article Processing Charges
    • Submission Guidelines
    • Manuscript Types
    • Download Template (Word)
    • Download Copyright Form (Word)
  • Editorial Board
    • Editorial Board
    • Editors Responsibilities
    • Join as Editor / Reviewer
  • More
    • Contact
    • Pay Publication Fee Online
    • Bank / UPI Payment Details
    • Blogs
  • Submit Paper

Recent Papers

Dedicated to advancing knowledge through rigorous research and scholarly publication

  1. Home
  2. Recent Papers

Verification of Generative AI Outputs in Computer Science Research: A Review of Hallucination, Code Reliability, Citation Integrity and Assessment Frameworks

Authors: Sonal Anilkumar Tiwari, Manav Thakur

Published in: IJRRETAS, Volume 11, Issue 8, pp. 20-30 (02 Aug 2025)

Impact Factor: 7.95 (2025)

License: CC BY 4.0 CC BY 4.0 (Open Access)

Certificate: View Certificate

Abstract

Generative artificial intelligence (GAI) has become part of everyday practice in computer science research, supporting code writing, literature searching, drafting and reasoning. The fluency of large language models, however, is not a guarantee of correctness, and unverified outputs can introduce fabricated references, insecure programs and invalid arguments into the scholarly record. This review synthesises peer-reviewed and archival research published between 2017 and 2024 on the reliability of GAI outputs and on the means of verifying them. It first outlines the architectural and training foundations that explain why such systems produce probable rather than verified text. It then examines the taxonomy, causes and detection of hallucination, the empirical evidence on defects in generated source code and scholarly citations, and the evaluation frameworks, benchmarks and measurement practices used to judge model output. Three comparison tables summarise the main studies and their limits. The review finds that detection methods are capable but fragmented, that benchmarks assess systems rather than individual artefacts, that human evaluation is rarely supported by reliability testing, and that no validated instrument supports a researcher deciding whether a specific output may be used. The paper concludes that verification must be treated as a multi-dimensional, human-supervised and statistically validated activity, and it proposes directions for future work on performance-based assessment frameworks.

Keywords
  • generative artificial intelligence
  • large language models
  • hallucination
  • verification
  • code security
  • citation fabrication
  • evaluation frameworks
References
  1. Athaluri, S. A., Manthena, S. V., Kesapragada, V. S. R. K. M., Yarlagadda, V., Dave, T., & Duddumpudi, R. T. S. (2023). Exploring the boundaries of reality: Investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references. Cureus, 15(4), Article e37432.
  2. Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P. S., Yang, Q., & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3), Article 39. https://doi.org/10.1145/3641289
  3. Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017), 625–630.
  4. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730
  5. Kalai, A. T., & Vempala, S. S. (2024). Calibrated language models must hallucinate. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing (pp. 160–171). ACM.
  6. Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.
  7. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems (Vol. 33, pp. 9459–9474). Curran Associates.
  8. Liu, J., Xia, C. S., Wang, Y., & Zhang, L. (2023). Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems (Vol. 36, pp. 21558–21572). Curran Associates.
  9. Manakul, P., Liusie, A., & Gales, M. J. F. (2023). SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 9004–9017). Association for Computational Linguistics.
  10. Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020). On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 1906–1919). Association for Computational Linguistics.
  11. Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-t., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 12076–12100). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.741
  12. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems (Vol. 35, pp. 27730–27744). Curran Associates.
  13. Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., & Karri, R. (2022). Asleep at the keyboard? Assessing the security of GitHub Copilot's code contributions. In 2022 IEEE Symposium on Security and Privacy (pp. 754–768). IEEE.
  14. Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023). Do users write more insecure code with AI assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (pp. 2785–2799). ACM.
  15. Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., & Bowman, S. R. (2024). GPQA: A graduate-level Google-proof Q&A benchmark. In Proceedings of the First Conference on Language Modeling.
  16. Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems (Vol. 36). Curran Associates.
  17. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30, pp. 5998–6008). Curran Associates.
  18. Venable, J., Pries-Heje, J., & Baskerville, R. (2016). FEDS: A framework for evaluation in design science research. European Journal of Information Systems, 25(1), 77–89.
  19. Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, Article 14045.
  20. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (Vol. 36, pp. 46595–46623). Curran Associates.
How to Cite
APA: Tiwari, S. A., & Thakur, M. (2025). Verification of Generative AI Outputs in Computer Science Research: A Review of Hallucination, Code Reliability, Citation Integrity and Assessment Frameworks. International Journal for Rapid Research in Engineering Technology & Applied Science, 11(8), 20–30. https://ijrretas.com/research-paper/verification-of-generative-ai-outputs-in-computer-science-research-a-review-of-hallucinati-328
More from this Issue
  • Biodiversity and Ecological Status of Selected Plant Communities in Northern Maharashtra: A ReviewGoswami Arupkumar Asit, Chandrashekhar
  • Non-linear optical Properties and Thermodynamic functions of 2,5-Dimethoxy-4-nitroaniline by using DFT and HFVishrut Chaudhary
More by these Authors
  • A Hybrid Light GBM Framework Integrating Feature Selection and Cost-Sensitive Learning for Network Outlier Detection2026 • Vol. 12, Issue 2

Download Paper

Paper Id: IJRRETAS328

Publish Date: 2025-08-02

ISSN: 2455-4723

Publisher Name: IJRRETAS

About IJRRETAS

ijrretas is a leading open-access, peer-reviewed journal dedicated to advancing research in applied sciences and engineering. We provide a global platform for researchers to disseminate innovative findings and technological breakthroughs.

ISSN
2455-4723
Established
2015

Quick Links

Home Submit Paper Track Paper Status Author Guidelines Editorial Board Past Issues Topics
Fees Structure Scope & Topics Terms & Conditions Privacy Policy Refund and Cancellation Policy

Contact Us

Tilak Nagar, Indore, Madhya Pradesh, India

info@ijrretas.com

+91 77710 84928

www.ijrretas.com

Indexed In
Google Scholar Crossref ResearchGate CiteFactor
© 2026 ijrretas. All Rights Reserved.
Privacy Policy Terms of Service