Generation of Impact Factor-Driven Security Rating Questionnaire Using LLMs for AIoT Applications

  • Han, Yuna; 
  • Shim, Simon; 
  • Gajulamandyam, Deva Kumar; 
  • Choi, Yeji; 
  • Lee, Hyunwoo; 
  • ... Chang, Hangbae
Citations

WEB OF SCIENCE

2
Citations

SCOPUS

1

초록

Industry 4.0 has transformed industries by accelerating the adoption of artificial intelligence of things (AIoT); however, it has also led to security risks, such as data leakage. Existing data protection research focuses on network layers, whereas application-layer rating systems rely on subjective evaluations, limiting consistency and applicability. This study proposes a scalable and objective AIoT security rating framework that clarifies ambiguities in the five-question rating system of the Korean Intellectual Property Office and unifies fragmented managerial and technical rating systems. By leveraging large language models (LLMs), the framework integrates a security rating model based on 14 impact factors with automated questionnaire generation. A novel percentage-based measurable constraint ensures objectivity and consistency. The fine-tuned Llama 3.1 8B Instruct model, optimized via direct preference optimization, can enhance customization and question relevance. Results across 13 metrics, including G-Eval and Security G-Eval, highlighted its superiority in aligning questions with prompts, thereby improving specificity and clarity over existing LLMs. A user survey validated its effectiveness with a score of 4.1 out of 5 for correlation and answerability, supported by a Cronbach's alpha of 0.878. This study thus introduces a robust and practical AIoT security rating framework, particularly for the manufacturing, healthcare, and transportation domains, reducing subjective biases while improving applicability.

키워드

Security Rating Questionnaire; AIoT-based Questionnaire; Impact Factor Scaling; LLM Alignment; Security; G-Eval Metric; MODEL
제목
Generation of Impact Factor-Driven Security Rating Questionnaire Using LLMs for AIoT Applications
저자
Han, Yuna; Shim, Simon; Gajulamandyam, Deva Kumar; Choi, Yeji; Lee, Hyunwoo; Chang, Hangbae
DOI
10.22967/HCIS.2026.16.006
발행일
2026-01
유형
Article
저널명
Human-centric Computing and Information Sciences
권
16

파일 다운로드

Thumbnail