Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT

Nir Daniel; Florian Klaus Kaiser; Anton Dzega; Aviad Elyashar; Rami Puzis

doi:10.1007/978-3-031-54129-2_5

Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT

Nir Daniel, Florian Klaus Kaiser, Anton Dzega, Aviad Elyashar, Rami Puzis

Department of Software and Information Systems Engineering

Ben-Gurion University of the Negev

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

A typical analyst spends much time and effort investigating alerts from network intrusion detection systems (NIDS). Available NIDS rules for enterprise and industrial control systems are not always accompanied by high-level explanations that allow for building valid hypotheses about the attacker’s techniques and intentions. The plethora of rules and the lack of high-level information necessitates new automated methods for alert enrichment. Large language models, such as ChatGPT, encompass a vast amount of knowledge, including cyber threat intelligence such as ports and protocols (low-level) and MITRE ATT &CK techniques (high-level). Despite being a very new technology, ChatGPT is increasingly used in order to automate processes that experts previously performed. In this paper, we explore the ability of ChatGPT to reason about NIDS rules while labeling them with MITRE ATT &CK techniques. We discuss prompt design and present results on ChatGPT-3.5, ChatGPT-4, and a keyword-based approach. Our results indicate that both versions of ChatGPT outperform a baseline that relies on a-priori frequencies of the techniques. ChatGPT-3.5 is much more precise than ChatGPT-4, with a little reduction in recall.

Original language	American English
Title of host publication	Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers
Editors	Sokratis Katsikas, Habtamu Abie, Silvio Ranise, Luca Verderame, Enrico Cambiaso, Rita Ugarelli, Isabel Praça, Wenjuan Li, Weizhi Meng, Steven Furnell, Basel Katt, Sandeep Pirbhulal, Ankur Shukla, Michele Ianni, Mila Dalla Preda, Kim-Kwang Raymond Choo, Miguel Pupo Correia, Abhishta Abhishta, Giovanni Sileno, Mina Alishahi, Harsha Kalutarage, Naoto Yanai
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	76-91
Number of pages	16
ISBN (Print)	9783031541285
DOIs	https://doi.org/10.1007/978-3-031-54129-2_5
State	Published - 1 Jan 2024
Event	International Workshops which were held in conjunction with 28th European Symposium on Research in Computer Security, ESORICS 2023 - The Hague, Netherlands Duration: 25 Sep 2023 → 29 Sep 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14399 LNCS

Conference

Conference	International Workshops which were held in conjunction with 28th European Symposium on Research in Computer Security, ESORICS 2023
Country/Territory	Netherlands
City	The Hague
Period	25/09/23 → 29/09/23

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure

Keywords

Alerts investigation
Cyber threat intelligence
Natural language processing

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-54129-2_5

Cite this

Daniel, N., Kaiser, F. K., Dzega, A., Elyashar, A., & Puzis, R. (2024). Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. In S. Katsikas, H. Abie, S. Ranise, L. Verderame, E. Cambiaso, R. Ugarelli, I. Praça, W. Li, W. Meng, S. Furnell, B. Katt, S. Pirbhulal, A. Shukla, M. Ianni, M. Dalla Preda, K.-K. R. Choo, M. Pupo Correia, A. Abhishta, G. Sileno, M. Alishahi, H. Kalutarage, ... N. Yanai (Eds.), Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers (pp. 76-91). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14399 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-54129-2_5

Daniel, Nir ; Kaiser, Florian Klaus ; Dzega, Anton et al. / Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers. editor / Sokratis Katsikas ; Habtamu Abie ; Silvio Ranise ; Luca Verderame ; Enrico Cambiaso ; Rita Ugarelli ; Isabel Praça ; Wenjuan Li ; Weizhi Meng ; Steven Furnell ; Basel Katt ; Sandeep Pirbhulal ; Ankur Shukla ; Michele Ianni ; Mila Dalla Preda ; Kim-Kwang Raymond Choo ; Miguel Pupo Correia ; Abhishta Abhishta ; Giovanni Sileno ; Mina Alishahi ; Harsha Kalutarage ; Naoto Yanai. Springer Science and Business Media Deutschland GmbH, 2024. pp. 76-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{fd74fa15ec09484397af831b23c4a0c1,

title = "Labeling NIDS Rules with MITRE ATT \&CK Techniques Using ChatGPT",

abstract = "A typical analyst spends much time and effort investigating alerts from network intrusion detection systems (NIDS). Available NIDS rules for enterprise and industrial control systems are not always accompanied by high-level explanations that allow for building valid hypotheses about the attacker{\textquoteright}s techniques and intentions. The plethora of rules and the lack of high-level information necessitates new automated methods for alert enrichment. Large language models, such as ChatGPT, encompass a vast amount of knowledge, including cyber threat intelligence such as ports and protocols (low-level) and MITRE ATT \&CK techniques (high-level). Despite being a very new technology, ChatGPT is increasingly used in order to automate processes that experts previously performed. In this paper, we explore the ability of ChatGPT to reason about NIDS rules while labeling them with MITRE ATT \&CK techniques. We discuss prompt design and present results on ChatGPT-3.5, ChatGPT-4, and a keyword-based approach. Our results indicate that both versions of ChatGPT outperform a baseline that relies on a-priori frequencies of the techniques. ChatGPT-3.5 is much more precise than ChatGPT-4, with a little reduction in recall.",

keywords = "Alerts investigation, Cyber threat intelligence, Natural language processing",

author = "Nir Daniel and Kaiser, \{Florian Klaus\} and Anton Dzega and Aviad Elyashar and Rami Puzis",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; International Workshops which were held in conjunction with 28th European Symposium on Research in Computer Security, ESORICS 2023 ; Conference date: 25-09-2023 Through 29-09-2023",

year = "2024",

month = jan,

day = "1",

doi = "10.1007/978-3-031-54129-2\_5",

language = "American English",

isbn = "9783031541285",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "76--91",

editor = "Sokratis Katsikas and Habtamu Abie and Silvio Ranise and Luca Verderame and Enrico Cambiaso and Rita Ugarelli and Isabel Pra{\c c}a and Wenjuan Li and Weizhi Meng and Steven Furnell and Basel Katt and Sandeep Pirbhulal and Ankur Shukla and Michele Ianni and \{Dalla Preda\}, Mila and Choo, \{Kim-Kwang Raymond\} and \{Pupo Correia\}, Miguel and Abhishta Abhishta and Giovanni Sileno and Mina Alishahi and Harsha Kalutarage and Naoto Yanai",

booktitle = "Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers",

address = "Germany",

}

Daniel, N, Kaiser, FK, Dzega, A, Elyashar, A & Puzis, R 2024, Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. in S Katsikas, H Abie, S Ranise, L Verderame, E Cambiaso, R Ugarelli, I Praça, W Li, W Meng, S Furnell, B Katt, S Pirbhulal, A Shukla, M Ianni, M Dalla Preda, K-KR Choo, M Pupo Correia, A Abhishta, G Sileno, M Alishahi, H Kalutarage & N Yanai (eds), Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14399 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 76-91, International Workshops which were held in conjunction with 28th European Symposium on Research in Computer Security, ESORICS 2023, The Hague, Netherlands, 25/09/23. https://doi.org/10.1007/978-3-031-54129-2_5

Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. / Daniel, Nir; Kaiser, Florian Klaus; Dzega, Anton et al.
Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers. ed. / Sokratis Katsikas; Habtamu Abie; Silvio Ranise; Luca Verderame; Enrico Cambiaso; Rita Ugarelli; Isabel Praça; Wenjuan Li; Weizhi Meng; Steven Furnell; Basel Katt; Sandeep Pirbhulal; Ankur Shukla; Michele Ianni; Mila Dalla Preda; Kim-Kwang Raymond Choo; Miguel Pupo Correia; Abhishta Abhishta; Giovanni Sileno; Mina Alishahi; Harsha Kalutarage; Naoto Yanai. Springer Science and Business Media Deutschland GmbH, 2024. p. 76-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14399 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT

AU - Daniel, Nir

AU - Kaiser, Florian Klaus

AU - Dzega, Anton

AU - Elyashar, Aviad

AU - Puzis, Rami

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024/1/1

Y1 - 2024/1/1

N2 - A typical analyst spends much time and effort investigating alerts from network intrusion detection systems (NIDS). Available NIDS rules for enterprise and industrial control systems are not always accompanied by high-level explanations that allow for building valid hypotheses about the attacker’s techniques and intentions. The plethora of rules and the lack of high-level information necessitates new automated methods for alert enrichment. Large language models, such as ChatGPT, encompass a vast amount of knowledge, including cyber threat intelligence such as ports and protocols (low-level) and MITRE ATT &CK techniques (high-level). Despite being a very new technology, ChatGPT is increasingly used in order to automate processes that experts previously performed. In this paper, we explore the ability of ChatGPT to reason about NIDS rules while labeling them with MITRE ATT &CK techniques. We discuss prompt design and present results on ChatGPT-3.5, ChatGPT-4, and a keyword-based approach. Our results indicate that both versions of ChatGPT outperform a baseline that relies on a-priori frequencies of the techniques. ChatGPT-3.5 is much more precise than ChatGPT-4, with a little reduction in recall.

AB - A typical analyst spends much time and effort investigating alerts from network intrusion detection systems (NIDS). Available NIDS rules for enterprise and industrial control systems are not always accompanied by high-level explanations that allow for building valid hypotheses about the attacker’s techniques and intentions. The plethora of rules and the lack of high-level information necessitates new automated methods for alert enrichment. Large language models, such as ChatGPT, encompass a vast amount of knowledge, including cyber threat intelligence such as ports and protocols (low-level) and MITRE ATT &CK techniques (high-level). Despite being a very new technology, ChatGPT is increasingly used in order to automate processes that experts previously performed. In this paper, we explore the ability of ChatGPT to reason about NIDS rules while labeling them with MITRE ATT &CK techniques. We discuss prompt design and present results on ChatGPT-3.5, ChatGPT-4, and a keyword-based approach. Our results indicate that both versions of ChatGPT outperform a baseline that relies on a-priori frequencies of the techniques. ChatGPT-3.5 is much more precise than ChatGPT-4, with a little reduction in recall.

KW - Alerts investigation

KW - Cyber threat intelligence

KW - Natural language processing

UR - http://www.scopus.com/inward/record.url?scp=85188683063&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-54129-2_5

DO - 10.1007/978-3-031-54129-2_5

M3 - Conference contribution

SN - 9783031541285

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 76

EP - 91

BT - Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers

A2 - Katsikas, Sokratis

A2 - Abie, Habtamu

A2 - Ranise, Silvio

A2 - Verderame, Luca

A2 - Cambiaso, Enrico

A2 - Ugarelli, Rita

A2 - Praça, Isabel

A2 - Li, Wenjuan

A2 - Meng, Weizhi

A2 - Furnell, Steven

A2 - Katt, Basel

A2 - Pirbhulal, Sandeep

A2 - Shukla, Ankur

A2 - Ianni, Michele

A2 - Dalla Preda, Mila

A2 - Choo, Kim-Kwang Raymond

A2 - Pupo Correia, Miguel

A2 - Abhishta, Abhishta

A2 - Sileno, Giovanni

A2 - Alishahi, Mina

A2 - Kalutarage, Harsha

A2 - Yanai, Naoto

PB - Springer Science and Business Media Deutschland GmbH

T2 - International Workshops which were held in conjunction with 28th European Symposium on Research in Computer Security, ESORICS 2023

Y2 - 25 September 2023 through 29 September 2023

ER -

Daniel N, Kaiser FK, Dzega A, Elyashar A, Puzis R. Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. In Katsikas S, Abie H, Ranise S, Verderame L, Cambiaso E, Ugarelli R, Praça I, Li W, Meng W, Furnell S, Katt B, Pirbhulal S, Shukla A, Ianni M, Dalla Preda M, Choo KKR, Pupo Correia M, Abhishta A, Sileno G, Alishahi M, Kalutarage H, Yanai N, editors, Computer Security. ESORICS 2023 International Workshops - CPS4CIP, ADIoT, SecAssure, WASP, TAURIN, PriST-AI, and SECAI, 2023, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2024. p. 76-91. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-54129-2_5

Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT

Abstract

Publication series

Conference

UN SDGs

Keywords

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this