Other formats:
BibTeX
LaTeX
RIS
@inproceedings{46806, author = {Švábenský, Valdemar and Weiss, Richard and Cook, Jack and Vykopal, Jan and Čeleda, Pavel and Mache, Jens and Chudovský, Radoslav and Chattopadhyay, Ankur}, address = {New York, NY, USA}, booktitle = {Proceedings of the 53rd ACM Technical Symposium on Computer Science Education (SIGCSE '22)}, doi = {http://dx.doi.org/10.1145/3478431.3499414}, keywords = {cybersecurity education; command-line history; educational data mining; learning analytics; assessment; modeling}, howpublished = {elektronická verze "online"}, language = {eng}, location = {New York, NY, USA}, isbn = {978-1-4503-9070-5}, pages = {787-793}, publisher = {ACM}, title = {Evaluating Two Approaches to Assessing Student Progress in Cybersecurity Exercises}, url = {https://youtu.be/rzJ0AG_FsvU}, year = {2022} }
TY - JOUR ID - 46806 AU - Švábenský, Valdemar - Weiss, Richard - Cook, Jack - Vykopal, Jan - Čeleda, Pavel - Mache, Jens - Chudovský, Radoslav - Chattopadhyay, Ankur PY - 2022 TI - Evaluating Two Approaches to Assessing Student Progress in Cybersecurity Exercises PB - ACM CY - New York, NY, USA SN - 9781450390705 KW - cybersecurity education KW - command-line history KW - educational data mining KW - learning analytics KW - assessment KW - modeling UR - https://youtu.be/rzJ0AG_FsvU N2 - Cybersecurity students need to develop practical skills such as using command-line tools. Hands-on exercises are the most direct way to assess these skills, but assessing students' mastery is a challenging task for instructors. We aim to alleviate this issue by modeling and visualizing student progress automatically throughout the exercise. The progress is summarized by graph models based on the shell commands students typed to achieve discrete tasks within the exercise. We implemented two types of models and compared them using data from 46 students at two universities. To evaluate our models, we surveyed 22 experienced computing instructors and qualitatively analyzed their responses. The majority of instructors interpreted the graph models effectively and identified strengths, weaknesses, and assessment use cases for each model. Based on the evaluation, we provide recommendations to instructors and explain how our graph models innovate teaching and promote further research. The impact of this paper is threefold. First, it demonstrates how multiple institutions can collaborate to share approaches to modeling student progress in hands-on exercises. Second, our modeling techniques generalize to data from different environments to support student assessment, even outside the cybersecurity domain. Third, we share the acquired data and open-source software so that others can use the models in their classes or research. ER -
ŠVÁBENSKÝ, Valdemar, Richard WEISS, Jack COOK, Jan VYKOPAL, Pavel ČELEDA, Jens MACHE, Radoslav CHUDOVSKÝ and Ankur CHATTOPADHYAY. Evaluating Two Approaches to Assessing Student Progress in Cybersecurity Exercises. Online. In \textit{Proceedings of the 53rd ACM Technical Symposium on Computer Science Education (SIGCSE '22)}. New York, NY, USA: ACM, 2022, p.~787-793. ISBN~978-1-4503-9070-5. Available from: https://dx.doi.org/10.1145/3478431.3499414.
|