-
Structural damage detection via hierarchical damage information with volumetric assessment
Authors:
Isaac Osei Agyemang,
Jianwen Chen,
Liaoyuan Zeng,
Isaac Adjei-Mensah,
Daniel Acheampong,
Gordon Owusu Boateng,
Adu Asare Baffour
Abstract:
Image environments and noisy labels hinder deep learning-based inference models in structural damage detection. Post-detection, there is the challenge of reliance on manual assessments of detected damages. As a result, Guided-DetNet, characterized by Generative Attention Module (GAM), Hierarchical Elimination Algorithm (HEA), and Volumetric Contour Visual Assessment (VCVA), is proposed to mitigate…
▽ More
Image environments and noisy labels hinder deep learning-based inference models in structural damage detection. Post-detection, there is the challenge of reliance on manual assessments of detected damages. As a result, Guided-DetNet, characterized by Generative Attention Module (GAM), Hierarchical Elimination Algorithm (HEA), and Volumetric Contour Visual Assessment (VCVA), is proposed to mitigate complex image environments, noisy labeling, and post-detection manual assessment of structural damages. GAM leverages cross-horizontal and cross-vertical patch merging and cross foreground-background feature fusion to generate varied features to mitigate complex image environments. HEA addresses noisy labeling using hierarchical relationships among classes to refine instances given an image by eliminating unlikely class categories. VCVA assesses the severity of detected damages via volumetric representation and quantification leveraging the Dirac delta distribution. A comprehensive quantitative study, two robustness tests, and an application scenario based on the PEER Hub Image-Net dataset substantiate Guided-DetNet's promising performances. Guided-DetNet outperformed the best-compared models in a triple classification task by a difference of not less than 3% and not less than 2% in a dual detection task under varying metrics.
△ Less
Submitted 29 July, 2024;
originally announced July 2024.
-
AfricAIED 2024: 2nd Workshop on Artificial Intelligence in Education in Africa
Authors:
George Boateng,
Victor Kumbol
Abstract:
Recent AI advancements offer transformative potential for global education, yet their application often overlooks Africa's unique educational landscape. AfricAIED 2024 will address this gap, spotlighting efforts to develop AI in Education (AIED) systems tailored to Africa's needs. Building on the success of the inaugural workshop, AfricAIED 2024 will feature an online AI Hackathon focused on democ…
▽ More
Recent AI advancements offer transformative potential for global education, yet their application often overlooks Africa's unique educational landscape. AfricAIED 2024 will address this gap, spotlighting efforts to develop AI in Education (AIED) systems tailored to Africa's needs. Building on the success of the inaugural workshop, AfricAIED 2024 will feature an online AI Hackathon focused on democratizing preparation for Ghana's National Science & Maths Quiz (NSMQ). Participants will create open-source AI tools leveraging resources from the Brilla AI project to level the academic playing field and enhance science and math education across Africa. The workshop will showcase top competitors' solutions, invite discussions on AIED opportunities and challenges in Africa, and highlight the latest advancements in AI education integration. AfricAIED 2024 aims to foster collaboration and innovation, amplifying African voices in the AIED community and driving positive change in African education through AI.
△ Less
Submitted 30 April, 2024;
originally announced May 2024.
-
Brilla AI: AI Contestant for the National Science and Maths Quiz
Authors:
George Boateng,
Jonathan Abrefah Mensah,
Kevin Takyi Yeboah,
William Edor,
Andrew Kojo Mensah-Onumah,
Naafi Dasana Ibrahim,
Nana Sam Yeboah
Abstract:
The African continent lacks enough qualified teachers which hampers the provision of adequate learning support. An AI could potentially augment the efforts of the limited number of teachers, leading to better learning outcomes. Towards that end, this work describes and evaluates the first key output for the NSMQ AI Grand Challenge, which proposes a robust, real-world benchmark for such an AI: "Bui…
▽ More
The African continent lacks enough qualified teachers which hampers the provision of adequate learning support. An AI could potentially augment the efforts of the limited number of teachers, leading to better learning outcomes. Towards that end, this work describes and evaluates the first key output for the NSMQ AI Grand Challenge, which proposes a robust, real-world benchmark for such an AI: "Build an AI to compete live in Ghana's National Science and Maths Quiz (NSMQ) competition and win - performing better than the best contestants in all rounds and stages of the competition". The NSMQ is an annual live science and mathematics competition for senior secondary school students in Ghana in which 3 teams of 2 students compete by answering questions across biology, chemistry, physics, and math in 5 rounds over 5 progressive stages until a winning team is crowned for that year. In this work, we built Brilla AI, an AI contestant that we deployed to unofficially compete remotely and live in the Riddles round of the 2023 NSMQ Grand Finale, the first of its kind in the 30-year history of the competition. Brilla AI is currently available as a web app that livestreams the Riddles round of the contest, and runs 4 machine learning systems: (1) speech to text (2) question extraction (3) question answering and (4) text to speech that work together in real-time to quickly and accurately provide an answer, and then say it with a Ghanaian accent. In its debut, our AI answered one of the 4 riddles ahead of the 3 human contesting teams, unofficially placing second (tied). Improvements and extensions of this AI could potentially be deployed to offer science tutoring to students and eventually enable millions across Africa to have one-on-one learning interactions, democratizing science education.
△ Less
Submitted 30 April, 2024; v1 submitted 3 March, 2024;
originally announced March 2024.
-
Leveraging AI to Advance Science and Computing Education across Africa: Challenges, Progress and Opportunities
Authors:
George Boateng
Abstract:
Across the African continent, students grapple with various educational challenges, including limited access to essential resources such as computers, internet connectivity, reliable electricity, and a shortage of qualified teachers. Despite these challenges, recent advances in AI such as BERT, and GPT-4 have demonstrated their potential for advancing education. Yet, these AI tools tend to be depl…
▽ More
Across the African continent, students grapple with various educational challenges, including limited access to essential resources such as computers, internet connectivity, reliable electricity, and a shortage of qualified teachers. Despite these challenges, recent advances in AI such as BERT, and GPT-4 have demonstrated their potential for advancing education. Yet, these AI tools tend to be deployed and evaluated predominantly within the context of Western educational settings, with limited attention directed towards the unique needs and challenges faced by students in Africa. In this chapter, we discuss challenges with using AI to advance education across Africa. Then, we describe our work developing and deploying AI in Education tools in Africa for science and computing education: (1) SuaCode, an AI-powered app that enables Africans to learn to code using their smartphones, (2) AutoGrad, an automated grading, and feedback tool for graphical and interactive coding assignments, (3) a tool for code plagiarism detection that shows visual evidence of plagiarism, (4) Kwame, a bilingual AI teaching assistant for coding courses, (5) Kwame for Science, a web-based AI teaching assistant that provides instant answers to students' science questions and (6) Brilla AI, an AI contestant for the National Science and Maths Quiz competition. Finally, we discuss potential opportunities to leverage AI to advance education across Africa.
△ Less
Submitted 26 April, 2024; v1 submitted 11 February, 2024;
originally announced February 2024.
-
Cooperative Resource Trading for Network Slicing in Industrial IoT: A Multi-Agent DRL Approach
Authors:
Gordon Owusu Boateng,
Guisong Liu
Abstract:
The industrial Internet of Things (IIoT) and network slicing (NS) paradigms have been envisioned as key enablers for flexible and intelligent manufacturing in the industry 4.0, where a myriad of interconnected machines, sensors, and devices of diversified quality of service (QoS) requirements coexist. To optimize network resource usage, stakeholders in the IIoT network are encouraged to take pragm…
▽ More
The industrial Internet of Things (IIoT) and network slicing (NS) paradigms have been envisioned as key enablers for flexible and intelligent manufacturing in the industry 4.0, where a myriad of interconnected machines, sensors, and devices of diversified quality of service (QoS) requirements coexist. To optimize network resource usage, stakeholders in the IIoT network are encouraged to take pragmatic steps towards resource sharing. However, resource sharing is only attractive if the entities involved are able to settle on a fair exchange of resource for remuneration in a win-win situation. In this paper, we design an economic model that analyzes the multilateral strategic trading interactions between sliced tenants in IIoT networks. We formulate the resource pricing and purchasing problem of the seller and buyer tenants as a cooperative Stackelberg game. Particularly, the cooperative game enforces collaboration among the buyer tenants by coalition formation in order to strengthen their position in resource price negotiations as opposed to acting individually, while the Stackelberg game determines the optimal policy optimization of the seller tenants and buyer tenant coalitions. To achieve a Stackelberg equilibrium (SE), a multi-agent deep reinforcement learning (MADRL) method is developed to make flexible pricing and purchasing decisions without prior knowledge of the environment. Simulation results and analysis prove that the proposed method achieves convergence and is superior to other baselines, in terms of utility maximization.
△ Less
Submitted 26 August, 2023;
originally announced August 2023.
-
Towards an AI to Win Ghana's National Science and Maths Quiz
Authors:
George Boateng,
Jonathan Abrefah Mensah,
Kevin Takyi Yeboah,
William Edor,
Andrew Kojo Mensah-Onumah,
Naafi Dasana Ibrahim,
Nana Sam Yeboah
Abstract:
Can an AI win Ghana's National Science and Maths Quiz (NSMQ)? That is the question we seek to answer in the NSMQ AI project, an open-source project that is building AI to compete live in the NSMQ and win. The NSMQ is an annual live science and mathematics competition for senior secondary school students in Ghana in which 3 teams of 2 students compete by answering questions across biology, chemistr…
▽ More
Can an AI win Ghana's National Science and Maths Quiz (NSMQ)? That is the question we seek to answer in the NSMQ AI project, an open-source project that is building AI to compete live in the NSMQ and win. The NSMQ is an annual live science and mathematics competition for senior secondary school students in Ghana in which 3 teams of 2 students compete by answering questions across biology, chemistry, physics, and math in 5 rounds over 5 progressive stages until a winning team is crowned for that year. The NSMQ is an exciting live quiz competition with interesting technical challenges across speech-to-text, text-to-speech, question-answering, and human-computer interaction. In this ongoing work that began in January 2023, we give an overview of the project, describe each of the teams, progress made thus far, and the next steps toward our planned launch and debut of the AI in October for NSMQ 2023. An AI that conquers this grand challenge can have real-world impact on education such as enabling millions of students across Africa to have one-on-one learning support from this AI.
△ Less
Submitted 8 August, 2023;
originally announced August 2023.
-
Real-World Deployment and Evaluation of Kwame for Science, An AI Teaching Assistant for Science Education in West Africa
Authors:
George Boateng,
Samuel John,
Samuel Boateng,
Philemon Badu,
Patrick Agyeman-Budu,
Victor Kumbol
Abstract:
Africa has a high student-to-teacher ratio which limits students' access to teachers for learning support such as educational question answering. In this work, we extended Kwame, a bilingual AI teaching assistant for coding education, adapted it for science education, and deployed it as a web app. Kwame for Science provides passages from well-curated knowledge sources and related past national exa…
▽ More
Africa has a high student-to-teacher ratio which limits students' access to teachers for learning support such as educational question answering. In this work, we extended Kwame, a bilingual AI teaching assistant for coding education, adapted it for science education, and deployed it as a web app. Kwame for Science provides passages from well-curated knowledge sources and related past national exam questions as answers to questions from students based on the Integrated Science subject of the West African Senior Secondary Certificate Examination (WASSCE). Furthermore, students can view past national exam questions along with their answers and filter by year, question type, and topics that were automatically categorized by a topic detection model which we developed (91% unweighted average recall). We deployed Kwame for Science in the real world over 8 months and had 750 users across 32 countries (15 in Africa) and 1.5K questions asked. Our evaluation showed an 87.2% top 3 accuracy (n=109 questions) implying that Kwame for Science has a high chance of giving at least one useful answer among the 3 displayed. We categorized the reasons the model incorrectly answered questions to provide insights for future improvements. We also share challenges and lessons with the development, deployment, and human-computer interaction component of such a tool to enable other researchers to deploy similar tools. With a first-of-its-kind tool within the African context, Kwame for Science has the potential to enable the delivery of scalable, cost-effective, and quality remote education to millions of people across Africa.
△ Less
Submitted 26 April, 2024; v1 submitted 21 February, 2023;
originally announced February 2023.
-
Can an AI Win Ghana's National Science and Maths Quiz? An AI Grand Challenge for Education
Authors:
George Boateng,
Victor Kumbol,
Elsie Effah Kaufmann
Abstract:
There is a lack of enough qualified teachers across Africa which hampers efforts to provide adequate learning support such as educational question answering (EQA) to students. An AI system that can enable students to ask questions via text or voice and get instant answers will make high-quality education accessible. Despite advances in the field of AI, there exists no robust benchmark or challenge…
▽ More
There is a lack of enough qualified teachers across Africa which hampers efforts to provide adequate learning support such as educational question answering (EQA) to students. An AI system that can enable students to ask questions via text or voice and get instant answers will make high-quality education accessible. Despite advances in the field of AI, there exists no robust benchmark or challenge to enable building such an (EQA) AI within the African context. Ghana's National Science and Maths Quiz competition (NSMQ) is the perfect competition to evaluate the potential of such an AI due to its wide coverage of scientific fields, variety of question types, highly competitive nature, and live, real-world format. The NSMQ is a Jeopardy-style annual live quiz competition in which 3 teams of 2 students compete by answering questions across biology, chemistry, physics, and math in 5 rounds over 5 progressive stages until a winning team is crowned for that year. In this position paper, we propose the NSMQ AI Grand Challenge, an AI Grand Challenge for Education using Ghana's National Science and Maths Quiz competition (NSMQ) as a case study. Our proposed grand challenge is to "Build an AI to compete live in Ghana's National Science and Maths Quiz (NSMQ) competition and win - performing better than the best contestants in all rounds and stages of the competition." We describe the competition, and key technical challenges to address along with ideas from recent advances in machine learning that could be leveraged to solve this challenge. This position paper is a first step towards conquering such a challenge and importantly, making advances in AI for education in the African context towards democratizing high-quality education across Africa.
△ Less
Submitted 30 January, 2023;
originally announced January 2023.
-
Multimodal Emotion Recognition among Couples from Lab Settings to Daily Life using Smartwatches
Authors:
George Boateng
Abstract:
Couples generally manage chronic diseases together and the management takes an emotional toll on both patients and their romantic partners. Consequently, recognizing the emotions of each partner in daily life could provide an insight into their emotional well-being in chronic disease management. The emotions of partners are currently inferred in the lab and daily life using self-reports which are…
▽ More
Couples generally manage chronic diseases together and the management takes an emotional toll on both patients and their romantic partners. Consequently, recognizing the emotions of each partner in daily life could provide an insight into their emotional well-being in chronic disease management. The emotions of partners are currently inferred in the lab and daily life using self-reports which are not practical for continuous emotion assessment or observer reports which are manual, time-intensive, and costly. Currently, there exists no comprehensive overview of works on emotion recognition among couples. Furthermore, approaches for emotion recognition among couples have (1) focused on English-speaking couples in the U.S., (2) used data collected from the lab, and (3) performed recognition using observer ratings rather than partner's self-reported / subjective emotions. In this body of work contained in this thesis (8 papers - 5 published and 3 currently under review in various journals), we fill the current literature gap on couples' emotion recognition, develop emotion recognition systems using 161 hours of data from a total of 1,051 individuals, and make contributions towards taking couples' emotion recognition from the lab which is the status quo, to daily life. This thesis contributes toward building automated emotion recognition systems that would eventually enable partners to monitor their emotions in daily life and enable the delivery of interventions to improve their emotional well-being.
△ Less
Submitted 21 December, 2022;
originally announced December 2022.
-
"Are you okay, honey?": Recognizing Emotions among Couples Managing Diabetes in Daily Life using Multimodal Real-World Smartwatch Data
Authors:
George Boateng,
Xiangyu Zhao,
Malgorzata Speichert,
Elgar Fleisch,
Janina L�scher,
Theresa Pauly,
Urte Scholz,
Guy Bodenmann,
Tobias Kowatsch
Abstract:
Couples generally manage chronic diseases together and the management takes an emotional toll on both patients and their romantic partners. Consequently, recognizing the emotions of each partner in daily life could provide an insight into their emotional well-being in chronic disease management. Currently, the process of assessing each partner's emotions is manual, time-intensive, and costly. Desp…
▽ More
Couples generally manage chronic diseases together and the management takes an emotional toll on both patients and their romantic partners. Consequently, recognizing the emotions of each partner in daily life could provide an insight into their emotional well-being in chronic disease management. Currently, the process of assessing each partner's emotions is manual, time-intensive, and costly. Despite the existence of works on emotion recognition among couples, none of these works have used data collected from couples' interactions in daily life. In this work, we collected 85 hours (1,021 5-minute samples) of real-world multimodal smartwatch sensor data (speech, heart rate, accelerometer, and gyroscope) and self-reported emotion data (n=612) from 26 partners (13 couples) managing diabetes mellitus type 2 in daily life. We extracted physiological, movement, acoustic, and linguistic features, and trained machine learning models (support vector machine and random forest) to recognize each partner's self-reported emotions (valence and arousal). Our results from the best models (balanced accuracies of 63.8% and 78.1% for arousal and valence respectively) are better than chance and our prior work that also used data from German-speaking, Swiss-based couples, albeit, in the lab. This work contributes toward building automated emotion recognition systems that would eventually enable partners to monitor their emotions in daily life and enable the delivery of interventions to improve their emotional well-being.
△ Less
Submitted 22 August, 2022; v1 submitted 16 August, 2022;
originally announced August 2022.
-
Kwame for Science: An AI Teaching Assistant Based on Sentence-BERT for Science Education in West Africa
Authors:
George Boateng,
Samuel John,
Andrew Glago,
Samuel Boateng,
Victor Kumbol
Abstract:
Africa has a high student-to-teacher ratio which limits students' access to teachers. Consequently, students struggle to get answers to their questions. In this work, we extended Kwame, our previous AI teaching assistant, adapted it for science education, and deployed it as a web app. Kwame for Science answers questions of students based on the Integrated Science subject of the West African Senior…
▽ More
Africa has a high student-to-teacher ratio which limits students' access to teachers. Consequently, students struggle to get answers to their questions. In this work, we extended Kwame, our previous AI teaching assistant, adapted it for science education, and deployed it as a web app. Kwame for Science answers questions of students based on the Integrated Science subject of the West African Senior Secondary Certificate Examination (WASSCE). Kwame for Science is a Sentence-BERT-based question-answering web app that displays 3 paragraphs as answers along with a confidence score in response to science questions. Additionally, it displays the top 5 related past exam questions and their answers in addition to the 3 paragraphs. Our preliminary evaluation of the Kwame for Science with a 2.5-week real-world deployment showed a top 3 accuracy of 87.5% (n=56) with 190 users across 11 countries. Kwame for Science will enable the delivery of scalable, cost-effective, and quality remote education to millions of people across Africa.
△ Less
Submitted 10 July, 2022; v1 submitted 27 June, 2022;
originally announced June 2022.
-
Development, Deployment, and Evaluation of DyMand -- An Open-Source Smartwatch and Smartphone System for Capturing Couples' Dyadic Interactions in Chronic Disease Management in Daily Life
Authors:
George Boateng,
Prabhakaran Santhanam,
Elgar Fleisch,
Janina L�scher,
Theresa Pauly,
Urte Scholz,
Tobias Kowatsch
Abstract:
Dyadic interactions of couples are of interest as they provide insight into relationship quality and chronic disease management. Currently, ambulatory assessment of couples' interactions entails collecting data at random or scheduled times which could miss significant couples' interaction/conversation moments. In this work, we developed, deployed and evaluated DyMand, a novel open-source smartwatc…
▽ More
Dyadic interactions of couples are of interest as they provide insight into relationship quality and chronic disease management. Currently, ambulatory assessment of couples' interactions entails collecting data at random or scheduled times which could miss significant couples' interaction/conversation moments. In this work, we developed, deployed and evaluated DyMand, a novel open-source smartwatch and smartphone system for collecting self-report and sensor data from couples based on partners' interaction moments. Our smartwatch-based algorithm uses the Bluetooth signal strength between two smartwatches each worn by one partner, and a voice activity detection machine-learning algorithm to infer that the partners are interacting, and then to trigger data collection. We deployed the DyMand system in a 7-day field study and collected data about social support, emotional well-being, and health behavior from 13 (N=26) Swiss-based heterosexual couples managing diabetes mellitus type 2 of one partner. Our system triggered 99.1% of the expected number of sensor and self-report data when the app was running, and 77.6% of algorithm-triggered recordings contained partners' conversation moments compared to 43.8% for scheduled triggers. The usability evaluation showed that DyMand was easy to use. DyMand can be used by social, clinical, or health psychology researchers to understand the social dynamics of couples in everyday life, and for developing and delivering behavioral interventions for couples who are managing chronic diseases.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Emotion Recognition among Couples: A Survey
Authors:
George Boateng,
Elgar Fleisch,
Tobias Kowatsch
Abstract:
Couples' relationships affect the physical health and emotional well-being of partners. Automatically recognizing each partner's emotions could give a better understanding of their individual emotional well-being, enable interventions and provide clinical benefits. In the paper, we summarize and synthesize works that have focused on developing and evaluating systems to automatically recognize the…
▽ More
Couples' relationships affect the physical health and emotional well-being of partners. Automatically recognizing each partner's emotions could give a better understanding of their individual emotional well-being, enable interventions and provide clinical benefits. In the paper, we summarize and synthesize works that have focused on developing and evaluating systems to automatically recognize the emotions of each partner based on couples' interaction or conversation contexts. We identified 28 articles from IEEE, ACM, Web of Science, and Google Scholar that were published between 2010 and 2021. We detail the datasets, features, algorithms, evaluation, and results of each work as well as present main themes. We also discuss current challenges, research gaps and propose future research directions. In summary, most works have used audio data collected from the lab with annotations done by external experts and used supervised machine learning approaches for binary classification of positive and negative affect. Performance results leave room for improvement with significant research gaps such as no recognition using data from daily life. This survey will enable new researchers to get an overview of this field and eventually enable the development of emotion recognition systems to inform interventions to improve the emotional well-being of couples.
△ Less
Submitted 16 February, 2022;
originally announced February 2022.
-
SuaCode Africa: Teaching Coding Online to Africans using Smartphones
Authors:
George Boateng,
Prince Steven Annor,
Victor Kumbol
Abstract:
There is a burgeoning trend of smartphone ownership in Africa due to the low costs of Android smartphones and the global increase in social media usage. Building upon previous works that introduced a smartphone-based coding course to secondary and tertiary students in Ghana via an in-person program and an online course, this work introduced Africans in 37 countries to our online smartphone-based c…
▽ More
There is a burgeoning trend of smartphone ownership in Africa due to the low costs of Android smartphones and the global increase in social media usage. Building upon previous works that introduced a smartphone-based coding course to secondary and tertiary students in Ghana via an in-person program and an online course, this work introduced Africans in 37 countries to our online smartphone-based course in 2019. Students in this 8-week course read lesson notes, submitted assignments, collaborated with peers, and facilitators in an online forum and completed open and closed-ended surveys after the course. We performed qualitative and quantitative analyses on the data from the course.
Out of the 709 students that applied, 210 were officially admitted to the course after passing the preliminary assignments. And at the end of the course, 72% of the 210 students completed the course. Additionally, students' assignment submissions and self-reports showed an understanding of the programming concepts, with comparable performance between males and females and across educational levels. Also, students mentioned that the lesson notes were easy to understand and they enjoyed the experience of writing code on their smartphones. Moreover, students adequately received help from peers and facilitators in the course forum. Lastly, results of a survey sent to students a year after completing this program showed that they had developed various applications, wrote online tutorials, and learned several tools and technologies. We were successful at introducing coding skills to Africans using smartphones through SuaCode Africa.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
-
AutoGrad: Automated Grading Software for Mobile Game Assignments in SuaCode Courses
Authors:
Prince Steven Annor,
Samuel Boateng,
Edwin Pelpuo Kayang,
George Boateng
Abstract:
Automatic grading systems have been in existence since the turn of the half-century. Several systems have been developed in the literature with either static analysis and dynamic analysis or a hybrid of both methodologies for computer science courses. This paper presents AutoGrad, a novel portable cross-platform automatic grading system for graphical Processing programs developed on Android smartp…
▽ More
Automatic grading systems have been in existence since the turn of the half-century. Several systems have been developed in the literature with either static analysis and dynamic analysis or a hybrid of both methodologies for computer science courses. This paper presents AutoGrad, a novel portable cross-platform automatic grading system for graphical Processing programs developed on Android smartphones during an online course. AutoGrad uses Processing, which is used in the emerging Interactive Media Arts, and pioneers grading systems utilized outside the sciences to assist tuition in the Arts. It also represents the first system built and tested in an African context across over thirty-five countries across the continent. This paper first explores the design and implementation of AutoGrad. AutoGrad employs APIs to download the assignments from the course platform, performs static and dynamic analysis on the assignment to evaluate the graphical output of the program, and returns the grade and feedback to the student. It then evaluates AutoGrad by analyzing data collected from the two online cohorts of 1000+ students of our SuaCode smartphone-based course. From the analysis and students' feedback, AutoGrad is shown to be adequate for automatic assessment, feedback provision to students, and easy integration for both cloud and standalone usage by reducing the time and effort required in grading the 4 assignments required to complete the course.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
-
BERT meets LIWC: Exploring State-of-the-Art Language Models for Predicting Communication Behavior in Couples' Conflict Interactions
Authors:
Jacopo Biggiogera,
George Boateng,
Peter Hilpert,
Matthew Vowels,
Guy Bodenmann,
Mona Neysari,
Fridtjof Nussbeck,
Tobias Kowatsch
Abstract:
Many processes in psychology are complex, such as dyadic interactions between two interacting partners (e.g. patient-therapist, intimate relationship partners). Nevertheless, many basic questions about interactions are difficult to investigate because dyadic processes can be within a person and between partners, they are based on multimodal aspects of behavior and unfold rapidly. Current analyses…
▽ More
Many processes in psychology are complex, such as dyadic interactions between two interacting partners (e.g. patient-therapist, intimate relationship partners). Nevertheless, many basic questions about interactions are difficult to investigate because dyadic processes can be within a person and between partners, they are based on multimodal aspects of behavior and unfold rapidly. Current analyses are mainly based on the behavioral coding method, whereby human coders annotate behavior based on a coding schema. But coding is labor-intensive, expensive, slow, focuses on few modalities. Current approaches in psychology use LIWC for analyzing couples' interactions. However, advances in natural language processing such as BERT could enable the development of systems to potentially automate behavioral coding, which in turn could substantially improve psychological research. In this work, we train machine learning models to automatically predict positive and negative communication behavioral codes of 368 German-speaking Swiss couples during an 8-minute conflict interaction on a fine-grained scale (10-seconds sequences) using linguistic features and paralinguistic features derived with openSMILE. Our results show that both simpler TF-IDF features as well as more complex BERT features performed better than LIWC, and that adding paralinguistic features did not improve the performance. These results suggest it might be time to consider modern alternatives to LIWC, the de facto linguistic features in psychology, for prediction tasks in couples research. This work is a further step towards the automated coding of couples' behavior which could enhance couple research and therapy, and be utilized for other dyadic interactions as well.
△ Less
Submitted 18 September, 2021; v1 submitted 2 June, 2021;
originally announced June 2021.
-
"You made me feel this way": Investigating Partners' Influence in Predicting Emotions in Couples' Conflict Interactions using Speech Data
Authors:
George Boateng,
Peter Hilpert,
Guy Bodenmann,
Mona Neysari,
Tobias Kowatsch
Abstract:
How romantic partners interact with each other during a conflict influences how they feel at the end of the interaction and is predictive of whether the partners stay together in the long term. Hence understanding the emotions of each partner is important. Yet current approaches that are used include self-reports which are burdensome and hence limit the frequency of this data collection. Automatic…
▽ More
How romantic partners interact with each other during a conflict influences how they feel at the end of the interaction and is predictive of whether the partners stay together in the long term. Hence understanding the emotions of each partner is important. Yet current approaches that are used include self-reports which are burdensome and hence limit the frequency of this data collection. Automatic emotion prediction could address this challenge. Insights from psychology research indicate that partners' behaviors influence each other's emotions in conflict interaction and hence, the behavior of both partners could be considered to better predict each partner's emotion. However, it is yet to be investigated how doing so compares to only using each partner's own behavior in terms of emotion prediction performance. In this work, we used BERT to extract linguistic features (i.e., what partners said) and openSMILE to extract paralinguistic features (i.e., how they said it) from a data set of 368 German-speaking Swiss couples (N = 736 individuals) who were videotaped during an 8-minutes conflict interaction in the laboratory. Based on those features, we trained machine learning models to predict if partners feel positive or negative after the conflict interaction. Our results show that including the behavior of the other partner improves the prediction performance. Furthermore, for men, considering how their female partners spoke is most important and for women considering what their male partner said is most important in getting better prediction performance. This work is a step towards automatically recognizing each partners' emotion based on the behavior of both, which would enable a better understanding of couples in research, therapy, and the real world.
△ Less
Submitted 18 September, 2021; v1 submitted 2 June, 2021;
originally announced June 2021.
-
Kwame: A Bilingual AI Teaching Assistant for Online SuaCode Courses
Authors:
George Boateng
Abstract:
Introductory hands-on courses such as our smartphone-based coding course, SuaCode require a lot of support for students to accomplish learning goals. Online environments make it even more difficult to get assistance especially more recently because of COVID-19. Given the multilingual context of SuaCode students - learners across 42 African countries that are mostly Anglophone or Francophone - in t…
▽ More
Introductory hands-on courses such as our smartphone-based coding course, SuaCode require a lot of support for students to accomplish learning goals. Online environments make it even more difficult to get assistance especially more recently because of COVID-19. Given the multilingual context of SuaCode students - learners across 42 African countries that are mostly Anglophone or Francophone - in this work, we developed a bilingual Artificial Intelligence (AI) Teaching Assistant (TA) - Kwame - that provides answers to students' coding questions from SuaCode courses in English and French. Kwame is a Sentence-BERT (SBERT)-based question-answering (QA) system that we trained and evaluated offline using question-answer pairs created from the course's quizzes, lesson notes and students' questions in past cohorts. Kwame finds the paragraph most semantically similar to the question via cosine similarity. We compared the system with TF-IDF and Universal Sentence Encoder. Our results showed that fine-tuning on the course data and returning the top 3 and 5 answers improved the accuracy results. Kwame will make it easy for students to get quick and accurate answers to questions in SuaCode courses.
△ Less
Submitted 13 June, 2021; v1 submitted 21 October, 2020;
originally announced October 2020.
