Artificial Educators

Welcome to the Artificial Educators project, where four studies are being conducted on how digital technologies can undertake educator roles:

1. Artificial Educators study, exploring to what degree artificial intelligence can perform the AITSL standards expected of a teacher;

2. Robotic Educators study, exploring how androids can be used to undertake educator roles in teaching; 

3. Avatar Educators study, exploring how digital representations can be used to undertake educator roles in cognitive development;  and

4. Virtual Educators study, exploring how generative AI can be used to undertake educator roles in assessment.

The Artificial Educators research project incorporates a range of studies conducted with cohorts of students. The remaining studies draw upon other sources of data to explore the potential of Artificial Educators and contribute to a wider Meta Study of Educational Technology.

Each study will begin with an exploration of the research literature followed by a study exploring the research question, followed by extension studies to explore the research question in more detail and verify the findings.

Collectively, each study is part of a project, and the projects develop a systems simulation to model the interactions between various aspects of educational technologies and computer education, culminating in a metastudy simulation of the field.

Studies in development and research will be conducted with student cohorts.

Dr Jason Zagami

Artificial Educators study 

AI capacity to meet AITSL standards expected of a teacher study has six cohorts each year, three of ~10 postgraduate students undertaking studies in educational technologies over a 12-week period, a cohort of n~15 of secondary ITE students over two semesters, and a cohort of n~250 primary ITE students over a 12-week period.

Robotic Educators study 

AI support for teaching study has two cohorts each year, a cohort n~15 of secondary ITE students over two semesters, including an industry placement, and a cohort of n~250 primary ITE students over a 12-week period.

Avatar Educators study 

AI support for cognition study has three cohorts each year, each of ~10 postgraduate students undertaking studies in educational technologies over a 12-week period.

Virtual Educator study

AI support for assessment study has six cohorts each year, three of ~10 postgraduate students undertaking studies in educational technologies over a 12-week period, a cohort of n~15 of secondary ITE students over two semesters, and a cohort of n~250 primary ITE students over a 12-week period.

Researchers and PhD students are welcome to request to join the research project team. You are asked to familiarise yourself with the Theoretical Perspective and Methodology used in the project, and you will be expected to contribute a profile of yourself to the Researcher Model (You may also wish to conduct a Self Study of your involvement in the project). You can then select which of the models you wish to contribute to and the associated studies that you will be involved in. 

Coordinating and writing meetings are organised by each project team, with intensive writing periods generally occurring in June/July and December/January periods depending on the particular studies. 

Request to join the project team

Project team members (Restricted Access)

Researchers agree to abide by the Australian Code for the Responsible Conduct of Research and make a significant intellectual or scholarly contribution to the research and its output and agree to be listed as an author. 

Researchers to be listed as authors in research outputs will be expected to contribute to the:

• analysis or interpretation of research data; and

• drafting significant parts of the research output and assisting in the critical revision of team member contributions. 

The order of authorship will reflect the relative contribution of the researchers who collaborated on the project and research output, determined by team members in an anonymous conjoint pair ranking process.

Researchers agree to treat fellow researchers and others involved in the research fairly and with respect, raising any concerns or conflicts of interest, and clearly informing team members of intentions in regular team meetings. 

Any disputes within the project team should be resolved by the team where possible,  but agree to follow the formal Griffith University institutional processes to resolve disputes if necessary, including mediation.

All Research Studies projects comply with the Australian National Statement on Ethical Conduct in Human Research. You should only provide your consent to participate in a study when fully informed of what the study involves.

Where data matching is required, e.g. from multiple surveys, only the minimum information that is needed to identify you is collected using a unique contact identifier (UCI) to minimise the risk that this information is associated with you.

In some studies, you may elect to be identified as a participant in the study on the project website and/or in publications, but this is never a requirement. Identifying information is otherwise deleted when no longer required, and all other research data is deidentified and made available in open data repositories for reuse. 

This research is being led by:

Dr Jason Zagami (Griffith University) 0755528454  j.zagami@griffith.edu.au

who can answer any questions you have about the studies.

Research Studies

Each study involves a different research process, described in the study's invitation and informed consent, and you should only complete a request to participate in a study (or participate anonymously) once you are fully informed of what is involved in the study.

GU Ref No: 2023/TBA

Each study is conducted in accordance with the Griffith University Research Integrity framework and all studies involving participants have been approved by the Griffith University Human Research Ethics Committee. 

If you have any concerns or complaints about the ethical conduct of the research project, you should contact the Manager, Research Ethics (07) 37354375 or research-ethics@griffith.edu.au

Self Study

Using the Self Study framework to document participation in the project and publish a study on reflective practice and researcher experiences for academic review in December, with feedback expected in March (review comments (3 months)) with publication in May (6 Months).

Systematic Literature Review Study

In February, a Systematic Literature Review Study will revise the Domain Model and evidence gaps. 

A Systematic Literature Review will be submitted to a journal as a revision of the study's Systematic Literature Review for academic review in March, with feedback expected in May (review comments (3 months)) with publication in August (6 Months).

Artificial Educators study 

From March - October for the Secondary ITE cohort (24 weeks) and from July - October for the Primary ITE cohort (12 weeks), the study will monitor students learning about AITSL standards with comparisons made with a Large Language Model (LLM) to determine if it can demonstrate the same standard. The Standard Turing Test method will be applied in which student C, the interrogator, is given the task of trying to determine which – A or B – is a computer response and which is a human student response. The interrogator is limited to submitting written questions to a facilitator, who passes the question to be answered by a Large Language Model (LLM) or another student.

An Expert Model and Influence Model will be developed from the study.

Robotic Educators study 

From March - October for the Secondary ITE cohort (24 weeks), the study will monitor students learning to construct lesson plans using Large Language Model (LLM) generated responses for Lower Secondary computing education from March - May, during industry placement (practicum) in June, and for Upper Secondary computer science in July - October.

From July - October for the Primary ITE cohort (12 weeks), the study will monitor students learning to construct lesson plans using Large Language Model (LLM) generated responses for primary school Technologies education.

An Expert Model and Influence Model will be developed from the study.

Avatar Educators study 

From March - June, for two Post-grad cohorts (12 weeks), the study will monitor student use of AI tools to complete a range of cognitive tasks.

From July - October, for a Post-grad cohort (12 weeks), the study will monitor student use of AI tools to complete a range of cognitive tasks.

An Expert Model and Influence Model will be developed from the study.

Virtual Educators study 

From March - June, for two Post-grad cohorts (12 weeks), the study will monitor student use of AI tools to complete a range of cognitive tasks.

From July - October, for a Post-grad cohort (12 weeks), the study will monitor student use of AI tools to complete a range of cognitive tasks.

An Expert Model and Influence Model will be developed from the study.

The Artificial Educator, Robotic Educator, Avatar Educator and Virtual Educator studies will be submitted to a journal for academic review in October, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Overview Study

In August, researchers will conduct an Overview Study to develop a connection circle and causal relationships from the Expert and Influence Models and develop a Social Ecological Model to categorise and associate causal relationships in an Interaction Model.

The Overview Study will be submitted to a journal for academic review in October, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Simulation Study

In September, a Simulation Study will refine a simulation from the Interaction Model, and explore a range of simulations of the project research questions using the Process Model. 

The Simulation Study will be submitted to a journal for academic review in November, with feedback expected in January (review comments (3 months)) with publication in April (6 Months).

Validation Study

In Early January, researchers in the Validation Study will explore the effectiveness of the Process Model simulation to replicate the results of the AI support for teaching, AI support for cognition, and AI support for assessment studies.

The Validation Study will be submitted for academic review in February, with feedback expected in April (review comments (3 months)) with publication in July (6 Months).

Metastudy

In February, researchers in the Meta Study will meet to combine the Process Models, informed by Validation Models, of a range of studies to refine a dynamic simulation model of the field into a Meta Study Model. 

The Metastudy research team will coordinate a range of simulation explorations of the field using the Meta Study Model and allocate team members to develop studies, with drafts of these studies completed and shared by Late May. 

In June, researchers will meet again to combine individual studies into an edited publication or field study, with sections allocated for completion and sharing by Late July and final edits by Late August. 

The Metastudy will be submitted for academic review in September, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Artificial Educators study 

March - October: Secondary ITE cohort (24 weeks)

July - October: Primary ITE cohort (12 weeks)

Robotic Educators study 

March - October: Secondary ITE cohort (24 weeks)

July - October: Primary ITE cohort (12 weeks)

Avatar Educators study 

March - June: 2 x Postgrad cohorts (12 weeks)

July - October: Postgrad cohort (12 weeks)

Virtual Educator study

March - October: Secondary ITE cohort (24 weeks)

March - June: 2 x Postgrad cohorts (12 weeks)

July - October: Primary ITE cohort (12 weeks)

July - October: Postgrad cohort (12 weeks)

The study is a post-positivist exploration of the field of education with a focus on the application of technology to teaching and learning processes.

Accepting that the theories, hypotheses, background knowledge, and researchers' values will influence what is observed, the study pursues objectivity by recognising the effects of such biases and uses a combination of quantitative and qualitative methods to mitigate bias and improve the validity of research outcomes.

Ontologically, reality exists, but our understanding of it is imperfect and socially constructed. 

Axiologically, while bias is undesirable, it is also inevitable, and mitigation is required, but this in itself will be influenced by the values and beliefs held by researchers.

(Optional) To stay informed of project calls to participate, recommendations, workshops, symposiums, presentations and publications, subscribe to the Artificial Educators project newsletter with your Unique Contact Identifier (UCI) to maintain your anonymity. No more than one email a month.

Overall Research Studies project posts will be made to the Research Projects Facebook and LinkedIn groups.

The project incorporates ten methods of research inquiry (Self Study, Systematic Literature Review, Delphi Study, Confidence Intervals, Social-Ecological Modelling, Simulation Study, Quasi-Experiemental, Most Significant Change, Validation Study, and Mixed Method) that contribute to a metastudy that draws from the ten research projects. 

Focused studies (Artificial, Robotoic, Avatar and Virtual Educators) use Quasi-Experiemental research to contribute actors, influences and relationships to Expert and Influence models, and instead of the refinement process of the Change study used in other projects, using confidence factors to support validation.

Participants are undergraduate and postgraduate students with studies incorporated into specific courses.

The studies use a quasi-experimental pre-post research methodology to explore the research questions, with a document analysis method used to identify actors and their properties in an Expert Model and provide weightings for the influence of these properties on other actors to inform an Influence Model.

The Trend studies contribute to a systems simulation methodology identifying actors and their properties in an Expert Model and providing weightings for the influence of these properties on other actors in an Influence Model.

The Overview study categorises the relationships between actors in the Expert and Influence model into a Social Ecological Model, which is then developed into systems simulations expressed as a Process Model. 

The effectiveness of the Process Model in replicating Trend study outcomes is used to validate the model in a Validation study. 

The models developed in each of the ten research studies projects are finally combined into a Metamodel detailing aspects of the field of educational technologies and computer education. 

Each research project is replicated annually to reflect changes in the field of educational technologies and computer education, and will involve the same methods but with an overall changed population of participants. The specific longitudinal analysis will be conducted in the Changes Study.

To better understand a complex research study, self-study or reflective practice research provides a mechanism for analysis of the researchers' theories, hypotheses, background knowledge, experiences and values as the research unfolded and assists in understanding the influence these had on what is observed.

Application of reflective analysis of research practice in Self Study.

A Researcher Model is generated from the Self Study narrative as a template for modelling other researchers as Actors within the Process and Metastudy models and enables the influence of researchers to be incorporated into these models.

A systematic examination of academic literature to establish a Domain Model and Evidence Gap Map. While focusing on white (peer-reviewed research) literature, grey literature (primarily reports) is included where it is deemed significant but is not treated systematically.

A systematic literature review with meta-analysis on reported research from 2000-2022: conducted using PRISMA 2020 framework using The Lens agglomeration database. 

Separate reviews are conducted on international literature and Australian-only literature and used to inform participants in the Trend Study of previous studies on trends and assist in the current study.

Systematic Literature Review on Artificial Educators

Systematic Literature Review on Robotic Educators

Systematic Literature Review on Avatar Educators

Systematic Literature Review on Virtual Educators

Overview of the research questions derived from Systematic Literature Review and generated summaries (using Generative Pre-trained Transformer) of identified topics and themes.

Details of evidence derived from peer-reviewed research and reports

Details Evidence Gap Map

Four Artificial Educators studies are being conducted on how digital technologies can undertake educator roles:

• Artificial Educators study, exploring to what degree artificial intelligence can perform the AITSL standards expected of a teacher;

• Robotic Educators study, exploring how androids can be used to undertake educator roles in teaching; 

• Avatar Educators study, exploring how digital representations can be used to undertake educator roles in cognitive development;  and

• Virtual Educators study, exploring how generative AI can be used to undertake educator roles in assessment.

March

Secondary ITE cohort (24 weeks), the study will monitor students learning about AITSL standards with comparisons made with a Large Language Model (LLM) to determine if it can demonstrate the same standard. The Standard Turing Test method will be applied in which student C, the interrogator, is given the task of trying to determine which – A or B – is a computer response and which is a human student response. The interrogator is limited to submitting written questions to a facilitator, who passes the question to be answered by a Large Language Model (LLM) or another student.

October 

Primary ITE cohort (12 weeks), the study will monitor students learning about AITSL standards with comparisons made with a Large Language Model (LLM) to determine if it can demonstrate the same standard. The Standard Turing Test method will be applied in which student C, the interrogator, is given the task of trying to determine which – A or B – is a computer response and which is a human student response. The interrogator is limited to submitting written questions to a facilitator, who passes the question to be answered by a Large Language Model (LLM) or another student.

An Expert Model and Influence Model will be developed from the study.

March

Secondary ITE cohort (24 weeks), the study will monitor students learning when instructional content is provided by an android instructor with a control group when the same content is provided by a human instructor.

October 

Primary ITE cohort (12 weeks), the study will monitor students learning when instructional content is provided by an android instructor with a control group when the same content is provided by a human instructor. After 6 weeks, the groups will switch, so that each group will be instructed by both android and human instructors.

An Expert Model and Influence Model will be developed from the study.

March 

For two Post-grad cohorts (12 weeks), the study will monitor students learning when instructional content is provided by an avatar instructor with a control group where the same content is provided by a live-streamed human instructor.

October 

For a third Post-grad cohort (12 weeks), the study will monitor student use of both human and avatar educators with participants given a choice of which to engage with.

An Expert Model and Influence Model will be developed from the study.

An Expert Model is generated from the Artificial Educators, Robotic Educators, Avatar Educators, and Virtual Educators studies to identify Actors (aka factors/elements/entities/objects) and Attributes (aka properties/variables) that influence the research questions. [Generated within AllOurIdeas]

From March - October for the Secondary ITE cohort (24 weeks) and three postgrad cohorts, and from July - October for the Primary ITE cohort (12 weeks), the study will monitor student use of Large Language Model (LLM) generated responses to learning activities; the perceived effectiveness of LLM feedback to student questions; and the accuracy of LLM responses to student questions as assessed by instructors. 

An Expert Model and Influence Model will be developed from the study.

The amount of influence each Actor has on various Attributes is determined from a correlative Influence Model achieved from analysis of Artificial Educators, Robotic Educators, Avatar Educators, and Virtual Educators studies. 

Systems research is conducted using the Expert and Influence Models to construct an Interaction Model of causal relationships in the research questions and categorise these actors and their relationships within a Social-Ecological framework.

Development of a Connection Circle of relationships between the actors identified in the Influence Model and subsequent identification of Causal Relationships (Loops) between actors.

From causal relationships, groupings of actors are derived from those actors with the strongest relationships. Generated as an onion diagram using the Social-Ecological Modelling (SEM) framework, with bands of Actors classified, grouped and clustered within bands.

A dynamic simulation model of changes in the research questions is developed from previous models to enable a range of scenarios to be explored and better understand the interaction between factors.

A systems model is developed from the Expert, Influence, Interaction and Change models, using the Insight Maker simulation software to create a dynamic simulation of the actors and their influence.

Using the causal relationships of actor attributes generated from connection circles and causal loop diagrams, informed by the larger relationships identified by the Social Ecological Model, a simulation model showing dynamic interactions between actor properties is generated. [Link to Agent-Based Simulation Model in Insight Maker]

Comparing the Robotic Educators, Avatar Educators, and Virtual Educators study outputs with the outputs of the Process Model simulation to validate the simulation effectiveness in modelling changes.

January

Researchers will explore the relationships identified in the Process Model [Link to Process Model simulation in Insight Maker] and the influence each actor has on the attributes of others and simulate the changes identified in the Robotic Educators, Avatar Educators, and Virtual Educators studies using the Process Model, contrasting the outcomes to validate the simulation.

The Process Model simulation is validated using a Most Significant Change (MSC) research framework on changes identified in Robotic Educators, Avatar Educators, and Virtual Educators studies with simulation comparisons used to compare identified changes with those generated by the simulation model. Through the hierarchical Most Significant Change process, change cases are grouped and categorised to determine those which are the most significant to research questions.

Models produced from various studies are integrated into a larger Meta Model researching the interactions between various aspects of Education Technologies.

The Simulation Model produced by the Metastudy enables a contextualisation of the Educational Technologies Trends study and a validation process on the predictions from the study over time with the predictions generated from the Metastudy simulation model.

A Metastudy model is developed to simulate field-level interactions of a wide range of actors and attributes.  [Link to Agent-Based Simulation Metastudy Model in Insight Maker]

TBA late 2023

TBA late 2023

Annual online symposia in early December to present research findings.

Online Writing Workshops in June/July and December/January to support research team publications.

TBA 2023

TBA 2024

TBA 2024