Programming Education

Welcome to the Programming Education Trends project, researching trends in how programming is taught in schools.

The five research questions being explored in this study are:

1. How is Visual Programming being used in schools?

2. How is Manipulative Programming being used in schools?

3. How is Robotics and Drone Programming being used in schools?

4. How is Social Programming being used in schools? and 

5. How is AI Supported Programming being used in schools?

The Programming Education research project incorporates a range of studies, and you are welcome to contribute to the Trend study, and if you do so, you can also participate in the subsequent Confirmation and Changes Studies. The Trend and Changes studies will each take less than an hour of your time over a few months, while the Confirmation study takes only a few minutes at a time, and occurs at the end of each term for the year. The remaining studies draw upon other sources of data to explore Educational Technology Trends and contribute to a wider Meta Study of Educational Technology.

Each study will begin with an exploration of the research literature followed by a study exploring the research question and drawing upon your expertise in the field, followed by extension studies to explore the research question in more detail and verify the findings.

Collectively, each study is part of a project, and the projects develop a systems simulation to model the interactions between various aspects of educational technologies and computer education, culminating in a metastudy simulation of the field.

Sign up to join the studies. 

You can use your email which we will keep private, or create a unique contact identifier (UCI) to make things more private.

You 'should' read the invitation to participate and informed consent. These are slightly different for each study, but essentially are the same and say we will keep your details private.

If you want your participation acknowledged, fill in a form with your personal details so we can acknowledge you on the website and in publications.

You can be involved in as many or as few of the projects and studies as you wish. They are designed to take as little of your time as possible.

For this project, the studies aim to answer:

1. How is Visual Programming being used in schools?

2. How is Manipulative Programming being used in schools?

3. How is Robotics and Drone Programming being used in schools?

4. How is Social Programming being used in schools? and 

5. How is AI Supported Programming being used in schools?

There are three parts to each study you can be involved in. The first is what is called a Trend Study, where each participant tells us what is happening, and we build a consensus so it is not just individual opinions. 

To do this we give you a summary of the research, you come up with ideas and contribute these, and then everyone votes on these ideas by being shown pairs of ideas and choosing between them. Each vote only takes a second, and in 5 minutes you can cast several hundred votes. Collectively, using a whole lot of complex maths, we build a consensus and a ranking of the ideas. 

We will then summarise this, show you the ranking, and you have an opportunity to rethink if new ideas should be added, some ideas combined or removed, and then we vote again. This builds the final ranking of ideas.

All up it will not take more than an hour over four months - we spread things out to give everyone time to contribute ideas and cast votes.

You can also sign up to let us know your thoughts on the top 5 of these ideas over the year in a Confirmation Study, confirming if your experience reflects the consensus. At the end of each school term, you will have a very short survey to complete, each taking less than 5 minutes.

Finally, we are using the study to build a computer simulation model so that more complex analyses and predictions can be made, and if you would like to be involved in a Changes Study, at the end of the year you will be asked to describe your experiences with the top 5 ideas, be provided with a simulation you can explore, and then participate in a videoconference focus group discussing how the simulation reflected your experiences.

If you sign up, we will let you know by email when each is occurring, and you can follow the progress of the study on this website if interested.

A lot more happens, but that is the gist of it. I hope you decide to be involved,

Dr Jason Zagami

Participants may be interested in postgraduate studies leading to a Graduate Certificate or coursework Master's Degree, majoring in Educational Technologies. The research questions in this study can be explored in greater detail in the program, which includes courses exploring Researching, Creating and Transforming Educational Technologies. Completion of the research pathway can lead to research higher-degree studies.

Otherwise, if you have completed an honours year as part of your undergraduate studies, or undertaken other advanced postgraduate studies that included a research component, you may be eligible to undertake a Research Master's or Doctoral degree that incorporates, in part, these research projects, studies, and/or methods. PhD students are required to make a unique contribution to human knowledge, and you will need to extend or incorporate new aspects to projects, but there is scope to explore how this can occur during your studies.

If interested in studying through Griffith University, you can make an application for study and are welcome to contact Dr Jason Zagami to discuss your proposal and potential supervision.

The following studies are available for participation:

Trends Studies

To participate in the Visual Programming Trends Study, you are asked to:

Read the Invitation to Visual Programming trends study 

Read the Informed Consent for the Visual Programming trends study

(Optional) Complete a Request to Participate in the Visual Programming trends study

(Optional) Complete an Acknowledgement of participation request

To participate in the Manipulative Programming Trends Study, you are asked to:

Read the Invitation to Manipulative Programming trends study 

Read the Informed Consent for the Manipulative Programming trends study 

(Optional) Complete a Request to Participate in the Manipulative Programming trends study

(Optional) Complete an Acknowledgement of participation request

To participate in the Robotics and Drone Programming Trends Study, you are asked to:

Read the Invitation to Robotics and Drone Programming trends study 

Read the Informed Consent for the Robotics and Drone Programming trends study 

(Optional) Complete a Request to Participate in the Robotics and Drone Programming trends study

(Optional) Complete an Acknowledgement of participation request

To participate in the Social Programming Trends Study, you are asked to:

Read the Invitation to Social Programming trends study 

Read the Informed Consent for the Social Programming trends study

(Optional) Complete a Request to Participate in the Social Programming trends study

(Optional) Complete an Acknowledgement of participation request

To participate in the AI Supported Programming Trends Study, you are asked to:

Read the Invitation to AI Supported  Programming trends study 

Read the Informed Consent for the AI-Supported Programming trends 

(Optional) Complete a Request to Participate in the AI Supported  Programming trends study

(Optional) Complete an Acknowledgement of participation request

Confirmation Study

To participate in a Confirmation Study, you must be participating in the Trend Study, and are asked to:

Read the Invitation to Participate in the Confirmation Study 

Read the Informed Consent for Confirmation Study

Complete Request to Participate in the Confirmation Study

(Optional) Complete an Acknowledgement of participation request

Changes Study

To participate in a Changes Study, you must have participated in the Trend Study, and are asked to:

Read the Invitation to Participate in the Changes Study 

Read the Informed Consent for Changes Study

Complete a Request to Participate in the Changes Study 

(Optional) Complete an Acknowledgement of participation request

Researchers and PhD students are welcome to request to join the research project team. You are asked to familiarise yourself with the Theoretical Perspective and Methodology used in the project, and you will be expected to contribute a profile of yourself to the Researcher Model (You may also wish to conduct a Self Study of your involvement in the project). You can then select which of the models you wish to contribute to and the associated studies that you will be involved in. 

Coordinating and writing meetings are organised by each project team, with intensive writing periods generally occurring in June/July and December/January periods depending on the particular studies. 

Request to join the project team

Project team members (Restricted Access)

Researchers agree to abide by the Australian Code for the Responsible Conduct of Research and make a significant intellectual or scholarly contribution to the research and its output and agree to be listed as an author. 

Researchers to be listed as authors in research outputs will be expected to contribute to the:

• analysis or interpretation of research data; and

• drafting significant parts of the research output and assisting in the critical revision of team member contributions. 

The order of authorship will reflect the relative contribution of the researchers who collaborated on the project and research output, determined by team members in an anonymous conjoint pair ranking process.

Researchers agree to treat fellow researchers and others involved in the research fairly and with respect, raising any concerns or conflicts of interest, and clearly informing team members of intentions in regular team meetings. 

Any disputes within the project team should be resolved by the team where possible,  but agree to follow the formal Griffith University institutional processes to resolve disputes if necessary, including mediation.

All Research Studies projects comply with the Australian National Statement on Ethical Conduct in Human Research. You should only provide your consent to participate in a study when fully informed of what the study involves.

Where data matching is required, e.g. from multiple surveys, only the minimum information that is needed to identify you is collected using a unique contact identifier (UCI) to minimise the risk that this information is associated with you.

In some studies, you may elect to be identified as a participant in the study on the project website and/or in publications, but this is never a requirement. Identifying information is otherwise deleted when no longer required, and all other research data is deidentified and made available in open data repositories for reuse. 

This research is being led by:

Dr Jason Zagami (Griffith University) 0755528454  j.zagami@griffith.edu.au

who can answer any questions you have about the studies.

Research Studies

Each study involves a different research process, described in the study's invitation and informed consent, and you should only complete a request to participate in a study (or participate anonymously) once you are fully informed of what is involved in the study.

Invitation to Visual Programming trends study

Informed Consent for the Visual Programming trends study GU Ref No: 2023/44

Informed Consent for the Manipulable Programming trends study GU Ref No: 2023/44

Invitation to Robotics and Drone Programming trends study 

Informed Consent for the Robotics and Drone Programming trends study GU Ref No: 2023/44

Invitation to Social Programming trends study

Informed Consent for the Social Programming trends study GU Ref No: 2023/44

Informed Consent for the AI-Supported Programming trends study GU Ref No: 2023/44

Invitation to Participate in the Confirmation Study 

Informed Consent for Confirmation Study GU Ref No: 2023/45

Invitation to Participate in the Changes Study 

Informed Consent for Changes Study GU Ref No: 2023/43

You can request to participate in the above "Request to Participate"

Each study is conducted in accordance with the Griffith University Research Integrity framework and all studies involving participants have been approved by the Griffith University Human Research Ethics Committee. 

If you have any concerns or complaints about the ethical conduct of the research project, you should contact the Manager, Research Ethics (07) 37354375 or research-ethics@griffith.edu.au

Self Study

By January, researchers in any of the project studies are expected to have completed a profile for inclusion in the Researcher Model that will contribute to the Metastudy research.

Researchers are welcome to use the Self Study framework to document their participation in the project and independently publish a study on their reflective practice, or researcher experiences may be aggregated by researchers to publish collective reflective studies exploring a team's participation in individual studies. Researchers should share with the project team their intentions to publish and Self Study outputs.

Review Study

In February, researchers in the Trend Study will revise the Systematic Literature Review of the Domain Model and highlight research publication trends that will assist participants in the Trends Study. 

A Systematic Literature Review will be submitted to a journal as a revision of the study's Systematic Literature Review for academic review in March, with feedback expected in May (review comments (3 months)) with publication in August (6 Months).

Trend Study

In March, researchers in the Trend Study will monitor participant suggestions in the Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1, review contributions, remove duplicates, combine similar alternatives, and remove inappropriate alternatives. Researchers will commence refining the Expert Model.

In April, participants will complete the Conjoint Round 1, choosing between alternatives and generating a ranked list. Researchers will review the lists in Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1 and revise a clarification guide for the alternatives that may not be clear to assist participants in completing Alternatives Survey 2. Researchers will commence refining the Influence Model.

In May, researchers will monitor participant review of the initial ranking using the clarification guide and as participants suggest new alternatives, combine alternatives, or revise wordings in Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2. Researchers will finalise the Expert Model.

In June, participants will complete Conjoint Round 2, Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2 which will generate the final rankings. Researchers will finalise the Influence Model and collaboratively develop an academic paper on the trends identified to answer the project research questions. Team members will be allocated sections to be completed by Late June.

In July, researchers will review each other's section drafts by Mid July, with final edits completed by Late July.

The Trend Study will be submitted to a journal for academic review in August, with feedback expected in October (review comments (3 months)) with publication in January (6 Months).

Confirmation Study

In June, researchers in the Confirmation Study will monitor responses from Term 2 in the confirmation survey, with sections allocated (Technologies, Challenges, Impacts).

Confirmation Study Survey (Responses) (Restricted Access)

In October, researchers in the Confirmation Study will monitor responses from Term 3 in the confirmation survey. Researchers will commence drafts of their allocated section.

In Early December, researchers in the Confirmation Study will finalise responses from the confirmation surveys. Researchers will share their drafts.

By Mid December, researchers will combine their sections to produce an Influence Model, and be allocated section drafts to complete for an academic paper on the Influence Model with drafts completed by Late December.

By Mid January, researchers will have reviewed each other's section drafts, with final edits completed by Late January.

The Confirmation Study will be submitted to a journal for academic review in January, with feedback expected in April (review comments (3 months)) with publication in June (6 Months).

Overview Study

In August, researchers in the Overview Study will meet and develop a connection circle and causal relationships from the Expert and Influence Models, and team members will be allocated sections of the SEM model to categorise and associate causal relationships. 

By Mid August, researchers will combine their sections of the SEM to produce an Interaction Model, and be allocated section drafts to complete for an academic paper on the Interaction Model with drafts completed by Late August.

By Mid September, researchers will have reviewed each other's section drafts, with final edits completed by Late September.

The Overview Study will be submitted to a journal for academic review in October, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Simulation Study

In September, researchers in the Simulation Study will refine a simulation from the Interaction Model, and explore a range of simulations of the project research questions using the Process Model. 

Researchers will be allocated sections of the simulation to refine and conduct localised simulations to be completed and added to the full simulation model by Mid September, with full model simulations completed by Late September. The team will collaboratively complete section drafts of an academic paper on how the simulation explores the project research questions by Early October, with section allocations as agreed to in team meetings.

By Mid October, researchers will have reviewed each other's section drafts, with final edits completed by Late October.

The Simulation Study will be submitted to a journal for academic review in September, with feedback expected in January (review comments (3 months)) with publication in April (6 Months).

Changes Study

In October, researchers in the Changes Study will be allocated participants, supporting them in developing their change narratives.

Case Study Survey (Responses) (Restricted Access)

In November, researchers conduct online video conference focus groups to clarify their narrative survey responses describing the changes they have experienced and how well the simulation model reflected these experiences. Teams or Zoom will be used for interviews with transcription to Nvivo for analysis.

Trend Study Simulation (Educational Technology Trends)

Study Participants in the Changes Study (Restricted Access)

In Early December, researchers will collaboratively combine the change cases using the Most Significant Change process and be allocated sections to draft in the Change Model. 

By Mid December, researchers will have reviewed each other's section drafts, with final edits to a Change Model completed by Late December.

The Changes Study will be submitted for academic review in January, with feedback expected in March (review comments (3 months)) with publication in June (6 Months).

Validation Study

In Early January, researchers in the Validation Study will explore the effectiveness of the Process Model simulation to replicate the results of the Change Model, with changes allocated, simulations completed and compared, and drafts shared by Mid January with final edits completed to a Validation Model by Late January.

The Validation Study will be submitted for academic review in February, with feedback expected in April (review comments (3 months)) with publication in July (6 Months).

Metastudy

In February, researchers in the Meta Study will meet to combine the Process Models, informed by Validation Models, of a range of studies to refine a dynamic simulation model of the field into a Meta Study Model. 

The Metastudy research team will coordinate a range of simulation explorations of the field using the Meta Study Model and allocate team members to develop studies, with drafts of these studies completed and shared by Late May. 

In June, researchers will meet again to combine individual studies into an edited publication or field study, with sections allocated for completion and sharing by Late July and final edits by Late August.

The Metastudy will be submitted for academic review in September, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Trend Studies

Each study is expected to take no more than 25 minutes of your time.

Participants in a Trend Study will be invited in February. 

In March, you will be asked to briefly examine the research on the study questions and suggest alternatives to the Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1. (10 minutes)

In April, you will be asked to choose between alternatives in the Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1. (5 minutes)

In May, you will be provided with the initial ranking and clarification guide; and asked to suggest new alternatives, combine alternatives, or revise their wording in  Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2. (5 minutes)

In June, you will be asked to choose between alternatives in the Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2. which will generate the final rankings. (5 minutes)

In July, you will be provided with the final rankings from each survey and asked if you wish to be acknowledged as a contributor to the study.

The Trends Study will be submitted for academic review in August, with publication expected in January.

Confirmation Study

You must be participating in the Trend Study.

The study is expected to take no more than 15 minutes of your time.

Participants in the Confirmation Study will be invited in February.

In July, you will be asked to complete the Term 2 Confirmation survey on which of the top five Visual, Manipulative, Robotics & Drone, Social, and AI Supported programming you have used the most in Term 2. Multiple choice questions based on Trend Study.

In October, you will be asked to complete the Term 3 Confirmation survey on which of the top five Visual, Manipulative, Robotics & Drone, Social, and AI Supported programming you have used the most in Term 3. Multiple choice questions based on Trend Study.

(5 minutes)

In December, you will be asked to complete the Term 4 Confirmation survey on which of the top five Visual, Manipulative, Robotics & Drone, Social, and AI Supported programming you have used the most in Term 4. Multiple choice questions based on Trend Study.

(5 minutes)

The Confirmation Study will be submitted for academic review in January, with publication expected in June.

Changes Study

You must have participated in the Trend Study.

The study is expected to take no more than an hour of your time.

Participants in the Changes Study will be invited in September.

In October, you will be asked to complete a Case Study Survey of questions to generate a change case. (20 minutes);

You will then be provided with access to an Online Simulation to compare with your experiences. (20 minutes);

In November, you will be asked to participate in an online focus group to clarify elements of your change case. (20minutes) and asked if you wish to be acknowledged as a contributor to the study and have your name and affiliation listed with your case study.

The Changes Study will be submitted for academic review in January, with publication expected in June.

The study is a post-positivist exploration of the field of education with a focus on the application of technology to teaching and learning processes.

Accepting that the theories, hypotheses, background knowledge, and researchers' values will influence what is observed, the study pursues objectivity by recognising the effects of such biases and uses a combination of quantitative and qualitative methods to mitigate bias and improve the validity of research outcomes.

Ontologically, reality exists, but our understanding of it is imperfect and socially constructed. 

Axiologically, while bias is undesirable, it is also inevitable, and mitigation is required, but this in itself will be influenced by the values and beliefs held by researchers.

(Optional) To stay informed of project calls to participate, recommendations, workshops, symposiums, presentations and publications, subscribe to the Programming Education project newsletter with your Unique Contact Identifier (UCI) to maintain your anonymity. No more than one email a month.

Overall Research Studies project posts will be made to the Research Projects Facebook and LinkedIn groups.

The project incorporates ten methods of research inquiry (Self Study, Systematic Literature Review, Delphi Study, Confidence Intervals, Social-Ecological Modelling, Simulation Study, Case Study, Most Significant Change, Validation Study, and Mixed Method) that contribute to a metastudy that draws from the ten research projects. 

Participants can contribute to the Trends, Confirmation and Changes studies. The Trends study uses a modified Delphi consensus research method to provide agreed insight on the issues impacting education and those likely to impact education. Online surveys are used to identify alternatives that are ranked using a conjoint pair method, each over two rounds of the typical Delphi study method.

The Confirmation study uses multiple surveys to repeatedly rank the top five responses of educators in their practice over the year with confidence intervals used to compare means and identify significant differences. 

The Trends study contributes to a systems simulation methodology identifying actors and their properties in an Expert Model and providing initial weightings for the influence of these properties on other actors. 

The Confirmation study samples key actors, and confirms the accuracy of the weightings applied to influence attributes for actors in an Influence Model.

The Overview study categorises the relationships between actors in the Expert and Influence model into a Social Ecological Model, which is then developed into systems simulations expressed as a Process Model. 

The Changes study uses Case Study and the Most Significant Change methods to explore in more detail the changes experienced by participants, developed through a survey with results incorporated into a Simulation model, and the model is used to conduct trials to simulate the changes experienced by participants. Participant experiences are then compared with the model and discussed in focus groups to improve the model, which is validated in a Model Validation study. 

The models developed in each of the ten research studies projects are finally combined into a Metamodel detailing aspects of the field of educational technologies and computer education. 

Each research project is replicated annually to reflect changes in the field of educational technologies and computer education, and will involve the same methods but with an overall changed population of participants. The specific longitudinal analysis will be conducted in the Changes Study.

To better understand a complex research study, self-study or reflective practice research provides a mechanism for analysis of the researchers' theories, hypotheses, background knowledge, experiences and values as the research unfolded and assists in understanding the influence these had on what is observed.

Application of reflective analysis of research practice in Self Study.

A Researcher Model is generated from the Self Study narrative as a template for modelling other researchers as Actors within the Process and Metastudy models and enables the influence of researchers to be incorporated into these models.

A systematic examination of academic literature to establish a Domain Model and Evidence Gap Map. While focusing on white (peer-reviewed research) literature, grey literature (primarily reports) is included where it is deemed significant but is not treated systematically.

A systematic literature review with meta-analysis on reported research from 2000-2022: conducted using PRISMA 2020 framework using The Lens agglomeration database. 

Separate reviews are conducted on international literature and Australian-only literature and used to inform participants in the Trend Study of previous studies on trends and assist in the current study.

Systematic Literature Review on Visual Programming

Systematic Literature Review on Manipulable Programming

Systematic Literature Review on Robotics and Drone Programming 

Systematic Literature Review on Community Supported Programming 

Systematic Literature Review on AI Supported Programming

Overview of the research questions derived from Systematic Literature Review and generated summaries (using Generative Pre-trained Transformer) of identified topics and themes.

Details of evidence derived from peer-reviewed research and reports

Details Evidence Gap Map

Drawing upon the expertise of teachers, researchers, industry and government, a consensus is sought on the factors that influence programming education in Australian schools:

1. How is Visual Programming being used in schools?

2. How is Manipulable Programming being used in schools?

3. How is Robotics and Drone Programming being used in schools?

4. How is Community Supported Programming being used in schools? and 

5. How is AI Supported Programming being used in schools?

March

In each of the AllOurIdeas surveys: Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1 - click on the view results tab to look at the list of suggested alternatives contributed by other participants, previous studies and the systematic qualitative literature review, and then back on the Cast Votes tab, in the Add your own idea here... field, suggest new alternatives; reword existing; or combining existing alternatives, in the three AllOurIdeas surveys in the study: Visual, Manipulable, Robotics and Drone, Social, and AI Supported Programming. Participants should not start choosing between the pairs of alternatives at this stage. 

April

Participants make choices from the presented paired alternatives, providing responses to 30-40 pairs of alternatives for each of the categories of the study: Visual Programming Survey 1, Manipulative Programming Survey 1, Robotics & Drone Programming Survey 1, Social Programming Survey 1, and AI Supported Programming Survey 1. Combined with results from other participants, this will provide a consensus ranking of the alternatives. 

May

Participants use the initial ranking results of the Conjoint Round 1, for each of the revised AllOurIdeas surveys: Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2 - clicking on the view results tab to look at the list of suggested alternatives from the first round, and then back on the Cast Votes tab, in the Add your own idea here... field, participants suggest new alternatives; rewording of existing; or combining existing alternatives in the three AllOurIdeas surveys in the study: Visual, Manipulable, Robotics and Drone, Social and AI Supported Programming. Do not start choosing between the pairs of alternatives at this stage. 

June

Participants make choices from the presented paired alternatives, providing responses to 30-40 pairs of revised alternatives for each of the categories of the study: Visual Programming Survey 2, Manipulative Programming Survey 2, Robotics & Drone Programming Survey 2, Social Programming Survey 2, and AI Supported Programming Survey 2. Combined with results from other participants, this will provide a refined consensus ranking of the alternatives.

An Expert Model is generated from experts in Educational Technologies to identify Actors (aka factors/elements/entities/objects) and Attributes (aka properties/variables) that influence the research questions using online surveys and literature review. [Generated within AllOurIdeas]

A confirmation is conducted by surveys on the accuracy of the Trend study to determine the technologies, challenges and impacts for educators over the 12 months following the commencement of the Trend study. 

Three Confirmation surveys conducted at the end of each school Term, June, September and December, investigate the accuracy of Trend Study rankings. Each survey ranks the top five responses to the research question over the previous three months. Confidence intervals are used to compare means and identify significant differences. 

The amount of influence each Actor has on researched Attributes as determined from consensus is confirmed at three monthly intervals to establish the accuracy of the Trend Study to measure this influence in the short term. The influencing attributes of each actor form an Influence Model of measured ratings of attribute influence for each actor. 

Systems research is conducted using the Expert and Influence Models to construct an Interaction Model of causal relationships in the research questions and categorise these actors and their relationships within a Social-Ecological framework.

Development of a Connection Circle of relationships between the actors identified in the Influence Model and subsequent identification of Causal Relationships (Loops) between actors.

From causal relationships, groupings of actors are derived from those actors with the strongest relationships. Generated as an onion diagram using the Social-Ecological Modelling (SEM) framework, with bands of Actors classified, grouped and clustered within bands.

A dynamic simulation model of changes in the research questions is developed from previous models to enable a range of scenarios to be explored and better understand the interaction between factors.

A systems model is developed from the Expert, Influence, Interaction and Change models, using the Insight Maker simulation software to create a dynamic simulation of the actors and their influence.

Using the causal relationships of actor attributes generated from connection circles and causal loop diagrams, informed by the larger relationships identified by the Social Ecological Model, a simulation model showing dynamic interactions between actor properties is generated. [Link to Agent-Based Simulation Model in Insight Maker]

Exploring the changes occurring over time in Programming Education, the Process Model simulation is compared to expert experiences of changes in Programming Education. Each round of the study, conducted annually, contributes to a longitudinal data set of educational changes and improvements to the Process Model.

November

Expert participants will be invited to complete a survey [Link to Survey] to contribute narratives on the ranked alternatives identified in the Trend Study and develop cases describing the changes they are experiencing in education. [Change Case narratives]

Early November

Participants will examine the relationships identified in the Process Model [Link to Process Model simulation in Insight Maker] and the influence each actor has on the attributes of others and contrast their change case with those generated by the Process Model. [Simulations of Change Case narratives]

Late November,

Participants will participate in an online interview to clarify elements of their change case and how it was represented by the simulation. [Schedule of Interviews] [Recording of Interviews] [Transcripts of Interviews] [Transcript Analysis]

Descriptive Multiple Case Study of Change Cases contributed by participants.

Comparing the Change Model narratives with the outputs of the Process Model simulation to validate the simulation effectiveness in modelling changes.

January

Researchers will explore the relationships identified in the Process Model [Link to Process Model simulation in Insight Maker] and the influence each actor has on the attributes of others and simulate the change cases identified in the Change Study [Change Case narratives] using the Process Model, contrasting the outcomes to validate the simulation.

The Process Model simulation is validated using a Most Significant Change (MSC) research framework on collected narratives (Change Model) with simulation comparisons and interviews used to compare expert experiences of changes with those generated by the simulation model. Through the hierarchical Most Significant Change process, change cases are grouped and categorised to determine those which are the most significant to research questions.

Models produced from various studies are integrated into a larger Meta Model researching the interactions between various aspects of Education Technologies.

The Simulation Model produced by the Metastudy enables a contextualisation of the studies and a validation process on the predictions from the study over time with the predictions generated from the Metastudy simulation model.

A Metastudy model is developed to simulate field-level interactions of a wide range of actors and attributes.  [Link to Agent-Based Simulation Metastudy Model in Insight Maker]

TBA late 2023

TBA late 2023

Annual online symposia in early December to present research findings.

Online Writing Workshops in June/July and December/January to support research team publications.

TBA 2023

TBA 2024

TBA 2024