Research on Research

Welcome to the Research on Research project, researching trends in approaches to research in the field of Educational Technology.

The four research questions being explored in this study are:

1. What Educational Technologies are being researched in Australia?

2. Who are the Australian researchers of Educational Technologies? and

3. How are various Research Methods being applied in the study of Educational Technologies? and

4. What replication is being conducted of Educational Technology research findings?
   
   

Each study will begin with an exploration of the research literature followed by a study exploring the research question and drawing upon your expertise in the field to explore the research question.

Collectively, each study is part of a project, and the projects develop a systems simulation to model the interactions between various aspects of educational technologies and computer education, culminating in a metastudy simulation of the field.

Still in development, but the studies are mapping Australian research, researchers, and methodologies for the field of Educational Technologies and K12 Computer Education.

Dr Jason Zagami

Participants may be interested in postgraduate studies leading to a Graduate Certificate or coursework Master's Degree, majoring in Educational Technologies. The research questions in this study can be explored in greater detail in the program, which includes courses exploring Researching, Creating and Transforming Educational Technologies. Completion of the research pathway can lead to research higher-degree studies.

Otherwise, if you have completed an honours year as part of your undergraduate studies, or undertaken other advanced postgraduate studies that included a research component, you may be eligible to undertake a Research Master's or Doctoral degree that incorporates, in part, these research projects, studies, and/or methods. PhD students are required to make a unique contribution to human knowledge, and you will need to extend or incorporate new aspects to projects, but there is scope to explore how this can occur during your studies.

If interested in studying through Griffith University, you can make an application for study and are welcome to contact Dr Jason Zagami to discuss your proposal and potential supervision.

Researchers contributing to Educational Technologies research projects are expected to contribute their research profile to building a comprehensive understanding of what Educational Technology research is being undertaken in Australia and who is undertaking it. This can be as simple as providing an Orcid ID and generally does not involve any data not available publicly. 

Researchers contributing to Educational Technologies research projects are also expected to contribute to improvements to projects through an iterative Design Research methodology on the methods used in each project, conducted by analysis of research team discussions, progress and annual evaluation surveys.

As this process will be conducted as formal research, consent is required as a precondition of participation in the research projects. In exceptional circumstances, this could be waived and allowances made, but data collected on research team meetings, progress and evaluation surveys, etc., while these will be anonymised in any publications, would include all member contributions. 

Consent Form

Self Study

By January, researchers in any of the project studies are expected to have completed a profile for inclusion in the Researcher Model that will contribute to the Metastudy research.

Researchers are welcome to use the Self Study framework to document their participation in the project and independently publish a study on their reflective practice, or researcher experiences may be aggregated by researchers to publish collective reflective studies exploring a team's participation in individual studies. Researchers should share with the project team their intentions to publish and Self Study outputs.

Review Study

In February, researchers in the Trend Study will revise the Systematic Literature Review of the Domain Model and highlight research publication trends that will assist participants in the Trends Study. 

A Systematic Literature Review will be submitted to a journal as a revision of the study's Systematic Literature Review for academic review in March, with feedback expected in May (review comments (3 months)) with publication in August (6 Months).

Trend Study

In March, researchers in the Trend Study will monitor participant suggestions in the Screen Time Survey 1, Mobile Devices Survey 1, AI Tools Survey 1 and AI Filters survey 1, review contributions, remove duplicates, combine similar alternatives, and remove inappropriate alternatives. Researchers will commence refining the Expert Model.

In April, participants will complete the Conjoint Round 1, choosing between alternatives and generating a ranked list. Researchers will review the lists in Screen Time Survey 1, Mobile Devices Survey 1, AI Tools Survey 1 and AI Filters survey 1 and revise a clarification guide for the alternatives that may not be clear to assist participants in completing Alternatives Survey 2. Researchers will commence refining the Influence Model.

In May, researchers will monitor participant review of the initial ranking using the clarification guide and as participants suggest new alternatives, combine alternatives, or revise wordings in Screen Time Survey 2, Mobile Devices Survey 2, AI Tools Survey 2 or AI Filters survey 2. Researchers will finalise the Expert Model.

In June, participants will complete Conjoint Round 2, Screen Time Survey 2, Mobile Devices Survey 2, AI Tools Survey 2 or AI Filters survey 2 which will generate the final rankings. Researchers will finalise the Influence Model and collaboratively develop an academic paper on the trends identified to answer the project research questions. Team members will be allocated sections to be completed by Late June.

In July, researchers will review each other's section drafts by Mid July, with final edits completed by Late July.

The Trend Study will be submitted to a journal for academic review in August, with feedback expected in October (review comments (3 months)) with publication in January (6 Months).

Confirmation Study

In June, researchers in the Confirmation Study will monitor responses from Term 2 in the confirmation survey, with sections allocated (Technologies, Challenges, Impacts).

Confirmation Study Survey (Responses) (Restricted Access)

In October, researchers in the Confirmation Study will monitor responses from Term 3 in the confirmation survey. Researchers will commence drafts of their allocated section.

In Early December, researchers in the Confirmation Study will finalise responses from the confirmation surveys. Researchers will share their drafts.

By Mid December, researchers will combine their sections to produce an Influence Model, and be allocated section drafts to complete for an academic paper on the Influence Model with drafts completed by Late December.

By Mid January, researchers will have reviewed each other's section drafts, with final edits completed by Late January.

The Confirmation Study will be submitted to a journal for academic review in January, with feedback expected in April (review comments (3 months)) with publication in June (6 Months).

Overview Study

In August, researchers in the Overview Study will meet and develop a connection circle and causal relationships from the Expert and Influence Models, and team members will be allocated sections of the SEM model to categorise and associate causal relationships. 

By Mid August, researchers will combine their sections of the SEM to produce an Interaction Model, and be allocated section drafts to complete for an academic paper on the Interaction Model with drafts completed by Late August.

By Mid September, researchers will have reviewed each other's section drafts, with final edits completed by Late September.

The Overview Study will be submitted to a journal for academic review in October, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

Simulation Study

In September, researchers in the Simulation Study will refine a simulation from the Interaction Model, and explore a range of simulations of the project research questions using the Process Model. 

Researchers will be allocated sections of the simulation to refine and conduct localised simulations to be completed and added to the full simulation model by Mid September, with full model simulations completed by Late September. The team will collaboratively complete section drafts of an academic paper on how the simulation explores the project research questions by Early October, with section allocations as agreed to in team meetings.

By Mid October, researchers will have reviewed each other's section drafts, with final edits completed by Late October.

The Simulation Study will be submitted to a journal for academic review in September, with feedback expected in January (review comments (3 months)) with publication in April (6 Months).

Changes Study

In October, researchers in the Changes Study will be allocated participants, supporting them to develop their change narratives.

Case Study Survey (Responses) (Restricted Access)

In November, researchers conduct online video conference focus groups to clarify their narrative survey responses describing the changes they have experienced and how well the simulation model reflected these experiences. Teams or Zoom will be used for interviews with transcription to Nvivo for analysis.

Trend Study Simulation (Educational Technology Trends)

Study Participants in the Changes Study (Restricted Access)

In Early December, researchers will collaboratively combine the change cases using the Most Significant Change process and be allocated sections to draft in the Change Model. 

By Mid December, researchers will have reviewed each other's section drafts, with final edits to a Change Model completed by Late December.

The Changes Study will be submitted for academic review in January, with feedback expected in March (review comments (3 months)) with publication in June (6 Months).

Validation Study

In Early January, researchers in the Validation Study will explore the effectiveness of the Process Model simulation to replicate the results of the Change Model, with changes allocated, simulations completed and compared, and drafts shared by Mid January with final edits completed to a Validation Model by Late January.

The Validation Study will be submitted for academic review in February, with feedback expected in April (review comments (3 months)) with publication in July (6 Months).

Metastudy

In February, researchers in the Meta Study will meet to combine the Process Models, informed by Validation Models, of a range of studies to refine a dynamic simulation model of the field into a Meta Study Model. 

The Metastudy research team will coordinate a range of simulation explorations of the field using the Meta Study Model and allocate team members to develop studies, with drafts of these studies completed and shared by Late May. 

In June, researchers will meet again to combine individual studies into an edited publication or field study, with sections allocated for completion and sharing by Late July and final edits by Late August.

The Metastudy will be submitted for academic review in September, with feedback expected in December (review comments (3 months)) with publication in March (6 Months).

All Research Studies projects comply with the Australian National Statement on Ethical Conduct in Human Research. You should only provide your consent to participate in a study when fully informed of what the study involves.

Where data matching is required, e.g. from multiple surveys, only the minimum information that is needed to identify you is collected using a unique contact identifier (UCI) to minimise the risk that this information is associated with you.

In some studies, you may elect to be identified as a participant in the study on the project website and/or in publications, but this is never a requirement. Identifying information is otherwise deleted when no longer required, and all other research data is deidentified and made available in open data repositories for reuse. 

Each study is conducted in accordance with the Griffith University Research Integrity framework and all studies involving participants have been approved by the Griffith University Human Research Ethics Committee. 

If you have any concerns or complaints about the ethical conduct of the research project, you should contact the Manager, Research Ethics (07) 37354375 or research-ethics@griffith.edu.au

The study is a post-positivist exploration of the field of education with a focus on the application of technology to teaching and learning processes.

Accepting that the theories, hypotheses, background knowledge, and researchers' values will influence what is observed, the study pursues objectivity by recognising the effects of such biases and uses a combination of quantitative and qualitative methods to mitigate bias and improve the validity of research outcomes.

Ontologically, reality exists, but our understanding of it is imperfect and socially constructed. 

Axiologically, while bias is undesirable, it is also inevitable, and mitigation is required, but this in itself will be influenced by the values and beliefs held by researchers.

(Optional) To stay informed of project calls to participate, recommendations, workshops, symposiums, presentations and publications, subscribe to the Research on Research project newsletter with your Unique Contact Identifier (UCI) to maintain your anonymity. No more than one email a month.

Overall Research Studies project posts will be made to the Research Projects Facebook and LinkedIn groups.

The project incorporates ten methods of research inquiry (Self Study, Systematic Literature Review, Delphi Study, Confidence Intervals, Social-Ecological Modelling, Simulation Study, Case Study, Most Significant Change, Validation Study, and Mixed Method) that contribute to a metastudy that draws from the ten research projects. 

Participants can contribute to the Trends, Confirmation and Changes studies. The Trends study uses a modified Delphi consensus research method to provide agreed insight on the issues impacting education and those likely to impact education. Online surveys are used to identify alternatives that are ranked using a conjoint pair method, each over two rounds of the typical Delphi study method.

The Confirmation study uses multiple surveys to repeatedly rank the top five responses of educators in their practice over the year with confidence intervals used to compare means and identify significant differences. 

The Trends study contributes to a systems simulation methodology identifying actors and their properties in an Expert Model and providing initial weightings for the influence of these properties on other actors. 

The Confirmation study samples key actors, and confirms the accuracy of the weightings applied to influence attributes for actors in an Influence Model.

The Overview study categorises the relationships between actors in the Expert and Influence model into a Social Ecological Model, which is then developed into systems simulations expressed as a Process Model. 

The Changes study uses Case Study and the Most Significant Change methods to explore in more detail the changes experienced by participants, developed through a survey with results incorporated into a Simulation model, and the model is used to conduct trials to simulate the changes experienced by participants. Participant experiences are then compared with the model and discussed in focus groups to improve the model, which is validated in a Model Validation study. 

The models developed in each of the ten research studies projects are finally combined into a Metamodel detailing aspects of the field of educational technologies and computer education. 

Each research project is replicated annually to reflect changes in the field of educational technologies and computer education, and will involve the same methods but with an overall changed population of participants. The specific longitudinal analysis will be conducted in the Changes Study.

To better understand a complex research study, self-study or reflective practice research provides a mechanism for analysis of the researchers' theories, hypotheses, background knowledge, experiences and values as the research unfolded and assists in understanding the influence these had on what is observed.

Application of reflective analysis of research practice in Self Study.

A Researcher Model is generated from the Self Study narrative as a template for modelling other researchers as Actors within the Process and Metastudy models and enables the influence of researchers to be incorporated into these models.

A systematic examination of academic literature to establish a Domain Model and Evidence Gap Map. While focusing on white (peer-reviewed research) literature, grey literature (primarily reports) is included where it is deemed significant but is not treated systematically.

A systematic literature review with meta-analysis on reported research from 2000-2022: conducted using PRISMA 2020 framework using The Lens agglomeration database. 

Separate reviews are conducted on international literature and Australian-only literature and used to inform participants in the Trend Study of previous studies on trends and assist in the current study.

Systematic Literature Review on What EduTech research is being done in Australia 

Systematic Literature Review on Who is doing EduTech research is being done in Australia 

Systematic Literature Review on how EduTech research is being done in Australia  

Systematic Literature Review on what replication studies are being done on EduTech research in Australia

Overview of the research questions derived from Systematic Literature Review and generated summaries (using Generative Pre-trained Transformer) of identified topics and themes.

Details of evidence derived from peer-reviewed research and reports

Details Evidence Gap Map

Drawing upon the expertise of teachers, researchers, industry and government, a consensus is sought on the factors that influence Australian educational technologies through the research questions:

1. What are the most pressing challenges related to technology and education?

2. What are the most significant emerging trends related to technology and education? and

3. Which educational technologies will likely have the most significant impact over the next five years?

March

In each of the AllOurIdeas surveys: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication - participants click on the view results tab to look at the list of suggested alternatives contributed by other participants, previous studies and the systematic qualitative literature review, and then back on the Cast Votes tab, in the Add your own idea here... field, suggest new alternatives; reword existing; or combining existing alternatives, in the four AllOurIdeas surveys in the study: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication. Participants should not start choosing between the pairs of alternatives at this stage. [AllOurIdeas link WhatR1, WhoR1,HowR1,RepR1]

April

Participants make choices from the presented paired alternatives, providing responses to 30-40 pairs of alternatives for each of the three categories of the study: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication. Combined with results from other participants, this will provide a consensus ranking of the alternatives.  [AllOurIdeas link WhatR1, WhoR1,HowR1,RepR1]

May

Participants use the initial ranking results of the Conjoint Round 1, for each of the revised three AllOurIdeas surveys: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication - clicking on the view results tab to look at the list of suggested alternatives from the first round, and then back on the Cast Votes tab, in the Add your own idea here... field, participants suggest new alternatives; rewording of existing; or combining existing alternatives in the four AllOurIdeas surveys in the study: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication. Participants do not start choosing between the pairs of alternatives at this stage. [AllOurIdeas link WhatR2, WhoR2,HowR2,RepR2]

June

Participants make choices from the presented paired alternatives, providing responses to 30-40 pairs of revised alternatives for each of the four categories of the study: Educational Technologies being researched in Australia, by whom, using what methods, and with what replication. Combined with results from other participants, this will provide a refined consensus ranking of the alternatives.  [AllOurIdeas link WhatR2, WhoR2,HowR2,RepR2]

An Expert Model is generated from experts in Educational Technologies to identify Actors (aka factors/elements/entities/objects) and Attributes (aka properties/variables) that influence the research questions using online surveys and literature review. [Generated within AllOurIdeas]

A confirmation is conducted by surveys on the accuracy of the Trend study to determine the technologies, challenges and impacts for educators over the 12 months following the commencement of the Trend study. 

Three Confirmation surveys conducted at the end of each school Term, June, September and December, investigate the accuracy of Trend Study rankings. Each survey ranks the top five responses to the research question over the previous three months. Confidence intervals are used to compare means and identify significant differences. 

The amount of influence each Actor has on researched Attributes as determined from consensus is confirmed at three monthly intervals to establish the accuracy of the Trend Study to measure this influence in the short term. The influencing attributes of each actor form an Influence Model of measured ratings of attribute influence for each actor. 

Systems research is conducted using the Expert and Influence Models to construct an Interaction Model of causal relationships in the research questions and categorise these actors and their relationships within a Social-Ecological framework.

Development of a Connection Circle of relationships between the actors identified in the Influence Model and subsequent identification of Causal Relationships (Loops) between actors.

From causal relationships, groupings of actors are derived from those actors with the strongest relationships. Generated as an onion diagram using the Social-Ecological Modelling (SEM) framework, with bands of Actors classified, grouped and clustered within bands.

A dynamic simulation model of changes in the research questions is developed from previous models to enable a range of scenarios to be explored and better understand the interaction between factors.

A systems model is developed from the Expert, Influence, Interaction and Change models, using the Insight Maker simulation software to create a dynamic simulation of the actors and their influence.

Using the causal relationships of actor attributes generated from connection circles and causal loop diagrams, informed by the larger relationships identified by the Social Ecological Model, a simulation model showing dynamic interactions between actor properties is generated. [Link to Agent-Based Simulation Model in Insight Maker]

Exploring the changes occurring over time in Educational Technologies, the Process Model simulation is compared to expert experiences of changes in Educational Technologies. Each round of the study, conducted annually, contributes to a longitudinal data set of educational changes and improvements to the Process Model.

November

Expert participants will be invited to complete a survey [Link to Survey] to contribute narratives on the ranked alternatives identified in the Trend Study and develop cases describing the changes they are experiencing in education. [Change Case narratives]

Early November

Participants will examine the relationships identified in the Process Model [Link to Process Model simulation in Insight Maker] and the influence each actor has on the attributes of others and contrast their change case with those generated by the Process Model. [Simulations of Change Case narratives]

Late November,

Participants will participate in an online interview to clarify elements of their change case and how it was represented by the simulation. [Schedule of Interviews] [Recording of Interviews] [Transcripts of Interviews] [Transcript Analysis]

Descriptive Multiple Case Study of Change Cases contributed by participants.

Comparing the Change Model narratives with the outputs of the Process Model simulation to validate the simulation effectiveness in modelling changes.

January

Researchers will explore the relationships identified in the Process Model [Link to Process Model simulation in Insight Maker] and the influence each actor has on the attributes of others and simulate the change cases identified in the Change Study [Change Case narratives] using the Process Model, contrasting the outcomes to validate the simulation.

The Process Model simulation is validated using a Most Significant Change (MSC) research framework on collected narratives (Change Model) with simulation comparisons and interviews used to compare expert experiences of changes with those generated by the simulation model. Through the hierarchical Most Significant Change process, change cases are grouped and categorised to determine those which are the most significant to research questions.

Models produced from various studies are integrated into a larger Meta Model researching the interactions between various aspects of Education Technologies.

The Simulation Model produced by the Metastudy enables a contextualisation of the studies and a validation process on the predictions from the study over time with the predictions generated from the Metastudy simulation model.

A Metastudy model is developed to simulate field-level interactions of a wide range of actors and attributes.  [Link to Agent-Based Simulation Metastudy Model in Insight Maker]

TBA late 2023

TBA late 2023

Annual online symposia in early December to present research findings.

Online Writing Workshops in June/July and December/January to support research team publications.

TBA 2023

TBA 2024

TBA 2024