Trial Review

Did you know?

The ANZCTR now automatically displays published trial results and simplifies the addition of trial documents such as unpublished protocols and statistical analysis plans.

These enhancements will offer a more comprehensive view of trials, regardless of whether their results are positive, negative, or inconclusive.

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been endorsed by the ANZCTR. Before participating in a study, talk to your health care provider and refer to this information for consumers
Trial registered on ANZCTR


Registration number
ACTRN12621001125819
Ethics application status
Approved
Date submitted
14/07/2021
Date registered
23/08/2021
Date last updated
23/08/2021
Date data sharing statement initially provided
23/08/2021
Type of registration
Retrospectively registered

Titles & IDs
Public title
Randomised hybrid implementation-effectiveness trial of a school-based mental health program in Australian schools
Scientific title
Randomised hybrid implementation-effectiveness trial of a school-based mental health program in Australian schools
Secondary ID [1] 303332 0
Nil known
Universal Trial Number (UTN)
Trial acronym
Linked study record

Health condition
Health condition(s) or problem(s) studied:
Emotional problems 320575 0
Behavioural problems 320576 0
Mental health 320578 0
Condition category
Condition code
Public Health 318437 318437 0 0
Health promotion/education
Mental Health 320591 320591 0 0
Studies of normal psychology, cognitive function and behaviour

Intervention/exposure
Study type
Interventional
Description of intervention(s) / exposure
In this trial all primary schools will receive a universal prevention program known as the PAX Good Behaviour Game [PAX GBG], which is a school-based, classroom intervention used by teachers to teach students self-regulation and prosocial behaviours in the classroom. For teachers to be qualified to deliver the program they must complete the 1-day didactic PAX GBG initial teacher training course. This is a 1-day course, delivered virtually (via Zoom) by master trainers from the PAXIS Institute, USA. The training ensures proficiency in each of the component parts of the program (PAX Kernels). In addition to learning about how to implement the strategies (PAX kernels), teachers learn the importance of a nurturing environment and its effects on trauma, self-regulation, and mental health outcomes throughout the lifespan.

The program itself is comprised of nine PAX kernels, which are evidence-based strategies that are designed to improve social-emotional learning and regulation. The main component of the program which teaches these outcomes is ‘Beat-the-Timer’. In Beat-the-Timer, all 9 PAX kernels are designed to be used together – and to do so, students are divided into small teams and a timer is set during which students are expected to work within their teams to engage in on-task/desirable behaviours and focus on their set task until the timer ends. Any off-task or less desirable/unwanted behaviour is tallied, and at the end of the playing period, teams with three or fewer points are rewarded. When the program is first delivered, rewards are given immediately following a ‘beat the timer’ game, and then are increasingly delayed, to teach delayed self-gratification.

There is no standard program dosage, although for Beat-the-Timer to be most beneficial, teachers are expected to deliver it up to 3 times per day, for between 10 and 40 minutes. In addition to playing Beat-the-Timer, teachers are expected to deliver each individual kernel daily, and the duration for this can vary for between 1 minute to 20 minutes. The dosage and frequency of the strategies should increase over time, from every two days in the beginning to up to three times per day once students and teachers are comfortable with the program. It should not be played at the same time each day, and any rewards associated with the Beat the Timer component should be increasingly delayed over time to support students’ self regulation. As the overall PAX GBG program is designed to be integrated and to support usual classroom lessons, there is no finite delivery period. The minimum duration to see benefits at the student level would be 12 weeks, however, the longer the duration the greater the benefits. Delivering the PAX GBG program for a minimum of 12 months is ideal.

Schools in both the intervention and control conditions will receive the PAX GBG program in this trial. As such, the exposure being tested is a multi-component implementation strategy [PAX Plus] to see if this leads to higher levels of PAX GBG adoption in schools and greater effectiveness in relation to student emotional and behavioural changes. The PAX Plus enhanced implementation strategy was iteratively co-designed by the research team, the NSW Department of Education and an advisory group of teachers and principals. Schools in the PAX Plus condition will have access to all the same resources as in PAX Standard (control condition) and in addition, will have access to our implementation approach which includes the following strategies:
1. Fortnightly e-newsletter: The purpose of the newsletters will be to provide information about specific PAX kernels (why they are important, how to deliver them, and tips to make use easier) to encourage teachers to use the full range of kernels. Between 2 and 3 kernels will be addressed in each newsletter. Newsletters will be emailed out fortnightly across a 4-month period. The newsletters will be between 2 and 3 pages in length, including visualisations, and will be expected to take between 5 and 10 minutes to read.
2. Peer Learning network: School staff will have access to an online peer learning network, hosted on Microsoft Teams. It is a Questions and Answers forum for teachers from different schools to share their experiences with implementation challenges. Questions can be posted anytime, and once a week, a member of the research team will moderate the forum and answer the week’s questions.
3. Promoting PAX Chats: PAX Chats are live discussions between teachers and the PAXIS Institute (program owners) in the USA to resolve common challenges with kernel implementation. The school leadership team will be encouraged to send reminders for teachers to participate in these PAX Chats available once or twice a term.
4. Executive Support: A member of the research team will schedule a 15-minute phone call with the principal of each school every 4 weeks to check in with, and gain an understanding of, their progress.
5. School champions: One teacher or other staff member trained in the PAX GBG program will be nominated as a ‘program champion’. Their role will be to promote the innovation and to identify strategies to enhance engagement within the school. As part of this, the champion may chair weekly meetings with the relevant teachers and staff to share learning, problem solve, and maintain motivation. The school principal will be contacted to identify the most appropriate person for this role in their school.
6. Recognition system: Schools will be provided with certificates of achievement certified by the NSW Department of Education to acknowledge the commitment of staff to the PAX GBG program. Schools will also be provided with a staff 'tootle board' (which is adaptation of one PAX GBG kernel delivered to students) where they can anonymously post positive affirmations about their peers. In addition, the research team will set up an online portal which will publicly recognise teachers and staff doing an outstanding job of program delivery. Schools will be asked to nominate staff for this and to provide a brief background of the staff member.
7. Audit and provide feedback: Schools will be provided with access to an online survey system (using Qualtrics) to use to monitor the outcomes of the PAX GBG for their own school and feedback progress to staff as a motivation tool. Schools will have a unique link to this tool for their own school, and are able to download data directly.

PAX Plus schools will be provided a toolkit (manual) describing each of these strategies in detail. Schools will be directed to use these implementation supports for 6 months from when they join the trial. Some strategies have a clear, specified timepoint for implementation (e.g. executive support which occurs every 4 weeks) while others can be used as frequently as suits the needs of individual schools (e.g. recognition system).
Intervention code [1] 319640 0
Prevention
Comparator / control treatment
In addition to control (comparator) schools receiving the PAX GBG program, they will have access to any standard implementation supports that were developed by the program owners. These supports include:
• 4 x 2-hour classroom visits from a coach for the first four weeks of delivery of PAX GBG program to observe the fidelity of the teacher's delivery, and will provide constructive feedback post-observation.
• Teachers will be provided a hard-copy training manual and access to online resources supplied by the program developers to support delivery.
• A short handbook outlining these strategies/resources will be provided to schools so that they are aware of what is being offered and how to access it if they wish.

This standard implementation arm is unguided; there will be no formal requirements for schools in this condition to implement these implementation strategies, nor will any formal implementation period be defined. The decision to provide no formal guidelines for what has to be delivered or for how long is because there currently is no formal implementation framework for this program; accordingly, this arm is testing implementation in the wild, and will instead capture data on the natural variance in implementation style when schools are in control of deciding how much they want to invest in structured (or not) implementation processes.
Control group
Active

Outcomes
Primary outcome [1] 326404 0
Program adoption: Adoption will be defined as a school making a formal decision to commit to using PAX Good Behaviour Game as part of their routine practices. We have developed a bespoke self-report measure for adoption which is a binary item asking teachers whether the PAX GBG program is currently being delivered in their classrooms.
Timepoint [1] 326404 0
6-months post-baseline (primary timepoint) and 12-months post-baseline.
Secondary outcome [1] 391353 0
Change in total score on teacher-Rated 25-item Strengths and Difficulties Questionnaire (SDQ).
Timepoint [1] 391353 0
6-months post-baseline
Secondary outcome [2] 398259 0
To assess how well the PAX GBG was integrated into the routine practices of schools we will examine change in total scores on the 19-item Normalisation Measure Development (NoMad) instrument.
Timepoint [2] 398259 0
6-weeks post-baseline, 6-months post-baseline and 12-months post-baseline
Secondary outcome [3] 398927 0
The fidelity of program delivery will be assessed using total scores on a 18-item custom scale designed to capture core components of the PAX GBG intervention.
Timepoint [3] 398927 0
6-weeks post-baseline, 6-months post-baseline, 12-months post-baseline.
Secondary outcome [4] 399886 0
Identification of which implementation strategies worked best to support adoption in which schools will be assessed via thematic analysis of qualitative data and guided by a realist evaluation framework. In total, 40 teachers and 20 support staff (i.e., principals/school leadership, school counsellors) will be interviewed and their data combined to identify key implementation strategies.
Timepoint [4] 399886 0
Semi-structured qualitative interviews will be conducted at 6-months and 12-months post-baseline.

Eligibility
Key inclusion criteria
All New South Wales government primary schools (K - Year 6) are eligible to enrol in the trial as long as at least one teacher in the school has trained in the PAX Good Behaviour Game program.
Minimum age
4 Years
Maximum age
70 Years
Sex
Both males and females
Can healthy volunteers participate?
Yes
Key exclusion criteria
None.

Study design
Purpose of the study
Prevention
Allocation to intervention
Randomised controlled trial
Procedure for enrolling a subject and allocating the treatment (allocation concealment procedures)
Allocation is not concealed
Methods used to generate the sequence in which subjects will be randomised (sequence generation)
Schools will be randomised with a 1:1 allocation in blocks of 4 using a computer-generated randomisation table (i.e. computerised sequence generation). Randomisation will occur at the school level to avoid contamination that could occur with teachers implementing different strategies within the same school.
Masking / blinding
Open (masking not used)
Who is / are masked / blinded?



Intervention assignment
Parallel
Other design features
Phase
Not Applicable
Type of endpoint/s
Efficacy
Statistical methods / analysis
Implementation effectiveness: The extent and variation in changes in primary implementation outcomes induced by PAX Plus will be analysed at 6-months using linear mixed effects models, with a fixed effect of condition, and a random effect of school to account for clustering. Analysis of change in secondary continuous implementation outcomes (fidelity, uptake, adherence) from 6-weeks to 6- and 12-months post baseline will similarly use linear mixed effects models, including a repeated effect of time point. School characteristics will be tested as potential mediating factors of implementation effectiveness (e.g. social economic advantage, size, geography), as will implementation exposure condition, through inclusion of interaction terms in the models. Descriptive analyses will be used to provide an overview of the implementation scales across the collective study sample; linear regression models will be used to examine associations between the range of implementation scales and SDQ scores. T-tests will be used to examine associations between implementation factor scales and adoption. Alpha will be set at 0.05 for all analyses.

Emotional and/or behavioural outcomes: We will use mixed models repeated measures analyses, with maximum likelihood estimation and an appropriate covariance structure, to evaluate the change in emotional and behavioural problems - assessed by the SDQ - over time (baseline to 6-months) in the intervention relative to control condition. Analyses will accord with the intention-to-treat principle and will include a random effect of school to account for clustering.

Qualitative data
Thematic analysis will be undertaken to identify, interpret, and report on patterns of meaning in the data, using Braun and Clark’s six phase model. Transcriptions will be separated based on participant group. First, two researchers will read and re-read the data, taking initial memos. Second, they will review the same two transcripts and generate initial codes. Third, they will consolidate these codes into a codebook based on themes included in the semi-structured interview guide informed by critical realism and supplemented by inductive thematic analysis to capture identified themes. Fourth, initial themes will be reviewed by the wider research team and consolidated into a thematic coding tree. Fifth, the tree will be iteratively refined as coding continues. Finally, data will be extracted and reported.

Recruitment
Recruitment status
Recruiting
Date of first participant enrolment
Anticipated
Actual
1/04/2021
Date of last participant enrolment
Anticipated
31/07/2022
Actual
Date of last data collection
Anticipated
31/08/2023
Actual
Sample size
Target
240
Accrual to date
120
Final
Recruitment in Australia
Recruitment state(s)
NSW

Funding & Sponsors
Funding source category [1] 307750 0
Government body
Name [1] 307750 0
NSW Department of Education
Country [1] 307750 0
Australia
Funding source category [2] 309145 0
Charities/Societies/Foundations
Name [2] 309145 0
Andrew & Geraldine Buxton Foundation
Country [2] 309145 0
Australia
Funding source category [3] 309146 0
Government body
Name [3] 309146 0
National Health & Medical Research Council
Country [3] 309146 0
Australia
Primary sponsor type
University
Name
University of New South Wales
Address
UNSW Sydney, NSW 2052. Australia.
Country
Australia
Secondary sponsor category [1] 308451 0
Other Collaborative groups
Name [1] 308451 0
Black Dog Institute
Address [1] 308451 0
Hospital Road, Randwick NSW 2031
Country [1] 308451 0
Australia

Ethics approval
Ethics application status
Approved
Ethics committee name [1] 307772 0
University of New South Wales Human Research Ethics Committee
Ethics committee address [1] 307772 0
UNSW Sydney, NSW 2052. Australia.
Ethics committee country [1] 307772 0
Australia
Date submitted for ethics approval [1] 307772 0
21/09/2020
Approval date [1] 307772 0
04/12/2020
Ethics approval number [1] 307772 0
HC200759
Ethics committee name [2] 309013 0
State Education Research Applications Process (SERAP)
Ethics committee address [2] 309013 0
State Education Research Applications Process (SERAP)
NSW Department of Education
GPO Box 33 SYDNEY NSW 2001
Ethics committee country [2] 309013 0
Australia
Date submitted for ethics approval [2] 309013 0
28/09/2020
Approval date [2] 309013 0
02/12/2020
Ethics approval number [2] 309013 0
2020364

Summary
Brief summary
Implementation of evidence-based programs in school settings can be challenging and these challenges undermine the benefits these programs deliver for children. The primary aim of this study is to assess the potential effectiveness of an implementation intervention designed to enhance the adoption of the PAX Good Behaviour Game in New South Wales (NSW) primary schools. A secondary outcome is to investigate the impact of the PAX Good Behaviour Game on children’s mental health in the Australian context.

The study uses a cluster randomised hybrid III effectiveness-implementation design and will involve 60 NSW primary schools. Randomisation will occur at the school level. All NSW public schools trained in the PAX Good Behaviour Game are eligible for participation. The intervention is a multicomponent implementation strategy that has been iteratively co-designed by the research team and local stakeholders. Intervention schools will have access to multi-component implementation support strategy in addition to the coaching received as usual care to assist with implementation challenges. Primary and secondary implementation and effectiveness outcomes using self-report online surveys with teachers, support staff and parents at baseline until 6-months follow up, and where relevant, from baseline to 12-months follow up. Qualitative interviews with teachers, support staff and principals will be used to examine which implementation strategies worked for whom and under what conditions.
Trial website
Trial related presentations / publications
Public notes

Contacts
Principal investigator
Name 108442 0
Dr Michelle Tye
Address 108442 0
Black Dog Institute
Hospital Road, Randwick NSW 2031
Country 108442 0
Australia
Phone 108442 0
+61 02 9065 9152
Fax 108442 0
Email 108442 0
[email protected]
Contact person for public queries
Name 108443 0
Michelle Tye
Address 108443 0
Black Dog Institute
Hospital Road, Randwick NSW 2031
Country 108443 0
Australia
Phone 108443 0
+61 02 9065 9152
Fax 108443 0
Email 108443 0
[email protected]
Contact person for scientific queries
Name 108444 0
Michelle Tye
Address 108444 0
Black Dog Institute
Hospital Road, Randwick NSW 2031
Country 108444 0
Australia
Phone 108444 0
+61 02 9065 9152
Fax 108444 0
Email 108444 0
[email protected]

Data sharing statement
Will individual participant data (IPD) for this trial be available (including data dictionaries)?
No
No/undecided IPD sharing reason/comment
NSW Department of Education prohibits public sharing of individual participant data.


What supporting documents are/will be available?

Doc. No.TypeCitationLinkEmailOther DetailsAttachment
12526Study protocol    Trial protocol is currently under review, we will ... [More Details]
12527Ethical approval    381346-(Uploaded-14-07-2021-11-59-07)-Study-related document.pdf



Results publications and other study-related documents

Documents added manually
No documents have been uploaded by study researchers.

Documents added automatically
SourceTitleYear of PublicationDOI
EmbaseA cluster randomised effectiveness-implementation trial of an intervention to increase the adoption of PAX Good Behaviour Game, a mental health prevention program, in Australian primary schools: Study protocol.2022https://dx.doi.org/10.1016/j.conctc.2022.100923
N.B. These documents automatically identified may not have been verified by the study sponsor.