Mater Research Support Centre

Research Implementation

The information on research implementation is designed to be used by novice researchers and are friendly and simple to use.  MRSC provides some basic information on research implementation and the appropriate standards and regulations that need to be followed.  Currently there are three people at the MRSC who can assist with the details of trial implementation. They are Sue Jenkins-Manning, Vicki Flenady and Prof. Allan Chang. The best way to make contact is to send emails to Kate.Triggs@mater.org.au

Assistance can be given in the form of email correspondence, phone advice if the questions is simple (eg where do I find the template for the consent form? etc) or an informal consultation. Advice can be given on formatting HREC submissions including content, formatting of the document, budget application advice and  clinical study agreements. We can also offer advice on all stages of research coordination and implementation including recruitment, data collection, assistance with research assistants etc. However, for advice on  Mater HREC submissions, please contact Odette Petersen prior to contacting the MRSC for advice.

Resources:

• Screening Log Book
  • Why Use A Screening Log Book?
  • Information to Include in A Log book.
• Study Procedure Manual
• The data Collection form
• Purpose of the data collection form:
• Objectives of designing data collection forms:
• Standardisation of Forms:
• Testing the data collection forms:
• References:
• Maximising recruitment
  • Prior to implementation:
  • During recruitment phase:
  • References:
• Assessing recruitment progress
• Minimising drop outs
• Data collection and management
  • Accuracy of data collection forms:
  • Access of Data
  • Paper Data Entry
  • Computer data entry
  • Advantages of Direct Electronic Data Entry
  • Disadvantages of Direct Electronic Data Entry
  • Use of electronic scanning devices
  • References:
• Storage of Documentation

Screening Log Book

Why Use A Screening Log Book?

A log book provides vital information to assist with recruitment to  the study by  not only  identifying the characteristics of those who enter the trial but also those who refuse participation. Publishing of  this data in the study results  also provides other researchers with  essential information on which to base a similar study.   One of the major purposes of the log book is as an aid to recruitment. Therefore a screening log provides the figures on which to assess and evaluate the progress of a study.  A log book can provide information on refusal rates so that in the case of a high refusal rate a new approach may be adopted. In addition by using the information in the log to define the types of patients who are consenting to participate the recruitment strategy can be modified and refined.

Information to Include in A Log book.

Identifying number eg UR number of all potential recruits
Demographic characteristics eg DOB, sex, ethnicity, level of education (some of these may be able to be directly extracted from a hospital database)
Date screened for eligibility Reason not screened eg missed seeing at appointment
Reason found not eligible eg exclusion criteria not previously detected Reason for refusal if offered (However an eligible recruit is not required to give any reason for their refusal)
Name of person who approached potential recruit
Place of recruitment eg ANC (if several areas).

Back to Top

Study Procedure Manual

A study procedure manual should be placed in each relevant clinical area for each separate study.

Included in the manual should be

• the study protocol
• a detailed list of the procedures to be followed
• a short staff information sheet with an overview of the study
• contact details of the researchers / study team
• Standard Operating Procedures (SOP) for each area
• flow charts to determine eligibility- procedure to follow if unblinding is required
• procedure for reporting adverse events
• all relevant CRF / data collection forms
• detailed description of the procedure

In addition to the generic information each area should have their specific SOPs included in the study procedure manual.

For example:

• pharmacy -  what happens when drugs arrive, disposal of expired supplies etc.
• clinical staff - what investigations should be performed and when, how to determine patient eligibility, instructions on how to take and record data measurements, how adverse events are reported etc.
• laboratory – how samples are processed, who is notified of the results.
• data collection personnel – what units the data is collected in, what to do  if an error occurs etc.

Back to Top

The data Collection form

Purpose of the data collection form:

• Facilitate easy collection of the raw data
• Collect data in a way which facilitates monitoring and auditing of a study
• Expedite the steps of data processing  

Objectives of designing  data collection forms:

The overall objectives of designing data collection forms are to produce an instrument and supporting instructions that allow collection and processing of data that are:

• Relevant to study objectives
• Accurate
• Complete
• Standardized across collection sites
• Standardized across related studies
• Efficient for data recording
• Efficient for data processing
• Appropriate for statistical analysis  
• The objectives of a study should determine what data will be collected.
• List the study objectives and work back from there the clear data that will need to be collected to make the calculations at the end of the trial
• Care should be taken to identify the data that actually are needed to meet these objectives.
• Remember the amount of data collected has a major impact on the effort required for data collection and management.
• As the volume of data increases, it becomes increasingly difficult to ensure the accuracy of what is recorded.
• Additional  data to be collected may also be determined from previous trials.  

During the forms development process, it is important that the draft forms be reviewed carefully for completeness, accuracy, and ability to be processed efficiently at the field sites or resource centres and at the coordinating centre.  Data collection forms are also referred to as case report forms and case record forms.  

Back to Top

Standardisation of Forms:

• Ensuring the form has a standard, coherent format will help ensure that the data collected is unambiguous with minimal errors. Developing a layout that is user friendly with logical isions within the form  will help  facilitate the collection of  accurate data and minimise missing data.  
• Data collection forms from previous studies may be able to be used as a blank template to assist with formatting of the form for another study. Starting with forms that have been proven in the field will greatly reduce the effort to develop the forms for a new study and in some cases facilitate comparison or pooling of results across studies by ensuring the same data is collected in the same manner.
• Standardization within a study is achieved primarily by consistency in the format and appearance of data collection forms and by ensuring the consistency of items that are collected on more than one form or at more than one time. A small number of item formats should be chosen and used consistently. For example, multiple choice items should require the user to respond in a consistent way, e.g., checking a box, circling a response, etc.
• The study procedure manual should include all standard operating procedures for the method that measurements are to be taken and recorded.
• If a measurement can be taken more than one way it should be specified in the manual and on the form how the measurement should be  taken eg BP taken lying supine.
• When collecting categorical data values it is important to specify the units and manner in which the data should be collected. The correct number of boxes should be provided on the form, the relevant decimal points, commas and units to be used printed on the form. For example, for a participant’s height  the following could be used:   __ . __ __ m      or     __ __ __ cm.
• If whole numbers are to be used the study procedure manual  should explain when the value should be rounded up and when it should be rounded down.
• Date and times must be entered consistently.
  The following is a suggested format :__ __ / __ __ / __ __ __ __     __ __ : __ __   dd     /    mm   /       yyyy   24 hour
• For multiple choice questions the instructions for answering the question must be consistent across all questions, either circling an answer or placing a tick in a box.
• For some multi choice questions you may need a ‘don’t remember’ or  ‘unknown’ box. Check that you are only asking one question at a time.
• Pose the question so a definitive answer is required. For example: Has the patient ever smoked cigarettes? is better than Is the person a smoker?
• Page formats also should be consistent. Each form should have an identical header area. Instructions should be positioned in the same way on each form. Skip rules should be indicated in a consistent manner.
• When appropriate, provide codes for “unknown” or “not applicable”. Consider the implications for statistical analysis of various reasons for missing data when deciding on the level of detail to capture concerning reasons for missing data.
• When it is necessary to provide an “other” category, provide a space to record the unanticipated response. It is highly desirable to ensure that the other category is seldom used by appropriate pretesting of the form.
• Every item should require a response.
• Every form must contain a series of fields that allow that form to be uniquely identified and correctly related to other data forms. Every form used in a study should begin with a standard header section containing these fields. This section should be identical in format and content for each form type. For hospital patient the unique identifier could be the hospital UR number.
• Forms should be printed on one side only to avoid missing pages.
• There should be an easy ‘path’ to follow through the form which makes data entry logical.
• For different stages of the data collection process it may be easier to colour code forms eg blue forms for the maternal data / green forms for the neonatal data.  

Back to Top

Testing  the data collection forms:

• There is little doubt that the effort required to test the data collection forms prior to production proves to be a wise investment.
• Initial testing may be done by having some project personnel play the role of participants while others administer the forms.
• However, it is highly desirable to test the forms on persons as similar as possible to the intended study participants.
• Terminology familiar to the study staff may not be understood by persons with less medical knowledge.
• Whenever possible, the forms completed during the test period also should be used to test the data entry and data management system developed for the study.
• The use of “live data” as test cases often reveals problems in software and procedures that were not anticipated by the programming staff during system testing.  

References:

Good Phillip. The design and Conduct of Clinical Trials. 2002.New York : John Wiley and Sons. 

Hosking J,  Newhouse M,  Bagniewska A, Hawkins B. Data Collection and Transcription. Controlled Clinical Trials 16:66S-103s, 1995  

McFadden E. Management of Data in Clinical Trials. 1998.New York : John Wiley and Sons.  

Meinert C. Clinical trials design, Conduct and Analysis. 1986.New York : Oxford University Press Inc.  

Mulay Marilyn. A Step-by-Step Guide to Clinical Trials. 2001. Sudbury : Jones and Bartlett Inc.  National Statement 2000. Commonwealth ofAustralia . Canberra . AGPS.

TGA. Note for Guidance on Good Clinical practice ( CPMP/ICH/135/95). Annotated with TGA Comments. 2000. Canberra . Department of Health and Aged care.  

The Belmont Report Ethical Principles and Guidelines for the Protection of HumanSubjects of Research. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 1979. http://ohrp.osophs.dhhs.gov/humansubjects/guidance/belmont.htm

Spilker B. Guide to Clinical trials.1996.Pennsylvania : Lippincott-Raven Publishers.  

Williams J, Cheung W, Cohen D, Hutchings H, Long M, Russell I (2003) Can randomised trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment. Health Technology Assessment Vol 7: No 26.  

World Medical Association Declaration Of Helsinki Ethical Principles for Medical Research Involving Human Subjects. 2002 www.wma.net/e/policy/b3.htm

Back to Top

Maximising recruitment

The recruitment goal in a fixed sample size design will have been fixed in the planning stage of the trial. It is imperative when working out the sample size that factors such as loss to follow up should be taken into account. The recruitment timeframe should take into consideration all factors which may influence recruitment such as lag time for centres to set up and commence recruiting, anticipated number of recruits from eligible patients (although 100% recruitment would be optimal in most cases this is not a feasible or realistic goal) and seasonal changes (recruits may be lower during holiday periods due to clinics being closed, eligible patients putting off appointments etc)

Prior to implementation:

• Discussion with the clinical staff in the planning stage of the trial is essential to ascertain the availability of resources, eligible participants, impact on workload and interest and support of the staff.
• Education of the clinical staff in regards to the importance of conducting well designed clinical trials needs to be undertaken well before the commencement of the trial.
• Each clinical area involved in the trial should have a study procedure manual
• Prior to commencement of the trial the recruitment timeframe should be determined, and the actual enrolment plotted against the expected timeframe.        Mailing to inidual clinicians regarding the commencement of a trial has shown to have a poor response rate. However face to face interviews are more successful although these are expensive and time consuming and may not be cost effective.
• Distribution of small palm cards with trial particulars such as inclusion/ exclusion criteria, contact phone numbers etc.
• If the budget allows pens, buttons , stick it note pads etc can be produced with the trial logo or acronym printed on them. 

Back to Top

During recruitment phase:

• Without the assistance of a dedicated research assistance clinicians may recruit as few as one-fifth the number of the patients they anticipated they would and some will recruit none at all.

• Research staff should be available to conduct in service sessions out of office hours with a member of the research team available on a phone number or pager.

• Use of wall charts/posters in prominent places to remind clinicians about the trial and alert potential recruits about the trial.

• Weekly verbal reminders to treating clinicians help to counteract the natural tendency of busy clinicians to forget about a trial.

• In long term trials a schedule of regular meetings to discuss and update the trial progress with clinical staff needs to be established. These need to coincide with routine staff changes eg  new registrars commencing  each year.

• Enrolment should be monitored on a regular basis and reports sent out to each site/department involved in recruiting. This can be easily be done in the form of a graph incorporated into a regular newsletter sent to all clinicians.

• Determine what motivates the clinicians who are participating well in the trial  and address this issue with other clinicians.

• If possible for multi centred trials have all site investigators and coordinators  meet to discuss recruitment and site specific problems.

• To aid those recruiting study packs with all the required forms such as the patient consent form, information sheet, data collection forms, lab slips etc should be prepared in advance and placed in easily accessible areas.  Small incentives such as chocolates etc can be offered for passing certain recruitment steps eg first 50 patients recruited. 

References:

McBride P, Massoth K 1996. Recruitment of private practices for primary care research: Experience in a preventive services clinical trial. J Fam Prac 43: 389 – 395.

Spilker B. 1992. Patient recruitment in clinical trials. Raven Press, New York .

Back to Top

Assessing recruitment progress

• A recruitment strategy is essential for the smooth running of a trial.

• A database should be implemented to enable each recruit to be followed.

• Drop outs should be carefully monitored and the reason for dropping out asked and documented.

• High drop out rates may indicate problems with the requirements of the trial, inability of recruits to attend clinic, problems with the research staff etc. See minimising drop outs.

• If several methods of screening eligible recruits are in progress there needs to be careful documentation on which method was used so that the methods can be assessed and compared to ascertain the more effective method.

• There should be a formalised system for reviewing and evaluating the degree of success of the strategy.

• The use of Gant charts may be helpful in planning the timeline and selecting when assessments of the recruitment need to be performed.

• Assessing the recruitment progress may be performed after a specific time period, after a specific number of recruits have been entered into the study and/or if recruitment lags behind the anticipated rate for a defined length of time.

• Graphs (Fig 1 & Fig 2) etc can be used to monitor recruitment progress and also for teaching sessions to aid in recruitment.

Fig1

Fig2

Back to Top

Minimising drop outs

Back to Top

Data collection and management

Accurate completion of the data collection forms is integral to successfully carrying out a clinical trial. There are two methods that data can be entered: paper data entry which is then transferred to a computer for storage and analysis and direct electronic data entry onto a computer.

Accuracy of data collection forms:

• Paper based data collection forms which are then transferred to a computer approx. 70 – 85% accuracy

• Online data entry onto a computer approx. 95% accuracy

• Double entry data entry onto a computer approx. 99% accuracy

Access of Data

• Ideally all trial data should be accessed from the primary source which in a hospital setting is usually but not always the patient medical chart.

• Electronically stored, routinely collected data from administration and clinical databases can be used for collection of some data however studies have confirmed that the validity of routinely collected data is suspect, particularly in systems that are not under clinical and professional control.

• If data is to be obtained from sources other than the admitting hospital this needs to be refected on the consent form  so that other institutions and clinicians have the patient’s consent to release the required data.

• Data collection forms should be included in the HREC proposal so that committee members can review exactly what data is to be collected.        For all data collection forms including electronic forms the person entering the data should sign and date the form.

• A study procedure manual should be prepared for every trial which includes the Standard Operating Procedures on how the documentation should be recorded. 

Back to Top

Paper Data Entry

• All entries should be filled out in black pen.

• Any change or correction to a CRF must be made in black pen and should be dated, initialled and explained (if necessary) and should not obscure the original entry.

• The use of ‘Whiteout’ or similar correction fluid products is not permitted.

• A list of those iniduals authorised to make data changes and those data changes that are permitted  should be kept by the chief investigator.

• All boxes should be filled in.

• Boxes should not be left blank as it is then unclear if this value is zero, not applicable or missing data. If the question is not applicable then N/A should be documented by the question or a line should be placed through the question.

• For Australia all dates should be recorded as dd/mm/yyyy unless otherwise requested by the trial investigators.

• All data entry forms should be checked by a second person.

• Written trial documents should be kept in a locked room and/or filing cabinet.  

Back to Top

Computer data entry

• For electronic data entry the system must be designed to permit data changes in such a way that there is no deletion of previously entered data.

• There must be set Standard Operating Procedures (SOP) for the use of the systems including the maintenance of a security system that prevents unauthorised access to the data and adequate backup of the data.

• Researchers must be responsible for ensuring appropriate security for any confidential material, including that held in computing systems. Where computing systems are accessible through networks, particular attention to security of confidential data is required. Electronic data must be password protected.

• Boxes should not be left blank – a value should be allocated to be placed in a box to signify that the question is not applicable. Another value should be allocated to signify missing data where the source data has been checked and the data is unable to be obtained. In this way when data analysis is performed it is easy to see which data has been verified as truly missing.

• Data entry personnel should be given explicit instructions regarding the types of data changes they may make.

• Data should be transcribed as recorded on the paper data form even if it appears incorrect and then a query should be made regarding the data. In this way there is an audit trail from the original data records to the computer data file.  ‘On the spot’ changes without verification lead to discrepancies between the original data records and the computer data file, thus leading to questions regarding the integrity and validity of the data.

• All data should be backed up at regular intervals.

Back to Top

Advantages of Direct Electronic Data Entry

• The computer system can be programmed to recognise unacceptable data or errors or incomplete data. This enables immediate detection and correction of errors.

• Saves on time printing and distributing data collection forms.

• Allows earlier data entry completion.

• Allows early detection of common data entry errors thereby enabling specific education to be implemented.  

Disadvantages of Direct Electronic Data Entry

• Expensive set up costs with computer and training.
• Security may be an issue.
• Technical problems may slow or halt the process and may be expensive to fix.  

Use of electronic scanning devices

Software is now available for use with such data collection forms as surveys and questionnaires.

Advantages

• Reduces the time taken to load data – large amounts of surveys can be processed very quickly.
• Reduces  the element of human error.
• Software can be programmed to do simple data analysis and format text fields easily.  

Disadvantages

• Expensive to set up
• Forms need to be formatted in a certain manner for the scanner to read and interpret them  

The MRSC now have  a photocopier with document reader capabilities for processing of questionnaires and surveys. However to use this facility it is essential that the MRSC be involved with the design and formatting of the survey and/or questionnaire to ensure it is compatible with the software. Please email the MRSC atAnne-Maree_Stout@mater.org.au if you have a query regarding the use of the optical scanner

Back to Top

References:

Good Phillip. The design and Conduct of Clinical Trials. 2002.New York : John Wiley and Sons. 

Hosking J,  Newhouse M,  Bagniewska A, Hawkins B. Data Collection and Transcription. Controlled Clinical Trials 16:66S-103s, 1995  

McFadden E. Management of Data in Clinical Trials. 1998.New York : John Wiley and Sons.  

Meinert C. Clinical trials design, Conduct and Analysis. 1986.New York : Oxford University Press Inc.  

Mulay Marilyn. A Step-by-Step Guide to Clinical Trials. 2001. Sudbury : Jones and Bartlett Inc.  National Statement 2000. Commonwealth ofAustralia . Canberra . AGPS.

TGA. Note for Guidance on Good Clinical practice ( CPMP/ICH/135/95). Annotated with TGA Comments. 2000. Canberra . Department of Health and Aged care.  

The Belmont Report Ethical Principles and Guidelines for the Protection of HumanSubjects of Research. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 1979. http://ohrp.osophs.dhhs.gov/humansubjects/guidance/belmont.htm

Spilker B. Guide to Clinical trials.1996.Pennsylvania : Lippincott-Raven Publishers.  

Williams J, Cheung W, Cohen D, Hutchings H, Long M, Russell I (2003) Can randomised trials rely on existing electronic data? A feasibility study to explore the value of routine data in health technology assessment. Health Technology Assessment Vol 7: No 26.  

World Medical Association Declaration Of Helsinki Ethical Principles for Medical Research Involving Human Subjects. 2002 www.wma.net/e/policy/b3.htm

Back to Top

Storage of Documentation

• Written trial documents should be kept in a locked room and/or filing cabinet.

• The Mater Health Services currently retain patient medical records (charts) for 10 years for adults and 28 years for children after the last presentation to the facility.

• The TGA requires trial documentation to be retained for 15 years following completion of a clinical trial.

• It should be possible to validate data collected on CRFs against another source of data such as patient medical charts. The supporting document is referred to as a ‘source document’. Section 1.52 of the Note for Guidance on Good Clinical Practice (CPMP/ICH/135/95) defines a source document as: “Original documents, data, and records (eg hospital records, clinical and office charts, laboratory notes, memoranda, subjects’ diaries or evaluation check lists, pharmacy dispensing records, recorded data from automated instruments, copies or transcriptions certified after verification as being accurate copies, microfiches, photographic negatives, microfilm or magnetic media, x rays, subjects files and records kept at the pharmacy, at the laboratories and at medic-technical departments involved in the clinical trial.”

• The hospital records for each participant must be annotated to indicate that the patient medical record also must be kept for a minimum of 15years from the completion of the trial, as it contains information relevant to the trial.

• Data management should comply with relevant privacy protocols, such as the Australian Standard on personal privacy protection.

• Data related to publications must be available for discussion with other researchers. Where confidentiality provisions apply (for example, where the researchers or institution have given undertakings to third parties, such as the subjects of the research), it is desirable for data to be kept in a way that reference to them by third parties can occur without breaching such confidentiality.

• Researchers must be responsible for ensuring appropriate security for any confidential material, including that held in computing systems. Where computing systems are accessible through networks, particular attention to security of confidential data is required.

• Please refer to Section 8 of the Note for Guidance on Good Clinical Practice (CPMP/ICH/135/95) on what is considered essential documents for the conduct of a clinical trial and therefore what documents must be retained.

References:

National Statement 2000. Commonwealth of Australia .Canberra. AGPS.

TGA. Note for Guidance on Good Clinical practice ( CPMP/ICH/135/95). Annotated with TGA Comments. 2000.Canberra . Department of Health and Aged care.  

The National Privacy Principles in the Privacy Amendment (Private Sector) Act 2000 www.privacy.gov.au/publications

Back to Top