Design

When selecting a suitable a PELA, design issues to be considered include:

Reach: There are many different types of PELA in use in Australia's universities. In terms of their reach, they can be university wide, faculty-wide, discipline-wide or unit specific.

Content: The wider its reach, the more generic in content a PELA is likely to be. Almost all wide-scale PELAs currently in place in Australian universities have an emphasis on one or more language macro skill (reading, writing, listening or speaking), with the vast majority focusing on writing. The decision on content has to take into account the kind of language students are required to use, the uses to which language is put in tertiary courses, and the overall approach taken by the institution.

Construct: ‘Authenticity’, in terms of the similarity of a PELA to discipline-based tasks, is no guarantee of construct validity. Construct validity is a complex phenomenon, and considerable expertise in this field of research is required in PELA design.

Format: Some PELAs are available online and some are paper-based. The greater the reach, the more cost-effective online delivery usually is, although some very local PELAs are also available online through learning management systems such as Blackboard. Online instruments have the advantage that they can potentially be automatically marked, depending on the format of the assessment. At the same time, staff in some universities with online PELAs report that there have been issues with the technology, either because systems could not cope with large-scale assessment or because individual students did not have the technical know-how to complete the assessment.

Other issues to consider are whether the PELA should be timed or untimed (and if timed, how long it should be), and whether it should be held under test conditions, supervised or self-administered. Those decisions will be linked to the ultimate purpose of the assessment.

Whatever its ultimate design, a newly developed PELA will need to be carefully trialled, reviewed and evaluated by experts if it is to be valid, reliable and have credibility in the long term. The higher the stakes (for example, if the PELA is compulsory or if its results may affect a student's academic marks), the more rigorous and defensible this process will need to be.