http://www.jiscinfonet.ac.uk/case-studies/tangible/nottingham/index_html1/view
Main issues
-the move towards increased online assessment could exacerbate any potential disadvantages for individuals with certain forms of disability.
- need to focus on the measurement of subject matter understanding rather than a students' ability to interact with a particular assessment format.
Requirements
- a system that accommodates different needs( depending on the disability)
- a system that can be used quickly
- and that works on any computer
Use of Touchstone which allows the customisation An extra table within the main system has been added; it has the advantage that accommodations can be 'distributed' to any client machine that the candidate sits at for an exam and is easy to centrally administer. A new simple interface has been added to the system to allow non-technical staff to easily set student preferences.
Users can change a number of different colours that are used: font size, background, foreground, headings, number of marks, and notes, time allowed.
Benefits
- system caters for several disabilities
- can be centrally controlled through the creation of personal profiles within the system
- Touchstone caters for students who cannot use a mouse
- the system calculates the extra time and display this to users
Disadvantages
- upfront development time to alter the CAA system to be more flexible
- IT support personnel must work closely together to ensure that students are made aware of what adjustments can be made, what would be most suitable for them, and then to ensure that any that are performed work as intended
Showing posts with label Case studies. Show all posts
Showing posts with label Case studies. Show all posts
Sunday, 22 February 2009
Saturday, 21 February 2009
Use of summative computer assisted assessment in Applied Technology and Finance
http://www.jiscinfonet.ac.uk/case-studies/tangible/leeds/index_html
Context
- module of 'Applied Technology and Finance' taken in semester one of level 1 by students on both tourism and events management courses
- 350 students
- many students are weak in numeracy and IT literacy
-change driven by staff
Problems/reasons to introduce new method
-paper tests took a considerable amount of time to mark and feedback to 350 students
- impossible to give the students multiple opportunities to receive feedback
- need to introduce students to the rigid deadlines demanded by the University
New method
- 5 stage tests worth 10% each and an end exam worth 50%
- Tests have both formative and summative aspects.
-Every two weeks starting at week three the students had to take a test (summative assessment) with the questions based on the activities of the previous two weeks along with a small number of the questions based on the supplementary reading and activities
- immediate results, feedback at the end of the week
- design implemented at the beginning by WebCT experts and later by all tutors
-students were thoroughly briefed before assessments and before the end exam
- students would get 20 questions presented randomly from a bank of 30 questions to avoid cheating. Also answers randomised in multiple choice questions.
- only plausible excuses accepted for missed tests
- WebCT Vista and Respondos were used; students were expected to know how to use thse tools for use in other modules
Benefits
- tutor time reduced to 20 hrs from 120 hrs (supervison, marking, standardisation..)
-learning objects are re-usable
--staff satisfaction
- possibility to use theis method to other modules
- marked improvement in attendance during weeks where tests took place
- results were sent to personal tutors immediately - possibility to follow up on any problems
- mean mark increase from 53% to 63%
- past questions for final exam posted online: those who did them gained 15% more
- students less apprehensive of taking a Finance subject since they get regular feedback on how they are doing.
- savings in costs and time
-students were engaged because of the small amounts of summative assessment spread over the module
Disadvantages
- initial cost and time to develop the learning objects
- new staff teaching on the module require an induction (general approach, how to develop questions, quality assurance)but skills are re-usable
- one type of feedback does not necessarily fit all students.
Conditions
- This approach works best in modules with large numbers of students where the initial outlay of time and effort will have a bigger pay off at a later date.
- important to have a few technologists among the tutors
- tutors need to modify their teaching to embed the new assessment method
Context
- module of 'Applied Technology and Finance' taken in semester one of level 1 by students on both tourism and events management courses
- 350 students
- many students are weak in numeracy and IT literacy
-change driven by staff
Problems/reasons to introduce new method
-paper tests took a considerable amount of time to mark and feedback to 350 students
- impossible to give the students multiple opportunities to receive feedback
- need to introduce students to the rigid deadlines demanded by the University
New method
- 5 stage tests worth 10% each and an end exam worth 50%
- Tests have both formative and summative aspects.
-Every two weeks starting at week three the students had to take a test (summative assessment) with the questions based on the activities of the previous two weeks along with a small number of the questions based on the supplementary reading and activities
- immediate results, feedback at the end of the week
- design implemented at the beginning by WebCT experts and later by all tutors
-students were thoroughly briefed before assessments and before the end exam
- students would get 20 questions presented randomly from a bank of 30 questions to avoid cheating. Also answers randomised in multiple choice questions.
- only plausible excuses accepted for missed tests
- WebCT Vista and Respondos were used; students were expected to know how to use thse tools for use in other modules
Benefits
- tutor time reduced to 20 hrs from 120 hrs (supervison, marking, standardisation..)
-learning objects are re-usable
--staff satisfaction
- possibility to use theis method to other modules
- marked improvement in attendance during weeks where tests took place
- results were sent to personal tutors immediately - possibility to follow up on any problems
- mean mark increase from 53% to 63%
- past questions for final exam posted online: those who did them gained 15% more
- students less apprehensive of taking a Finance subject since they get regular feedback on how they are doing.
- savings in costs and time
-students were engaged because of the small amounts of summative assessment spread over the module
Disadvantages
- initial cost and time to develop the learning objects
- new staff teaching on the module require an induction (general approach, how to develop questions, quality assurance)but skills are re-usable
- one type of feedback does not necessarily fit all students.
Conditions
- This approach works best in modules with large numbers of students where the initial outlay of time and effort will have a bigger pay off at a later date.
- important to have a few technologists among the tutors
- tutors need to modify their teaching to embed the new assessment method
Friday, 20 February 2009
Phased online summative assessment in undergraduate accounting
CAMEL (tangible benefits of e-learning) Case Study: University of Glamorgan, Phased online summative assessment in undergraduate accounting
http://www.jiscinfonet.ac.uk/case-studies/tangible/glamorgan/index_html
Context
-first year accounting students
Problem
- Students' poor performance
- lack of participation in the learning process
- superficial engagement in the topic area
Aims
- measure development
- identify area that need support
- provide timely feedback to promote self reflection
- use feedback to promote learning
End of course assessment
- an in-class time-constrained case study, weighted 30%,
- an end of year 3 hours' time-constrained examination.
New online assessment method in addition to end of course texts
- phased: 5 texts from October to March
- 60 questions per test: multiple-choice, multiple-response, true/false, yes/no and text match questions
- Feedback provided:score for the test, a summary of the questions asked and the student answers to those questions and an explanation of the incorrect answers.
- 50 minutes, supervised, several sittings due to the size of the labs.
-questions shuffled to prevent cheating
Benefits
- improved performance at end of course assessment
- students liked to be tested on small areas of the syllabus rather than one test covering many topics
- students thought that this assessment encouraged them to allocate their study time appropriately throughout the year
- students thought that testing gave opportunities for self-assessment, evaluation and reflection at an early stage and throughout the year
- it allowed them to get timely feedback
- it helped them get organised and do a bit at a time/easier to revise at the end
- after the initial effort to create the materials, tutors have no extra workload
- incerased motivation and time management
- possibility to diagnose problems early and give support
- increased student learning and participation in the learning process.
My comments
- It is not clear if the marks count towards the final mark
- it is not necessary to use the technology for a phased assessment approach
- the technology makes the teachers' work easier
- the technology allows the quality of materials and feedback to be the same for all students
- even in accounting the use of multiple choice questions is not sufficient to test understanding
Formative e-assessment in economics
Formative e-assessment in Economics
http://www.jiscinfonet.ac.uk/case-studies/tangible/derby/index_html/view
Problems
- high failure rates because:
SS haven't studied economics before and lack the necessary skills for model
building approach
SS have inadequate self-regulated learning skills in first year
Lectures don't promote self-regulating skills
- subject requires regular practice and development of knowledge and skills over short time span
- low retention rate
Solution
'... to reduce failure rates amongst full-time students through improving self-regulating learning through computer-based formative private study.'
Challenges
- need to use computers with a lot of memory: programme accessible from univ. only.
- part time and distance SS would not easily access the materials
- many SS never consider the online element: advertising was needed
- many SS would use the formative materials instead of going to lectures
What was included in the materials:
- extensive feedback loops( errors generate more explanations+ examples+ return )
- interactive learning activities and interactive formative assessment
- conceptual knowledge
- automated advice and activities for improvement,
- multiple attempts,
- staged development and continuous formative grades
Method
- simulations, drag and pull diagrams, recognition exercises,
- calculation activities,
- and concept identification activities,
Activities were linked to lectures (before as preparation and after as reinforcement and additional learning), other materials and summative assessment. They kept the student active and provide continuous feedback whilst building knowledge and skills.
Monitoring
Student engagement was monitored (number of times they accessed, the time spent at each access, the level of progression at each access and a summary score) Inactive students were contacted and offered a clinic.
Advantages
- increase in mean scores for all assessments relative to both 2003 and 2004
- rise in the top end performance at A and B grades and a fall in the failure rates in the final assessment
- positive informal feedback (materials help understand and apply the information and prepares for the exam) and formal feedback (incerased interest in topic, gave them confidence)
- reduced drop out rate
- no negative effect on attendance
- weak students spotted and helped early
- supported students whose first language is not English
- promoted staff development
- e-learning was adopted by other department as a result
-the formative materials lead directly to a new computer-based final assessment
- reduction in marking time; more time for materials development and monitoring
Disadvantages
- materials development, error resolution, evaluation time consuming
- possible only with a very good IT support team and large number of students
- materials need continuous development
- coursework results not affected
The materials now form the central component of the teaching and learning strategy for first year economics and data analysis within the business school.
Notes/lessons learned
- changes in self-regulated learning were not seen (inappropriately measures) yet these were the real objectives!
-SS must see a direct and obvious benefit or they will not use the materials
- essential to discuss benefits/ dissemination in class
http://www.jiscinfonet.ac.uk/case-studies/tangible/derby/index_html/view
Problems
- high failure rates because:
SS haven't studied economics before and lack the necessary skills for model
building approach
SS have inadequate self-regulated learning skills in first year
Lectures don't promote self-regulating skills
- subject requires regular practice and development of knowledge and skills over short time span
- low retention rate
Solution
'... to reduce failure rates amongst full-time students through improving self-regulating learning through computer-based formative private study.'
Challenges
- need to use computers with a lot of memory: programme accessible from univ. only.
- part time and distance SS would not easily access the materials
- many SS never consider the online element: advertising was needed
- many SS would use the formative materials instead of going to lectures
What was included in the materials:
- extensive feedback loops( errors generate more explanations+ examples+ return )
- interactive learning activities and interactive formative assessment
- conceptual knowledge
- automated advice and activities for improvement,
- multiple attempts,
- staged development and continuous formative grades
Method
- simulations, drag and pull diagrams, recognition exercises,
- calculation activities,
- and concept identification activities,
Activities were linked to lectures (before as preparation and after as reinforcement and additional learning), other materials and summative assessment. They kept the student active and provide continuous feedback whilst building knowledge and skills.
Monitoring
Student engagement was monitored (number of times they accessed, the time spent at each access, the level of progression at each access and a summary score) Inactive students were contacted and offered a clinic.
Advantages
- increase in mean scores for all assessments relative to both 2003 and 2004
- rise in the top end performance at A and B grades and a fall in the failure rates in the final assessment
- positive informal feedback (materials help understand and apply the information and prepares for the exam) and formal feedback (incerased interest in topic, gave them confidence)
- reduced drop out rate
- no negative effect on attendance
- weak students spotted and helped early
- supported students whose first language is not English
- promoted staff development
- e-learning was adopted by other department as a result
-the formative materials lead directly to a new computer-based final assessment
- reduction in marking time; more time for materials development and monitoring
Disadvantages
- materials development, error resolution, evaluation time consuming
- possible only with a very good IT support team and large number of students
- materials need continuous development
- coursework results not affected
The materials now form the central component of the teaching and learning strategy for first year economics and data analysis within the business school.
Notes/lessons learned
- changes in self-regulated learning were not seen (inappropriately measures) yet these were the real objectives!
-SS must see a direct and obvious benefit or they will not use the materials
- essential to discuss benefits/ dissemination in class
Subscribe to:
Posts (Atom)
