Category Archives: Assessment

When It’s Really NOT Good to Know

No Good to KnowIn my current position as someone who handles data for the school division, I often get asked to create tools, run analyses, develop theories, or present recommendations on matters concerning student performance. The thrust of my work is to help decision makers (teachers, administrators, etc.) make better decisions in their practice through reflection on evidence.  At the end of the day, it’s about improving student learning.

Usually, requests for help appear to link quite strongly to students’ wellbeing in school, such as their academic achievement, behaviour, attendance, or self-efficacy. Sometimes, requests are a little further removed, but still have tangible links, such as an HR project on a representative workforce.

I typically ask the purpose of the project. Sometime, colleagues want to see how well their students are reading.  Other times, an administrator wants to know if resources are being allocated appropriately.  Maybe a school team is trying to see if a targeted intervention is having the desired effect.  All these are great things to know.

I’ve also started asking why we want to collect this data. Occasionally, the answer I get is that “it would be good to know.”  As much as I am dedicated to helping colleagues (in fact, that is my primary role), I feel that this is the absolute worst reason to collect and analyze data.  In the immortal words of that great philosopher G.I. Joe, “Knowing is half the battle.”  I should say, knowing is only half the battle, and it’s also the most dangerous.

Why? Let’s take the example of students reading.  What happens when we put in the time to assess, graph, statistically analyze, and interpret the data on student reading levels in a classroom, and it turns out that most students are struggling?  It is not merely ‘good to know’ that students are struggling; knowing that students are struggling is only half the battle.  We are now obligated to do something about it.

I use the word obligation here very deliberately. As teachers, we are expected to help our students improve, and more than just a little bit (since small performance improvements over the course of one school year are more likely associated with normal effects of aging).  So if we know that our students are struggling, then we must be prepared to support, to intervene, to adjust, to strategize, to seek additional help.  On the flip side, what does it say if you know you students are struggling and aren’t prepared to do make any changes?

Herein lies the danger of knowing. Either you must be prepared to change something that you thought was working well, or you must be prepared to say that you refuse to help.  Knowing is only half the battle.  It’s not just good to know unless you are ready to do as well.

So to that end, I’ve decided to return to why I’m here in this role.  When a colleague asks me for data, I’ll first do them a favour: I’ll ask them how it will be used to help improve student learning.  After all, I want to help them, not expose them to danger for which they are unprepared.  That, I think, is good to know.

February 23, 2015Permalink Leave a comment

School IS a Student’s Real World

runningI like to think I live in the real world even though I am an educator and for years my students thought I lived, slept and ate in the school. Students would stare at me when I was in public and for the longest time I would check for the roll of toilet paper stuck to the bottom of my shoe, or look in the mirror to see if I still had a milk mustache (or something worse), or was I in one of those dreams where I forgot to get dressed and was still in my pyjamas? And then it dawned on me that they were used to seeing me in their world at school and the world at Wal Mart was a seemingly different place. Many people say that in school we need to present real world problems for students to solve and I for a long time I went along with it. But then I heard Tom Hierck at SELU say that “school is a student’s real world”.

In education I hear people say that school does not reflect the real world and with the latest spotlight on assessment, I really hear the complaints that our current practices do not reflect the real world nor prepare children for the real world. And that makes me think about the real world I live in.

I enjoy running. Most of the time. Sometimes I simply enjoy the scenery, the weather, or I feel grateful that I am able to run where I want, when I want. That I am capable of running. And sometimes I think. What I like the most about running is allowing my thoughts to run wild as I just try to put one foot in front of the other even if it is at a turtle’s pace.

One day this summer I was thinking about running while I was running. I was preparing for a run in Banff. Each time I went out for a run I would set a new mini goal or challenge. Sometimes the goal was to run a certain distance, sometimes my goal was to run at a certain pace, sometimes my goal was to run hills and sometimes my goal was simply to get out there and move.

All of these mini goals or challenges could be measured in different ways. This made me think about education. Each year we set broad goals for students and challenges are created to test and prepare for those goals along the way.

I measure my running progress in different ways and there are different benefits to achieving my goals. I set smaller goals just as a teacher may scaffold learning. Some days I can easily accomplish my challenges when my body is rested and sometimes I am unable to achieve my challenges when the weather is windy or too hot or the goal I have set is beyond my zone of proximal development. When I achieve a goal I feel good about myself and I am ready to continue to challenge myself. When I feel good about myself I feel like I can conquer anything I set my mind to. I think about our students who achieve and feel confident to continue to challenge themselves. When I run I am more cognizant of my habits and I drink more water and eat healthier foods. Why would I work so hard exercising and ruin the benefits by eating junk food? When our students experience success and achievement in one area, they are more apt to be ready to learn in other areas. When I don’t feel successful I try something else or try what I was doing a different way and to be honest, sometimes I quit. I purchased an app which was supposed to improve my pace. The voice told me to run faster. I couldn’t run faster; I didn’t know how to run faster so I quit for awhile because I didn’t know how to improve and I felt like a failure. Then I downloaded an app that tracks my data; the app tracks the time I run, the distance, the elevation, the number of work-outs per week, it allows me to set goals and tracks those goals and every once in awhile I get an e-mail certificate notifying me an accomplishment.

In terms of measuring my progress, there are many ways I can determine my progress. One of my goals for running is weight loss so of course; I can set goals and measure my progress with the scale. But that is not the only way I can measure my progress. I can also self-assess by asking myself how I feel (health/unhealthy), seeing how my clothes fit (tight, loose, just right), taking measurements and collecting data on my speed and distance when I run. Triangulation of data – the same reason we don’t simply use one piece of assessment evidence to determine student learning, I can’t only use the scale to determine my progress.

Although I am an adult and in more control of my life than many of our students, it is still up to me to get myself off the couch and out running. I don’t get a 0 or feel like a failure if I miss a run. If I miss some runs I may not progress as quickly as I might like. If I miss a lot of runs, I won’t be ready for the road race and I will feel foolish. I know this because I have been unprepared for races and I have learned that I need to train and prepare for road races. There will always be people around me that will run faster and farther regardless of how much I train and that is why I have to set goals for myself. There are people that will challenge me and whom I will provide a challenge. A couple of years ago my friends and I were running in Banff and we all agreed to wear our Rider jerseys. Shortly after the start line we split up to run at our own paces. Near the finish line a man ran up beside me and said he had been trying to catch up all race because he didn’t want a Rider fan to beat him. I had no idea I was providing a challenge for this man and I had no idea I would be trying to beat a stranger to the finish line but a little healthy competition provided me an opportunity to push myself further than I thought I could. (And yes, I did beat him across the finish line!)

Running is a part of my real world. I set goals for myself, I assess my progress of those goals regularly and I respond to the data to continue to make improvements. In a classroom, we have outcomes as broad goals for students. We chunk or break down those goals into smaller achievable pieces and provide opportunities and experiences for student learning, understanding and growth. Sometimes students are unable to meet those goals and we need to respond to those needs by possibly changing our instructional strategies, teaching the material in a different way or using different tools. We find different ways to measure the goals of our students and use various pieces of evidence to determine student learning in relation to the goals that have been set. We know that when we provide meaningful feedback to kids that they can see what they have to do to improve and when we don’t provide feedback students disengage in their own learning and progress. When students achieve a goal, we need to celebrate and that celebration can simply be the internal feeling of accomplishing that goal (I climbed that 5km mountain in Banff and finished the 10k run) or having someone share in that celebration (a teacher commenting on a job well done) and sometimes that extrinsic certificate along the way gives some required prompting to continue along the journey to achieving a personal goal.

math is real lifeWhat do you think? Do you live in the real world? What kinds of goals do you set for yourself? How do you measure your goals? How do you respond to your successes and failures?

Is school a real world?

October 9, 2014Permalink Leave a comment

Mastery: Promoting Fires in Students’ Bellies

by Cathy Herrick

The idea of Mastery has been around for centuries, in many fields, and has many definitions.  In education writers, researchers, administrators, and teachers have their own pre-determined ideas as to what constitutes mastery.  A review of some of the most recent thinking on this topic reveals that there is no one common definition, nor a “right” definition, but rather some agreement as to what constitutes mastery and what does not constitute mastery relative to student learning and achievement.

Mastery, Mastery Learning, and a Mastery designation on a rubric all have different connotations.  Mastery is defined in the Oxford dictionary as, “an action demonstrating or involving great skill or power…. to perform a notable deed, or wonderful feat.”  Benjamin Bloom describes the concept of mastery learning as a possible outcome for any student who was provided adequate time and “favourable learning conditions” to perform and be assessed on a criterion-based scale determined by a teacher (Bloom,1977).  As a school division we are working to establish some common criteria to assess deeper levels of understanding of curricular outcomes, which we have labeled as Mastery on a rubric.

Teachers, administrators, coaches, consultants, and superintendents in our school division are exploring, collaborating, and doing some deep thinking about what Mastery as a concept means relative to student achievement.  Challenges to develop criteria, learning tasks, and exemplars are initiating some consternation and uncertainty for teachers when facing pressures from students and parents who are used to a point system to measure success.  However, this inquiry, discussion, experimentation, and reflection are all necessary components to understanding and developing the ways and means of ensuring that students have access to authentic learning activities that build the “fire in their bellies” to learn and to continue to learn.

Engaging students in learning has a fundamental tie to the Mastery level. Students demonstrating a desire to work hard, to think strategically, and to accept some ambiguity also clearly know themselves well as learners. Tomlinson worries about college students who have no experience of the “fire in their belly” to drive them to do hard thinking, to read, to debate ideas,  and to push themselves toward excellence in a real world pursuit” (Tomlinson, 2013 ). She worries that students don’t know how to reason or to think abstractly because they have learned to “jump through the hoops” of schooling, doing what is necessary to get the points.

At the Mastery level on a rubric we want to promote:

  • Students accomplishing novel challenges through the application of skills and understandings
  • Students “effective transfer of learning done with creativity, polish, and grace” with an understanding of what has been done and why (Wiggins, 2013 p. 13)
  • A student’s desire to get better at something that matters to them
  • Authentic tasks at the heart of “doing” the subject

We have taken a risk, developing rubrics that challenge students to demonstrate mastery of an outcome.  We are modeling the “messiness”, the risk-taking, the inquiry, the struggles, and the tenacity that we expect of our students at their highest levels.  We continue to learn; we have a “fire in our belly” to “get it right”, to assist our students in achieving the highest levels possible.

Though we can’t define or measure “mastery” only as an attitude/behaviour, nor can we completely ignore it as we observe students in their pursuit of Level 4 – Mastery.  Our Olympic athletes have provided us with some insight these past few weeks as they achieve mastery levels in their sport.  Not only have they learned the fundamental concepts and skills of their sport, but they have applied them and their understandings in races, in games, and during performances.  Every athlete has a fire in their belly to succeed. These athletes have not attained a level of Mastery focusing only on bits of knowledge, isolated drills, or by not knowing what excellence looks like.  Thus, on our rubrics we want to ensure that Mastery does not look like:

  • A march through facts and sub skills, dotted with quizzes
  • A high grade on a test/quiz of low level questions
  • A simple recall of facts and skills
  • Proficiency with drills and exams
  • Local norms and low expectations, nor
  • Arbitrary calculations

Forging ahead as educators we must not be satisfied with learning that is just ‘good enough’. We chose this profession because we love to see the ‘light bulbs’ turn on, the excitement in the eyes of learners who make new discoveries, and we relish the moments when tough questions with no answers get asked and debated.  Developing classroom climates and practices that inspire kids to achieve mastery levels of learning involves:

  • Students believing that the learning task matters,
  • Regular inquiry, problem solving, and innovation,
  • Timely and descriptive feedback (including re-assessments),
  • Fostering intrinsic motivation rather than pressure to get the grade,
  • Self-comparisons rather than comparisons to others,
  • Environments where students can safely take risks and believe that they can succeed, and
  • Planning that begins with the end in mind.

Gusky (2013) promotes that Mastery can only be measured in light of a world standard, that students deserve the opportunity to know where the bar is.  An Olympic athlete knows, or has an idea, of what Mastery looks like in his/her sport.   While, we in education may not be all the way there yet, the goal is worthy of the pursuit, which begins as a school division, extends to the province, the nation, and beyond.

References

Cushman, K. (2013). Minds on fire. Educational Leadership, 71(4), 38-43.

Gusky, T., Anderson E. (2013). In search of a useful definition of mastery. Educational Leadership,
71(4), 19-23.

Tomlinson, C. (2013). Let’s not dilute mastery. Educational Leadership, 71(4), 88-89.

Tucker, C. (2013). Five musts for mastery. Educational Leadership, 71(4), 57-60.

Wiggins, G. (2013). How good is good enough? Educational Leadership, 71(4), 10-16.

February 26, 2014Permalink

Formative Assessment

As a teacher I used formative assessment on a daily basis before I knew it was called formative assessment.  I recorded some of the formative data I collected and reported it to parents on report cards.  I also included student behaviors in report card marks such as completed/incomplete work, attendance and participation.  Sometimes the feedback I provided my students was irrelevant and provided little, if any opportunity for growth.  I could beat myself up about it or I could learn from my experiences and apply new understandings to help improve student learning while also improving my teaching practice.

Often times the education system is criticized for not being relevant to the real world.  During my daughter’s basketball practice I started to see how formative assessment is related.  Players have an opportunity to practice specific skills while coaches provide feedback for improvement.   Eventually individual skills are combined to prepare for application in a game situation.  My daughter played her first game on Saturday.  I saw this as a summative experience because the players demonstrated the skills learned during practice.  The coaches could still provide feedback and make notes about what to work on at the next practice.

Then I started to think about swim club.   Again, my children attended practice to learn skills and set goals.  Coaches provided timely and specific feedback on various skills.  At the first swim meet the goal was for my child to make it to the end of the pool without touching the bottom.  After a race, the coach would know what to work on at the next practice to improve upon personal goals.  Eventually the goal was working towards standard times set for swimmers.  There were some meets which my husband was unable to attend but he still wanted to know how our children were doing.  When I let him know that our child beat a personal best or were three tenths of a second from an ‘A’ time or made an efficient flip turn, he was able to understand how she was swimming.  When I shared my opinion and told him that she looked so cute on the starting block with bug-eye goggles, huge smile and proud waves to the crowd, I didn’t really share how she was doing.  I can’t imagine how anyone could disagree with my opinion but it was irrelevant to how she swam her races.  Just as my husband wanted specific information according to the goal, parents of the students in our classes need information on how their children perform according to the outcome.  This shared information is summative.  My husband and I didn’t require daily practice updates – formative assessment – but if our children were not improving from one swim meet to the next, the coach might have had a discussion with us for an improvement plan that we could be aware of and support. However, the implementation and use of formative assessment was critical to the process.

Sports and school aren’t the only places where formative assessment is practiced.  A driver’s license is earned after a test but not before practice and feedback from driving instructors and parents (sometimes positive, sometimes not!).  After completing a written test, all drivers are subject to a road test.  Not all drivers pass the first road test.  The road test results are not averaged.  When a driver passes the road test, they earn a license regardless of how many road tests were required.

As teachers we also receive feedback from our students, colleagues and parents.  Some feedback is requested, some is positive and some is negative.  When I think about the positive feedback I received from parents, rarely was it specific.  “Thank you for being a good teacher.”  What does that really mean?  What do parents think I am good at?  When I received negative feedback from parents and colleagues it really stung and I tended to dwell on it but after much reflection I was able to learn from that feedback because often times the negative feedback was more specific.  Although our students may not have the finesse or skills to critically share specific feedback, non-verbal communication can also provide feedback from our students as to how we are meeting student needs.

Just as it is difficult to hear negative feedback, it is difficult to share feedback sometimes.  It may be difficult to find something a student has done well on an assignment just as it may be difficult to suggest something for improvement.  I know that I need to practice providing feedback to students and right now it feels and sounds somewhat robotic but with practice, I know that I will improve on providing specific, timely feedback to students.

Although formative assessment is not always formally recorded in real life, in real life we practice, we perform, and we learn from our mistakes.  Mistakes can be realized internally and mistakes can be shared through feedback.  Teaching is a practice and I know that I require practice to help students achieve their learning goals.  I also know that what I do and understand today will change but that today I am doing the best I can with what I know and understand.

October 28, 2013Permalink

Student Generated Comics Using the Comic Life App

by Donna DesRoches

Recently I have had the pleasure of working with groups of students using the iPads and an app called ComicLife. I watched theirComicLife App Icon enthusiasm and excitement while first creating their story, mapping out a short storyboard and then creating the comic.

In the article, Using Student Generated Comic Books in the Classroom, the author believes that because kids are familiar and comfortable with comic books several benefits emerge when they create their own comics particularly in the areas of reading, writing and research skills.

Designing a comic book provides an opportunity for students to be creative in the presentation of their writing.  It also allows them to apply and demonstrate their knowledge in an innovative and imaginative way:

  • Through comics students can investigate the use of dialogue, succinct and dramatic vocabulary and nonverbal communication.
  • Designing comic books can generate expository composition including historical and biographical writing
  • The creative process allows students to determine what is most important from their reading, to rephrase it succinctly, and then to organize it logically.

There is a natural fit for creating comics and using the app ComicLife within the grade 9 ELA curriculum but it can be used as a form of summative assessment across all subject areas.  The article, Using Comic Life at Every Level of Bloom’s, provides some very specific examples of how the ComicLife App can be integrated into a variety of subject areas including math.

Quality projects using ComicLife occur when students are well prepared both in content knowledge and provided with a clearly designed process.

Some basic steps enable students to use ComicLife to its full advantage:

  1. Provide a number of comics or graphic novels to students and have them analyze them for comic book elements:
    1. Flow of images through panels or frames
    2. Borders and gutters
    3. Captions (voice of the narrator)
    4. Speech and thought balloons or bubbles
    5. Tone shown through shape, bold or italics
    6. Use of symbols to represent concepts or ideas
    7. Sound effects represented by words
    8. Provide students with time to play with and explore the ComicLife app
      1. Select templates
      2. Menu buttons to select and change
      • Font styles
      • Page layouts
      • Insert and delete pages
      • Sharing options
    9. Access the camera
    10. Access the photo gallery
  2. Provide students with a template or process for creating a rough draft or storyboard of the comic.  I usually have students draw panels on a blank sheet of paper. Another way to storyboard is the use of cards as illustrated in this post.  Students can then manipulate the sequence of the cards to create the best story flow. Often they need to be encouraged to keep their drawings simple – stick figures are best – and to focus on the dialogue.  They should also know that once they begin creating their comic some points in the storyboard will and should change.
  3. Once students have completed their storyboard they can begin to create their comic.  I am always surprised at how important the storyboard is to students as they refer to it often in the midst of the creative process.
  4. The finished comics can be printed.  They can also be shared with their classmates via an Apple TV.
October 21, 2013Permalink