Comments on: An empirical test of blogging in the classroom http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/ transforming academic communities with new tools of the social web Sat, 04 Feb 2012 03:31:52 +0000 http://wordpress.org/?v=2.2.3 By: Nicole Ellison http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-106 Nicole Ellison Mon, 10 Apr 2006 19:48:34 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-106 Hi Vicki, Thanks for your questions. I’ve tried to answer them thoroughly but let me know if you have additional points of clarification. 1) It sounds like the blog assignments were mirror images of paper assignments? Is this correct? - The writing prompts were as similar as possible – although each one dealt with a different topic and set of readings. The only difference between the paper and the blog without comments was the way in which they delivered their work to the instructor. For the blogging with comments condition, they had to provide two comments to others in their group for full credit. 2) Did you allow students to use first person in the blog assignments and papers? - Yes, many of the prompts asked explicitly for their thoughts and opinions, so first person was appropriate. 3) How did the professor grade and give feedback on the blog assignment versus the paper? - This was a large course, so feedback was not as robust as it would be in a class with lower enrollment. In general they were given full credit for assignments that met the criteria listed in the syllabus: * Most importantly, does the written work address all aspects of the assignment? * Is the writing free from glaring typographic, grammar, and spelling mistakes? Was care taken with word choice, sentence construction, and argument presentation? * Does the writing assignment (WA) demonstrate a familiarity with and understanding of the readings? Are specific points and arguments from the readings and lectures incorporated into the paper? * Does the WA make a thoughtful, meaningful and earnest contribution to understanding the material? * Does the WA properly credit ideas and phrases contributed by others? * When appropriate, does the WA reference ideas and information discovered by doing outside research? 4) Do you think that Assignment #5 showed far lower quiz and participation scores because it was around Thanksgiving break? (I recall in college that was a time many people just didn’t do their work!) - WA5 was due Nov 22, so that is a possible explanation. We will do further analysis to see whether relationships we found hold true if this WA is excluded. 5) How did students find one another’s blogs? Was there a master listing or wiki? - For the commenting WAs, I formed students into groups of 7 or 8. Using our CMS, ANGEL, they were able to post their URLs to their group members. 6) I see that the survey was optional, would this weed out “unmotivated” students? - Potentially – although they reported their GPAs and there was a wide range. Students received extra credit for the survey, and participation was fairly even across surveys. 7) There are typically students who don’t turn in assignments, did you see any variation in the % of students turning in assignments for each assignment? (More students could have done #5 which then brought the score down?) - The N was pretty stable around 50 (out of 68 total students) for each survey. Specifically: WA1 (n=52), WA2 (n=50), 3 (n=49), 4 (n=49), 5 (n=50), WA6 (n=51). In the fall, I am going to repeat the study but in a way that will generate more useful data (ie random assignment). I also plan to do more with commenting and to aggregate responses in a class blog. The study I describe here is just one step and I look forward to collecting more data – and to see what other research is being done in the area! Thank you, Nicole Hi Vicki,

Thanks for your questions. I’ve tried to answer them thoroughly but let me know if you have additional points of clarification.

1) It sounds like the blog assignments were mirror images of paper assignments? Is this correct?
- The writing prompts were as similar as possible – although each one dealt with a different topic and set of readings. The only difference between the paper and the blog without comments was the way in which they delivered their work to the instructor. For the blogging with comments condition, they had to provide two comments to others in their group for full credit.

2) Did you allow students to use first person in the blog assignments and papers?
- Yes, many of the prompts asked explicitly for their thoughts and opinions, so first person was appropriate.

3) How did the professor grade and give feedback on the blog assignment versus the paper?
- This was a large course, so feedback was not as robust as it would be in a class with lower enrollment. In general they were given full credit for assignments that met the criteria listed in the syllabus:
* Most importantly, does the written work address all aspects of the assignment?
* Is the writing free from glaring typographic, grammar, and spelling mistakes? Was care taken with word choice, sentence construction, and argument presentation?
* Does the writing assignment (WA) demonstrate a familiarity with and understanding of the readings? Are specific points and arguments from the readings and lectures incorporated into the paper?
* Does the WA make a thoughtful, meaningful and earnest contribution to understanding the material?
* Does the WA properly credit ideas and phrases contributed by others?
* When appropriate, does the WA reference ideas and information discovered by doing outside research?

4) Do you think that Assignment #5 showed far lower quiz and participation scores because it was around Thanksgiving break? (I recall in college that was a time many people just didn’t do their work!)
- WA5 was due Nov 22, so that is a possible explanation. We will do further analysis to see whether relationships we found hold true if this WA is excluded.

5) How did students find one another’s blogs? Was there a master listing or wiki?
- For the commenting WAs, I formed students into groups of 7 or 8. Using our CMS, ANGEL, they were able to post their URLs to their group members.

6) I see that the survey was optional, would this weed out “unmotivated” students?
- Potentially – although they reported their GPAs and there was a wide range. Students received extra credit for the survey, and participation was fairly even across surveys.

7) There are typically students who don’t turn in assignments, did you see any variation in the % of students turning in assignments for each assignment? (More students could have done #5 which then brought the score down?)
- The N was pretty stable around 50 (out of 68 total students) for each survey. Specifically: WA1 (n=52), WA2 (n=50), 3 (n=49), 4 (n=49), 5 (n=50), WA6 (n=51).

In the fall, I am going to repeat the study but in a way that will generate more useful data (ie random assignment). I also plan to do more with commenting and to aggregate responses in a class blog. The study I describe here is just one step and I look forward to collecting more data – and to see what other research is being done in the area!
Thank you, Nicole

]]> By: Nicole Ellison http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-97 Nicole Ellison Sun, 09 Apr 2006 00:25:25 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-97 Hi, Rowland, and thanks for the insights into commenting. I agree that peer review is a powerful tool and that this particular assignment (in the study described) may not have capitalized on the potential benefits of peer review and evaluation. Although I'm still analyzing, my sense isn't that comments had a negative impact on learning goals per se, but rather that time on task may have been affected by students' knowledge that they'd be commenting. If this is the case and commenting is done in such a way that no additional benefits accrue, then the end result is negative. We didn't provide training on constructive criticism, nor did we give students guidelines about what their comments should or shouldn't do. Next semester I will remedy this. A common theme from my focus group data is that in situations where students aren't given many guidelines about how to provide feedback, often they resort to bland "I agree" or "that's interesting" comments because they don't want to "hurt someone's feelings." Obviously the benefit of such feedback – for both giver and receiver- is limited. It sounds like you've had success using blogs to increase student understanding of the material, and I'd love to hear details about how you incorporating peer commenting into your course design. Thanks, Nicole Hi, Rowland, and thanks for the insights into commenting. I agree that peer review is a powerful tool and that this particular assignment (in the study described) may not have capitalized on the potential benefits of peer review and evaluation. Although I’m still analyzing, my sense isn’t that comments had a negative impact on learning goals per se, but rather that time on task may have been affected by students’ knowledge that they’d be commenting. If this is the case and commenting is done in such a way that no additional benefits accrue, then the end result is negative. We didn’t provide training on constructive criticism, nor did we give students guidelines about what their comments should or shouldn’t do. Next semester I will remedy this. A common theme from my focus group data is that in situations where students aren’t given many guidelines about how to provide feedback, often they resort to bland “I agree” or “that’s interesting” comments because they don’t want to “hurt someone’s feelings.” Obviously the benefit of such feedback – for both giver and receiver- is limited.
It sounds like you’ve had success using blogs to increase student understanding of the material, and I’d love to hear details about how you incorporating peer commenting into your course design. Thanks, Nicole

]]> By: Nicole Ellison http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-94 Nicole Ellison Sat, 08 Apr 2006 20:04:16 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-94 Reply to Bob Hughes' comments: Hi Bob, Thanks for your note. I see Hua has posted the references - let me know if you need PDFs. Regarding your comment about the structure of the blogs: Yes, each student created their own blog using Blogger or another tool of their choice. I created groups of 7 or so students for the commenting assignment. Aggregating the blogs would have made it easier to manage, I agree, although in focus groups I've conducted with students, sometimes this backfires in that older posts get pushed off the front page and therefore become "invisible." The students point out that right before the assignment is due, there is a flurry of activity. In a large class (>100 students), ironically, this meant that the postings of students who waited until the last minute received more attention from their peers in comparison with the postings of students who completed the assignment a few days before it was due. In my case, in the future, I plan to use another system which aggregates the posts, while keeping the small group structure since I teach a large class and do not want the students to be overwhelmed (a common refrain in the focus groups). Reply to Bob Hughes’ comments:
Hi Bob, Thanks for your note. I see Hua has posted the references - let me know if you need PDFs. Regarding your comment about the structure of the blogs: Yes, each student created their own blog using Blogger or another tool of their choice. I created groups of 7 or so students for the commenting assignment. Aggregating the blogs would have made it easier to manage, I agree, although in focus groups I’ve conducted with students, sometimes this backfires in that older posts get pushed off the front page and therefore become “invisible.” The students point out that right before the assignment is due, there is a flurry of activity. In a large class (>100 students), ironically, this meant that the postings of students who waited until the last minute received more attention from their peers in comparison with the postings of students who completed the assignment a few days before it was due. In my case, in the future, I plan to use another system which aggregates the posts, while keeping the small group structure since I teach a large class and do not want the students to be overwhelmed (a common refrain in the focus groups).

]]> By: Vicki Davis http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-92 Vicki Davis Sat, 08 Apr 2006 13:02:04 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-92 Mrs. Ellison -- this was an excellent presentation and I enjoyed it greatly. I too have some thoughts but was hoping that you could answer some of these questions for clarification purposes for me as I put my thoughts together for a blog entry. We need more meaningful empirical research and this is a beginning. 1) It sounds like the blog assignments were mirror images of paper assignments? Is this correct? 2) Did you allow students to use first person in the blog assignments and papers? 3) How did the professor grade and give feedback on the blog assignment versus the paper? 4) Do you think that Assignment #5 showed far lower quiz and participation scores because it was around Thanksgiving break? (I recall in college that was a time many people just didn't do their work!) 5) How did students find one another's blogs? Was there a master listing or wiki? 6) I see that the survey was optional, would this weed out "unmotivated" students? 7) There are typically students who don't turn in assignments, did you see any variation in the % of students turning in assignments for each assignment? (More students could have done #5 which then brought the score down?) Mrs. Ellison — this was an excellent presentation and I enjoyed it greatly. I too have some thoughts but was hoping that you could answer some of these questions for clarification purposes for me as I put my thoughts together for a blog entry.

We need more meaningful empirical research and this is a beginning.

1) It sounds like the blog assignments were mirror images of paper assignments? Is this correct?

2) Did you allow students to use first person in the blog assignments and papers?

3) How did the professor grade and give feedback on the blog assignment versus the paper?

4) Do you think that Assignment #5 showed far lower quiz and participation scores because it was around Thanksgiving break? (I recall in college that was a time many people just didn’t do their work!)

5) How did students find one another’s blogs? Was there a master listing or wiki?

6) I see that the survey was optional, would this weed out “unmotivated” students?

7) There are typically students who don’t turn in assignments, did you see any variation in the % of students turning in assignments for each assignment? (More students could have done #5 which then brought the score down?)

]]> By: Yuehua Wu http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-86 Yuehua Wu Fri, 07 Apr 2006 16:29:47 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-86 Hi, Bob, Here is the list of references. Du, H. S. and C. Wagner (2005). Learning with weblogs: An empirical investigation. the 38th Hawaii International Conference on System Sciences. Glogoff, S. (2005). "Instructional Blogging: Promoting interactivity, student-centered learning, and peer Input." Retrieved March 25, 2006, from http://www.innovateonline.info/index.php?view=article&id=126. Oravec, J. A. (2002). "Bookmarking the world: weblog applications in education; weblogs can be used in classrooms to enhance literacy and critical thinking skills." Journal of Adolescent & Adult Literacy Retrieved May 8, 2003, from Expanded Academic ASAP database. Reinhart, J. M., A. L. Whicker, et al. (2005). "News Blogs in Distance Education Programs." Distance Learning 2(5): 23-28. Richardson, W. (2004). "Metablognition." Weblogg-Ed. http://www.weblogg-ed.com/2004/04/27 Tryon, C. (2006). "Writing and Citizenship: Using blogs to teach first-year composition." Pedagogy 6(1): 128-132. Hi, Bob,
Here is the list of references.

Du, H. S. and C. Wagner (2005). Learning with weblogs: An empirical investigation. the 38th Hawaii International Conference on System Sciences.

Glogoff, S. (2005). “Instructional Blogging: Promoting interactivity, student-centered learning, and peer Input.” Retrieved March 25, 2006, from http://www.innovateonline.info/index.php?view=article&id=126.

Oravec, J. A. (2002). “Bookmarking the world: weblog applications in education; weblogs can be used in classrooms to enhance literacy and critical thinking skills.” Journal of Adolescent & Adult Literacy Retrieved May 8, 2003, from Expanded Academic ASAP database.

Reinhart, J. M., A. L. Whicker, et al. (2005). “News Blogs in Distance Education Programs.” Distance Learning 2(5): 23-28.

Richardson, W. (2004). “Metablognition.” Weblogg-Ed. http://www.weblogg-ed.com/2004/04/27

Tryon, C. (2006). “Writing and Citizenship: Using blogs to teach first-year composition.” Pedagogy 6(1): 128-132.

]]> By: Rowland Gallop http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-67 Rowland Gallop Thu, 06 Apr 2006 10:15:20 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-67 Thanks for a very interesting presentation. As you say it is no surprise that time on task is the key variable as there is so much research to support this and it is one of the "Seven Principles". It seems to me that to make blogging comments a useful part of the learning there has to be some point in doing them as far as the learning is concerned. In other words, rather than just a requirement to comment, there needs to be a dialogue set up where students have a conversation based on developing a deeper understanding of the material. I would be interested in seeing what sort of comments the students made. In my experience, at least at first, you get comments like "nice work John" and it takes some time and practice building an on-line community before the comments become a useful dialogue. If the commenting does not explicitly add to the understanding then it is not surprising that it did not aid the students in their tests. Again it depends on the level of the interaction for which you are asking but I have found that a carefully sequenced set of blogging exercise based on specific criteria can aid learning provided the goal of the learning is to encourage the students to think deeply. If the goal is shallow learning then getting a meaningful interaction on a blog either does not happen or does not help the learning. I have found that getting students to use a blog to reflect on their own learning or to share ideas has been a useful tool in helping students to develop their understanding. I hope you find these comments helpful and thanks again for the presentation. Rowland Thanks for a very interesting presentation.

As you say it is no surprise that time on task is the key variable as there is so much research to support this and it is one of the “Seven Principles”.

It seems to me that to make blogging comments a useful part of the learning there has to be some point in doing them as far as the learning is concerned. In other words, rather than just a requirement to comment, there needs to be a dialogue set up where students have a conversation based on developing a deeper understanding of the material.

I would be interested in seeing what sort of comments the students made. In my experience, at least at first, you get comments like “nice work John” and it takes some time and practice building an on-line community before the comments become a useful dialogue.

If the commenting does not explicitly add to the understanding then it is not surprising that it did not aid the students in their tests.

Again it depends on the level of the interaction for which you are asking but I have found that a carefully sequenced set of blogging exercise based on specific criteria can aid learning provided the goal of the learning is to encourage the students to think deeply. If the goal is shallow learning then getting a meaningful interaction on a blog either does not happen or does not help the learning.

I have found that getting students to use a blog to reflect on their own learning or to share ideas has been a useful tool in helping students to develop their understanding.

I hope you find these comments helpful and thanks again for the presentation.

Rowland

]]> By: Bob Hughes http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-64 Bob Hughes Wed, 05 Apr 2006 23:07:59 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-64 You are to be congratulated on actually putting your hypotheses about the value of blogs to an empirical test. This is important for all of us. I have acomment and a request. comment: it sounds like from your presentation that the students' blogs were not all in the same place. Wouldn't it make more sense to have a common blog that everyone contributed or a wiki with multiple branches that students could work from. It seems like this would make the sheer management of the process easier for everyone. Am I missing somethign? Request: Could you post your complete referecences? Again thanks for helping us think about blogging. You are to be congratulated on actually putting your hypotheses about the value of blogs to an empirical test. This is important for all of us.

I have acomment and a request.

comment: it sounds like from your presentation that the students’ blogs were not all in the same place. Wouldn’t it make more sense to have a common blog that everyone contributed or a wiki with multiple branches that students could work from. It seems like this would make the sheer management of the process easier for everyone. Am I missing somethign?

Request: Could you post your complete referecences?

Again thanks for helping us think about blogging.

]]> By: Nicole Ellison http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-62 Nicole Ellison Wed, 05 Apr 2006 18:55:23 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-62 Hi Polly, Thanks for your kind words. Since recording the presentation, we’ve had a chance to dig a little deeper into the dataset and are beginning to believe that time on task is really the key. Time on task may be a mediating variable that explains a lot of the variance in the comprehension (quiz scores). So, it’s not that blogging in and of itself led to lowered scores (in some cases), but rather format -> time on task -> comprehension. One explanation is that students made a conscious trade-off about how they would spend their time because they anticipated blogging would take longer (in the commenting condition particularly) and therefore spent less time composing. Another explanation which we don’t discuss in the presentation is that students have not yet determined what their “blogging in school” writing voice should be. They have written short papers before, but most haven’t blogged as part of their coursework. They may be importing conceptual frameworks about what educational blogging should look like based on other kinds of informal online writing practices (i.e. IM, personal blogs) and therefore may adopt a more casual (and less time-consuming) written style when blogging for instructional purposes. I think this is an issue that instructors will really have to grapple with. That is, when students blog for us in the classroom, are they essentially writing a traditional paper which is submitted in a new format? This doesn’t seem to really capitalize on the affordances of the medium and I am not advocating it. But, if we are to really take this up as a new pedagogical tool, what kinds of writing practices are we expecting – and what are their implications for learning outcomes? Will students take their writing less seriously because it’s associated with blogging, a format that often consists of shorter entries, more casual tone, and more personal opinion (sometimes with pointers to evidence, but sometimes not)? I hope to explore these questions in future research and don’t have any answers, but I’d love to hear others’ thoughts on the subject. Thanks again, Nicole Hi Polly,

Thanks for your kind words. Since recording the presentation, we’ve had a chance to dig a little deeper into the dataset and are beginning to believe that time on task is really the key. Time on task may be a mediating variable that explains a lot of the variance in the comprehension (quiz scores). So, it’s not that blogging in and of itself led to lowered scores (in some cases), but rather format -> time on task -> comprehension.
One explanation is that students made a conscious trade-off about how they would spend their time because they anticipated blogging would take longer (in the commenting condition particularly) and therefore spent less time composing.
Another explanation which we don’t discuss in the presentation is that students have not yet determined what their “blogging in school” writing voice should be. They have written short papers before, but most haven’t blogged as part of their coursework. They may be importing conceptual frameworks about what educational blogging should look like based on other kinds of informal online writing practices (i.e. IM, personal blogs) and therefore may adopt a more casual (and less time-consuming) written style when blogging for instructional purposes.
I think this is an issue that instructors will really have to grapple with. That is, when students blog for us in the classroom, are they essentially writing a traditional paper which is submitted in a new format? This doesn’t seem to really capitalize on the affordances of the medium and I am not advocating it. But, if we are to really take this up as a new pedagogical tool, what kinds of writing practices are we expecting – and what are their implications for learning outcomes? Will students take their writing less seriously because it’s associated with blogging, a format that often consists of shorter entries, more casual tone, and more personal opinion (sometimes with pointers to evidence, but sometimes not)?
I hope to explore these questions in future research and don’t have any answers, but I’d love to hear others’ thoughts on the subject.
Thanks again, Nicole

]]> By: Polly Hoover http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-52 Polly Hoover Wed, 05 Apr 2006 16:46:15 +0000 http://www.higheredblogcon.com/index.php/an-empirical-test-of-blogging-in-the-classroom/#comment-52 I thought that this was a great presentation. I'm currently working on a course which will include blogging as part of the writing assignments and the research that you've presented is very helpful in thinking about setting up this aspect of the course. The time on task component is particularly valuable in organizing the assignments. Thank you! I thought that this was a great presentation. I’m currently working on a course which will include blogging as part of the writing assignments and the research that you’ve presented is very helpful in thinking about setting up this aspect of the course. The time on task component is particularly valuable in organizing the assignments. Thank you!

]]>