John Walkup

I recently reviewed the draft for the national common core mathematics standards.  Precision and accuracy form concepts that interest me on a regular basis, so the common core standards lit me up when I read:

Above all, examples of precision need to clarify the distinction between precision and accuracy.  In this view, the standards writers unwisely selected the U.S. Census for an example.

First of all, we know the measurement is not accurate to within a single count because the census failed to count many individuals that comprise part of the world population.  But this error is not inherent in any given instance of a measurement: When I see an individual, I mark him or her down in my log with infinite precision.  The grand sum off all data points (that is, when I add all the counts to arrive at the total population) remains infinitely precise. On this count, if the census measurements reflect raw counts rather than interpolations and extrapolations, then even the last digit is precise.

Now if we dive deeper we realize that census takers don’t just make head counts. They ask heads of households the number of people living in the home, which we could cast as an imprecise measurement if the responses reflected a non-systematic error. But the common core standards are only correct on a technicality: we cannot expect students to know this. A better example of imprecision in measurement would be the width of the universe expressed to the nearest inch, an obvious overestimation of the precision capabilities of even our most sophisticated equipment.

This misunderstanding between precision and accuracy is pervasive, but also damaging. Failing to report raw counts in their entirety implies the use of an ill-defined round-off procedure. If I count the number of cars in my city and arrive at 23,492, do I report the result as 23,000, 23,500, or 23,490?  How could one justify one number over the other?

So let’s revisit the population census mentioned in the common core standards.  What does the number 304,059,724 represent? Answer: A raw count of people encountered by the census takers. The fact that this measurement does not correlate to the term population (that is, the actual number of people living in the U.S.) is a matter of accuracy.  The number 304,059,724 itself is precise.

Speaking of interpolation and extrapolation, I saw nothing in the common core standards that addresses these two
topics on the conceptual level, but the fact that interpolating is typically safer than extrapolating is important to know in mathematical and scientific reasoning.

In a recent blog, I discussed a model for improving the odds that talented students of physics and chemistry will choose to teach at the public school level.  This plan focused on improving the pedagogical skill of laboratory teaching assistants. I want to discuss this plan from a new angle — the manner in which it can improve the teaching in college and university classrooms.

College professors’ teaching skills (no matter the subject area) are largely formed by the manner in which they were taught themselves, which is more often than not substandard.  Teaching is a learned skill.  Even those considered “born teachers” are only so labeled because of their inherent ability to make content interesting or clearly understood — the skills needed to reinforce knowledge and overcome barriers to learning so often fail to manifest when faced with adversity.

If we want to understand the problem, we need to address the viewpoint of many (although not all)  college instructors.

I want to be a good teacher, but I was hired to publish and receive grant money. I have a family to feed, so taking time away from research to learn proper teaching strategies places me in a compromising situation. Do I want to take a chance on not receiving tenure and having to seek another faculty position when I had to fight like Hell to get this one?  Simply put, my college doesn’t place much emphasis on teaching because there is no money in it.  It’s “publish or perish,” so don’t blame me. I didn’t write the rules.

If we devote our attention to the professional development of college faculty members, further complications arise.  For one, many of the current college faculty have developed habits that extend far back to their graduate teaching experiences. Breaking old habits requires a concerted effort on part of the instructor each time they teach; such efforts fade when faculty face pressures from their department’s research requirements.

Even greater is the reluctance on the part of many faculty members to allow third parties to observe their teaching and offer formative feedback. Without implementation monitoring, however, newly learned skills fail to gain a toehold as instructors gradually fall back on their traditional teaching habits.

A Shift in Focus

One arena in which the professional development community can intercede and develop adequate teaching skills is in the laboratory.  Student lab assistants do not face the same research pressures as their college faculty counterparts. Furthermore, it is far easier to compel teaching assistants to attend professional development training sessions. Since they have little power on campus, there are few barriers to monitoring their use of newly learned strategies in the lab, a critical component of the professional development cycle.

So, one area we can focus our efforts is improving the teaching skills of graduate students. The skills they learn will remain once they depart graduate school and join college faculties. Over time, the teaching quality of college faculty will rise.  For this reason, we should seriously consider mobilizing a national effort to improve the teaching skill of teaching assistants.

Who’s with me?

I recently participated in The Renaissance Group’s annual conference in Washington DC. The Renaissance Group is a collection of colleges and universities that feature high-quality teacher preparation programs. The goal of The Renaissance Group is to elevate public awareness of the need for good K-12 teacher training and to instill the idea that teacher preparation is a campus-wide effort involving every campus discipline.

The Renaissance Group’s conferences are always a joy to attend because they are so well organized and the participants are fully committed to the Group’s mission.  The list of speakers at this year’s event was impressive, with  Secretary of Education Arne Duncan and Under Secretary of Education Martha Kanter leading the way.

The Problem

One of the problems discussed during Dr. Kanter’s presentation is the dearth of math and science teachers, a problem that threatens to grow and leave the U.S. behind in global competition.  The problem is most severe in the physical sciences—physics and chemistry. As a K-12 professional developer with a physics background, I would like to propose a solution to this problem.

For the most part, the educational community has looked at the inadequate number of science teachers as largely a recruitment problem.  And it is, but only in the most general sense.

So what is a typical recruitment process? How does the Army recruit soldiers?  How does IBM rake in talent? How does the education field recruit teachers? Like the Army and IBM, we first try to convince recruits that joining our enterprise will reap them personal rewards—which can involve everything from money to self-fulfillment. We then try to explain away the drawbacks, either by explaining that the drawbacks do not apply to them or that they are relatively minor in comparison to the rewards.

This approach faces limitations when applied to potential physical science teachers for a number of reasons. To illustrate these reasons, let me put myself in the shoes of a hypothetical physics student contemplating a career in teaching. (I suspect chemistry students resemble physics in their attitudes toward teaching careers.) Please keep in mind that I don’t necessarily agree with the sentiments expressed below.

1. “For me, the money simply isn’t there. My physics classes are hard and I think I spend far more time and effort than most others completing my coursework. To accept a job where I receive the same low pay as those who studied ‘easier’ subjects is a slap in the face.”

2. “I consider myself a good person but I am not enticed by the self-fulfillment argument. I work hard and I would love to see my work eventually find practical applications that the public can appreciate. But I’m not out to save the world.”

3. “I have little patience for those who fail to appreciate scholarship, so teaching unmotivated students does not appeal to me one iota. I see the movies where the inspirational teacher drives students achievement to success based on his or her commanding personality and leadership. But I don’t have the personal skills to do that.”

4. “I taught a lab and hated it. I explained the science and lab procedures as clearly as I could, but students would never listen to me. After screwing up the experiment, they would then turn in garbage for their reports.  My teaching evaluations were unpleasant to say the least.  Do I want to inflict this torture on myself for the rest of my career?”

5. “I want an outlet for intellectual pursuits and I just don’t consider teaching an intellectual exercise. What is there to it? You lecture on the content, then you hand out some workbook exercises and walk around helping students that raise their hand. My off hours will be spent grading papers, a brainless activity I loathe. Simply put, teaching doesn’t require the thinking skills and decision-making I like to perform.”

6. “I want respect. With advanced knowledge of mathematics and physics, I don’t want to be known simply as a `teacher.’ People at parties will probably tell me that they really respect my profession, but I don’t buy it. They’re just being polite.”

The Sheldon Cooper model of physical science student is largely a caricature, so are these characterizations fair? Perhaps not.  However, I think many of the characterizations mentioned above will resonate with many physical science students, even though they are founded on misconceptions.

Central to any reform effort is the focus on issues that form realistic chances for change within the scope of the reformers’ jurisdiction.  Yes, the educational community can implement new salary scales that will make science teaching more lucrative, but that decision is largely out of our hands as education reformers.  Also, little can be done on a practical level to instill a heightened sense of fulfillment in potential recruits.

However, we can offer a holistic solution to the sentiment expressed in the final four statements above.

When performed well, teaching is a satisfying intellectual exercise.  The educational community, in my opinion, has not gone far enough to link the intellectual aspect of higher-order decision-making to effective teaching practice. Let me give you an example:

Student questioning is an art and science founded on common sense, adherence to well-reasoned principles, a body of research knowledge, and most importantly for this discussion a continual reliance on decision-making. The proper method to employ at any given time relies on numerous classroom conditions. Unfortunately, I rarely see effective student questioning when observing classrooms, but rather traditional habits. Teachers call on raised hands (volunteers) for no apparent reason, provide inadequate wait time, and group students according to no discernible strategy. Worse yet, when asked to explain their questioning methods, teachers often fail to conjure any image of metacognition—the ability to know why they are doing something when they are doing it. When asked to explain their decision-making in terms of Bloom’s Taxonomy or depth of knowledge, I often get blank stares.

Why is this example illustrative? With inadequate formal training and no corrective feedback on their teaching effectiveness, teaching assistants employ substandard instructional strategies that feed many of their misconceptions about the teaching profession. They end the year with negative teaching experiences and never garner respect for education as a discipline.

In short, the education community relies too much on trying to talk budding scientists into becoming teachers. Although these efforts should not subside, we need a fresh approach.

Holistic Solution Focusing on Input Processes

Teaching assistants (both undergraduate and graduate) form a sizable pool of potential teaching candidates. Unfortunately, they traditionally receive at best only a modicum of training comprising reviews of (1) grading policies, (2) lesson content, (3) safety policies, and (4) lab equipment operation. All four are important, but missing is the most crucial component of being an effective teaching assistant: How to teach.

Rather than rely solely on traditional teaching assistant preparation, I advocate a substantive preparation program that will focus on raw, bare-knuckled teaching skills. Such a program, to be effective, must implement a full professional development cycle that sets reasonable targets, trains potential teaching assistants in research-based strategies, and monitors implementation on a continual basis.

Based on the above familiar Deming cycle used in business, the professional development cycle frames our approach to solving the science teacher recruitment dilemma. The program begins with the act stage, in which teaching assistant candidates receive formal training in tangible research-based instructional strategies. The plan stage involves target setting based on feedback garnered during the training session, followed by the implementation of the learned skills in the do stage. The check stage follows, in which we (a) measure the instructional processes precisely to determine strengths and deficiencies and (b) measure teacher behaviors solely with respect to the targets established during the plan stage. The cycle repeats perpetually for as long as the teaching assistant serves in his or her role.

Failure to implement a full professional development cycle spells death for any education initiative, and the proposed model here poses no exception.  Without targets, there is no means for measuring the effectiveness of the program. Without implementation monitoring, teaching assistants will gradually fall back on natural habits over the course of the semester. And without training, teaching assistants never learn the new skills in the first place. The message is clear: Implement a complete professional development cycle or watch the reform fade.

In case the plan remains unclear, here is one step-by-step procedure for implementing the model:

Step 1. During the summer semester, a professional consultant (either a third-party or faculty member) trains teaching assistant candidates in instructional strategy. (I will post the scope of the training in another blog. For now, let me just say that the skills taught during the training must be fundamental, effective, and easy to implement.)

Step 2. Approximately one week before the semester begins, teacher candidates and the professional consultant convene to set targets for implementation.

Step 3. Once lab instruction begins, academic coaches (usually fellow teaching assistants or graduate education students who completed the training sessions) observe teaching assistants deliver instruction and provide post-instruction feedback to the teaching assistants.

Naturally, we will need to develop supporting materials such as training manuals and video vignettes of effective teaching.

Long-term Success Defined According to Output Processes

At the risk of establishing a straw man argument, we should now revisit the barriers to student recruitment we expressed earlier.

1.  As scientists and educators, there is little we can do about the salary structure of teachers, largely a community/district issue. Therefore the proposed project does not address the issue of teacher pay.

2. The proposed project does not address the issue of self-fulfillment, a personality trait that requires a long time to adjust.

3. Student misbehavior, motivation, and engagement are largely matters of teaching skill.  By training teaching assistants skills in student questioning and active participation, we provide them the means of instilling effective classroom management and boosting student motivation.

4. The perceptions we hold about our own careers are heavily influenced by the skill we apply when performing related activities. Teachers are no exception. When teachers do their job well, student appreciation for their efforts rises, inducing a concomitant rise in self-confidence and attitude toward the profession.

5. Done well, teaching is an intellectual pursuit, replete with higher-order thinking and decision-making. One primary goal of this program is to draw out this scholarship and place it squarely in the center of all training. The result is new-found satisfaction with the practice of teaching among teaching assistants.

6. Attitudinal changes toward education will form as undergraduate students become exposed in their laboratory classes to proven teaching methods. Admittedly, these attitudes would only evolve over time scales on the order of many years.

Throughout the professional development cycle, goals remain fixed on input processes—the teacher behaviors exhibited by the teaching assistants when teaching.  All observations and targets center on these learned behaviors.

Note that the goal of the professional development cycle is not to increase student learning, a notion that is unfortunately heresy in education. Sure, the students in the laboratories will likely learn more when taught using properly trained teaching assistants, but we should avoid making the classic mistake we see in education of measuring success through quiz and test scores (which are not within the direct control of the teaching assistants).

Once a teaching assistant finishes their teaching requirements for the year, we measure the output processes—the changes in attitudes toward the teaching profession expressed by the teaching assistants involved in the program. However, we must be careful in jumping to conclusions based on the results of a pilot year project. Change takes time and we may not realize the success of the program until it has been in place for at least two years.

In a nutshell, we are proposing to recruit physical science students into the teaching field by turning them into good teachers. And as for those that decide to stay in the physical sciences and become college professors? Well, they will become good teachers too.

Lesson design is a crucial component of science.  With the recent emphasis on 21st-century skills, perhaps now is a good time to rethink the way we teach students how to design experiments.  Although this would naturally cover a lot of topics, let me discuss one in particular: the interplay between theory and hypothesis.

I have been surfing the Web and discussing experimental design with my colleagues. It is clear to me that many of us have widely varying views of what constitutes proper experimental design.

To formulate a proper experimental design, we can start by going back to its purpose. Why experiment in the first place?  What does it do for us?

When we perform experiments, we are typically trying to understand a broader picture of ordinary events we see in nature.  Let me offer an example that we refer to repeatedly throughout this discussion: the classic-water-drops-on-a-dime experiment.

I have seen the dime-drop experiment performed in numerous classroom sessions. But often the purpose of the experiment is misstated.  When we perform this experiment, we are not trying to satisfy the question, “How many drops of water can we place on a dime?”  Who cares?  Even if we were able to find the answer to high precision, our results are completely worthless as soon as we swap the dime out for another coin, such as a penny.

In science, we try not to simply collect data. An experiment whose sole purpose is to find the number of drops we can place on a dime is worthless. A table of water drop experiments, all performed on various coins, is only slightly more useful.

What we are seeking is the answer to a bigger question, such as, “What accounts for the number of liquid drops that can be placed on a disk?” Is it the perimeter?  Is it the surface area?

The quarter has a much larger surface area than a dime, but only a slightly larger perimeter. To some people, a quarter that has twice the surface area of a dime should therefore be able to “hold” twice as much water since there is twice as much area for the water to reside.

However, others will pick the factor perimeter.  Sure, the quarter has a much larger surface area than a dime, but water has to spill over the edges.  The dime only has a slightly smaller perimeter than the quarter, so the length of coin-edge available for water to spill over is not so great . So perimeter should be the dominant factor.

So the question is much grander than simply “how many water drops?”  If we can answer the question “perimeter or diameter” the impact can be much larger and can possibly explain other phenomenon we see in nature.  How much condensation can accumulate on a portion of the Space Shuttle when flying through icy clouds?  A properly designed experiment can allow us to extrapolate the results (within reason) to answer this question.

So now we have arrived at the theory stage. Those in the “surface area” camp will theorize: The amount of liquid a disk can hold is directly proportional to its surface area.

Notice that we have used the term liquid instead of water.  Again, we are seeking answers to grand questions, not minutiae.  We will use water in the experiment and be careful to note that the experimental result has its limitations.

So what does this all have to do with the hypothesis?  Well, to test a theory we need to perform an experiment.  But the experiment has to be conclusive. As we always do, we revert to what we already know:  For example, suppose we earlier we performed a water-drop experiment on the dime and found it held on average 7 drops.

The hypothesis is now clear.  If a dime can hold 7 drops (which we already know from prior observations), then our theory predicts a quarter can hold 14 drops (assuming that the quarter is twice the surface area of a dime. It isn’t, but we’ll pretend it is for the sake of argument).

Hypothesis:  Based on earlier results of water-drop experiments on dimes, theory predicts that a quarter will hold on average 14 drops.

So the combination theory/hypothesis serves two purposes:

1. It establishes a well-defined outcome for the experiment.
2. It establishes the big picture of what the experiment is all about.

Often experiments, especially at the lower grade levels, omit the theory entirely, which turns hypotheses into nothing more than guesses.  Let me be clear:  A hypothesis is not a guess.  It is also not a mere prediction.  It is a prediction based on (1) prior experimental evidence and (2) theory.

Before I move on to another example, we should note that coins, which have uneven surfaces, are not the best objects to use in this experiment. Again, we seek broad understandings, so we should use disks that have smooth surfaces.

Let’s turn our attention to another classic experiment.

Hypothesis:  I think if I plant the honeydews in moist soil, they will grow better.

Based on what?  What big picture will this experiment attempt to answer?

Instead, consider the following:

Prior knowledge:  Watermelons and cantaloupes grow better in moist soil (prior observation).  Watermelons and cantaloupes are both melons (factual knowledge).

Theory:  Melons grow better in moist soil.

Hypothesis:  A honeydew (also a melon) planted in moist soil will grow better than a honeydew planted in dry soil.

Experimental results now answer the question definitively (although they do not really prove or disprove anything). If the honeydew grows better in moist soil, then we have produced strong evidence that melons (the bigger picture) grow better in moist soil. The implications are important since the melon family is rather large.  A farmer wanting to grow sugar melons can look at the results of the experiments and make an educated guess on whether his crop will grow if he knows the moisture of the soil.

I look forward to hearing your comments.

Our company released the Community Edition of its Sapphire classroom observation software for free download a few months ago. Since then, administrators have been downloading the software and using it in their classrooms.  I want to take this time to explain why the feedback provided by Sapphire can enhance data-driven decision making and drive professional development to improve the amount of academic engagement time (often mistakenly confused with time on task) available to students.

We used state-of-art tools like Java FX to create Sapphire, which has granted the software considerable functionality. I want to focus this entry on the timeline feature, which uses real-time charting to map the time usage of classroom sessions.

Suppose a principal steps in and observes a classroom for 50 minutes.  Using Sapphire, she obtains the following timeline.  (In my experience, this timeline describes a typical classroom; I have certainly observed far worse.)

The timeline suggests to her considerable areas for recovering lost instructional time between the bells.

1. The first three minutes devoted to transition is too long and completely wasted because the assessment appearing between 3-18 minutes could have begun at the 1 minute mark (or even earlier) while the teacher took roll.

2. Partial student engagement from 12-18 minutes (and especially between 15-18 min) indicates many students finished early. Can the teacher shorten the next quiz? Possibly.  Can she assign academic work to students who finish early? Probably. Do her students need to spend 16 minutes on a quiz? Maybe, but if this is a common occurrence she would likely review her teacher’s checking-for-understanding techniques.

3. The teaching of new content (a good thing) was interrupted by the PA system (a school-centered disruption) at the 24-minute mark. Never minding the disruption factor (which can irritate teachers and distract students for the remainder of the class session), students lost three minutes of instruction. She questions why the school is interrupting instruction during the middle of a classroom session, and if all three minutes consisted of truly important information.

4. The transition to review activities at the 36 minute mark took three minutes, which is probably too long. She might ask the entire teaching staff to discuss means of shortening transitions. Do the staff have a consistent method for transitions? Can they reduce this time to (say) one minute?

5. The review session featured one-fifth of the students disengaged throughout the last ten minutes of the observation. This points to a classroom management/engagement issue.  Did the teacher realize that one-fifth of the class was disengaged?

The attentive engagement index is 0.82, indicating that students, no matter how intent they were on learning, could not have been engaged more than 82% of the observation session.  The non-attentive engagement index, 0.49, indicates that even the most disengaged students were engaged at least 49% of the time — a small consolation given that this is less than half of the observation session.  The true engagement index for this session is therefore a range between 49% and 82%, depending on the motivation of the student. Note the engagement index for the typical student, 66%.  We’ll come back to this value later.

After considerable discussion and a bit of professional development in questioning strategies (which The Standards Company LLC can provide), the principal steps back into the classroom and observes the following:

Notice the differences:

1. The school secretarial staff now interrupt the class near the beginning of class, not during the middle.  And they have trimmed their announcement to only include vital information. (They have decided to refrain from calling out of the names of birthdays, which was a habit in the past.)  During the announcement, the teacher took roll and handed out the quiz.

2. The teacher examined the Sapphire timeline and decided that trimming the quiz by three problems would not seriously detract from its purpose but would save three minutes. (Students who finished early completed word searches to keep them busy while the rest finished.  This does not count as academic engagement, but at least it keeps them from tearing into each other. Perhaps the teacher can improve even more by having them review grade-appropriate vocabulary words. )

3. The teacher proceeds with the same lesson as before (but without the school-centered interruption).  However, the transition to the subsequent review session is shorter.  After undergoing professional development in student questioning and classroom management, the teacher maintains more engagement during the review session.

4. Because of the saved time, the teacher has roughly ten minutes near the end of the session to teach new vocabulary words.

So what did the students gain?  The attentive students are now engaged 91% of the time, an increase of 9% (about 7 minutes).  The big gains were made in the non-attentive group, an increase from 49% to 86%.  Wow!   And since this group comprises many of the students that lie on the fringe of proficiency on state tests, the gains in achievement scores can be enormous.

What about the engagement index for the typical group?  This is a weighted average that takes into account the portion of the students that are engaged at any one time.   This value is now 89%, an improvement of 23% over the previous session.  In a sense, this teacher has improved his or her classroom time usage by 23% with a little professional development and changes in classroom management. (The secretarial staff helped too; classroom time usage is a schoolwide problem needing cooperation from every staff member.)

The teacher can improve the results even more. The principal thinks this teacher can trim another lost minute from the transition at the beginning of the classroom session and eliminate the transition at the end entirely.  And the time lost t partial engagment during the last portion of the quiz can be eliminated by providing students with true academic activities instead of word searches.

So what can we expect schools to achieve when improving classroom time usage? I am not sure what values the three engagement indices would exhibit if studied on a wide scale. I do know that schools, over a period of a full week, can achieve an attentive engagement index of 95%, for I have witnessed it personally.  But it takes a concerted effort on everyone’s part. (Professional learning communities, anyone?)

So how much does it cost to fix problems with classroom time usage?  You be the judge, but I think most of the readers will see that the cures are relatively cheap, with most being free.  But to raise academic engagement time at a school, the staff has to first know where the problems exist. Sapphire is the most powerful tool available for achieving this end and monitoring improvements.

You can download Sapphire for free from our official web site, http://www.standardsco.com.

Calling on random non-volunteers to answer in-class questions is becoming increasingly popular, with many teachers using numbered Popsicle sticks or playing cards to select students. Calling on random non-volunteers effectively engages all students on academic content, holds all students accountable for learning, and provides a more accurate barometer of teaching effectiveness.

On some occasions, however, teachers may find it more appropriate to call on volunteers (raised hands). As one important example, teachers often ask questions (especially highly rigorous questions) to engage students on academic content, not to check for understanding. In such a situation, the very act of thinking and discussing on the part of the students becomes more important than the answer itself.

Unfortunately, teachers often call on volunteers carelessly, allowing uninterested students to disengage mentally and, as an unfortunate consequence, misbehave. In this teaching tip, I want to illustrate a manner in which teachers can call on raised hands, yet maintain the benefits of random, non-volunteer questioning.

Selection method

We can describe in four steps a powerful technique you can use to question students when the volunteer selection strategy is more appropriate:

1. Query the entire class and make every student think that you will call on random-nonvolunteers.

“I already provided an analogy that compares power and work. In a few minutes, I will call on some of you to share your own analogy.”

2. Place students in informal groups (such as peer-shares) or formal groups to discuss possible responses.

“Get in your groups and discuss your ideas.”

3. Provide adequate wait time for your students to think about the content and discuss possible responses.

“I will call on the first student in about five minutes.”

4. Call on volunteers to respond.

“Rather than calling on some of you, does anyone here have an analogy they want to share?”

Since students thought we would select them, we were able to compel them to engage the question and discuss responses in earnest. However, we also recognized that our weaker students might have struggled to create an analogy—no matter, the very act of discussing possible analogies forced all students to engage the academic content, thus reinforcing their understanding of the current lesson and enhancing active participation.

In the example above, we performed the first three steps because we wanted to compel all students to at least try to conjure an analogy and to provide them support for their efforts. Yet, we chose to call on volunteers in the final step for three reasons: (1) we were not trying to check for understanding, (2) the act of thinking and discussing was more important than the answer, and (3) some of the weaker students would have struggled to arrive at an analogy within the alloted time.

Another interesting application of this method pertains to questions on content learned previously that students may have forgotten.  For example, a teacher teaching a lesson on literary review may want students to try and remember the definition of mood. Calling on volunteers with no real strategy frustrates those students who take longer to remember.  The steps outlined above work equally well in such a situation.

The key to effective questioning centers on meta-cognition—understanding why we choose a particular strategy when we do—the strategy itself becomes another wrench in the professional teachers’ toolbox.

When The Standards Company LLC provides reports on the state of the enacted curriculum, it is always careful to avoid stating unwarranted personal judgments. Instead, educational agencies should judge the results of educational research for themselves with respect to their own goals. In what follows, I will describe one reason why dogmatic targets often fail to take into consideration complex issues.

There is little doubt that the rise of content standards has changed the face of education, especially in regards to expectations. While in the past, teachers often claimed that they held high expectations for their students, state and national standards have recently delineated what the term “high expectations” constitutes.

To some teachers, high expectations now means teaching at what the state defines as “grade level.”  To other teachers, however, state content standards merely set the norm; high expectations correlate to above-grade-level content.

When I first began training teachers in professional development workshops, a few teachers would state that they didn’t need to teach on grade level because they felt the standards were too high for their students. Some teachers would even say that they focused instead on basic skills (a euphemism for “far below grade level”) because their students (get this) “did not go to college.”

My response was simple:  If my students don’t go to college, I want that to be their decision.  If I teach students content that is below grade level, I am making the decision for them.  I find that unacceptable.

Today, more teachers recognize the importance of teaching students grade-level content. But should we teach students “above grade level”?  The answer is similar to the answer to many education questions: “It depends.”

We should first recognize that state content standards define learning objectives, not teaching objectives. The distinction between the two terms will not surface until we explore the relationship between state content standards and depth of knowledge.

In many states, the English language arts standards increase in rigor according to the procedural knowledge needed to perform the standard.  For example, in one grade level (let’s say, Grade 7) students may only be asked to identify metaphors in poetry; the state assessment would likely provide a list of phrases from a poem and ask students to choose the one that corresponds to a metaphor. In Grade 8, students could be asked to explain the meaning of metaphors; in this case, students could identify the metaphor and ask students which explanation best describes its meaning.  Both standards relate to the same concept: metaphors. The difference between the two grade levels centers primarily on their levels of depth of knowledge, a model of rigor first described by Norman Webb.[1]  The Grade 7 example (identify) corresponds to Level 1; the Grade 8 example (explain) corresponds to Level 2.  (The Bloom’s Taxonomy levels [2] would differ as well.)

Should we teach students skills associated with Grade 8 in Grade 7?  I certainly think so, because a necessary means of cementing understanding at a particular level of rigor (whether it be described by Bloom’s Taxonomy or depth of knowledge) is to expose students to content at higher levels of rigor. In other words, when we teach students to analyze metaphors, even to a limited extent, we strengthen their ability to identify the appearance of metaphors.

Therefore, when a set of content standards increase from year to year through increased rigor, the appearance of a moderate amount of above-grade-level work should not be too alarming and could even be justified. (However, a predominance of above-grade-level work could indicate that the teaching staff is not sufficiently acquainted with the grade level standards targeted for their students.)

Now consider the case of mathematics, which in many states increases from year to year by changing to more advanced concepts, not necessarily through increased rigor.  For example, students could be asked to calculate integers raised to powers in Grade 5, but asked to calculate square roots in Grade 6.  If taught with an emphasis purely on procedural knowledge, the rigor would be the same in both standards: an apply-level Bloom’s Taxonomy and lowest level of depth of knowledge.  Here, above-grade-level work is not more rigorous, but rather off topic — a more serious situation. The same condition typically applies to science and history/social studies, where above-grade-level work often means students are taught concepts not even related to the adopted curriculum.

So, should students be taught above grade level work?  Teachers should base this decision on a firm understanding of rigor and the structure of their own state’s content standards.

[1] Webb, N. L., Alignment study in language arts, mathematics, science, and social studies of state standards and assessments for four states, Council of Chief State School Officers, Washington D.C., 2002. See http://facstaff.wcer.wisc.edu/normw/TILSA/TILSA.pdf.

[2] Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., Wittrock, M. C., A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Addison Wesley Longman, Inc., 2001.

Technical writing has long interested me. When I taught physics at Cal Poly, Melody DeMeritt (a member of the English Department faculty) and I co-facilitated a writing workshop as part of the university’s WinGED (Writing in General Education) program.  I not sure who, but someone introduced me to the writing method E-Prime during the workshop. I forgot about E-Prime for the next few years, but picked up the method again roughly two years ago. I am glad I did.

The E-Prime writing method involves recasting sentences to remove all forms of the verb “to be.”  (I have boldfaced and colored red such verbs throughout this blog entry.) Verbs of the form “to be” point to the concept of existence, but dynamic writing typically targets more meaningful actions.

I consider E-Prime challenging, but I think such efforts produce cleaner prose. The process of transforming a passage into E-Prime often forces the writer to re-evaluate his or her position, often to substantial benefit.

Let me demonstrate the process using a passage from an article Gerlinde Olvera* and I recently submitted to a research journal for publication. First, we start with the original passage:

I note two instances of the verb form “to be.”  The verb “is” dominates the first sentence over the action verb “facilitate,” an unfortunate construction. I can recast the first sentence by propelling the main verb “facilitate” to a more prominent position, eliminating “is” altogether.

I love the directness of this sentence, but it contains an ambiguity: Teachers mainly facilitate… as opposed to what?  Or should we interpret the sentence as “Teachers mainly facilitate Level-3 activities…” as opposed to Level-2 activities? In other words, should we consider the verb “facilitate” or the direct object “Level-3 activities” the focus of the sentence? (Note that the original sentence contained this flaw as well.) Let’s try to clarify our position:

Although longer than the original sentence (by roughly one word), our result contains more information.

Now we can consider the second sentence, where I have inserted one of my own proofreading comments in parentheses:

The first problem centers on the verb “expected.” Although stronger than verbs of the form “to be,” the verb “expected” relates to a passive educational process. As teachers, we do not merely “expect” students to think in certain ways; we “guide” or “teach” them.

Students are guided by whom?  The teacher? The school? Here, appearance of the verb “are” points to a passive construction and, therefore, begs for recasting into E-Prime.

Since both sentences begin with the same subject “teacher,” we can simplify the second equation using the pronoun “they”:

Looks good to me! The result removes considerable ambiguity and saves space–a win-win situation. The reader may object that the last instance of the word “them” could refer to the subject “teachers” or the direct object “students,” but to me the sentence reads just fine.

Those seeking help with this writing method should consider visiting the Wiki Encyclopedia entry and the links at the bottom of the page.

Oh, if you think you can offer suggestions for improving my prose or simply want to comment on my efforts, please do so.

_______

* Gerlinde analyzes curricular materials and contributes to our knowledge base (especially in regards to English language learners and 21st Century Skills) here at The Standards Company LLC.

I am becoming more involved in the California Mathematics Council (http://www.cmc-math.org/) and recently presented at both the South and North division conferences.

The South conference was hosted in Palm Springs and I took my daughter with me.  She had been begging to see her favorite metal band Turisas and they were appearing in Pomona, so why not?  They are Finnish and don’t appear in the U.S. very often.  They are also incredible and put together one of the most vivid (and loudest) concerts I have ever witnessed. Another LA-area band called Ironklad was extraordinary as well, and I urge anyone that loves nu-metal (e.g. Disturbed) and lives in Southern California to go see them.

In both Palm Springs and Asilomar, I presented “Avoiding the Most Common Mistakes in Math Instruction,” a compilation of boo-boos that I have witnessed over the years as a professional developer.  Since math is my second main love (behind physics), I especially enjoy presenting this talk, and it was well received.  In summary, here is one portion of the Walk of Shame (I will post more later):

A.  Ineffective questioning strategies — calling on volunteers (raised hands) is a particularly troublesome questioning method that causes much of the discipline problems and mental disengagement taking place inside our classrooms.  With a few exceptions, there is little reason to ask for volunteers to answer a question.  I advocate the following four-step procedure:

1.  Question the entire class (not just a single student)

2.  Wait sufficiently long before choosing students to respond.  (This is where I advocate letting students discuss the answers in pair shares (for DOK-2 questions) or small groups (for DOK-3 and DOK-4 questions).

3.  Sample the class by randomly selecting non-volunteers to answer the question.

4.  Leave No Child Behind–do not simply proceed if the responses indicate students are struggling.

B. Insufficient Concept Development — math teachers often focus too much on procedural knowledge development, which is not only limiting and easily forgotten, it is also a signature of the drill-and-kill method of solving math problems that turns students away from math and science.  (One of my curriculum specialists, Lisa Gibson, wrote an article based on this issue and was recently published by the journal Communicator.)

C. Not Teaching the Importance — too many mathematicians think that students will appreciate the inner beauty of mathematics and love the discipline for its own sake.  As far as students are concerned, Elizabeth Hurley is beautiful, Whittaker functions are not.  My opinion is that if we don’t teach students why the lesson they are learning is important, then why would they want to learn it?  (And no, telling students “You will need to know this on the test” does not count.)

D. Teaching subskills at the expense of grade-level content — yes, students often lack the subskills we think they need to learn grade-level content. However, we can scaffold these subskills while teaching them standards-based content.  For example, when teaching students to calculate powers of integers, I can provide them a multiplication table — the lesson is focused on calculating the powers of integers, not basic arithmetic. Remember, if we have taught students to calculate powers and what the concept “exponential expression” means, they have a chance on the state test; if we have instead focusing on reteaching them basic multiplication, they have no chance at all.

That’s enough for now. If you want the PowerPoint of my presentation, just shoot me a note. And go see Ironklad; they’re great.

Lately I have been reading an article by Andrew Ho in the journal Educational Researcher titled “The problem with `proficiency’: limitations of statistics and policy under No Child Left Behind.” It is a worthwhile article and I encourage others (hint: employees of The Standards Company) to read it.

Briefly, Ho describes a statistical analysis of student proficiency on state assessments. Now, each state defines “proficiency” differently. So while one state may decide that 50% correctness is worthy of a student to be denoted “proficient,” another state may decide 70% is the better choice. Naturally, the cut-off between “proficient” and “not proficient” would dramatically change the percentage of students considered proficient in one state in comparison to another. Who would argue otherwise?

What is not so clear is whether changes in the percentage of students scoring proficient would change between one state and the next if the cut-off point was changed. At first glance it shouldn’t matter: where two race cars begin on a track has little to do with how rapidly one car would gain on the other.

But complications set in when one considers that the race car analogy does not quite hold when examining state test proficiency, which does not map the gains from the same students from one year to the next but rather the same graduating class of students from one year to the next. In other words, we measure a batch of students designated as Grade 6 in one year, and another batch of students designated as Grade 6 the next year. In this case, the arbitrary cut-off point between “proficient” and “not proficient” can affect the growth in the number of students scoring proficient. It all comes down to a basic issue: how many students were on the edge of the cut-off point?

Ho performed a statistical analysis of data collected from a blind state (that is, Ho may know which state is under consideration, but he isn’t telling us) to measure the rate-of-change in student proficiency across two years. He then changed the cut-off point and re-performed the analysis.

Guess what? The rate-of-change changed dramatically, indicating that rate-of-change is overly sensitive to the choice of cut-off point.

There is a direct analogy of this issue in computer simulation. A common measure of the worth of a computer simulation is the sensitivity analysis. For example, suppose I have a computer simulation designed to indicate weather patterns appearing ten years from today. Naturally, I have to feed initial conditions into the computer before I can start the simulation. Suppose that I input the temperature, pressure, humidity, and wind speeds appearing at 5,000 locations spread across the globe. When I run the simulation, I find that a hurricane is blowing through College Station, TX, ten years from today. I can certainly publish my results and have Aggie fans begin upgrading their building codes in preparation for the onslaught of weather.

Now suppose that, for kicks, I decide to run the simulation once again. If I choose the same exact initial conditions, I should get the same result, a hurricane over College Station. (Actually, most simulations feature randomness, either by design or through random error, that would produce a different result if ran again.) However, I decide to modify the initial temperature at one of the 5,000 points by one degree. When I run the simulation once again, I instead get a hurricane over China.

So of what use is my computer simulation? Very little, because there is a fat chance that I would be able to input enough data describing the initial conditions to such precision that would produce a reliable result. Miss one temperature measurement by one degree and the hurricane appears on the other side of the globe.

According to Ho, the same problem exists in state test score proficiency. Small changes in what is arbitrarily termed “proficient” produces marked changes in improvement. So one state that improves rapidly in comparison to the others could be doing so simply because of its choice in the cut-off value, and little else.

As an education researcher, Ho’s findings reinforce my assertion that the real gains in education are made not through test score analysis (which Ho has shown are highly suspect) but through upgrades in curriculum and instruction. Many would then ask how, if we disregard test scores, we can tell if gains are being made in curriculum and instruction? The answer to that question is simple and cuts directly to the core of the beliefs of The Standards Company: direct measurement of curricular materials and teaching based on objective criteria. I would rather judge the teaching taking place in Mississippi or Kansas based on the levels of Bloom’s taxonomy and depth of knowledge appearing in their curricular materials than on student testing proficiency. After all, as a teacher I can control the curriculum I deliver to my students and the strategies I use to teach them.