Insights in Learning Science and Learning – Steve Ritter and a short summary with bullet points

Steve Ritter founder of Carnegie Learning.

Thanks Peter and Barry, you know as Barry said we are really excited about the merger with scientific learning because of both Carnegie Learning and scientific learning have such a deep background in learning science and how people learn. What I want to do is give you some insight into the way we think about learning science and research gets Incorporated into MATHia and hopefully how that relates to what you know about Fast ForWord.

Research Background

I’m going to start talking about a little bit of the research background. As I mentioned before I’m one of the founders of the company Carnegie Learning. I’ve been with the company since 1998. And on the screen is one of my co-founders John Anderson and its really, I was supposed to talk with John Anderson it’s really like his research that is foundational in the way we think about learning science and what gets incorporated into MATHia and our pedagogical approach. You can see this screenshot is from John winning the Atkinson prize for psychological and cognitive sciences. He’s really won pretty much every major award this is from the National Academy of Sciences in the United States.

John’s primary work is this model called ACT-R. What ACT-R is, is a model the general model about how people think learning perform. Applied in all areas but you know prior to the beginning of the work that led to Carnegie Learning, John was challenged to say “Well if you know so much about how people learn you should be able to apply this as education” and so that was really the origin of the what we called cognitive tutors at the time which became MATHia which basically building a model of how students learn and then using that model to understand how each individual student learns and providing appropriate experiences and feedback to each student so that student learns fast.

When this work came in and having been to Carnegie Mellon University that’s where I got my Doctorate and did my postdoc with John Anderson and we were actually colleagues in the psychology department and John ACT-R theory is complex but is really like three major elements that we use and apply in our product:

• Any complex knowledge is composed of simple knowledge component. That’s what the brain does when you’re doing a complex task it’s going to break that complex tasks down it’s a simple knowledge component skill.
• Knowledge is strengthened through active use. So, the more you do something the better you getting better at it. But the question is what are you getting better at, like what are you repeating? the answer is we have a lot of data to support this.
• What you really learn is these smaller knowledge components. It’s not so much that you’re learning to solve equations. let’s say as you learn to solve equations with negative coefficients of a certain type. Those kinds of things. So, this fine-grained view of how people learn is essential to the way MATHia works and what makes it so effective.

Talk about another mentor of mine – John Anderson. He is amazing. This guy Herb Simon is probably either really more amazing he’s considered a founder of the field of organizational behavior artificial intelligence in cognitive psychology and he also won a Nobel prize in economics and he wasn’t even an economist he was also in the psychology department.

But his Nobel Prize was for applying psychology to economics and he had this Knack. I tiled a class for him and he had this Knack of saying things that seem really simple but in fact were really profound and this is one of my favorite quotes from him and he said “Learning results from what the student does and thinks and only from what the student does and thinks. The teacher can advance learning only by influencing what the student does to learn.”

That’s really the sort of the origin of this a student-centered approach and when we start out at Carnegie learning it was actually pretty rare for educators to think about learning from the student’s perspective. Educators would think about what materials do I get to students, how do I organize classes, how do I train teachers you know what are my objectives? and all those things are important but is very different thinking about that from an Institutional perspective as opposed to think about it from a student perspective. You’re only going to achieve learning if all those things that you put around the student actually impact the student’s grade what the student thinks and what the student does.

So that was a big influence on what we put together and we ended up with this guiding principle which is basically saying the more we understand about how students think and learn the better we can help them think and learn. That’s been the foundation of Carnegie Learning since the beginning its really a continuing quest to learn more about how people learn and apply that as learning within our product. This is a simple screenshot of MATHia on the left I will talk a little more detail about how we think about MATHia, but the first thing to notice is, this is a complex problem giving students to solve this. Word problem on the left, the student is going to construct a table of values representing the relationships the mathematical relationships describe in the word problem and the student is going to solve equations you can see the student’s going to craft. It’s a multi-step problem. The reason we do that is we want the students to expose the steps of their thinking so that we can provide them with appropriate feedback on each step of the thinking process.

Understand student thinking.

Most products and certainly assessments focus on the answer that the student gives. This is an example of a typical test question in mathematics

What is 1/2 * 1/5.

The student has answered 1 which is incorrect. Traditional test or traditional software that focuses on the answer you would just mark that as incorrect and wouldn’t have very much insight into what the students knew about fractions and in fact you probably conclude that the student knows very little about fractions.

But if you ask the student to think out loud you can actually understand a lot about what the students thinking. This is a transcript of the student what we did was we asked the students to solve this equation and just tell us what you want what’s on your mind as you solve this equation. The student says

One half times one fifth

Now, I have to find a multiple of 10

So, one-half would go to five-tenths

and one-fifth would go to two-tenths

and multiply that and

that would be one whole.

That’s the student’s description of how he came to the solution that 1/2 * 1/5 is 1 and its really you know this is indicative how we take student’s thinking seriously. That if you take seriously what the student says it’s totally understandable what the student did.

Student said okay need to multiply fractions the first thing I’m going to do he says I need to find a multiple of 10 which is not quite right terminology math but what he means is I need to find a common denominator and that common denominator is 10 which is correct.

He also says 1/2 is 5/10 and 1/5 is 2/10 and it does that really rapidly and fluently. The student really knows a lot about fraction what he doesn’t know is when you multiply fractions you don’t need to find common denominators and find equivalent fractions like that.

He says multiply that I think that’s what that’s about as multiplying the numerator is 5 * 2 is 10 you keep that denominator of 10, you end up with a fraction of 10 over 10 which simplifies to 1  that’s exactly how this student came up with answer one and it’s perfectly reasonable to student to come up with that answer because it’s exactly like the procedure that you would use to add fractions when you’re adding fractions the correct procedure is to find common denominators create these equivalent fractions apply the operation in this case addition to the numerators you keep that common dominator in this case you don’t need to simplify but if you did you need to simplify at the end.

So, what the students doing is making a mistake that’s not a mistake that the student kind of intuitively came up with, the student just missed applying prior knowledge and the reason the students missed applying the prior knowledge is at the student doesn’t understand where the procedure for adding fractions came from.

Or whether procedure for multiplying fractions came from. So, the student doesn’t have the conceptual understanding to know WHEN, so you can generalize the addition procedure to subtraction but not to multiplication because they’re fundamentally different operation. And so, one of the things we do in MATHia is to give student a model and I’m just going to present kind of video of a student solving an addition problem and what the students doing is dragging over fraction bars the student can intuitively say see that the sum of those fractions is the length of this bar. But you can’t really measure the length of that bar without coming to common denominators. So, what we were able to show the student is in an addition procedure the reason you need, common denominators is you need to have a way to measure the length of this combined bar and then at the bottom you can see where relating that concrete process with a fraction bars of the students are doing to the kind of abstract and procedure that you would execute without drawing a diagram at the bottom.

And that’s the essence of what we do. In MATHia as the student goes through each of these steps dragging each of these bars over selecting the common denominator interpreting that to an equivalent fraction interpreting that in terms of the process each of those steps just like he just has some filling in a Cell in the spreadsheet that I showed in word problem earlier is related to one or more of the eligible products that come from that kind of ACT-R model.

Mastery Learning

And what we’re doing is evaluating the students current state of knowledge at a very fine-grain level and then putting that together to do what we call Mastery learning to understand when is the student ready to the progress to the next topic in the curriculum.

And this is what it looks like in MATHia again this is a different kind of word problem on the left side here on the right side is what we call this kilometer. This is a representation that’s available to students and teachers. The student’s kind of jog as they’re solving problems is to fill out all these dials.  The dials are sensitive to the particular step in the problem that the students solving and also to problem characteristics because different varieties of ratio problems like this involve different skills. So, this ratio problem is an interesting skill which is: –

You are hiking at Appalachian Trail in the Eastern US. You plan to hike the section of the trail across New Jersey at 9 miles per day and the hike is going to take you 8 days. The question is what’s the length of the trail in New Jersey? One of the interesting things about this problem is a phrasing of 9 miles per day. The student needs to express at a rate as a ratio of 9/1. Some students would actually find this problem easier if it said nine miles every two days.

Because it says 9 miles per day that number one is implicit in this phrase and that’s one of the kinds of skills that we pay attention to because what are data has told us is there are some students who actually struggle more with problems that have these kinds of unit rates and applied 1 as the denominator and so by specifying that as a separate skill we can direct instruction particularly to students who are struggling with that variants of this type of problem and we can also ensure that every student gets enough practice with problems that have the unit rate that they’re able to understand it and executed well and fluently.

This kind of approach of Mastery learning of ensuring that students understand the prerequisites before they move on was codified by this guy Benjamin Bloom another kind of influence of ours.  In 1968, he kind of formalized with series about the idea of mastering prerequisites before moving on is certainly an ancient idea. I’ve been reading a lot of Blooms old writings and I love this quote from him. Where he says and this is based on his data in working with thousands of students in the 60s and 70s, He says “Most students become very similar with regard to learning ability, rate of learning and motivation for learning -when provided with favorable learning conditions.”

That’s really a different way of saying what we mean by student-centered learning. We believe Mastery learning is kind of a statement that we really believe that every student can learn if the student’s not learning or even if the student is not motivated for learning it’s really kind of on us as teachers right, we haven’t provided the student with a favorable learning conditions that allow that student to learn.

Those favorable learning conditions you know embedded within our software about personalizing experience for students understanding what the student knows and doesn’t know so that we give a student an experience of success over time.

Research

Focus on research the other part of it is being willing to be judged in these kinds of formal studies. This study was conducted by the Rand Corporation couple of years ago. What they did a really formal randomized trial so they actually took 147 schools that were teaching algebra 1, they randomly assign those schools to either use our curriculum or to use an alternative curriculum. This was funded by the US Department of Education. It costs six million dollars to run a study of this size and figure.

And what you can see at the end of the study is, student using our curriculum roughly doubled their growth and learning. As compared to the control group. And that includes all students regardless of whether they used MATHia in our curriculum well or not. If you look at students who essentially use MATHia fairly minimal amount of time you get a two and a half times improvement relative to the traditional attraction control group so we’re really proud of those results.

Ongoing Research

Another thing that’s really sort of unique about Carnegie Learning is a commitment to ongoing research and this comes from that guiding principle that I talked about before that the more we learn and understand about student learning the better we can help them learn.

And so, we are active participants in externally funded research that’s fundamental research about how students learn and how to apply that knowledge to student learning. For example, current research, The Learner Data Institute is a partnership with the University of Memphis funded by the National Science Foundation which is all about developing new techniques to learn about data and understands student learning based on Big Data.

We have a grant from the US Department of Education with Carnegie Mellon University that’s looking at teacher orchestration so what kind of – how do we use that the data that’s coming from MATHia to help guide the teacher to work with the students who most need help. One really unique aspect of that is implementing augmented reality. This is not productized yet, it’s a vision of where we might go but we’ve developed the system with the Microsoft HoloLens which is an augmented reality headset with a teacher can kind of look out over the classroom and you can see these indicators floating over students’ heads so the teacher knows I need to go talk to Johnny over there because he’s struggling. We have a grant from The Gates Foundation at Schmidt Family Foundation he used to be the chairman of Google, which is focused on AB testing which is basically being able to run more field test that give us solid evidence about what’s working within MATHia and Fast ForWord as well.

And we have another Grant from the National Science Foundation with the University of Illinois in Pennsylvania. That’s focus on algorithmic Bias. Which is a big concern and is making sure that our algorithms are fair to all students.

I will stop sharing there and thank you very much for your time. And I’m happy to talk about questions at the end.

https://docs.google.com/presentation/d/1fKVREpjBPkL-IXwCrIBvPQTrqdPcDA-YnN6EYc51_vE/edit?usp=sharing