Alston Road

ANALYSIS, STRATEGY & PLANNING FOR DIGITAL HIGHER ED
03/05/2013

Technology, Transparency and MOOCs

Summary

MOOCs have introduced a greater level of transparency in online higher education. They offer students a chance to evaluate and compare institutions to a degree previously unheard of in higher ed. The focus of the evaluations is, primarily, instructional content and related activities. This focus may create new opportunities for less prestigious institutions to compete.

::

Technology’s Unintended Consequences (Strike Again)

Before the concept of the MOOCs was adopted by elite universities and became “a thing” in 2012, it had  a decidedly anti-establishment posture.  These courses had a DIY quality. They were created and run by people excited by the possibilities of forming ad hoc online communities of learners that could use the Net to learn what and how they wanted. By design, MOOCs operated outside of the constraints of traditional higher education. Today, of course, MOOCs are associated with institutions like Stanford, Harvard, McGill and other institutions which see an opportunity to combine their international brand recognition with open courseware in order to stake out a large slice of the future of digital higher education.

This may not be exactly what the people who started experimenting with MOOCs had in mind, but innovations have a history of leading to unintended, even contradictory consequences. MOOCs are no different. Indeed, the trajectory from DIY to “upmarket” may not be the most interesting unintended consequence; MOOCs may have also inadvertently ushered in a new degree of transparency in higher education.

By providing free and public access to courses and faculty – MOOCs and other “open” initiatives, such as OER – enable learners and other stakeholders to review, evaluate and compare an institution’s core “product” without ever being admitted to the institution. Comments by students are beginning to appear across the Web on the relative merits of different MOOC courses and platforms. And new portals have been created that allow students to select and rate courses from different institutions, much as they would a Hollywood film (see here and here).  The change can’t be overstated. A MOOC can be witnessed by 100,000+ people and discussed in the New York Times. On the other hand, faculty have often been hesitant to have a colleague sit in on their lectures. Night and day.

Transparency & Evaluation of Quality

This is new territory for higher education. We’ve not been subject to the transparency and “perfect information” that many sectors of the economy have faced. For example, customers seeking to purchase appliances at the local mega-appliance store now come armed with more information on-hand than the salespeople. Vacation resorts can’t stop tourists from sharing their bad experiences in online forums. By and large, though, students still make decisions about which institution to attend with virtually no direct exposure to the quality of teaching they will face over the next few years.

If the MOOC phenomenon continues to pick up steam, and more institutions see value in making one or more of their courses available through these public platforms, then MOOCs may become a key platform by which the value and differences of institutions of higher education are evaluated and, as a result, the terrain for competition between institutions.

But on what basis are institutions competing?  To answer this we need, first, to recognise that MOOCs expose certain parts of the  participating universities and not others. MOOCs place certain qualities and features of the universities on display, while other features are hidden. As it has throughout history, technology changes what we pay attention; what “matters” and doesn’t. (For example, the advent of television changed what kinds of people were “electable” in our political systems.)

Greater Focus on Instructional Content

The feature that is privileged in MOOCs is instructional content – the material that is presented to the students in the course.

Instructional content is privileged for a couple of reasons. First, because MOOCs are stripped of many of the other common elements and experiences that usually come packaged with being a university student: loans, registration processes, socialising, and concern about grades. (Because of how students are currently using MOOCs, the vast majority of MOOC students are less concerned with grades than in their college or university courses.). Consequently, instructional content is proportionally a larger part of the overall experience. Second, instructional content is a relatively tangible part of the learning experience. While learning is as much a “process” than artefact, evaluating a process is relatively difficult. Instructional content, on the other hand, is tangible and can be compared with relative ease to content in other courses.

On some level, participating institutions already recognise that these courses are being used to showcase the institution and its faculty. Universities are putting more effort into their MOOCs than is typical of online courses. Duke University recently provided a recap of the process they went through creating and launching a MOOC. They reported a total of 620 hours of labour for the development stage – well above sector norms. That included over 11 hours of video (12 individual videos per week) and more than 1000 files for an eight hour course.

But the effort at Duke will likely pale in comparison to the type we will see in future course developments. Once it becomes obvious to university leadership that these courses are serving as a calling card/front door/flagship for the institution we may see what amounts to another variety of the university “arms race” – this time focussed on instructional content in publicly available courses. I think it’s safe to assume that the President of one Ivy League school won’t be thrilled if the courses offered by their institution look shoddy and home-made in comparison to what’s coming out of another Ivy League school.

The significant impact that MOOCs can have on a university’s reputation was nicely illustrated earlier this month when Georgia Tech decided to pull the plug on its Coursera MOOC after only a few weeks, due to challenges with its design and execution. Those that work daily with online courses in traditional colleges and universities can vouch for the fact that many, many bad courses are designed and offered each semester, many of them incomplete at the start of term – no different from campus-based courses, in this respect. But these courses are not pulled from the shelves mid-stream. The difference, of course, is that the Georgia Tech course was part a high-profile initiative. An institution’s reputation was on the line.

Content: “It’s a Good Thing”

Instructional content has received very little attention in digital higher ed, to date. Some equate concern with the quality of instructional content with passive, one-way learning. They see interaction as the primary basis of learning.

While interaction is fundamental, so is content; the importance of one feature does not mean that the other is irrelevant. No matter what the subject matter, high quality, thoughtfully presented instructional content – whether it is illustrations, videos or well designed activities – is an absolutely powerful component of learning.  In fact, I expect the role of content to grow more important as the two currently distinct spheres of content and software merge (e.g. adaptive software, simulations), and as higher education moves beyond the current “cottage model” of content development in which much of the burden for content development falls to lone instructors without the time, incentives or necessary skill sets. I find dismissals of content’s importance quite simplistic, frankly, and when these arguments are put in the form of high quality content, humorous.

The changing status of instructional content can be seen in the trajectory of open educational resources (OER). When individual academics began in the late 90s to make components of their courses available on repositories like Merlot, it was of little significance. Anecdotal evidence suggests that academic leadership were rarely aware of the faculty’s involvement in these initiatives. They reasoned that the intellectual property resided with the faculty member, and if they wanted to dedicate the time to participating in these initiatives, this was the faculty’s concern. Today, decisions about participating in Coursera and other open content/course platforms involves the University Board, investigations by General Counsel, and planning from the VP of Marketing.

The extent to which an institution will seek to use MOOCs as a showcase for their online courses will be influenced by the degree to which the course is affiliated with the institution. Some MOOCs are presented clearly as output of the institution. The most direct path to communicating this direct affiliation is to (a) give the MOOC course the same title as the course within the university, (b) have it taught by an academic of the university, (c ) have the academic identified as a member of the faculty at the university. For illustrative purposes, consider the particular way that Udacity defines the origins of its courses (Figure A).

Compare Udacity’s approach approach to labelling the origins of the courses to Coursera’s, which links the MOOC directly and fully to the institution. Both models rely on the credentials of the instructors (s) behind the course, but Coursera aims to define the course more closely with the partnering institution (Figure B).

The difference stance in relation to universities taken by these two platforms is significant as the closeness of the affiliation to prestigious institutions was the basis of the original excitement about MOOCs in 2012. Private vendors had been doing MOOCs since the 1990s, after all, but to little excitement. Nor would we have read about MOOCs in the New York Times or The Guardian had they come to us via Pocatello Junior College.

The excitement about MOOCs was a by-product and reinforcement of the logic long used to evaluate and rank higher education institutions: the more exclusive, the better. The assumption is that the instructional content made available through these initiatives are of value because they are the product of these prestigious, highly selective institutions. That exclusivity and research productivity doesn’t necessarily correlate with instructional quality is well . . . interesting. And herein lies an opportunity (read on).

A Different Basis for Competition

There is very little stopping less prestigious institutions from producing higher quality courses than the elite institutions. Because the basis of competition has changed, and instructional content is now a key driver of value for learners, other colleges and universities that have made a significant investment in online education could produce high quality courses – as good or better than today’s MOOCs. In fact, as I suggested in a post last spring, the elite institutions may be less well suited to producing high-end instructional media. These institutions established their strong brands through research. Less prestigious institutions have generally focussed more on teaching and dedicated more of their resources to online learning, on average, than the elite institutions that currently dominate the MOOC space.

Technology has the power to change the basis upon which institutions compete. The oil crisis of the 1970s changed what mattered to car buyers; they wanted fuel efficiency. Honda and other then marginal players in the auto industry seized the opportunity and offered fuel efficient cars. Honda and Nissan (then Datsun) likely couldn’t compete head to head with the major US auto manufacturers, but when the way in which cars were evaluated changed, they took full advantage.

Similarly, technology is starting to change how value is defined in higher education. By deciding to take advantage of the technology’s capacity to distribute their courses, elite institutions have provided a previously unheard level of exposure to their core “product” – courses. But in doing this, the participating institutions also provide the means for learners and other stakeholders to determine quality for themselves. In turn, this creates opportunities for ambitious institutions of higher education, just as it has in other sectors, to compete in new ways.

By Keith Hampson, PhD. Analyst and Consultant to the Digital Higher Education Industry

02/28/2013

Five Big Issues: Hanover Research

Sometimes the best way to understand what’s going on in higher education, or any field for that matter, is to ask a research firm. Because they work with a range of colleges and universities daily, research firms enjoy a particularly useful view of the landscape. They get to hear what’s keeping academic leaders up at night.

Case in point: I recently spoke with Cam Wall, Content Director for Higher Education at Hanover Research (Washington DC). Cam touched on five of the stickier issues – some predictable, others surprising.

ROI

Institutions are striving to identify new ways to make their programs and operations financially sustainable. For the last couple of years, particularly, institutions want to know what new business models and which markets they need to focus on to ensure they address the new budget realities.

Student Support

What kinds of student support do today’s students need? What’s the best way to deliver this support and how might the support translate into student success and retention?

Workforce Readiness

How do we best go about preparing students for the workforce? What constitutes workforce readiness in 2013? And how can the institution demonstrate to regulators and other stakeholders that their learners are properly prepared?

Logo_hanover

Assessment

How can the institution do a better job of assessing student success? What are the most meaningful rubrics, how do we manage this data, and once we have the data in-hand, how do we use it to institute change across the institution?

Online Education

Not surprisingly, Cam reports that online education is major focus for many of the firm’s 450+ clients. Many institutions, particularly during the past year, are asking how best to invest their limited resources so that they can properly respond to the sector’s embrace of online education. Institutions also want to know where they stand visa vie other institutions; how far ahead or behind they are and what will it take to be competitive.

Big questions, all.

By Keith Hampson. Analysis, Strategy, and Workshops

01/10/2013

Digital Content (Finally) Gets Serious. An Interview with Dr. John Boersma

If there’s a weak link in digital higher education, it’s instructional media. Too often college students are presented with repurposed text, cheaply and quickly produced by people with limited skills, time and incentives.  This is using the Internet as a distribution mechanism – nothing more. 

A new company, Adapt Courseware, is taking a different approach to instructional media, though. I see great promise. I asked CEO John Boersma to provide some details. 

7D6E7C09-B9D9-4DBC-A552-8782F9760440-1Keith:  To the extent that we’ve paid any attention to instructional media, our focus has been on costs (e.g. OER). Given the degree to which your model focuses on quality, am I right to assume that you see a growing recognition of the importance of quality on the horizon? What do we know about the instructional value of high-end instructional media?

John: In general Adapt Courseware strives to deliver the best online learning experience in all its aspects. Effective multimedia is indeed one of our five core design principles, along with mastery learning, optimal challenge, student choice, and social learning.

There is quite a rich body of empirical research defining and supporting the value of effective multimedia, which is well detailed in Richard Mayer’s excellent book Multimedia Learning. For example, Mayer shows that five out of five studies conclude that students learn better from graphics and narration alone than from graphics, narration, and on-screen text. These studies generally show a large effect, and yet to this day you see far too much redundant onscreen text in online curriculum. 14 out of 14 studies show that students learn better when extraneous visuals are removed, and yet much MOOC content coming out even in 2012 distracts the student with talking-head videos of professors, or even “writing-hand” videos.

So you put this point in just the right way – we start by focusing on the quality we are trying to achieve, and then determine the cost to support that. This is quite different from most online curriculum development, which starts with a fixed budget – one that is generally too small. Quality for us ends up being defined in terms of a mix of effective visual styles that let the eyes and ears work together: good animation with high quality voice over, not too much text on screen, minimal on-screen time for the presenter, and so on. Good instructional video should approach the design and quality of good documentary film, where an audience that is not captive must be engaged.

At Adapt Courseware, we are currently engaged in a vigorous internal debate about these issues, because it is expensive. We ask – are we at risk for providing a higher level of quality than the market will pay for, especially in a climate of understandable downward pressure on curriculum costs? My view is that the answer will emerge from our efficacy testing – that we need to continuously assess the costs of production against the quality of the learning outcomes. PastedGraphic-2For now, we are making sure we err, if at all, on the side of quality. In the long run I feel confident that showing the benefits of what we do in producing better learning outcomes and higher course completion rates will support the needed costs.

Keith: “Adaptive” can mean many things. How exactly is Adapt Courseware adaptive?

John: “Adaptive” means that the learning environment adapts to the student, so that each student receives an individualized and optimized learning experience.

In a narrow sense, our platform is adaptive in that it delivers multimedia learning interactives adaptively as needed to support our mastery learning and optimal challenge design principles. We start by fine-graining course content – defining 200 or more learning topics for a typical three-credit course, each with its own set of learning objectives. Each topic gets its own multimedia suite – a three to six minute instructional video, an ebook section, and a set of learning interactives based on visuals from the video. These learning interactives are arranged in an “adaptive stack”. That is, they are arranged from the simplest single concept interactives – does a student remember and understand key terms – to the most complex, applied, multi-concept interactives. You can think of the adaptive stack aligning with Bloom’s taxonomy of learning, from simple remembering and understanding exercises to complex analysis, critical thinking, and application exercises.

course-statisticsAs students demonstrate mastery of concepts by successfully completing multimedia activities, they accumulate points on a “mastery meter” for that topic. We find that this real-time feedback to students is highly motivational. It’s like a video game – students can’t resist running up the score! That’s how we support mastery learning and how we can provide highly measurable learning outcomes to institutions. If we have an excellent learner like you, Keith, the system will quickly adapt and provide you only the three or four most complex, multi-concept activities you need to demonstrate topic mastery. A more typical student might require five, 10, or even 15 activities to reach mastery.

This is possible because the interactives are primarily designed to instruct, and only secondarily to assess. Any incorrect response will produce a hint designed to coach the student to the correct answer. Mastery points can’t be earned in this way – they are only available if an activity is completed correctly on the first try – but the hint engine will deliver to a student the most helpful hint based on the student’s incorrect response. The hint engine is really a second way in which the system is adaptive – different incorrect responses produce different hints.

However, getting an activity correct on any of three tries means you move up the stack to more challenging activities. Getting it wrong altogether means you move down the stack to more elementary activities. This is how we strive for optimal challenge for students – since each student gets a set of activities that steers toward their ability level, we reduce the extent to which students are bored by work that is too easy, or frustrated by work that is too hard. This helps with student retention and course completion.

That’s how the adaptive stack works today – it’s a good, effective start – but we are working now on far more sophisticated algorithms based on the data generated by large student populations.

Keith: The focus of analytics in digital higher ed has, to date, centered on the needs and interests of the faculty and the institution, rather than the student. But your system also provides students with analytics. Explain?

John:You are right; the student analytics have three audiences – the instructor, the adaptive system itself, and the student. By providing a rich set of analytics to instructors – more than 20 real-time global metrics per student, as well as per student performance on each learning topic – we give instructors the tools they need to monitor performance closely and intervene where helpful. It’s part of a larger strategy to allow instructors to focus on higher-value activities. There’s a lot of power latent in these metrics – we anticipate some really exciting announcements on that in the future.

For the students, real time feedback is key. We effectively say to a student, for example: Ok, you are currently at a B minus level of understanding of your current topic. Happy with that? Then it can be beer time. Or, you can work another 10-20 minutes to get that understanding up to 100%. Students generally hang in there – real time feedback strongly increases active learning time, which is a foundational element of student success.

There’s a theme here – we are always thinking about student motivation and how to support it. Think about a typical online student – let’s call him Joe. Joe works all day at his job, and then when he comes home, it’s kid time. Kid time goes on until eight or ten at night, and then Joe sits down to work on his online course, so he can get a better job.  A lot of online learning really does occur between ten in the evening and two in the morning. In many online courses today, Joe is confronted by a 30 page textbook chapter to read, perhaps online. It’s kind of brutal. No wonder course completion rates for general education courses aren’t great. Our approach is much, much more accessible.

Academic object analytics look at the same data through another lens – just how effective is that instructional video, text, or multimedia interactive across all students? We work on continuously improving our content based on this feedback loop.

Keith: There’s growing interest in competency-based learning in higher education; a shift away from “seat-time” and toward measuring learning. What role do adaptive learning systems have in competency-based learning?

John: This is a really fascinating area, and I think we are going to see change here at a pace that will surprise people.

The whole system of higher education today is based on authority: a professor says that a student has a B-level knowledge of accounting, say. A college says a student has learned enough to be educated at the Associate’s degree level. An accreditation body says a college has effective controls in place to make these claims. The system has all been based on authority precisely because actual learning was hard to measure – and it was hard to measure because it was all on paper.

Now, with online learning systems like Adapt Courseware, what the student has learned is objectively measured, defined in terms of hundreds of learning topics per course, and comparable across many institutions and large populations. We don’t need an authority to tell us that Joe has mastered depreciation in a financial accounting course but doesn’t really understand bonds. We can measure that directly, and so can Joe. Joe can then tell anyone he likes what he knows – precisely, accurately, and credibly.

Innovators like Western Governors University have taken important steps in this direction with their competency-based approach. The MOOCs are popularizing this idea in an important way. But only now are the tools needed to do this with a high degree of specificity becoming available. It’s going to be fascinating to watch the impact this trend will have on the accreditation model, and even on student assessment of the costs and benefits of traditional higher education altogether, over the next few years.

Once the system moves to a competency-based model, the goals of the institution flip from “has the student sat in the class long enough?” to “just how efficiently can this topic be taught?” If a student can master an introductory Psychology course in 75 hours, should they be required to do more hours of work? In a seat-time model, yes. In a competency-based model, no. If the curriculum can be improved so it only takes 60 hours, the student and society benefit. This is where adaptive systems come in – each student spends the time they need to reach mastery – no more, and no less.

Popular Related Posts

context + competition in digital higher education

an ode to content and first thoughts on adapt courseware

not quite right: higher ed’s business model and instructional technology

moocs: the prestige factor

12/29/2012

Recommended Post-Holiday Track

Bill Evans, Peace Piece (1958) from “Everybody Digs Bill Evans”.  Ideal for late night work.

12/27/2012

Week’s Most Interesting :: 12.27.2012

Hand-picked selections of articles, reports, blog posts and events from the last seven days (or so).

::

Learning New Lessons

I’ve been in the digital higher education arena long enough to still be shocked when major, mainstream news outlets pay attention to what we’re up to. It seems only yesterday that we were eating at the kids’ table. Last week’s article in the Economist offers the now oft-repeated claim that MOOCs will put some institutions out of business.

Excerpt: Top-quality teaching, stringent admissions criteria and impressive qualifications allow the world’s best universities to charge mega-fees: over $50,000 for a year of undergraduate study at Harvard. Less exalted providers have boomed too, with a similar model that sells seminars, lectures, exams and a “salad days” social life in a single bundle. Now online provision is transforming higher education, giving the best universities a chance to widen their catch, opening new opportunities for the agile, and threatening doom for the laggard and mediocre. Read the full article.

::

Online Education Whitepaper

University Ventures is a 2 year old private equity concern that invests in promising businesses that serve the needs of colleges and universities that wish to go online. Below, they offer an interesting whitepaper on digital higher education. Worth a read.

Excerpt: Since establishing University Ventures nearly two years ago, we have written and spoken on many aspects of higher education and online education in particular. With nearly 15% of U.S. students enrolled in higher education studying entirely online and earning degrees without ever setting foot on campus, and with online education in the headlines and popular consciousness like never before, this holiday season we thought it would be a nice gift (to ourselves, primarily) to organize our views on the evolution of online education and its impact on higher education more broadly in a handy whitepaper format. Read the full article.

::

The Best Ideas of 2012 from the New Economy

Higher education, despite its best efforts, has never been independent of the broader forces sweeping society. Not surprisingly, then, I find I learn at least as much from articles that appear irrelevant to higher ed. This post on Quartz by Glen Kelman is a good example. (Actually, one of his ideas addresses education directly; see below.)

Excerpt: In 2012, Silicon Valley stopped complaining about the shortage of educated talent and started doing something about it. Microsoft is sending software engineers into high school classrooms. A spin-out of Amazon engineers, Vittana, just raised money to support a wildly successful micro-finance site for funding education in developing countries. Universities launched the first large-scale massive open online courses on everything from math and cyptography to finance or a crash course in creativity. Read the full article.

::

Pearson & Rutgers Partnership

I’m in the last stages of writing a book. One aspect of the book’s thesis is that traditional colleges and universities are ill-prepared and, in certain respects, incapable of fully leveraging technology to improve the value of online education (cost and quality). When I first started writing the book in earnest, symptoms of the structural limitations of traditional higher ed were hard to come by. Today, I can barely keep up. Here’s one to consider: the growth of public-private partnerships.

Excerpt: For more than a decade, thousands of Rutgers University students have been able to broaden their educations by taking a wide range of online courses offered by the university. Now Rutgers is preparing to significantly expand the reach of the university’s online curriculum to even more students across New Jersey and beyond.

The university today announced a new public/private partnership with Pearson, provider of educational technology, content and services, which positions the university to significantly expand lifelong learning opportunities — including undergraduate and graduate degrees available entirely online — while maintaining access to the same level of academic quality that is offered in the traditional Rutgers classroom. Read the full article.

11/22/2012

Change in Higher Education: An Educator’s Perspective (Interview with Dr. Jesse Martin)

Dr. Jesse Martin is a thought-provoking educator. A Senior Lecturer at Bangor University in Wales, Dr. Martin focusses on the role of evidence-based in university education. Below, Jesse and I exchanged notes about the nature of change in higher education. 

* *

KCH: You wrote earlier this fall that many of the academics with whom you speak tend to be defensive when discussing the anticipated transformation of higher education; and that they “think that the world will change at their pace.” If this is the case, will learners – and those that fund higher education – wait for them?

JM: There is a great deal of unease in the sector because we can all see that things are changing. There is nothing new in change – that has always been with us. What is causing the unease is both the scale and speed of the change.

The first part of your question – will the learners wait – depends on what you mean by a learner. Most of us in higher education are painfully aware of the disconnect between learning and education today. A great many of the students don’t come to HE for learning, but come for a qualification. They want the points at the end of the game, and what they want from us is a clear set of rules and guidelines that will allow them to maximise their points and minimise their efforts. For these stakeholders, learning is one of the necessary challenges that occasionally gets thrown up as one of the obstacles in the game.

On the other hand, there are still some – I would guess the same relative proportion of the population – who have always wanted to learn, and that’s all they come to university for. Those who want to learn are satisfied with things as they are, and can’t really see the need for any massive overhauling of the system. They will succeed at the game, regardless of what we do to or for them. They will wait.

What massification of HE has done is increase (by a lot) the numbers who only want a qualification. I would say that the split is about 80/20, with the smaller number wanting to learn.

However, the larger group, the group that is in HE for a qualification, want to maximise their return with a minimal investment. For them they want as much prestige as they can afford, and as high a qualification as they can earn with as little effort as they can get away with. This is the group that I think will drive the greatest changes in HE. They will opt for non-traditional offerings if it works to their advantage. They know what they want, and will adapt to get it. They have no intention of waiting for anything (3 second wait for a website and they’re gone). If something that they think will suitably meet their needs comes along, they will opt for that.

The funders are another story. Private, for-profit providers are quickly moving into the non-traditional market, and as long as the sector is in a sellers market, the for-profits will charge a fortune and deliver a minimal experience – they are profit-oriented, and so will maximise their profits. As the number of available student places grow, and the availability begins to meet the demand, these for profit institutions will have to become more competitive, and the cost of a decent qualification from a for profit will drop dramatically. This will create high growth in this sub-sector as more and more students opt for private providers with better value for money (as far as qualifications go)

Governments are already withdrawing from the HE teaching sector, and are driving much of the change that we are experiencing. Although political expediency means that they pay lip service to teaching and learning, I think we will see a concentration of public monies in the big research-intensive institutions where the primary purpose is not teaching, but bringing prestige to the funder.

I think that the only significant stakeholders who will be willing to wait for lecturers to adapt are the students who really want to learn. The rest will (or are) simply react to the changing environment.

KCH. What are one or two of the more promising instructional approaches you are currently employing for your learners at Bangor University?

JM: The traditional approach to instruction involves a teacher reading, organising, and presenting information in bite sized chunks to the students for them to regurgitate back to me (with a little bit more) at some later date. My approach is to not do that.

In one of my classes (statistics in psychology) we’ve moved to a largely practical and problem based approach to teaching. The students produce data sets, and then I ask them what they want to find out from the data and how they think they might get it. I get some interesting approaches to data analyses (did you know that doing a find and replace in Excel returns the number of occurrences – an interesting way to count responses). The classes tend to be split on the approach with about half the students loving the freedom and half hating it that I’m not telling them what I want them to do. Regardless of what they feel about the approach, they learn how to manipulate numbers, and they understand the questions that they are asking of the data. I use a problem based, practical approach to learning a skill that they need in order to carry out research in psychology.

My most exciting class is about applying the principles of psychology to education. I provide loose guidelines about what I expect the students to bring back, and then I send them off to find out what they can. They give bi-weekly presentations (non-assessed) to each other (and to me), and write weekly blogs (assessed) about what they find. All I do is evaluate what they find.

As one of my students wrote last year: To put this bluntly, Jesse did nothing in this module; yet in doing nothing he did absolutely EVERYTHING… (The) end goal is learning. It’s the teacher allowing the students the freedom to make the classroom their own and to be there for them whenever they need a push in the right direction. This is proper teaching… (http://jackinabox2906.wordpress.com/2012/04/19/week-10-contemplating-educating/)

KCH: One of the aspects of the response to MOOCs that I have found most interesting is the implicit assumption that great research institutions necessarily produce great learning experiences. As an academic, how do you see the relationship between subject matter expertise among academics and educational value?

JM: I wrote about this earlier this year (see: http://hethoughts.wordpress.com/2012/05/25/the-dr-is-in-the-room/) and I still feel the same. I think that the two skills (teaching and research) are orthogonal. They have no relationship whatsoever. Great research is a mix of personality traits (inquisitiveness, attention to detail, methodological approach, etc.) and good training. Great teaching is a mix of personality traits (enthusiasm, empathy, positive outlook, creativity, etc.) and good training. There may appear to be a great deal of overlap, however the personality traits are different, and the training is different. Many will be good researchers, and many can be good teachers, with a few who end being great at both.

I would go so far as to suggest that being a hyper-expert can be (and often is) detrimental to good teaching. Often, the detail gets in the way of the story. Keeping in mind that the researcher is the one who knows about the detail, it is only natural that their interest gets in the way of good teaching.

I read somewhere (can’t remember where) that a topic area that could be covered in five minutes during a lecture in 1955 could be expanded to a full hours lecture by 1976. That same topic area can now be expanded to cover a full 15 week module. Hyper-experts end up so far out in the tendrils of their fields that the bigger picture gets lost along the way.

The second factor that this question alludes to is the ability to teach (my definition of teaching is to foster learning). Too often academics think that anyone can teach. Someone with years and years of experience in the field of education (first as a child, then as an undergraduate, and finally as a postgraduate) who eventually gains a PhD (or even a BA or BSc) knows what good teaching is (teacher cognition). They know what good teaching is because they have experienced it (as well as bad teaching). These pre-conceived ideas of good teaching by academics are highly resistant to change. However, just because you have experienced a lot of teaching doesn’t mean you are a good teacher.

Because institutions are made up of individual faculty or staff members who teach, the institutional reputation for research in no way reflects the quality of the teaching. Teaching is done by individuals, and although some researchers end up being good teachers, if the institutional focus is on research, good teaching is something usually happens by accident. Usually, leading research institutions put enough resources into teaching to achieve mediocrity and keep the student complaints to a minimum.

KCH: You’ve written a good deal about Christensen’s concept of disruptive innovation as it applies to higher education. Are there early symptoms of this disruption to which you think we ought to be paying more attention?

JM: The innovation that I think is the disruptive innovation or paradigm shift is the ubiquity of information. Digitisation has moved the world from information scarcity to information abundance. The symptoms of this change occur at both a micro and a macro level. The micro level symptoms include the general unease among lecturers (what is my value added), the rise in plagiarism, the demands of students for a better packaged product – these are things we are facing every day and should indicate (using John Naughton’s metaphor) that water is getting into the ship.

The macro symptoms that are easier to follow and notice as a community would include the encroachment of for profit institutions, the rise in open educational resources and open content, and the increase in teaching methods that take advantage of iniquitous content (e.g. MOOCs, blended learning, or flipped classrooms).

Christensen’s research demonstrates that the macro symptoms will be resisted by the established institutions to the point that so much resource has been put into defending themselves from the symptoms that we begin to see exhausted institutions collapsing in on themselves with no energy left to adopt.

The box has been opened, and the genie is out. We either figure out how to survive in this brave new world or we don’t.

Follow Jesse Martin’s blog HE Thoughts

2 comments
10/29/2012

The Pearson Think Tank (UK): An Exchange with Louis Coiffait

Reblogged from higher ed MANAGEMENT:

  • Click to visit the original post

Louis Coiffait is Head of Research at the London -based (UK) Pearson Think Tank. Louis and I had a chance to speak recently.  

KCH: It's quite unique to have a think tank directly associated with a private company. From where did the idea for launching a think tank at Pearson come? What are its main objectives?

LC: The Pearson Think Tank…

Read more… 1,241 more words

10/09/2012

Alfred Essa: The State of Analytics in Higher Education (Interview)

I’ve known Alfred Essa for a couple of years. From the start, Al struck me as one of those people that is truly focussed on improving higher education. After a career at various organizations in the US, including MIT and Minnesota State Colleges and Universities, Essa now serves as the Director of Innovation and Analytics Strategy at Desire2Learn, the Canadian education technology company (which recently accepted an 80 million dollar investment from OMERS). 

KCH: I suspect there is still considerable confusion about what the term “analytics” means when applied to higher education. Can you provide your own high-level perspective?

AE: Analytics is sometimes characterized misleadingly as “data-driven decision making”. Why is this misleading? Because we use data all the time to make decisions. Analytics is fundamentally not about having more data from a user perspective. It’s about having access to quality data, relevant data, and contextual data. That’s a very hard problem which can’t be solved by merely building very large databases. When it comes to analytics at Desire2Learn our mantra is: “Less Data, More Insights.” Analytics is as much about design as it is about technology. On the technology side it is now possible to leverage wonderfully new techniques such as machine learning to uncover hidden patterns. But that’s only the first step. On the design side we refine, shape, and expose those patterns visually so that they can be interpreted by humans. On the user side analytics means looking at the world empirically and statistically, which for most of us is not intuitive and requires training.

KCH: Are there different degrees to which institutions can use analytics?

AE: It’s useful to think of analytics as having three levels or degrees of maturity. Analytics (Level I), which is the current state of most tools and also organizational capability, is reporting. It is  the realm of traditional business intelligence. What are student retention, achievement, and completion rates? How does it break down by demographics? What is the trend? Analytics I is retrospective. It’s data about the past. At best, it’s data about the present. It’s like a rear-view mirror in a car. You need it but imagine having a car where you can see where you came from but not what is ahead of you. Analytics II is data about the future. Where will I end up being? With Analytics II we begin to use advanced techniques such as predictive modeling to forecast the future. Analytics III is the holy grail.  Anytime someone wants to go from point A to point B, there will be multiple routes. Analytics III is about finding the optimal path among many to reach one’s destination.  The most competitive institutions will be those who are either at Analytics III or well on their way. The least competitive one’s haven’t even begun.

KCH: Using data to plan and manage organizations is now fundamental in other sectors. What are the key obstacles to greater use of analytics in higher ed?

AE: There are two key obstacles. The first obstacle is technical. Underneath the hood analytics is technically very complex. Most organizations lack the resources and talent to do it on their own. The second obstacle is dogmatic. The reigning dogma is that we know how students learn. The reality is that we know very little. Erik Mazur’s work at Harvard illustrates this point. The promise of analytics is that we can finally take an empirical approach to understanding the learning process and use evidence, rather than guess work or speculation, as our guide for determining what works and what doesn’t. With analytics our understanding of learning is about to transition from alchemy to science.

KCH: Which segments of the higher education market have moved most quickly to implement analytics?

AE: The for-profit sector has taken an early lead, at least in recognizing that analytics can be a differentiator. Markets tend to concentrate the mind. The non-profits have taken longer to awaken from their dogmatic slumber. But they are waking up. In the long run an advantage that non-profits have is the value they place on research and collaboration. This will be a critical driver for successful adoption. There is also the sideshow going on with MOOCs, which will be interesting to watch

KCH: Faculty can be resistant to the idea of implementing analytics, as some interpret it as a tool for tracking their performance. How do you work past this issue?

AE: I remember a conversation several years ago with Mark Millirons on this topic. Mark is a pioneer in the field of analytics. His advice to me was you have to flip the conversation with faculty. The goal of analytics should be to empower students and instructors. If faculty perceive that analytics is yet another regime by administrators to use data about them against them, then analytics will not succeed. I think of analytics as real-time instrumentation for pilots and air-traffic controllers. The student is the pilot in their learning journey. They need real-time instruments in the cockpit to help them navigate through various obstacles. Air-traffic controllers also need real-time instruments to make sure all planes in their charge or on course. The measures and instrumentation for evaluating pilots and air-traffic controllers is different and not primary. The primary goal of analytics should be to empower, not evaluate. We often confuse the two.

09/23/2012

B.F. Skinner explains the value of “teaching machines”

The word you are looking for that describes this is . . . “creepy”.

07/25/2012

Sources of Information on Higher Education

If you are interested in how higher education operates, or are doing research on issues pertaining to higher education, these organizations may be of value:

05/20/2012

MOOCs: The Prestige Factor

Buried in the public responses to the news about MOOC (Massively Open Online Courses) and OER initiatives from Harvard, MIT, Stanford, Penn, Princeton and others is a deceptively important assumption. The assumption goes something like this: the open digital educational materials made available through these initiatives are of value because they are the product of these prestigious, highly selective institutions.

On the surface, this seems perfectly logical. It’s an interpretation of value based on the deeply engrained logic and criteria that people have long used to rank different institutions: the “best” institutions, like Harvard and Stanford, provide students (those with the money and grades) with access to the most respected academics, and in turn, to the latest and best thinking on academic subjects. The excitement about MOOCs from Princeton and others is that it gives the public access to materials that are otherwise available to a privileged few.

The Mourne Wall, Northern Ireland (From More Intelligent Life)

The traditional criteria for evaluating value in higher education may be misleading in this case, however. Prestigious institutions may, in fact, be the least well prepared and least well-suited of all types of institutions to lead the MOOC expansion. The particular orientation, interests, and market focus of these institutions, may limit their capacity to meet the needs that MOOCs typically seek to fulfill.

Prestigious University = High Quality Digital Instructional Materials?

First, let’s consider the specific “output” of these initiatives. Harvard and others are producing digital education content, wrapped with some form of assessment. This work will flow out of the institution’s teaching capacity and operations.

But these elite institutions earn their high ranking by placing their emphasis on research, not on teaching. This is true on an institutional as well as faculty level. Faculty members hired by these institutions know that their labour market value is based primarily on their research productivity;  the ability to garner research funds, conduct research, and publish. When it comes to teaching, there is remarkably little focus, as Derek Bok - former President of Harvard – writes of American colleges:

“A remarkable feature of American colleges is the lack of attention that most faculties pay to the growing body of research about how much students are learning and how they could be taught to learn more. . . .  One would think faculties would receive these findings eagerly. Yet one investigator has found that fewer than 10 percent of college professors pay any attention to such work when they prepare for their classes. Most faculties seem equally uninterested in research when they review the curriculum.” Derek Bok. 

We have, then, a general misalignment of institutional strengths and incentives with the project deliverables. Yet, it is the research productivity that is at the foundation of the excitement behind these initiatives. The ability and motivation to produce high quality educational media, particularly the type that requires considerable independent learning, is not the same as deep subject matter expertise that comes from a research focus.

This is not to suggest that there aren’t a number of great educators within these institutions. There are, of course. But the ability of an organization to deliver the best possible value – whatever the type – is always dependent on the focus of the organization; what kinds of work it incentivizes, the criteria used in hiring, how it defines excellence, etc. Whether we are analyzing the politics of global trade or higher education, it’s always important to “follow the money”.

One could counter this viewpoint by pointing out that these elite institutions have the resources to invest more heavily in teaching materials. Which is true, but it also irrelevant. Yes, more money can produce better results and compensate for the lack of focus on teaching and learning. But the goal is “value”, and value is always a balance between costs and quality, and superior value is less likely to come from institutions whose focus lies in research.

Learners and Content, A Misalignment

Again, the excitement generated about these materials and courses is based largely on the fact that they come to us from well-known, elite institutions. It then follows that the more similar these courses are to those traditionally offered from the elite institutions (for the “real” students), the better.  However, the “authenticity” of MOOC’s may actually conflict with the broader social and educational objectives that MOOCs serve.

The majority of people that don’t have access to higher education and who would most benefit from MOOCs are generally speaking not the same people for whom MIT-level material is appropriate. If a student is able learn MIT material via edX, then it is highly likely that they are more than capable of obtaining access to a college education, if they haven’t already. Moreover, to benefit from these initiatives, the learner must be relatively self-motivated and disciplined. The ideal learner for such initiatives, then, is someone that is at the top of the academic ladder and self-motivated; in other words, the cream of the crop.

If these initiatives, on the other hand, plan to pitch the material at a much lower academic level; a level well below what is normally taught at their institutions in order to serve the needs of students that are more likely to need access to free courseware, then the fact that they these initiatives come from elite institutions becomes of little significance, if not misleading.

Do Intentions Matter? 

MIT, Harvard and others are not launching these initiatives in order to grow their markets, expand revenue, or reduce costs. They are not doing this in order to survive a period of budget austerity, as might be the case at other, less well-established and financially solvent institutions. In fact, growth is generally not an objective for these schools; maintaining exclusivity remains their prime concern. In order for these institutions to maintain their relative standing in the higher education hierarchy, there must be exclusivity, lack of access and scarcity.

Rather, the motivation for Harvard and others is primarily social and reputational. While the initiatives may generate some benefits for their own students (going “digital” has a tendency to impose more discipline on course design, for example), they are “giving away” their wares because they can afford to, and because philanthropic acts such as these support their brands.

The motivations of these institutions matters because it influences, first, the likelihood of success and second, sustainability. If our broader interest is in finding new strategies that will improve the quality and cost of higher education, institutions whose success is based on exclusivity and who have the most to lose if the current model of higher education is disrupted may not be the best horse to bet on.

Conclusion

I applaud the efforts of these prestigious institutions. Their participation has generated considerable publicity for new models of higher education. And their initiative creates more movement, more flux in the higher education space which likely will be the impetus for more new ideas, which is exactly what’s needed. Nevertheless, our excitement about their participation in MOOC and OER; excitement based on the traditional logic for evaluating excellence in higher education, has little bearing and relevance in this case. If our objective is to find and support new models of higher education that are likely to address the most needy students, increase quality and reduce costs, I’m not sure that this philanthropic model, coming from institutions with little need to truly innovate, and that have a deeply vested interest in the status quo, will produce the best outcomes.

Thank you to Dr. Lloyd Armstrong whose post edX: A Step Forward or Step Backward stimulated my interest in this issue. 

05/04/2012

Josh Keller: Interactive Graphics for Higher Education

KCH: Josh, I’ve long been a fan of your information graphics that you’ve produced for the Chronicle. What do you see as the value and purpose for creating these kinds of materials for its readers?

Our interactive graphics are a natural continuation of the Chronicle’s journalistic mission: to help our readers understand higher education, get better at their jobs, and make more informed decisions. For some subjects, the traditional article – a long block of text with a photo or two – isn’t the most effective way to do that.

To give one example: all of us in higher ed are increasingly expected to make sense of difficult datasets: graduation rates, admissions trends, compensation, and so on. Is my college or department doing well? How can we do better? Where will our freshmen, our donors, and our employees come from in 10 years? Interactives can suggest specific answers to these questions, while stories can synthesize trends and provide context. The two forms often work best in tandem.

Being able to play around with information – making it personal, turning it around, taking it apart – helps you understand it, in the same way that you learn something different building with Legos than you do reading a book about architecture.

KCH: Why do we see so little of this kind of content in higher education curriculum?

Well, it’s expensive to produce. The costs are declining as the tools to make custom digital presentations become more advanced – and there are many ways colleges could do much more with less.

But cost isn’t really the problem. Building great educational software requires a high level of collaboration between instructors, designers, programmers, and marketers that’s rare in traditional higher education. It requires a commitment to building effective teams in two areas, design and programming, that many traditional colleges have ignored and continue to ignore. By contrast, in Silicon Valley, that kind of collaboration and investment is the foundation on which startups are built.

Even Stanford – one of the most tech-savvy places on earth – struggles to integrate interactive tools. Stanford’s medical school gave iPads to all incoming students in 2010, and instructional-technology staff worked with professors to create custom digital materials, including an interactive three-dimensional map of the the brain for anatomy students. But students told me last year that most professors didn’t take advantage of the iPads. And the custom map of the brain was no longer used after a supportive professor stopped teaching the course.

KCH: What kind of educational and professional background is best for people that want to do this kind of work? How did you find yourself doing this kind of work?

I’ve always had two career tracks, designing websites and writing newspaper stories. I ran a web design studio, Keller & Faber, and wrote as a freelance journalist on the side. Later I became the West Coast Correspondent for The Chronicle and did web projects on the side. The fact that these were separate jobs never made sense to me. Interactive news graphics is a compelling way to combine the two fields, to use the conceptual framework of journalism and the techniques of the web.

No particular formal education is needed to do this work. My background was very helpful to me, but I learned many of the day-to-day skills by coming up with a project, searching the internet, and slapping some code together. The exact tool used almost doesn’t matter. The essential skill is to be able to communicate ideas to other people (through words, design, or other means) and at the same time understand moderately technical concepts (such as data analysis or programming). There aren’t enough journalists comfortable in both of these areas.

KCH: What do you see as the next stage for interactive graphics?

I’d hate to make much of a prediction because the field is moving so quickly. But I’m guessing that at some point journalists will have to stop thinking of interactive graphics and news applications as niche categories distinct from articles, photos, and other more traditional forms of storytelling. Well-presented interactive data already forms a basic part of the identity of companies as diverse as Bloomberg, Simple, Nest, eTrade, and Square. It will become a basic part of our identity as news organizations too, and we’ll need to figure out how to support that.

04/23/2012

Barbarians at the Gate: “Welcome!”

Earlier this spring, 800 or so people converged on the Skysong campus of Arizona State University to discuss education. However, reports from the event note that the majority of participants, and almost all of the speakers were not educators, but entrepreneurs, technology company executives and investors. The Education Innovation Summit, now in its third year, has quickly become a key event for vendors to network, generate interest, and raise funds.

Demand for higher education is at record levels, public funds are tight, operating costs are rising, and the move to digital education offers the potential to scale-up services economically.  Not surprisingly, private investment is at levels not seen since the dot-com era.

Also not surprising is that the reception within higher education to private investment ranges from indifference to outright hostility. Higher ed has a long history of managing its needs internally, and is especially resistant to commercial interventions – due to the ideal of free inquiry and the political sensibilities of faculty, among other factors. This tone tends to flavour all discussions of commerce’s role in higher education.

The tension between commerce and higher education isn’t likely to get less provocative over the next few years. While campuses have become accustomed to having vendors assume responsibility for functions like bookstore management, catering and building maintenance, the current focus of education investors is on creating technologies that serve instructional functions – matters that were once seen as the exclusive territory of academics.

But higher education may need to get over its discomfort with vendors if it wishes to continue its migration toward technology-enabled education. There are very real limits to how effectively higher ed can meet it own needs for technology. Like end users in other sectors, universities are very good at the early stage work of identifying needs and crafting initial product designs. Many of the LMS in use today started in universities. But universities are less equipped to take these products to market, scale-up production/service, and drive down costs. It simply isn’t what the institutions were designed to do.

If we are going to have educational technology available that is as good as those in the consumer markets, we will need considerable involvement of the private sector. It’s not logical to put up with a clunky LMS in our schools, but expect to have the latest iPhone at home.

Educators have started voting with their feet. More and more are integrating tools like Youtube, Facebook, WordPress and Twitter into their teaching and course management practices.  Vendors, too, have moved away from building their own applications to using popular ones from the consumer market. See, for example, Inigral and Facebook, Google and Pearson, Instructure/Canvas and Twitter.

Educational technologies need to be just as good and just as inexpensive as technology in the consumer market. We are moving in this direction – not because we are being forced by evil vendors – but because these tools are familiar, simple to use, and inexpensive.