Skip navigation.

Michael Feldstein

Syndicate content
What We Are Learning About Online Learning...Online
Updated: 25 min 15 sec ago

Babson Study of Online Learning Released

Wed, 2015-02-04 23:52

By Phil Hill

Phil is a consultant and industry analyst covering the educational technology market primarily for higher education. He has written for e-Literate since Aug 2011. For a more complete biography, view his profile page.

Web | Twitter | LinkedIn | Google+ | More Posts (285)

Babson Survey Research Group (BSRG) just released its annual survey of online learning in US higher education (press release here). This year they have moved from use of survey methodology for the online enrollment section to use of IPEDS distance education data. Russ Poulin from WCET and I provided commentary on the two data sources as an appendix to the study.

The report highlights the significant drop in growth of online education in the US (which I covered previously in this e-Literate post). Some of the key findings:

  • Previous reports in this series noted the proportion of institutions that believe that online education is a critical component of their long-term strategy has shown small but steady increases for a decade, followed by a retreat in 2013.
  • After years of a consistently growing majority of chief academic officers rating the learning outcomes for online education “as good as or better” than those for face-to-face instruction, the pattern reversed itself last year.
  • This report series has used its own data to chronicle the continued increases in the number of students taking at least one online course. Online enrollments have increased at rates far in excess of those of overall higher education. The pattern, however, has been one of decreasing growth rates over time. This year marks the first use of IPEDS data to examine this trend.
  • While the number of students taking distance courses has grown by the millions over the past decade, it has not come without considerable concerns. Faculty acceptance has lagged, concerns about student retention linger, and leaders continue to worry that online courses require more faculty effort than face-to-face instruction.

BSRG looked at the low growth (which I characterized as ‘no discernible’ growth’ due to noise in the data) and broke down trends by sector.

Growth by sector

The report also found that more institutions are viewing online education as ‘critical to the long term strategy of my institution’.

Strategic online


There’s lots of good data and analysis available – read the whole report here.

I’ll write more about the critique of data sources that Russ and I provided in the next few days.

We are especially pleased that Phil Hill and Russ Poulin have contributed their analysis of the transition issues of moving to IPEDS data. Their clear and insightful description will be of value for all who track distance education.

I want to personally thank Jeff Seaman for the opportunity he and his team provided for us to provide this analysis.

The post Babson Study of Online Learning Released appeared first on e-Literate.

Is Standardized Testing a Pediatric Disease?

Sat, 2015-01-24 14:09

In my last post, I wrote about the tension between learning, with the emphasis on the needs and progress of individual human learners, and education, which is the system by which we try to guarantee learning to all but which we often subvert in our well-meaning but misguided attempts to measure whether we are delivering that learning. I spent a lot of time in that post exploring research by Gallup regarding the workplace performance of adults, various dimensions of personal wellbeing, and the links of both to each other and to college experiences. One of Gallup’s findings were that workers who are disengaged with their work are less healthy. They are more likely to get clinically depressed, more likely to get heart conditions, and more likely to die young. I then made a connection between disengaged adults and disengaged students. What I left implicit was that if being disengaged as an adult is bad for one’s health, it stands to reason that being disengaged as a child is also bad for one’s health. We could be literally making our children sick with schooling.

I am in the midst of reading Anya Kamenetz’s new book The Test. It has convinced me that I need to take some time making the connection explicit.

In that previous post, I wrote,

Also, people who love their jobs are more likely to both stay working longer and live longer. In a study George Gallup conducted in the 1950s,

…men who lived to see 95 did not retire until they were 80 years old on average. Even more remarkable, 93% of these men reported getting a great deal of satisfaction out of the work they did, and 86% reported having fun doing their job.

Conversely, a 2008 study the company found a link between employee disengagement and depression:

We measured their engagement levels and asked them if they had ever been diagnosed with depression. We excluded those who reported that they had been diagnosed with depression from our analysis. When we contacted the remaining panel members in 2009, we again asked them if they had been diagnosed with depression in the last year. It turned out that 5% of our panel members (who had no diagnosis of depression in 2008) had been newly diagnosed with depression. Further, those who were actively disengaged in their careers in 2008 were nearly twice as likely to be diagnosed with depression over the next year. While there are many factors that contribute to depression, being disengaged at work appears to be a leading indicator of a subsequent clinical diagnosis of depression.

Which is obviously bad for employer and employee alike.

In some cases, Gallup went all in with physiological studies. For example, they “recruited 168 employees and studied their engagement, heart rate, stress levels, and various emotions throughout the day,” using heart rate monitors, saliva samples, and handheld devices that surveyed employees on their activities and feelings of the moment at various points in the day.

After reviewing all of these data, it was clear that when people who are engaged in their jobs show up for work, they are having an entirely different experience than those who are disengage. [Emphasis in original.] For those who were engaged, happiness and interest throughout the day were significantly higher. Conversely, stress levels were substantially higher for those who were disengaged. Perhaps most strikingly, disengaged workers’ stress levels decreased and their happiness increased toward the end of the workday….[P]eople with low engagement…are simply waiting for the workday to end.

From here, the authors go on to talk about depression and heart attacks and all that bad stuff that happens to you when you hate that job. But there was one other striking passage at the beginning of this section:

Think back to when you were in school sitting through a class in which you had very little interest. Perhaps you eyes were fixed on the clock or you were staring blankly into space. You probably remember the anticipation of waiting for the bell to ring so you could get up from your desk and move on to whatever was next. More than two-thirds of workers around the world experience a similar feeling by the end of a typical workday.

I then went on to a point about preparing students to be engaged workers, but it’s worth pausing here and thinking for a moment. Schooling is the model, the archetype, for the workplace experience that literally causes people to lead shorter, sadder, sicker lives. Is that hyperbole? Is it a caricature of modern schooling? Actually, thanks to the current American obsession with standardized testing, the stereotype may actually understate the case.

In The Test, Kamenetz quotes the blog of a Chicago parent who had assisted her daughter’s class with computer-based testing. On the way home from the second day (?!) of testing, her daughter broke down in the car:

“I just can’t do this,” she sobbed. The ill-fitting headsets, the hard-to-hear instructions, the uncooperative mouse, the screen going to command modes, not being able to get clarification when she asked for it….It took just two days of standardized testing to doubt herself. “I’m just not smart, Mom. Not like everyone else. I’m just no good at kindergarten, just no good at all.”

I have read this paragraph a half dozen times now, and I still can’t get through it without tearing up.

Kamenetz then goes on to say that teacher and parents throughout the United States—especially the ones with elementary school-aged children—“report students throwing up, staying home with stomach aches, locking themselves in the bathroom, crying, having nightmares, and otherwise acting out on test days.”

A bit later in the book, she writes about a couple of Great Depression-era researchers named Harold Skeels and Harold Dye. They took a couple of one-year-old babies in an orphanage who had tested as “moderately to severely retarded” and moved them to a ward for mentally disabled young women, because the children were viewed as hopeless cases. Fourteen and sixteen months old, these girls were already discarded. But what happened next was anything but what the researchers expected. The girls became adopted by the residents and attendants of the ward. Kamenetz notes, “After just six months their IQ scores had improved to 77 and 87, and a few months after that their scores had climbed into the mid-90s, near average levels.”

The researchers were so taken aback that they repeated the experiment, bringing 13 “retarded” one- and two-year-old girls from orphanages to the adult women’s institution, where they were given foster mothers there.

According to an article discussing the case, the toddlers at the adult women’s home had toys bought for them by the attendants and clothes made for them by the residents. Their “mothers” cheerfully competed over which ones could be made to walk and talk first.

Meanwhile, a control group of supposedly low-IQ girls stayed at the orphanage, presumably living under the conditions one imagines in the kind of orphanage that would let some of its children be condemned to live out their lives in a mental institution when they were just 14 months old. What were the results?

The children [who were transferred to the mental institution] remained on the ward for a mean of nineteen months. All but two of the eleven gained more than 15 IQ points during that time. Once they tested at average intelligence they were moved to regular foster homes. A year after the experiment ended, of the thirteen original children, none was still classified as “feeble-minded.” At the first follow-up two and a half years later, in 1943, the mean IQ of the experimental group was exactly average, 101.4. Meanwhile the control group left at the orphanage had shown “marked deterioration” and now had an average IQ of 66.1, down from 86 at the beginning of the study.

Staying in the orphanage was actually more harmful to the young girls that putting them in an adult mental institution. This was not a short-term difference, either. In the 1960s, the researchers followed up with the girls from the original study.

Of the thirteen girls who had been adopted, first informally by developmentally disabled women[1] in the institution and then by families in the outside world, all of them were self-supporting. Eleven of them were married. They had a mean of 11.68 years of education. They earned an average wage of $4,224, which was in the range of average annual earnings for men in Iowa, their home state—not bad for a group of women from an institutional background in the 1960s.

Of the twelve girls in the control group, only four of them had jobs, all of them working in the institutions where they lived. Only three had been married. On average they had less than four years of schooling. The cost savings to the state for rescuing the girls who went on to live healthy, productive lives was approximately $200 million in today’s dollars.

Anya’s primary point for telling this story is to review the history of evidence that standardized tests are poor predictors of human potential. But the story is also a compelling illustration of the long-term harm to health and wellbeing that we do to humans when we subject them to inhumane conditions (and, on a more hopeful note, how just a little bit of human love and understanding can be so transformative in a person’s life). Note that the Gallup research shows long-term health effects for work situations that are likely a lot less stressful than those of living in a Depression-era orphanage and almost certainly not worse than the kind of stress that Chicago kindergartener endured.

As I was pondering this story, I was reminded of FDA Commissioner David Kessler. (Bear with me on this.) Kessler successfully argued that nicotine addiction is a pediatric disease based on the long-term harm that it does to children. On that basis, he was able to establish that regulating tobacco falls under the purview of the FDA and was therefore able to put a collar on the powerful tobacco industry and regulate it for the first time. Given the severe and long-term stress that American children endure today due to a testing regime that takes up to 25% of students’ total schooling time, I wonder whether similarly compelling evidence could be gathered showing that forcing students to endure endless rounds of high-stakes standardized testing has effects analogous to long-term exposure to hazardous waste.

  1. Michael’s note: Given the rest of the story that Anya is telling here, it makes one wonder how many of those women were really developmentally disabled.

The post Is Standardized Testing a Pediatric Disease? appeared first on e-Literate.

About Inside Higher Ed Selling Majority Stake

Sun, 2015-01-18 01:20

Update 1/21: See link and blurb at bottom of post from new Editor’s Note at Inside Higher Ed.

Last week the Huffington Post ran an article by David Halperin breaking the news that the private equity firm Quad Partners had acquired a controlling interest in Inside Higher Ed.

Quad Partners, a New York private equity firm that is invested heavily in the for-profit college industry, and whose founder has aggressively opposed regulation of that troubled industry, has acquired a controlling stake in the respected trade publication Inside Higher Ed (IHE), which often reports on for-profit colleges and the policy disputes surrounding them. There has been no public announcement, but the Quad Partners website now lists Inside Higher Ed as one of its investments, among a range of education-related companies, including for-profit trade schools Beckfield College, Blue Cliff College, Dorsey Schools, Pacific College of Oriental Medicine, and Marinello Schools of Beauty.

Doug Lederman, one of IHE’s two top editors, confirmed to me that Quad purchased a majority interest in IHE in November.

Quad Partner James Tieng is now an IHE board member. Quad also owns the influential college admissions management company Noel-Levitz and other education technology companies that contract with colleges and universities — another sector that IHE covers.

The rest of the article then goes full conspiracy theory, building off the for-profit connection of both Quad Partners and its founder. Halperin seems to believe mere indirect association with for-profits is evil and compromising in and of itself rather than finding any changes or compromises in IHE coverage.

The bigger issue in my mind was described by Keith Button at Education Dive.

While the list of potential conflicts of interest in such a sale is long, the fact that the deal wasn’t announced and the potential news coverage issues weren’t publicly addressed up-front raises more questions.

This issue of disclosure was partially addressed in the original article:

“I would expect people to be watching us” in light of this purchase, says Lederman. “Our credibility is hugely important to us, and ultimately it will rise or fall on the nature and tenor of our coverage.” He says IHE will go on as before: “The proof will be in what we publish.” If there are significant references in IHE to specific Quad-owned companies, the publication will disclose the relationship.

In my mind, IHE made a serious mistake by not publicizing the acquisition back in November and issuing a blanket disclosure. I don’t fault them for selling the controlling stake in the company, especially given the lack of a paywall. But I do fault them for not realizing how the lack of disclosure created the opportunity for a advocate to publicly challenge them. It’s actually ironic to see a full-fledged advocate (Halperin writes extensively attacking the for-profit sector as part of his funding and openly calls himself an advocate) require 100% pure financial independence for IHE.

There are two types of disclosure that are relevant – a blanket disclosure announcing a key event such as the sale of the majority of company shares, proactively distributed and available; and article-specific disclosures if IHE articles reference companies tied to their owners. IHE seems to be relying on the latter, but their credibility will take a hit by not doing the former.

IHE was caught off guard by the Huffington Post article, and they seem to have quickly put up an Ownership Statement on the same day the article ran (Jan 14th).

Inside Higher Ed is an independent journalism organization. The journalistic independence is critical in ensuring the fairness and thoroughness of our higher education coverage.

Inside Higher Ed Inc. is owned by its three founders, other individual investors, and Quad Partners, a private equity firm that invests in the education space. Quad purchased a controlling share of Inside Higher Ed in November 2014 from a group of venture capital firms that invested in the company originally a decade earlier.

Owners of Inside Higher Ed stock who are not editors play no role in the editorial policies of the company.

The problem is the following:

  • This statement comes across as a reaction to Halperin – you got us – leading to the appearance that IHE had something to hide; and
  • IHE has done little to actually disclose this ownership, as the statement is only linked on the About Us page and Doug Lederman’s page (no articles or prominent placement of significant news event).

I read and research quite a bit of higher ed news and it took me a while to find this statement, despite the fact that I was specifically looking for information. With the current placement, very few people would have seen it.

This news is relevant, more for the Quad Partners ownership of Noel-Levitz than for their ownership of Marinello Schools of Beauty. Higher ed enrollment in the US has been declining the past 2 years, and this change is shaping up to be one of the biggest drivers of change initiatives for institutions and associated markets. There might be no other organization with more influential on enrollment management than Noel-Levitz. In the past 12 months Inside Higher Ed has written eight articles where Noel-Levitz plays an important role, and this prominent Fortune article profiling the company states:

Noel-Levitz might be the most influential force in higher education pricing that you’ve never heard of, empowering what’s become a three-stage, market-distorting game for college administrators.

Readers should know about the ownership connection given the importance of enrollment management and college pricing, and readers should not have to find this if and only if they read an article with direct references.

Do I believe that Quad Partners has or will change IHE coverage, especially on enrollment management and pricing? No. In my experience, IHE’s leadership and the reporters I’ve dealt with have been very ethical and honest. Furthermore:

Lederman says that at the insistence of IHE, the purchase agreement includes a clause that precludes Quad Partners from any involvement in editorial operations. IHE was launched by Lederman and two co-founders in 2004, with a modest investment from three Washington DC-area venture funds, including the owners of the lead generation company Double Positive. Those three investors, who sold their shares to Quad in November, also had no role in editorial operations, says Lederman.

IHE does a great job covering important stories in higher ed, including a watch dog role of exposing problems that arise. We need them to be trusted, and they should quickly correct the mistake. My unsolicited advise:

  • Write an article disclosing the sale and linking to the Ownership Statement – don’t make this information hard to find;
  • Quote a portion of the purchase agreement clause in the article to clarify their statement of editorial independence; and
  • Create a separate page of editorial policies.

Update 1/19: In a separate Education Dive post from the weekend:

A top editor of Inside Higher Ed said Friday that, in hindsight, he wished there had been more transparency about the sale of the publication’s controlling interest to a private equity firm that has invested heavily in for-profit education.

“We were founded without any support, then we had one set of investors and we had never said anything about them,” Scott Jaschik, an Inside Higher Ed founder and editor, told Education Dive. “In hindsight, I wish we had, because clearly this is of interest to people.” [snip]

“I guess I would just say to anyone who has questions, read us and read our coverage and call me if you think we’re doing anything that we shouldn’t,” he said.

Excellent work by Education Dive, by the way. As for IHE, I still think they would benefit from a blanket disclosure.

Update 1/21: Inside Higher Ed has now posted a full blanket disclosure note. Good for them.

Some of you may have seen some recent blog posts and discussion on Twitter or elsewhere about Inside Higher Ed Inc.’s ownership status. We wanted you to have more information directly from us. [snip]

In November 2014, Quad Partners, a private equity firm that invests in numerous companies in the education space, including some small for-profit colleges, bought a controlling interest in our company by purchasing shares of Inside Higher Ed Inc.’s stock from our previous investors.

Quad intends to help Inside Higher Ed expand its staff, extend its reach, and improve its coverage and services. Its goal is to help Inside Higher Ed do what it does better. And yes, like all investors, it wants to make money.

Owners of Inside Higher Ed Inc. stock who are not editors play no role in the editorial policies of the company. Quad acknowledged explicitly in its agreement to invest in Inside Higher Ed Inc. that it would be precluded from any involvement in editorial operations.

The post About Inside Higher Ed Selling Majority Stake appeared first on e-Literate.

No Discernible Growth in US Higher Ed Online Learning

Tue, 2015-01-06 11:34

By 2015, 25 million post-secondary students in the United States will be taking classes online. And as that happens, the number of students who take classes exclusively on physical campuses will plummet, from 14.4 million in 2010 to just 4.1 million five years later, according to a new forecast released by market research firm Ambient Insight.

- Campus Technology, 2011

On the positive side, Moody’s notes that the U.S. Department of Education projects a 20-percent growth in master’s degrees and a 9-percent growth in associate degrees, opportunities in both online education and new certificate programs, and a rising earnings premium for those with college degrees.

- Chronicle of Higher Ed, 2014

Q.  How likely would it be that this fraction [% students taking online courses] would grow to become a majority of students over the next five years? A [from institutional academic leaders]. Nearly two-thirds responded that this was “Very likely,” with an additional one-quarter calling it “Likely.” [That’s almost 90% combined]

- Grade Change, Babson Survey 2013

More than two-thirds of instructors (68 percent) say their institutions are planning to expand their online offerings, but they are split on whether or not this is a good idea (36 percent positive, 38 percent negative, 26 percent neutral).

- Inside Higher Ed 2014

Still, the [disruptive innovation] theory predicts that, be it steam or online education, existing consumers will ultimately adopt the disruption, and a host of struggling colleges and universities — the bottom 25 percent of every tier, we predict — will disappear or merge in the next 10 to 15 years.

- Clayton Christensen in NY Times 2013

You could be forgiven for assuming that the continued growth of online education within US higher ed was a foregone conclusion. We all know it’s happening; the questions is how to adapt to the new world.

But what if the assumption is wrong? Based on the official Department of Education / NCES new IPEDS data for Fall 2013 term, for the first time there has been no discernible growth in postsecondary students taking at least one online course in the US.

From 2002 through 2013 the most reliable measure of this metric has been the Babson Survey Research Group (BSRG) annual reporting. While there are questions on absolute numbers due to questions on definition of what makes a course “online”, the year-over-year growth numbers have been quite reliable and are the most-referenced numbers available. Starting last year, using Fall 2012 data, the official IPEDS data started tracking online education, and last week they put out Fall 2013 data – allowing year-over-year changes.

I shared the recent overall IPEDS data in this post, noting the following:

By way of comparison, it is worth noting the similarities to the Fall 2012 data. The percentage data (e.g. percent of a sector taking exclusive / some / no DE courses) has not changed by more than 1% (rounded) in any of the data. This unfortunately makes the problems with IPEDS data validity all the more important.

It will be very interesting to see the Babson Survey Research Group data that is typically released in January. While Babson relies on voluntary survey data, as opposed to mandatory federal data reporting for IPEDS, their report should have better longitudinal validity. If this IPEDS data holds up, then I would expect the biggest story for this year’s Babson report to be the first year of no significant growth in online education since the survey started 15 years ago.

I subsequently found out that BSRG is moving this year to use the IPEDS data for online enrollment. So we already have the best data available, and there is no discernible growth. Nationwide there are just 77,493 more students taking at least one online class, a 1.4% increase.

Y-o-Y Analysis

Why The Phrase “No Discernible Growth”?

Even though there was a nationwide increase of 77,493 students taking at least one online course, representing a 1.4% growth, there is too much noise in the data for this to be considered real growth. Even with the drop in total enrollment, the percentage of students taking at least one online course only changed from 26.4% TO 27.1%.

Just take one school – Suffolk County Community College – who increased by roughly 21,600 student enrollments taking at least one online course from 2012 to 2013 due to a change in how they report data and not from actual enrollment increases. More than a quarter of the annual nationwide increase can be attributed to this one reporting change[1]. These and similar issues are why I use the phrase “no discernible growth” – the year-over-year changes are now lower than the ability of our data collection methods to accurately measure.

Combine Babson and IPEDS Growth Data

While we should not directly compare absolute numbers, it is reasonable to combine the BSRG year-over-year historical growth data (2003 – 2012) with the new IPEDS data (2012 – 2013).

Y-o-Y Growth Chart

One thing to notice is that is really a long-term trend of declining growth in online. With the release of last year’s BSRG report they specifically called out this trend.

The number of additional students taking at least one online course continued to grow at a rate far in excess of overall enrollments, but the rate was the lowest in a decade.

What has not been acknowledged or fully understood is the significance of this rate hitting zero, at least within the bounds of the noise in data collection.


Think of the implications here if online education has stopped growing in US higher education. Many of the assumptions underlying institutional strategic plans and ed tech vendor market data is based on continued growth in online learning. It is possible that there will be market changes leading back to year-over-year growth, but for now the assumptions might be wrong.

Rather than focusing just on this year, the more relevant questions are based on the future, particularly if you look at the longer-term trends. Have we hit a plateau in terms of the natural level of online enrollment? Will the trend continue to the point of online enrollments actually dropping below the overall enrollment? Will online enrollments bottom out and start to rise again once we get the newer generation of tools and pedagogical approaches such as personalized learning or competency-based education beyond pilot programs?

I am not one to discount the powerful effect that online education has had and will continue to have in the US, but the growth appears to be at specific schools rather than broad-based increases across sectors. Southern New Hampshire, Arizona State University, Grand Canyon University and others are growing their online enrollments, but University of Phoenix, DeVry University and others are dropping.

One issue to track is the general shift from for-profit enrollment to not-for-profit enrollment, even if the overall rates of online courses has remained relatively stable within each sector. There are approximately 80,000 fewer students taking at least one online course at for-profit institutions while there are approximately 157,000 more students in the same category at public and private not-for-profit sectors.

I suspect the changes will continue to happen in specific areas – number of working adults taking courses, often in competency-based programs, at specific schools and statewide systems with aggressive plans – but it also appears that just making assumptions of broad-based growth needs to be reconsidered.

Update: Please note that the data release is new and these are early results. If I find mistakes in the data or analysis that changes the analysis above, I’ll share in an updated post.

  1. Russ Poulin and I documented these issues in a separate post showing the noise is likely in the low hundreds of thousands.

The post No Discernible Growth in US Higher Ed Online Learning appeared first on e-Literate.

Fall 2013 IPEDS Data: New Profile of US Higher Ed Online Education

Mon, 2015-01-05 14:48

The National Center for Educational Statistics (NCES) and its Integrated Postsecondary Education Data System (IPEDS) provide the most official data on colleges and universities in the United States. Last year they released data on distance education (essentially online courses) starting with the Fall 2012 term. Last week they released new data for the Fall 2013 term. Below is a profile of online education in the US for degree-granting colleges and university, broken out by sector and for each state.

Please note the following:

  • For the most part distance education and online education terms are interchangeable, but they are not equivalent as DE can include courses delivered by a medium other than the Internet (e.g. correspondence course).
  • I have provided some flat images as well as an interactive graphic at the bottom of the post. The interactive graphic has much better image resolution than the flat images.
  • There are three tabs below in the interactive graphic – the first shows totals for the US by sector and by level (grad, undergrad); the second also shows the data for each state; the third shows a map view.
  • Please note that along with Russ Poulin from WCET we have identified some significant problems with IPEDS data validity – see this article for more information.

Fall Profile by Sector

By way of comparison, it is worth noting the similarities to the Fall 2012 data. The percentage data (e.g. percent of a sector taking exclusive / some / no DE courses) has not changed by more than 1% (rounded) in any of the data. This unfortunately makes the problems with IPEDS data validity all the more important.

It will be very interesting to see the Babson Survey Research Group data that is typically released in January. While Babson relies on voluntary survey data, as opposed to mandatory federal data reporting for IPEDS, their report should have better longitudinal validity. If this IPEDS data holds up, then I would expect the biggest story for this year’s Babson report to be the first year of no significant growth in online education since the survey started 15 years ago.

Fall 2012 Profile by Sector

If you select the middle tab, you can view the same data for any selected state. As an example, here is data for California.

Sector and State CA Example

There is also a map view of state data colored by number of, and percentage of, students taking at least one online class for each sector. If you hover over any state you can get the basic data. As an example, here is a view highlighting North Carolina public 4-year institutions.

Map NC public 4-year


Interactive Graphic

For those of you who have made it this far, here is the interactive graphic. Enjoy the data.

 Learn About Tableau

The post Fall 2013 IPEDS Data: New Profile of US Higher Ed Online Education appeared first on e-Literate.

Fall 2013 IPEDS Data: Top 30 largest online enrollments per institution

Mon, 2015-01-05 05:18

The National Center for Educational Statistics (NCES) and its Integrated Postsecondary Education Data System (IPEDS) provide the most official data on colleges and universities in the United States. Last year they released data on distance education (essentially online courses) starting with the Fall 2012 term. Last week they released new data for the Fall 2013 term.

Let’s look at the top 30 online programs for Fall 2013 (in terms of total number of students taking at least one online course) while showing both 2012 and 2013 data. Some notes on the data source:

  • I have combined the categories ‘students exclusively taking distance education courses’ and ‘students taking some but not all distance education courses’ to obtain the ‘at least one DE’ category;
  • The ‘All Students’ category combines those taking DE course with those taking no DE courses;
  • I have highlighted in red the not-for-profit sectors;
  • IPEDS tracks data based on the accredited body, which can differ for systems – I manually combined most for-profit systems into one institution entity as well as Arizona State University[1]; and
  • Please note that along with Russ Poulin from WCET we have identified some significant problems with IPEDS data validity – see this article for more information.

Fall 2013 Top 30 Largest Online Enrollments Per Institution All Students Taking At Least One Online Course (Graduate and Undergraduate, including Non-degree or certificate-seeking)

Top 30

First Impressions
  • With the exception of Grand Canyon University overall and Argosy University for DE, the for-profit sector took a beating with significant drops in both total enrollment and DE enrollment.
  • Western Governors University, University of Central Florida, Southern New Hampshire Univesity, Suffolk County Community College and Northern Virginia Community College seem to have the biggest DE enrollment increases; Suffolk is so dramatic a change in DE without a big change in total enrollment, however, that I assume they changed how they report the data (see the article mentioned above).

We’ll add more data tables and graphics breaking apart this data over the next few weeks here at e-Literate.

Update: Fixed the rank ordering for Suffolk County Community College and Northern Virginia Community College.

  1. ASU splits into various entities although the online programs are coordinated.

The post Fall 2013 IPEDS Data: Top 30 largest online enrollments per institution appeared first on e-Literate.

The Quotable Justin Reich: MOOC research needs to reboot

Sat, 2015-01-03 18:19

Thanks to Audrey Watters I just read a new article in Science Magazine and publicly posted here by Justin Reich, the lead researcher for HarvardX (Harvard’s implementation of edX and associated research team)[1]. Justin calls out the limitations of current MOOC research that focuses on A/B testing and engagement instead of learning, single-course context, and post hoc analysis with proper course design. While praising the field for making available cleansed data for any type of analysis, his core argument is that we need new approaches that cannot be solved just by research teams.
Update: Added link to publicly-available DOCX article.

While the whole article is worth reading, there are quite a few insightful quotes should get past the journal paywall.

  • Big data sets do not, by virtue of their size, inherently possess answers to interesting questions.
  • We have terabytes of data about what students clicked and very little understanding of what changed in their heads.
  • It does not require trillions of event logs to demonstrate that effort is correlated with achievement.

  • One reason that early MOOC studies have examined engagement or completion statistics is that most MOOCs do not have assessment structures that support robust inferences about learning.
  • Distinguishing between engagement and learning is particularly crucial in voluntary online learning settings, because media that provoke confusion and disequilibrium can be productive for learners.
  • Boosting motivation in well-designed courses is good, but if a MOOC’s overall pedagogical approach is misguided, then plug-in experiments can accelerate participation in ineffective practices.
  • For the first MOOC researchers, getting data cleaned for any analysis was an achievement. In early efforts, following the path of least resistance to produce results is a wise strategy, but it runs the risk of creating path dependencies.
  • For the first MOOC researchers, getting data cleaned for any analysis was an achievement. In early efforts, following the path of least resistance to produce results is a wise strategy, but it runs the risk of creating path dependencies.
Some e-Literate Context

This article is a welcome statement from one of the leading MOOC researchers, and it connects with some earlier posts and interactions at e-Literate. In June 2014 I wrote a post contrasting the MOOC research results with the approach taken at the University of Phoenix.

Beyond data aggregated over the entire course, the Harvard and MIT edX data provides no insight into learner patterns of behavior over time. Did the discussion forum posts increase or decrease over time, did video access change over time, etc? We don’t know. There is some insight we could obtain by looking at the last transaction event and number of chapters accessed, but the insight would be limited. But learner patterns of behavior can provide real insights, and it is here where the University of Phoenix (UoP) could teach Harvard and MIT some lessons on analytics. [snip]

UoP recognizes the value of learner behavior patterns, which can only be learned by viewing data patterns over time. The student’s behavior in a course is a long-running transaction, with data sets organized around the learner.[2]

Two days later I wrote a follow-up post based on commenters speculating that Harvard and MIT might have learning data that was just not released.

Granted, I am arguing without definitive proof, but this is a blog post, after all. I base my argument on two points – there is no evidence of HarvardX or MITx pursuing learner-centered long-running data, and I believe there is great difficulty getting non-event or non-aggregate data out of edX, at least in current forms.

Justin Reich replied in the comments, essentially agreeing about the lack of learner-centered long-running data analysis but disagreeing with my arguments on the effect of MOOC architecture and data availability. This comment from June aligns quite well with the current Science Magazine article.

My research presentation was not exhaustive, although generally belies my belief that we need advances in instrumentation and assessment. Fancy manipulations of of predictors (from the click stream) may be limited in value if we don’t have good measures of learning, or a rich understanding of the context of data. But I’m super excited, too, about people doing great work with the edX event log data, and it’ll get out.

It is very encouraging to see the HarvardX team pushing to move beyond clicks-as-engagement and get to actual learning analysis.

Additional Notes

Some additional notes:

  • I still maintain that the course-centric transactional design of MOOCs (as with most LMSs) plays a role in the current, limited MOOC research analysis. I have spoken to many MOOC researchers who lament the enormous amount of time it takes to parse JSON files to try and recreate patterns based on individual learners over time. While I believe that Harvard, MIT, and Stanford have research teams capable of this extraction, a learner-centered system architecture would do wonders to advance the state of art for learning analytics.
  • As mentioned above, I believe that the standard usage of an LMS in online or blended courses leads to many of the same limitations in learning analytics. You could apply many of Justin’s quotes outside of the MOOC world.
  • I wish Justin had moved beyond formal journal and conference proceedings articles in his references and included the results from the MOOC Research Initiative.[3] Although not peer-reviewed, several of these reports addressed deficiencies such as being discipline-specific and even including the assessment considered in the MOOC design (as opposed to post hoc analysis). These reports do not negate the points made in the Science Magazine article, but it would have been useful to include this set of reports as a basis to understand the current state of research.
  1. Note that Science Magazine access requires a subscription or purchase or individual article.
  2. Note that I based this argument on what UoP claims to be producing internally without being able to validate the results.
  3. Disclosure – MRI was funded by the Gates Foundation, which is also a sponsor of the next e-Literate TV series.

The post The Quotable Justin Reich: MOOC research needs to reboot appeared first on e-Literate.

Harmonizing Learning and Education

Thu, 2015-01-01 16:20

I’m the Whether Man, not the Weather Man, for after all it’s more important to know whether there will be weather than what the weather will be.

The Phantom Tollbooth

Dave Cormier has written a couple of great posts on our failure to take learner motivation seriously and the difference between improving learning and improving education. In the latter post—a response to Stephen Downes’ comment on the former post—Dave writes about the tension between improving an individual’s learning and improving our system of education, essentially positing that the reason why we as a society often fail to take learner engagement sufficiently seriously is because we become preoccupied with making the educational system accountable, a goal that we would be irresponsible not to take on but that we are also essentially doomed to fail at. (I may be putting words in his mouth on that last bit.) Dave writes,

There’s definitely something wrong if people are leaving their first degree and are not engaged in learning. We certainly need to address it. We totally want to be in the business of helping people do what they want to do. Try it. No really. Just try it. Sit down with a child and help them do what they want to do. And i don’t mean “hey this child has shown up with a random project they are totally passionate about and are asking me a question” I mean “stop them at a random time, say 8:25am, and just start helping them.” You will get blank stares. You’ll get resistance. You’ll get students who will say anything you want if it means you will go away/give them a grade. You will not enjoy this process. They will also not enjoy it.

There is something wrong. The problem is that we have built an education system with checks and balances, trying to make it accountable and progressive (in some cases), but we are building it without knowing why. We have not built an education system that encourages people to be engaged. The system is not designed to do it. It’s designed to get people to a ‘standard of knowing.’ Knowing a thing, in the sense of being able to repeat it back or demonstrate it, has no direct relationship to ‘engagement’. There are certainly some teachers that create spaces where engagement occurs, but they are swimming upstream, constantly battling the dreaded assessment and the need to cover the curriculum. The need to guarantee knowing.

He suggests that we need to redesign our education system around the goal of getting students to start caring and keep caring about learning. And his argument is interesting:

Give me a kid who’s forgotten 95% of the content they were measured in during K-12 and I will match that with almost every adult i know. Give me a kid who cares about learning… well… then i can help them do just about anything.

This is partly a workplace argument. It’s an economic value argument. It’s a public good argument. If Dave is right, then people who care about learning are going to be better at just about any job you throw at them than people who don’t. This is a critical argument in favor of public funding of a liberal arts education, personalized in the old-fashioned sense of having-to-do-with-individual-persons, that much of academia has ceded for no good reason I can think of. The sticky wicket, though, is accountability which, as Dave points out, is the main reason we have a schism between learning and education in the first place. Too bad we can’t demonstrate, statistically, that people who are passionate about learning are better workers. It’s a shame that we don’t have good data linking being excited about learning, being a high-performer in your job, and being a happy, fulfilled and economically well-off person. If we had that, we could largely resolve the tension between improving learning and improving education. We could give a compelling argument that it is in the taxpayers’ interest to build an education system whose purpose, as Dave suggests, is to increase the chances that students will start to care and continue to care about learning. It’s a tragedy that we don’t have proof of that link.

Oh, wait.

The Intuition Behind the Argument

Before I get into the numbers, I think it’s important to articulate the argument in a way that makes intuitive sense even to skeptics. As Dave points out, everybody agrees with the proposition that students should love learning if that proposition is presented to them as a platitude. Where people start to waffle is when we present the proposition to them as a priority, as in, “It is more important for students to learn to develop and nurture a passion for learning than it is for them to learn any particular thing.” And in order to resolve the tension between learning and education, we need to make an even stronger proposition: “A student who develops a passion for learning about subjects that are unrelated to her eventual career will, on balance, be a better employee and more successful professional than the same student who has studied content directly related to her eventual career with relative indifference.” Do you believe this proposition? Here’s a test:

Imagine that you could go back in time and choose an undergraduate major that was exactly tailored to the job that you do today. Would you be better or worse at your job than you are now? Would you be more or less happy?

Obviously, this test won’t work for people whose undergraduate major was the perfect pre-professional major for what they are doing now, which will include most faculty. But it should work for a majority of people, including lots of folks in business and government. In my case, I was a philosophy major, which prepared me well for a career in anything except philosophy. If I could have precognitively created a major for myself in educational technology back in the late 1980s, would I be more successful today? Would I be happier? The answer to both of those questions is almost certainly “no.” In fact, there is a good chance that I would have been less successful and less happy. Why? For one thing, I didn’t care about educational technology back then. I cared about philosophy. I pursued it with a passion. This gave me three things that I still have today. First, I have the intellectual tools of a philosopher. I don’t think I would have held onto the tools of another discipline if I didn’t care about them when I was learning about them. Second, I know what it feels like to pursue work that I am passionate about. I am addicted to that feeling. I am driven to find it in every job, and I am not satisfied until I do. This makes me more selective about the jobs I look at and much, much better at the ones that I take. And finally, though it was a long and winding road, my interest in philosophy led me to my interest in instructional technology in many ways. We tend to have a rather stunted notion of what it means for a subject we study to be “related” to our work. In my philosophy classes, I spent a lot of time thinking about what it means to “know” something, what it means to “learn” something, and what it means for something to be “good.” I got to see how these words are tangled up in logic, language, and culture, and how our notions of them change over time. I learned how to write and how to think, while I was simultaneously studying the first principles of language and cognition. All of these experiences, all of this knowledge, all of these skills have been directly valuable to me in my career as a professional non-philosopher (or a standup philosopher, as Mel Brooks might call me). I wouldn’t have them if I had majored in educational technology. I would have other things, but honestly, there are no deep skills in my work that I wish I had acquired through earlier specialization. Everything that I have needed to learn, I have been able to learn on the job. As Dave wrote, “Give me a kid who cares about learning… well… then i can help them do just about anything.”

If you are one of those people who majored in exactly what you ended up doing as a career, then try reversing the thought experiment. Suppose you could go back in time and major in anything you wanted. Something that you were passionate about, but something different from what you ended up majoring in. Would it have made a difference? Would you have been more or less successful in your current career? Would you have been more or less happy than you are now? For some folks, that pre-professional major was exactly what they needed to be doing. But I bet that, for a lot of folks, it wasn’t.

Survey says…?!

If any of this resonates with you at all, then you really must read the 2014 Gallup Purdue Index Report. You’ll have to register to get it, but trust me, this one is worth it. Gallup is most widely known for their political polling, but more broadly, their business is in collecting data that links people’s attitudes and beliefs to observable behaviors and objective outcomes. How likely is a person who thinks the “country is on the wrong track” to vote for the incumbent? Or to vote at all? Does believing that your manager is incompetent correlate with an increased chance of a serious heart problem? And conversely, does “having fun” at your job correlate with a higher chance of living into your 90s? Does having a “manager that cares about me as a person” mean that I am more likely to be judged a “top performer” at work and reduce the likelihood that I will be out sick? Does having a teacher who “makes me feel excited about learning” correlate with better workplace engagement when I graduate?

Ah. There it is.

To get the full impact of Gallup’s research, you have to follow it backwards from its roots. The company does significant business in employee satisfaction surveys. As with schooling, managers know that employee engagement matters but often fail to take it seriously. But according to research cited in Gallup’s book Wellbeing: The Five Essential Elements (which I also recommend), employees who could answer “yes” to the question about whether their manager cares about them as a person, are “more likely to be top performers, produce higher quality work, are less likely to be sick, less likely to change jobs, and less likely to get injured on the job.” Also, people who love their jobs are more likely to both stay working longer and live longer. In a study George Gallup conducted in the 1950s,

…men who lived to see 95 did not retire until they were 80 years old on average. Even more remarkable, 93% of these men reported getting a great deal of satisfaction out of the work they did, and 86% reported having fun doing their job.

Conversely, a 2008 study the company found a link between employee disengagement and depression:

We measured their engagement levels and asked them if they had ever been diagnosed with depression. We excluded those who reported that they had been diagnosed with depression from our analysis. When we contacted the remaining panel members in 2009, we again asked them if they had been diagnosed with depression in the last year. It turned out that 5% of our panel members (who had no diagnosis of depression in 2008) had been newly diagnosed with depression. Further, those who were actively disengaged in their careers in 2008 were nearly twice as likely to be diagnosed with depression over the next year. While there are many factors that contribute to depression, being disengaged at work appears to be a leading indicator of a subsequent clinical diagnosis of depression.

Which is obviously bad for employer and employee alike.

In some cases, Gallup went all in with physiological studies. For example, they “recruited 168 employees and studied their engagement, heart rate, stress levels, and various emotions throughout the day,” using heart rate monitors, saliva samples, and handheld devices that surveyed employees on their activities and feelings of the moment at various points in the day.

After reviewing all of these data, it was clear that when people who are engaged in their jobs show up for work, they are having an entirely different experience than those who are disengage. [Emphasis in original.] For those who were engaged, happiness and interest throughout the day were significantly higher. Conversely, stress levels were substantially higher for those who were disengaged. Perhaps most strikingly, disengaged workers’ stress levels decreased and their happiness increased toward the end of the workday….[P]eople with low engagement…are simply waiting for the workday to end.

From here, the authors go on to talk about depression and heart attacks and all that bad stuff that happens to you when you hate that job. But there was one other striking passage at the beginning of this section:

Think back to when you were in school sitting through a class in which you had very little interest. Perhaps you eyes were fixed on the clock or you were staring blankly into space. You probably remember the anticipation of waiting for the bell to ring so you could get up from your desk and move on to whatever was next. More than two-thirds of workers around the world experience a similar feeling by the end of a typical workday.

And here’s what Dave said in his first post:

Student separate into two categories… those that care and those that don’t care.

Our job, as educators, is to convince students who don’t care to start caring, and to encourage those who currently care, to continue caring.

All kinds of pedagogy happens after this… but it doesn’t happen until this happens.

So. In this case, we’re trying to make students move from the ‘not care’ category to the ‘care’ category by threatening to not allow them to stay with their friends. Grades serve a number of ‘not care to care’ purposes in our system. Your parents may get mad, so you should care. You’ll be embarrassed in front of your friends so you should care. In none of these cases are you caring about ‘learning’ but rather caring about things you, apparently, already care about. We take the ‘caring about learning’ part as a lost cause.

The problem with threatening people is that in order for it to continue to work, you have to continue to threaten them (well… there are other problems, but this is the relevant one for this discussion). And, as has happened, students no longer care about grades, or their parents believe their low grades are the fault of the teacher, then the whole system falls apart. You can only threaten people with things they care about.

I’m not suggesting that we shouldn’t hold kids accountable, but if we’re trying to encourage people to care about their work, about their world, is it practical to have it only work when someone is threatening them? Even if you are the most cynical personal imaginable, wouldn’t you like people to be able to do things when you aren’t actually threatening them? Are we promoting a ‘creative/knowledge economy’ by doing this? Are we building democracy? Unless you are a fascist (and i really mean that, unless you want a world where a couple of people tell everyone exactly what to do) you can’t really want the world to be this way.

It turns out that Dave actually overstates the case for Fascism. Fascist bosses get bad results from employees (in addition to, you know, killing them). If you want high-performing workers, you need engaged workers. And you can’t force people to engage.

Wellbeing isn’t just about work. It looks at five different types of personal “wellbeing”—career, social, financial, physical, and community—and shows how they are related to each other, to overall wellbeing, and to performance at work and in the world. (By the way, there’s a lot of good stuff in the sections on social and community wellbeing for the connectivists and constructionists in the crowd.)

We Don’t Need No Education

The Gallup Purdue Index Report picks up where Wellbeing leaves off. Having established some metrics that correlate both with overall personal happiness and success as well as workplace success, Gallup backs up and asks the question, “What kind of education is more likely to promote wellbeing?” They surveyed a number of college graduates in various age groups and with various measured levels of wellbeing, asking them to reflect back on their college experiences. What they didn’t find is in some ways as important as what they did find. They found no correlation between whether you went to a public or private, selective or non-selective school and whether you achieved high levels of overall wellbeing. It doesn’t matter, on average, whether you go to Harvard University or Podunk College. It doesn’t matter whether your school scored well in the U.S. News and World Report rankings. Student debt levels, on the other hand, do matter, so maybe that Harvard vs. Podunk choice matters after all. And, in a finding that will cheer my philosophy professors, it turns out that “[s]lightly more employed graduates who majored in the arts and humanities (41%) and social sciences (41%) are engaged at work than either science (38%) or business (37%) majors.”

What factors did matter? What moved the needle? Odds of thriving in all five areas of Gallup’s wellbeing index were

  • 1.7 times higher if “I had a mentor who encouraged me to pursue my goals and dreams”
  • 1.5 times higher if “I had at least one professor at [College] who made me excited about learning”
  • 1.7 times higher if “My professors at [College] cared about me as a person”
  • 1.5 times higher if “I had an internship or job that allowed me to apply what I was learning in the classroom”
  • 1.1 times higher if “I worked on a project that took a semester or more to complete”
  • 1.4 times higher if “I was extremely active in extracurricular activities and organizations while attending [College]”

Again, the institution type didn’t matter (except for students who went to for-profit private colleges, only 4% of which were found to be thriving on all five measures of wellbeing). It really comes down to feeling connected to your school work and your teachers, which does not correlate well with the various traditional criteria people use for evaluating the quality of an educational institution. If you buy Gallup’s chain of argument and evidence this, in turn, suggests that being a hippy-dippy earthy-crunchy touchy-feely constructivy-connectivy commie pinko guide on the side will produce more productive workers and a more robust economy (not to mention healthier, happier human beings who get sick less and therefore keep healthcare costs lower) than being a hard-bitten Taylorite-Skinnerite practical this-is-the-real-world-kid type career coach. It turns out that pursuing your dreams is a more economically productive strategy, for you and your country, than pursuing your career. It turns out that learning a passion to learn is more important for your practical success than learning any particular facts or skills. It turns out that it is more important to know whether there will be weather than what the weather will be.

So…what do we do with all this ed tech junk we just bought?

This doesn’t mean that ed tech is useless by any means, but it does mean that we have to think about what we use it for and what it can realistically accomplish. Obviously, anything that helps teachers and advisers connect with students, students connect with each other, or students connect with their passions is good. There’s also nothing inherently wrong with video lectures or adaptive learning programs as long as they are used as informational supplements once students start caring about what they learn or as tools to keep them caring about what they learn rather than substitutes for real engagement that shovel content in the name of “competency.” I’m interested in “flipping,” fad or no fad, because it emphasizes using the technology to clear the way for more direct human-to-human interactions with the students. Competencies themselves should be used more as markers of progress down a road that the student has chosen to travel rather than a set of hoops that the student must jump through (like a trained dog). Another thing that technologies can do is help students with what may be the only prerequisite to having passion to learn, which is believing that you can learn. In the places where I’ve seen adaptive learning software employed to most impressive effect, it has been in concert with outreach and support designed to help students who never learned to believe in themselves discover that they can, in fact, make progress in their education. Well-designed adaptive software lets them get help without feeling embarrassed and, perhaps more importantly, enables them to arrive at a confidence-building feeling of success and accomplishment quickly.

The core problem with our education system isn’t the technology or even the companies. It’s how we deform teaching and learning in the name of accountability in education. Corporate interests amplify this problem greatly because they sell to it, thus reinforcing it. But they are not where the problem begins. It begins when we say, “Yes, of course we want the students to love to learn, but we need to cover the material.” Or when we say, “It’s great that kids want to go to school every day, but really, how do we know that they’re learning anything?” It’s daunting to think about trying to change this deep cultural attitude. Nor does embracing Gallup’s train of evidence fully get us out of the genuine moral obligation to find some sort of real (but probably inherently deforming) measure of accountability for schools. But the most interesting and hopeful result from the Gallup research is this:

You don’t have to have every teacher make you feel excited about learning in order to have a better chance at a better life. You just need one.

Just one.

The post Harmonizing Learning and Education appeared first on e-Literate.

Year-end Updates on e-Literate News Posts

Tue, 2014-12-30 16:11

For my final 2014 post, I thought it would be interesting to provide year-end updates to some news posts on e-Literate over the past year. You’ll notice that there is somewhat of an emphasis on negative stories or implications. For most positive stories, companies and institutions are typically all too happy to send out press releases with the associated media paraphrasing, and we have little need here to cover as news. The following non-exhaustive list is in date order.

D2L Growth Claims

In December 2013 I described layoffs at Desire2Learn (now officially named D2L). The significance of this story is that it calls into question D2L’s growth claims and trumpeting of massive new investment of $85 million. Some updates:

IPEDS Data on Online Learning

In early January we covered the new federal data on online learning, eventually breaking out into graphical analysis of Top 20 schools, state-by-state, and sector-by-sector categories. Some updates:

  • Russ Poulin from WCET and I (with some excellent help from WCET researchers) did some further analysis showing “significant confusion over basic definitions of terms, manual gathering of data outside of the computer systems designed to collect data, and, due to confusion over which students to include in IPEDS data, the systematic non-reporting of large numbers of degree-seeking students”.
  • Based on this analysis, NCES essentially responded by saying to ‘follow the damn rules, we’re not changing our approach’ (paraphrase).
  • The new data for the Fall 2013 term should be available in the next week or two, but for reasons listed above, I would be very cautious about forming conclusions for year-over-year changes.
2U’s IPO

In February and March we ran several stories about 2U’s IPO, as it clarified the business side of Online Service Providers and was one of the rare ed tech IPOs. Some updates:

Coursera New CEO and Direction

In March we described Coursera’s hiring of a new CEO – Richard Levin from Yale (and formerly creator of AllLearn and Open Yale Courses). Some updates:


In what was probably our biggest “news” story of the year, Michael and I covered the creation and release of the Unizin consortium. Some updates:

  • After we broke the story on May 16th the Unizin consortium was officially announced on June 11th.
  • As of the end-of-year, Unizin has signed up 10 institutionsColorado State University, the University of Florida, Indiana University, the University of Michigan, Ohio State University, Pennsylvania State, the University of Iowa, the University of Minnesota, the University of Wisconsin-Madison, and Oregon State University. The highlighted institutions were listed as potential partners in the original May story. Purdue University, the University of Maryland, the University of Texas, and the University of Utah were listed in May but have not (yet) joined Unizin.
  • Other than the different list of schools that have joined, substantially all of the original details have been confirmed by later events.
Cal State Online Demise

In July we covered the demise of Cal State Online less than three years after its high-profile kickoff. Some updates:

Problems with University of California UCPath System

In July we covered the $220+ million delayed program to implement a systemwide Payroll system that promised to pay for itself within five years. Some updates:

Kuali 2.0

In August we described the described the big changes to Kuali – moving development to a for-profit entity – and proclaimed that “community source is dead”. Some updates:

  • The Kuali Student, Kuali Coeus, Kuali Financial System, and Kuali Ready projects have all voted to shift to the new model and run through KualiCo.
  • For Kuali Student, the University of Maryland has signed on as the first institutional partner.
  • Boston College has decided to not go with KualiCo and has issued an RFP with the following purpose:

Boston College was a Kuali Student (KS) partner until the KS Board decision on November 14, 2014 to stop the current development of Kuali Student 1.0 and move to KualiCo. Boston College would like to complete the current development of Kuali Student Enrollment 1.0, under the current ECL license and is seeking a development partner. This would involve taking the latest Kuali Student Enrollment release and building out the required functionality.

Now, on to 2015.

The post Year-end Updates on e-Literate News Posts appeared first on e-Literate.

A 2014 (Personal) Blogging Retrospective

Mon, 2014-12-22 09:50

Unlike many of the bloggers who I enjoy reading the most, I don’t often let my blogging wander into the personal except as a route to making a larger point. For some reason, e-Literate never felt like the right outlet for that. But with the holidays upon us, with some life cycle events in my family causing me to be a bit more introspective than usual, and with the luxury of having discovered Phil’s top 20 posts of the year post showing up in my inbox, I’m in the mood to ruminate about my personal journey in blogging, where it’s taken me so far, and what it means to me. In the process, I’ll also reflect a bit on what we try to do at e-Literate.

When I started the blog 10 years ago, I honestly didn’t know what I was doing. OK, I guess that’s still true in some ways. What I mean is that I was looking for a purpose in my life. I had been a middle school and high school teacher for five years. It was the by far the best job I had ever had, and in some ways is still the best job I ever had. I left for a few different reasons. One was financial. I had fallen in love with a woman who had two teenaged daughters and suddenly found myself having to support a family. Another was frustration with a lack of professional growth opportunities. I taught in a wonderful, tiny little private school that operated out of eight rooms in the back of the Hoboken public library. It was amazing. But I wanted to do more and there was really no place for me to grow at the small school. I was young and feeling my oats. Lacking teacher’s certification and having been spoiled by teaching in such an amazing environment, I despaired of finding the right opportunity that would be professionally exciting while also allowing me to support my family. Part of it, too, was that I was beginning to get drawn to larger, systemic and cultural questions. For example, in the United States we have strong local control over our school systems, and my experience was that the overwhelming majority of parents care deeply for their children and want what’s best for them. Theoretically, it should be simple for parents to demand and get better schools. But that rarely happens. Why not? Why was the wonderful place that I was working at so rare? So I went wandering. I tried a few different things, but none of them made me happy. I am a teacher from a family of teachers. I needed to be close to education. But I also needed to support my family. And I needed to spread my wings, intellectually. I kept getting drawn to the bigger, systemic issues.

I started e-Literate just before I got a job at the SUNY Learning Network, having wandered in the wilderness first of graduate school and then of corporate e-Learning and knowledge management for a number of years. I had hoped that writing in public would help me clarify for myself what I wanted to do next in education as well as find some fellow travelers who might help me identify some sort of a career path that made sense. Meanwhile, I made a few good friends at SUNY, but mostly I grew quickly frustrated with the many barriers to doing good educational work that, once again, just shouldn’t exist if we lived in any kind of a rational world. Blogging was an oasis for me. It was a place where I found the kind of community that I should have had in academia but mostly didn’t. As I learned from early ed tech bloggers like Stephen Downes, Alan Levine, D’Arcy Norman, Scott Leslie, Beth Harris, Steven Zucker, Joe Ugoretz, George Siemens, and Dave Cormier (who co-hosted a wonderful internet radio show in those pre-podcasting days), I felt like I had found a home. It’s hard to describe what those early times of edublogging felt like if you weren’t around then. It was much friendlier. Much cozier. Everybody was just trying to figure stuff out together. I was just another shmoe working in the coal mines at a public university system, but in the blogosphere, there were really smart, articulate, accomplished people who took what I had to say seriously and encouraged me to say more. We argued sometimes, but mostly it was the good kind of argument. Arguments over what matters and what is true, rather than over who matters and what is the correct thing to say. It was…magical. I owe a great debt of gratitude to the bloggers I have mentioned here as well as others. I am ashamed to realize that I probably haven’t expressed that publicly before now. Without the folks who were already here when I arrived, I wouldn’t be where I am and who I am.

That said, finding a community is not the same thing as finding a purpose. The blogging wasn’t part of a satisfying career doing good in education so much as it was an escape from an unsatisfying career of failing to do good in education.

Then Blackboard sued Desire2Learn over a patent.

Such a strange thing to change a person’s life. Like most people, I really didn’t know what to make of it at first. I have never been dogmatically anti-corporate, anti-patent, or even anti-Blackboard. That said, Blackboard had proven itself to be a nasty, hyper-competitive company in those days, and this sounded like more of the same at first blush. But really, what did it mean to assert a patent in ed tech? I decided to figure it out. I read up on patent law and studied the court documents from the case (which Desire2Learn was publishing). I got a lot of help from Jim Farmer and some folks in the law community. And what I learned horrified me. Blackboard’s patent, if it had been upheld, would have applied to every LMS on the market, both proprietary and open source. Much worse, though, was the precedent it would have set. The basic argument that Blackboard made in their patent application process was that their invention was novel because it applied specifically to education. It was a little bit like arguing that one could patent a car that was designed only to be driven to the grocery store. Even if you didn’t care about the LMS, a successful assertion of that patent would have opened up Pandora’s box for any educational software. And if companies perceived that they could gain competitive advantages over their rivals by asserting patents, it would be the end of creative experimentation in educational technology. The U.S. patent system is heavily tilted toward large companies with deep pockets. Blackboard was already in the process of assembling a patent portfolio that would have enabled them to engage in what’s known as “stacking.” This is when a company files a flurry of lawsuits over a bunch of patents against a rival. Even if most of those assertions are bogus, it doesn’t matter, because the vast majority of organizations simply can’t afford the protracted legal battle. It’s less expensive for them to fold and just pay the extortion money patent license fees, or to sell out to the patent holder (which is probably what Blackboard really wanted from Desire2Learn). All that’s left in the market is for the big companies to cut cross-licensing deals with each other. Whatever you may think about the current innovation or lack thereof in educational technology, whatever we have now would have been crushed had Blackboard succeeded. That includes open source innovation. If a college president was told by her legal counsel that running a campus installation of WordPress with some education-specific modifications might violate a patent, what do you think the institutional decision about running WordPress would be?

So I went to war. I may have been just some shmoe working in the coal mines of a public university system, but dammit, I was going to organize. I translated the legalese of the patent into plain English so that everybody could see how ridiculous it was. I started a Wikipedia page on the History of Virtual Learning Environments so that people could record potential prior art against the patent. Mostly, I wrote about what I was learning about patents in general and Blackboard’s patents in particular. I wrote a lot. If you look down at the tag cloud at the bottom of the blog page, you’ll see that “Blackboard-Inc.” and “edupatents” are, to this day, two of the most frequently used tags on e-Literate.

And then an amazing thing happened. People listened. Not just the handful of edubloggers who were my new community, but all kinds of people. The entries on the Wikipedia page exploded in a matter of days. Every time Blackboard’s Matt Small gave a statement to some news outlet, I was asked to respond. I began getting invited to speak at conferences and association meetings for organizations that I never even knew existed before. Before I knew it, my picture was in freakin’ USA Today. e-Literate‘s readership suddenly went off the charts. In a weird way, I owe the popularity of the blog and the trajectory of my career to Blackboard and Matt Small.

And with that, I finally found my purpose. I won’t pretend that the community outrage and eventual outcome of the patent fight were mostly due to me—there were many, many people fighting hard, not the least of which were John Baker and Desire2Learn—but I could tell that I was having an impact, in part because of the ferocity with which Matt Small attempted to get me into trouble with my employers. With the blog, I could make things happen. I could address systemic issues. It isn’t a good vehicle for everything, but it works for some things. That’s why, more often than not, the best question to ask yourself when reading one of my blog posts is not “What is Michael really trying to say?” but “What is Michael really trying to do?” A lot of the time, I write to try to influence people to take (or not take) a particular course of action. Sometimes it’s just one or a couple of particular people who I have in mind. Other times it may be several disparate groups. For me, the blog is a tool for improving education, first and foremost. Improvement only happens when people take action. Therefore, saying the right things isn’t enough. If my writing is to be worth anything, it has to catalyze people to do the right things.

Of course, it doesn’t always work. Once Blackboard gave up on their patent assertion, I tried to rally colleges and universities to take steps to protect against educational patent assertion in the future. There was very little interest. Why? For starters, it was easier for them to vilify Blackboard than it was to confront the much more complex reality that our patent system itself is deeply flawed. But also, the universities that were in the best position to take affirmative steps harbored fantasies of being Stanford and owning a piece of the next Google. Addressing the edupatent problem in a meaningful way would have been deeply inconvenient for those ambitions and forced them to think hard about their intellectual property transfer policies. With the immediate threat over, there was no appetite for introspection on college campuses. The patent suit was dropped, Michael Chasen eventually left the company, Matt Small was moved into another role, and life has gone on. I suspect that somewhere in some university startup incubator is a student who was still in middle school when the edupatent war was going on and is filing patent applications for a “disruptive” education app today. Cue the teaser for the sequel, “Lawyers for the Planet of the Apes.”

Meanwhile, my blogging had raised my profile enough to get me out of SUNY and land me a couple of other jobs, both of which taught me a great deal about the larger systemic context and challenges of ed tech but neither of which turned out to be a long-term home for me (which I knew was likely to be the case at the time that I took them). But at the second job in particular, I got too busy with work to blog as regularly as I wanted to. It really bothered me that I had built up a platform that could make a difference and was largely unable to do anything with it. So I decided to try to turn it into a group blog. The blogosphere had changed by then. The power law had really taken hold. There were a handful of bloggers who got most of the attention, and it was getting harder for new voices to break in. So I decided to invite people who maybe didn’t (yet) have the same platform that they deserved but who regularly taught me important things through their writing to come and blog on e-Literate, writing whatever they liked, whenever they liked, however often they liked. No strings attached. I’m proud to have posts here from people like Audrey Watters, Bill Jerome, David White, Kim Thanos, and Laura Czerniewicz, among others. Most of the people I invited wrote one or a few posts and then moved on to other things. Which was fine. I wasn’t inviting them because I wanted to build up e-Literate. I was inviting them because I wanted to expose more people to their good work. That’s something that we still try to do when we can. For example, the analysis that Mike Caulfield did raising doubts about some of the Purdue Course Signals research was hugely important to the field of learning analytics. I’m proud to have had the opportunity to draw attention to it.

Like I said, most of the bloggers wrote a few pieces and then moved on. Most. One of them just kept hanging around, like a relative you invite to dinner who never gets the hint when it’s time to leave. As with many of the others, I had not really met Phil Hill before and mainly knew him through his writing. Before long, he was writing more blog posts on e-Literate than I was. And—please don’t tell him I told you this—I love his writing. Phil is more of a natural analyst than I am. He has a head for details that may seem terribly boring in and of themselves but often turn out to have important implications. Whether he is digging through discrepancies on employee numbers to call BS on D2L’s claims of hypergrowth (and therefore their rationale for all the investment money and debt they are taking on) or collaborating with WCET’s Russ Poulin on an analysis of how the Federal government’s IPEDs figures are massively misreporting the size of online learning programs, he consistently goes dumpster diving and comes back with gold. At the same time, he shares my constitutional inability to restrain myself from saying something when I see something that I think is wrong. This is what, for example, led him to file a public records request for information that definitively showed how few students Cal State Online was reaching for all the money that was spent on the program. For the record, Cal State is a former consulting client of Phil’s. As consultants in our particular niche, any critical post that we write of just about anyone runs the risk of alienating a potential client. Anyway, Phil is now co-publisher of e-Literate. The blog is every bit as much his as it is mine.

And so e-Literate continues to evolve. When I look at Phil’s list of our top 20 posts from 2014, it strikes me that there are a few things we are trying to do with our writing that I think are fairly unusual in ed tech reporting and analysis at the moment:

  • We provide critical analysis and long-form reporting on ed tech companies. There are lots of good pieces written by academic bloggers on ed tech products or the behavior of ed tech companies, but many of them are essentially cultural studies-style critiques of either widely reported news items or personal experiences. There’s nothing wrong with that, but it doesn’t give us the whole picture without some supplementation. On the other hand, the education news outlets break stories but don’t do as much in-depth analysis as one would hope for. Because Phil and I have eclectic backgrounds, we have some insight into how these companies work that academics or even reporters often don’t. We’ve been doing this long enough that we have a lot of contacts who are willing to talk to us so, even though we’re not in the business of breaking stories, we sometimes get important details that others don’t. Also, as you can tell from this blog post (if you’ve made it this far), we’re not afraid of writing long pieces.
  • We provide critical analysis and long-form reporting on colleges’ and universities’ (mis)adventures in ed tech. One of things that really bugs me about the whole ed tech blogging and reporting world is that some of the most ferocious critics of corporate misbehavior are often strangely muted on the dysfunction of colleges and universities and completely silent on the dysfunction of faculty. I’m as proud of our work digging into the back room deals of school administrators that circumvent faculty governance or the ways in which faculty behavior impedes progress in areas like better learning platforms or OER as I am of our analysis of corporate misbehavior.
  • We demystify. I was particularly honored to be invited to write a piece on adaptive learning for the American Federation of Teachers. The AFT tends to take a skeptical view of ed tech, so I took their invitation as validation that at least some of the writing we do here is of as much value to the skeptics as to the enthusiasts. When I write a piece about a vendor and I get compliments on it from both people inside the company and people who despise the company, I know that I’ve managed to explain something in a way that clarifies while letting readers make their own judgments. A lot of the coverage of ed tech tends to be either reflexively positive or reflexively negative, and in neither case do we get a lot of details about what the product is, how it actually works, and how people are using it in practice.

One other thing that I feel good about on e-Literate and that I am completely amazed by is our community of commenters. We frequently get 5 or 10 comments on a given post (either in the WordPress comments thread or on Google+), and it’s not terribly uncommon for us to get 50 or even 100 comments on a post. And yet, I can count on one hand the number of times that we’ve ever had personalized attacks or unproductive behaviors from our commenters. I have no idea why this is so and take no credit for it. Even after ten years, I can’t predict which blog posts will generate a lot of discussion and which ones will not. It’s just a magic thing that happens sometimes. I’m still surprised and grateful every time that it does.

But of all the astonishing, wonderful things that have happened to me because of the blog, one of the most astonishing and wonderful is the way that it turned into a fulfilling job. When Phil asked me to join him as a consultant two years ago, I frankly didn’t give high odds that we would be be in business a for very long. I thought the most likely scenario was that we would fail, hopefully in an interesting way, and have some fun in the process. (Please don’t tell Phil I said that either.) But we have been not only pretty consistently busy with work but also growing the business about as fast as we would want it to grow, despite an almost complete lack of sales or marketing effort on our part. The overwhelming majority of our work comes to us through people who read our blog, find something helpful in what we wrote, and contacts us to see if we can help more. We’ve made it a policy to mostly not blog about our consulting except where we need to make conflict-of-interest disclosures, but sometimes I wonder if that’s the right thing to do. The tag line of e-Literate is “What We Are Learning About Online Learning…Online”, and a lot of what we are learning comes from our jobs. If anything, that is more true now than ever, given that so much of our work springs directly from our blogging. Our clients tend to hire us to help them with problems related to issues that we have cared about enough to write about. We also seem to gain more clients than we lose by writing honestly and critically, and our relationships with our clients are better because of it. People who come to us for help expect us to be blunt and are not surprised or offended when we offer them advice which is critical of the way they have been doing things.

Honestly, this is the most fulfilled that I have felt, professionally, since I left the classroom. I will go back to teaching at some point before I retire, but in the meantime I feel really good about what I’m doing for the first time in a long time. I get to work with schools, foundations, and companies on interesting and consequential education problems—and increasingly on systemic and cultural problems. I get to do a lot of it in an open way, with people who I like and respect. I get to speak my mind about the things I care about without fear that it will get me in (excessive) trouble. And I even get paid.

Who knew that such a thing is possible?

The post A 2014 (Personal) Blogging Retrospective appeared first on e-Literate.

e-Literate Top 20 Posts For 2014

Sat, 2014-12-20 12:17

I typically don’t write year-end reviews or top 10 (or 20) lists, but I need to work on our consulting company finances. At this point, any distraction seems more enjoyable than working in QuickBooks.

We’ve had a fun year at e-Literate, and one recent change is that we are now more willing break stories when appropriate. We typically comment on ed tech stories a few days after the release, providing analysis and commentary, but there are several cases where we felt a story needed to go public. In such cases (e.g. Unizin creation, Cal State Online demise, management changes at Instructure and Blackboard) we tend to break the news objectively, providing mostly descriptions and explanations, allowing others to provide commentary.

The following list is based on Jetpack stats on WordPress, which does not capture people who read posts through RSS feeds (we send out full articles through the feed). So the stats have a bias towards people who come to e-Literate for specific articles rather than our regular readers. We also tend to get longer-term readership of articles over many months, so this list also has a bias for articles posted a while ago.

With that in mind, here are the top 20 most read articles on e-Literate in terms of page views for the past 12 months along with publication date.

  1. Can Pearson Solve the Rubric’s Cube? (Dec 2013) – This article proves that people are willing to read a 7,000 word post published on New Year’s Eve.
  2. A response to USA Today article on Flipped Classroom research (Oct 2013) – This article is our most steady one, consistently getting around 100 views per day.
  3. Unizin: Indiana University’s Secret New “Learning Ecosystem” Coalition (May 2014) – This is the article where we broke the story about Unizin, based largely on a presentation at Colorado State University.
  4. Blackboard’s Big News that Nobody Noticed (Jul 2014) – This post commented on the Blackboard users’ conference and some significant changes that got buried in the keynote and much of the press coverage.
  5. Early Review of Google Classroom (Jul 2014) – Meg Tufano got pilot access to the new system and allowed me to join the testing; this article mostly shares Meg’s findings.
  6. Why Google Classroom won’t affect institutional LMS market … yet (Jun 2014) – Before we had pilot access to the system, this article described the likely market affects from Google’s new system.
  7. Competency-Based Education: An (Updated) Primer for Today’s Online Market (Dec 2013) – Given the sudden rise in interest in CBE, this article updated a 2012 post explaining the concept.
  8. The Resilient Higher Ed LMS: Canvas is the only fully-established recent market entry (Feb 2014) – Despite all the investment in ed tech and market entries, this article noted how stable the LMS market is.
  9. Why VCs Usually Get Ed Tech Wrong (Mar 2014) – This post combined references to “selling Timex knockoffs in Times Square” with a challenge to the application of disruptive innovation.
  10. New data available for higher education LMS market (Nov 2013) – This article called out the Edutechnica and ListEdTech sites with their use of straight data (not just sampling surveys) to clarify the LMS market.
  11. InstructureCon: Canvas LMS has different competition now (Jun 2014) – This was based on the Instructure users’ conference and the very different attitude from past years.
  12. Dammit, the LMS (Nov 2014) – This rant called out how the LMS market is largely following consumer demand from faculty and institutions.
  13. Why Unizin is a Threat to edX (May 2014) – This follow-on commentary tried to look at what market effects would result from Unizin introduction.
  14. State of the Anglosphere’s Higher Education LMS Market: 2013 Edition (Nov 2013) – This was last year’s update of the LMS squid graphic.
  15. Google Classroom: Early videos of their closest attempt at an LMS (Jun 2014) – This article shared early YouTube videos showing people what the new system actually looked like.
  16. State of the US Higher Education LMS Market: 2014 Edition (Oct 2014) – This was this year’s update of the LMS squid graphic.
  17. About Michael – How big is Michael’s fan club?
  18. What is a Learning Platform? (May 2012) – The old post called out and helped explain the general move from monolithic systems to platforms.
  19. What Faculty Should Know About Adaptive Learning (Dec 2013) – This was a reprint of invited article for American Federation of Teachers.
  20. Instructure’s CTO Joel Dehlin Abruptly Resigns (Jul 2014) – Shortly after the Instructure users’ conference, Joel resigned from the company.

Well that was more fun that financial reporting!

The post e-Literate Top 20 Posts For 2014 appeared first on e-Literate.

Helix Education puts their competency-based LMS up for sale

Thu, 2014-12-18 17:05

Back in September I wrote about the Helix LMS providing an excellent view into competency-based education and how learning platforms would need to be designed differently for this mode. The traditional LMS – based on a traditional model using grades, seat time and synchronous cohort of students – is not easily adapted to serve CBE needs such as the following:

  1. Explicit learning outcomes with respect to the required skills and concomitant proficiency (standards for assessment)
  2. A flexible time frame to master these skills
  3. A variety of instructional activities to facilitate learning
  4. Criterion-referenced testing of the required outcomes
  5. Certification based on demonstrated learning outcomes
  6. Adaptable programs to ensure optimum learner guidance

In a surprise move, Helix Education is putting the LMS up for sale.  Helix Education provided e-Literate the following statement to explain the changes, at least from a press release perspective.

With a goal of delivering World Class technologies and services, a change we are making is with Helix LMS. After thoughtful analysis and discussion, we have decided to divest (sell) Helix LMS. We believe that the best way for Helix to have a positive impact on Higher Education is to:

  • Be fully committed and invest properly in core “upstream” technologies and services that help institutions aggregate, analyze and act upon data to improve their ability to find, enroll and retain students and ensure their success
  • Continue to build and share our thought leadership around TEACH – program selection, instructional design and faculty engagement for CBE, on-campus, online and hybrid delivery modes.
  • Be LMS neutral and support whichever platform our clients prefer. In fact, we already have experience in building CBE courses in the top three LMS solutions.

There are three aspects of this announcement that are quite interesting to me.

Reversal of Rebranding

Part of the surprise is that Helix rebranded the company based on their acquisition of the LMS – this was not just a simple acquisition of a learning platform – and just over a year after this event Helix Education is reversing course, selling the Helix LMS and going LMS-neutral. From the earlier blog post [emphasis added]:

In 2008 Altius Education, started by Paul Freedman, worked with Tiffin University to create a new entity called Ivy Bridge College. The goal of Ivy Bridge was to help students get associate degrees and then transfer to a four-year program. Altius developed the Helix LMS specifically for this mission. All was fine until the regional accrediting agency shut down Ivy Bridge with only three months notice.

The end result was that Altius sold the LMS and much of the engineering team to Datamark in 2013. Datamark is an educational services firm with a focus on leveraging data. With the acquisition of the Helix technology, Datamark could expand into the teaching and learning process, leading them to rebrand as Helix Education – a sign of the centrality of the LMS to the company’s strategy. Think of Helix Education now as an OSP (a la carte services that don’t require tuition revenue sharing) with an emphasis on CBE programs.

Something must have changed in their perception of the market to cause this change in direction. My guess is that they are getting pushback from schools who insist on keeping their institutional LMS, even with the new CBE programs. Helix states they have worked with “top three LMS solutions”, but as seen in the demo (read the first post for more details), capabilities such as embedding learning outcomes throughout a course and providing a flexible time frame work well outside the core design assumptions of a traditional LMS. I have yet to see an elegant design for CBE with a traditional LMS. I’m open to being convinced otherwise, but count me as skeptical.

Upstream is Profitable

The general move sounds like the main component is the moving “upstream” element. To be more accurate, it’s more a matter of staying “upstream” and choosing to not move downstream. It’s difficult, and not always profitable, to deal with implementing academic programs. Elements built on enrollment and retention are quite honestly much more profitable. Witness the recent sale of the enrollment consulting firm Royall & Company for $850 million.

The Helix statement describes their TEACH focus as one of thought leadership. To me this sounds like the core business will be on enrollment, retention and data analysis while they focus academic efforts not on direct implementation products and services, but on white papers and presentations.

Meaning for Market

Helix Education was not the only company building CBE-specific learning platforms to replace the traditional LMS. FlatWorld Knowledge built a platform that is being used at Brandman University. LoudCloud Systems built a new CBE platform FASTrak – and they already have a traditional LMS (albeit one designed with a modern architecture). Perhaps most significantly, the CBE pioneers Western Governors University and Southern New Hampshire University’s College for America (CfA) built custom platforms based on CRM technology (i.e. Salesforce) based on their determination that the traditional LMS market did not suit their specific needs. CfA even spun off their learning platform as a new company – Motivis Learning.

If Helix Education is feeling the pressure to be LMS-neutral, does that mean that these other companies are or will be facing the same? Or, is Helix Education’s decision really based on company profitability and capabilities that are unique to their specific situation?

The other side of the market effect will be determined by which company buys the Helix LMS. Will a financial buyer (e.g. private equity) choose to create a standalone CBE platform company? Will a traditional LMS company buy the Helix LMS to broaden their reach in the quickly-growing CBE space (350 programs in development in the US)? Or will an online service provider and partial competitor of Helix Education buy the LMS? It will be interesting to see which companies bid on this product line and who wins.


If I find out more about what this change in direction means for Helix Education or for competency-based programs in general, I’ll share in future posts.

The post Helix Education puts their competency-based LMS up for sale appeared first on e-Literate.

Blackboard’s SVP of Product Development Gary Lang Resigns

Tue, 2014-12-16 12:46

Gary Lang, Blackboard’s senior vice president in charge of product development and cloud operations, has announced his resignation and plans to join Amazon. Gary took the job with Blackboard in June 2013 and, along with CEO Jay Bhatt and SVP of Product Management Mark Strassman, formed the core management team that had worked together previously at AutoDesk. Gary led the reorganization effort to bring all product development under one organization, a core component of Blackboard’s recent strategy.

Michael described Blackboard’s new product moves toward cloud computing and an entirely new user experience (UX) for the Learn LMS, and Gary was the executive in charge of these efforts. These significant changes have yet to fully roll out to customers (public cloud in pilot, new UX about to enter pilot). Gary was also added to the IMS Global board of directors in July 2014 – I would expect this role to change as well given the move to Amazon.

At the same time, VP Product Management / VP Market Development Brad Koch has also resigned from Blackboard.[1] Brad came to Blackboard from the ANGEL acquisition. Given his long-term central role leading product definition and being part of Ray Henderson’s team[2], Brad’s departure will also have a big impact. Brad’s LinkedIn page shows that he has left Blackboard, but it does not yet show his new company. I’m holding off reporting until I can get public confirmation.

Blackboard provided the following statement from CEO Jay Bhatt.

The decision to leave Blackboard for an opportunity with Amazon was a personal one for Gary that allows him to return home to the West Coast. During his time here, Gary has made significant contributions to the strategic direction of Blackboard and the technology we deliver to customers. The foundation he has laid, along with other leaders on our product development team, will allow us to continue to drive technical excellence for years to come. We thank him for his leadership and wish him luck as he embarks on this new endeavor.

  1. The two resignations are unrelated as far as I can tell.
  2. Starting at Pearson, then at ANGEL, finally at Blackboard

The post Blackboard’s SVP of Product Development Gary Lang Resigns appeared first on e-Literate.

Vendors as Traditional Revolutionaries

Sun, 2014-12-14 09:40

In a post titled “The LMS for Traditional Revolutionaries,” Instructure’s VP of Research and Education for Canvas Jared Stein responded to my LMS rant with some numbers and some thoughts about the role of the vendor in encouraging progressive teaching practices. First, the numbers on the use of open education features in Canvas:

  • 3.8% of courses are “public”; you don’t need a login to see them.
  • 0.6% of courses are Creative Commons-licensed.
  • 4.0% of assignments are URL submissions (suggesting that students are completing their assignments on their blogs or elsewhere on the open web).

On the one hand, as Jared acknowledges, these percentages are very low. On the other hand, as he points out, 4% of assignments is close to 250,000 assignments, which is non-trivial as an absolute number. And all of this raises the question: What is the role of the vendor in promoting progressive educational practices?

Let’s take the best-case scenario. Suppose you’re a good person and a thoughtful educator who happens to work for a vendor at the moment. (For those of you who don’t know him, Jared enjoys just such a reputation, having spent a number of years as an excellent academic ed tech blogger and practitioner before joining Instructure.) What can you do? What is your role? On the one hand, you will get criticized by educators who want more and faster change for being too conventional. I certainly have leveled that sort of criticism at vendors before. And maybe those criticisms will sting particularly hard if you were one of those educators yourself before you joined the company (and maybe still are, in your heart of hearts). On the other hand, you are likely to be criticized as arrogant, high-handed, and unwilling to listen to your customers if you put yourself in the position of lecturing to educators (or, at worst, bullying them) about what you, as a vendor, define as best teaching practices. I certainly have leveled this sort of criticism as well.

So what’s a vendor to do? Jared writes,

These [open education features] are just a few examples of capabilities in Canvas that we believe add flexibility and encourage different approaches to teaching and learning. I recognize that sharing this data is a little risky; some may use it to argue that Canvas shouldn’t worry so much about the small percentage of educators who may take advantage of these fringe capabilities. After all, won’t teachers who are actually invested in open educational practices just eschew the LMS for their own platforms anyway?

Focusing only on “users like us” and ignoring the others may work in the short-term, but for long-term success you have to build bridges, not walls.

To help education improve itself for all teachers and learners we have to try to connect with those teachers who aren’t comfortable with radical shifts in pedagogy or technology. We believe that the best way to encourage positive change in educational practices across the broad landscape of content areas, learning objectives, and teaching philosophies is by providing tools that are easy-to-use, flexible, and comfortable to the majority of teachers and learners. The door to change must be open and the doorkeeper must be deposed.

Some of the ways we do this is by having an open community, engaging with people who disagree with us, and investing in the open platform aspect of Canvas. We need both traditionalists, critical pedagogues, progressive researchers, and open educators to contribute to Canvas.That doesn’t have to be done through pull requests or by building LTI apps or integrations, though that’s a brilliant way to build solutions that are right for your context. But by dialoging what works in teaching and learning and what doesn’t. By debating what technology is best for, and when it leads us away from our shared goals of teaching and learning better in an open and connected world.

Shorter Jared: We put capabilities to support progressive practices in our product in the hopes that our users will discover, adopt, and promote them, but it’s not our place to push our preferred educational practices on our customers.

In many cases—particularly with a platform that serves a large and heterogeneous swath of the campus community—that’s the best attitude you can get from your vendor. That’s the most they can do without rightly pissing off (more) people.

All of which brings me back to a single point: If you want better educational technology, then work to make sure that your colleagues in your campus community are asking for the things that you think would make educational technology better. If 40% rather than 4% of assignments created by your colleagues were on the open web, then learning platforms like LMSs would look and work differently. I guarantee it. Likewise, as long as most educators tend to use the technology to reproduce existing classroom practices, LMSs will look the same. I guarantee that too. And that’s not a vendor thing. That’s a software development thing. Community-developed open source learning platforms generally haven’t broken the mold, and the few that have tend to be the ones that you probably have never heard of because they don’t get adopted. They build what their community members ask for and what they think will attract other community members. So if you want better tech, then the best thing you can do to get it is to create demand for it among your colleagues.

The post Vendors as Traditional Revolutionaries appeared first on e-Literate.

The Battle for Open and MOOC Completion Rates

Wed, 2014-12-10 23:21

Yesterday I wrote a post on the 20 Million Minds blog about Martin Weller’s new book The Battle for Open: How openness won and why it doesn’t feel like victory. Exploring different aspects of open in higher education – open access, MOOCs, open education resources and open scholarship – Weller shows how far the concept of openness has come, to the point where “openness is now such a part of everyday life that it seems unworthy of comment”. If you’re interested in OER, open courses, open journals, or open research in higher education – get the book (it’s free and available in a variety of formats).

Building on the 20MM post about the ability to reuse or repurpose the book itself, I would like to expand on a story from early 2013 where I happen to play a role. I’ll mix in Weller’s description (MW) from the book with Katy Jordan’s data (KJ) and my own description (PH) from this blog post.

(MW) I will end with one small example, which pulls together many of the strands of openness. Katy Jordan is a PhD student at the OU focusing on academic networks on sites such as Academia. edu. She has studied a number of MOOCs on her own initiative to supplement the formal research training offered at the University. One of these was an infographics MOOC offered by the University of Texas. For her final visualisation project on this open course she decided to plot MOOC completion rates on an interactive graph, and blogged her results (Jordan 2013).


(MW) This was picked up by a prominent blogger, who wrote about it being the first real attempt to collect and compile completion data for MOOCs (Hill 2013), and he also tweeted it.

(PH) How many times have you heard the statement that ‘MOOCs have a completion rate of 10%’ or ‘MOOCs have a completion rate of less than 10%’? The meme seems to have developed a life of its own, but try to research the original claim and you might find a bunch of circular references or anecdotes of one or two courses. Will the 10% meme hold up once we get more data?

While researching this question for an upcoming post, I found an excellent resource put together by Katy Jordan, a graduate student at The Open University of the UK. In a blog post from Feb 13, 2013, Katy described a new effort of hers to synthesize MOOC completion rate data – from xMOOCs in particular and mostly from Coursera.

(MW) MOOC completion rates are a subject of much interest, and so Katy’s post went viral, and became the de-facto piece to link to on completion rates, which almost every MOOC piece references. It led to further funding through the MOOC Research Initiative and publications. All on the back of a blog post.

This small example illustrates how openness in different forms spreads out and has unexpected impact. The course needed to be open for Katy to take it; she was at liberty to share her results and did so as part of her general, open practice. The infographic and blog relies on open software and draws on openly available data that people have shared about MOOC completions, and the format of her work means others can interrogate that data and suggest new data points. The open network then spreads the message because it is open access and can be linked to and read by all.

(PH) Once I had and shared Katy’s blog post, it seemed the natural move was to build on this data. What was interesting to me was that there seemed to be different student patterns of behavior within MOOCs, leading to this initial post and culminating (for now) in a graphical view of MOOC student patterns.


(PH) With a bit of luck or serendipity, this graphical view of patterns nicely fit together with research data from Stanford.

(MW) It’s hard to predict or trigger these events, but a closed approach anywhere along the chain would have prevented it. It is in the replication of small examples like this across higher education that the real value of openness lies.

Weller has a great point on the value of openness, and I appreciate the mention in the book.

Source: Weller, M. 2014. The Battle for Open: How openness won and why it doesn’t feel like victory. London: Ubiquity Press. DOI:

The post The Battle for Open and MOOC Completion Rates appeared first on e-Literate.

Is Kuali Guilty of “Open Washing”?

Wed, 2014-12-03 14:05

Phil and I don’t write a whole lot about Student Information Systems (SISs) and the larger Enterprise Resource Management (ERP) suites that they belong to, not because they’re unimportant but because the stories about them often don’t fit with our particular focus in educational technology. Plus, if I’m being completely honest, I find them to be mostly boring. But the story of what’s going with Kuali, the open source ERP system for higher education, is interesting and relevant for a couple of reasons. First, while we hear a lot of concern from folks about the costs of textbooks and LMSs, those expenses pale in comparison to what colleges and universities pay in SIS licensing and support costs. A decent sized university can easily pay a million dollars or more for their SIS, and a big system like UC or CUNY can pay tens or even hundreds of millions. One of the selling points of Kuali has been to lower costs, thus freeing up a lot of that money to meet other needs that are more closely related to the core university mission. So what happens in the SIS space has a potentially huge budgetary impact on teaching and learning. Second, there’s been a lot of conversation about what “open” means in the context of education and, more specifically, when that label is being abused. I am ambivalent about the term “open washing” because it encourages people to flatten various flavors of openness into “real” and “fake” open when, in fact, there are often legitimately different kinds of open with different value (and values). That said, there certainly exist cases where use of the term “open” stretches beyond the bounds of even charitable interpretation.

When Kuali, a Foundation-developed project released under an open source license, decided to switch its model to a company-developed product where most but not all of its code would be released under a different open source license, did they commit an act of “open washing”? Finding the answer to that question turns out to be a lot more complicated than you might expect. As I started to dig into it, I found myself getting deep into details of both open source licensing and cloud computing, since the code that KualiCo will be withholding is related to cloud offerings. It turned out to be way too long and complex to deal with both of those topics in one post, so I am primarily going to follow up on Phil’s earlier posts on licensing here and save the cloud computing part for a future post.

Let’s start by recapping the situation:

  • The Kuali Foundation is a non-profit entity formed by a group of schools that wanted to pool their resources to build a suite of open source components that together could make up an ERP system suitable for colleges and universities.
  • One of the first modules, Kuali Financials, has been successfully installed and run by a small number of schools which have reported dramatic cost savings, both in the obvious licensing but also in the amount of work it took to get the system running.
  • Other elements of the suite have been been more problematic. For example, the registrar module, Kuali Student, has been plagued with multi-year delays and members dropping out of the project.
  • Kuali code has historically been released under an open source license called the “Educational Community License (ECL)”, which is a slight variation of the Apache license.
  • The Kuali Foundation announced that ownership of the code would be transferred to a new commercial entity, KualiCo. Their reason for doing so is to accelerate the development of the suite, although they have not been entirely clear about what they think the causes of their development speed problem are and why moving to a commercial structure will solve the problems.
  • KualiCo intends to release most, but not all, of future code under a different open source license, the Affero Gnu Public License (AGPL).
  • Some of the new code to be developed by KualiCo will not be released as open source. So far the only piece they have named in this regard is their multitenant code.
  • These changes happened fairly quickly, driven by the board, without a lot of community discussion.

So is this “open washing”? Is it a betrayal of trust of an open source community, or deceptively claiming to be open and proving to be otherwise? In one sense, the answer to that question depends on the prior expectations of the community members. I would argue that the biggest and clearest act of open washing may have occurred over a decade ago with the creation of the term “community source.” Back in the earlier days of Sakai—another “community source” project—I made a habit of asking participants what they thought “community source” meant. Very often, the answer I got was something along the lines of, “It’s open source, but with more emphasis on building a community.” But that’s not what Brad Wheeler meant at all when he popularized the term. As I have discussed in an earlier post, “community source” was intended to be a consortial model of development with an open source license tacked onto the end product. Far from emphasizing community, it was explicitly designed to maximize the control and autonomy of the executive decision-makers from the consortial partners—and to keep a lid on the decision-making power of other community participants. Remember the motto: “If you’ve got the gold, then you make the rules.” Community source, as defined by those who coined the term, is a consortium with a license. But community source was always marketed as an improvement on open source. “It’s the pub between the cathedral and the bazaar where all the real work gets done,” Brad liked to say.

Different “community source” projects followed the centralized, financially-driven model to different degrees. Kuali, for example, was always explicitly and deliberately more centralized in its decision-making processes than Sakai. As an outside observer of Kuali’s culture and decision-making processes, and as a close reader of Brad Wheeler’s articles and speeches about community source, I can’t say that the move to KualiCo surprised me terribly. Nor can I say that it is inconsistent with what Brad has said all along about how community source works and what it is for. The consortial leaders, whose membership I assume was roughly defined by their financial contributions to the consortium, made a decision that supports what they believe is in their interest. All code that was previously released under an open source license will remain under an open source license. Presumably the Kuali Foundation and KualiCo will be clear going forward about which consortial contributions go toward creating functionality that will be open source or private source in the future. I am not privy to the internal politics of the foundation and therefore am not in a position to say whether some of those who brought the gold were left out of the rule-making process. To the degree that Brad’s golden rule was followed, the move to KualiCo is consistent with the clearly stated (if craftily marketed) philosophy of community source.

The question of how much these changes practically affect open source development of Kuali is a more complicated one to answer. It is worth stating that another tenet of community source was that it was specifically intended to be commercial-friendly, meaning that the consortia tried to avoid licensing or other practices that discouraged the development of a healthy and diverse ecosystem of support vendors. (Remember, community source frames problems with the software ecosystem as procurement problems. As such, its architects are concerned with maintaining a robust range of support contracting options.) Here the balancing act is more delicate. On the one hand, to the degree that the changes give KualiCo advantages over potential competitors, Kuali will be violating the commercial-friendly principle of community source. On the other hand, to the degree that the changes do not give KualiCo advantages over potential competitors, it’s not clear why one would think that KualiCo will be a viable and strong enough company to move development faster than the way it has been until now.

The first thing to point out here is that, while KualiCo has only said so far that it will keep the multitenant code private source, there is nothing to prevent them from keeping more code private in the future. Instructure Canvas, which started out with only the multitenant code as private source, currently has the following list of features that are not in the open source distribution:

  • Multi-tenancy extensions
  • Mobile integration
  • Proprietary SIS integrations
  • Migration tools for commercial LMSs
  • Other minor customizations that only apply to our hosted environment
  • Chat Tool
  • Attendance Tool (Roll Call)

I don’t think there is a clear and specific number of private source features that marks the dividing line between good faith open source practices and “open washing”; nor am I arguing that Instructure is open washing here. Rather, my point is that, once you make the decision to be almost-all-but-not-completely open source, you place your first foot at the top of a slippery slope. By saying that they are comfortable withholding code on any feature for the purposes of making their business viable, KualiCo’s leadership opens the door to private sourcing as much of the code as they need to in order to maintain their competitive advantage.

Then there’s the whole rather arcane but important question about the change in open source licenses. Unlike the ECL license that Kuali has used until now, AGPL is “viral,” meaning that anybody who combines AGPL-licensed code with other code must release that other code under the AGPL as well. Anybody, that is, except for the copyright holder. Open source licenses are copyright licenses. If KualiCo decides to combine  open source code to which they own the copyright with private source code, they don’t have to release the private source code under the AGPL. But if a competitor, NOTKualiCo, comes along and combines KualiCo’s AGPL-licensed code with their own proprietary code, then NOTKualiCo has to release their own code under the AGPL. This creates two theoretical problems for NOTKualiCo. First, NOTKualiCo does not have the option of making the code they develop a proprietary advantage over their competitors. They have to give it away. Second, while NOTKualiCo has to share its code with KualiCo, KualiCo doesn’t have the same obligation to NOTKualiCo. So theoretically, it would be very hard for any company to compete on product differentiators when they are building upon AGPL-licensed code owned by another company.

I say “theoretically” because, in practice, it is much more complicated than that. First, there is the question of what it means to “combine” code. The various GPL licenses recognize that some software is designed to work with other software “at arm’s length” and therefore should not be subject to the viral clause. For example, it is permissible under the license to run AGPL applications on a Microsoft Windows or Apple Mac OS X operating system without requiring that those operating systems also be released under the GPL. Some code combinations fall clearly into this category, while others fall clearly into the category of running as part of the original open source program and therefore subject to the viral clause of the GPL. But there’s a vast area in the murky middle. Do tools that use APIs specifically designed for integration fall under the viral clause? It depends on the details of how they integrate as well as who you ask. It doesn’t help that the language used to qualify what counts as “combining” in Gnu’s documentation uses terms that are specific to the C programming language.

KualiCo has said that they will specifically withhold multitenant capabilities from future open source distributions. If competitors developed their own multitenant capabilities, would they be obliged to release that code under the AGPL? Would such code be “combining” with Kuali, or could it be sufficiently arm’s-length that it could be private source? It depends on how it’s developed. Since KualiCo’s CEO is the former CTO of Instructure, let’s assume for the sake of argument that Kuali’s multitenant capabilities will be developed similarly to Canvas’. Zach Wily, Instructure’s Chief Architect, described their multitenant situation to me as follows:

[O]ur code is open-source, but only with single-tenancy. The trick there is that most of our multi-tenancy code is actually in the open source code already! Things like using global identifiers to refer to an object (instead of tenant-local identifiers), database sharding, etc, are all in the code. It’s only a couple relatively thin libraries that help manage it all that are kept as closed source. So really, the open-source version of Canvas is more like a multi-tenant app that is only convenient to run with a single tenant, rather than Canvas being a single-tenant app that we shim to be multi-tenant.

The good news from NOTKualiCo’s (or NOTInstructureCo’s) perspective is that it doesn’t sound like there’s an enormous amount of development required to duplicate that multitenant functionality. Instructure has not gone through contortions to make the development of multitenant code harder for competitors; I will assume here that KualiCo will follow a similar practice. The bad news is that the code would probably have to be released under the AGPL, since it’s a set of libraries that are intended to run as a part of Kuali. That’s far from definite, and it would probably require legal and technical experts evaluating the details to come up with a strong conclusion. But it certainly seems consistent with the guidance provided by the Gnu Foundation.

OK, so how much of a practical difference does this make for NOTKualiCo to be able to compete with KualiCo? Probably not a huge amount, for several reasons. First, we’re not talking about an enormous amount of code here; nor is it likely to be highly differentiated. But also, NOTKualiCo owns the copyright on the libraries that they release. While anybody can adopt them under the AGPL, if KualiCo wanted to incorporate any of NOTKualiCo’s code, then the viral provision would have to apply to KualiCo. The practical net effect is that KualiCo would almost certainly never use NOTKualiCo’s code. A third competitor—call them ALSONOTKualiCo—could come in and use NOTKualiCo’s code without incurring any obligations beyond those that they already assumed by adopting KualiCo’s AGPL code, so there’s a disadvantage there for NOTKualiCo. But overall, I don’t think that withholding multitenant code from KualiCo’s open source releases—assuming that it’s done the way Instructure has done it—is a decisive barrier to entry for competitors. Unfortunately, that may just mean that KualiCo will end up having to withhold other code in order to maintain a sustainable advantage.

So overall, is Kuali guilty of “open washing” or not? I hope this post has helped make clear why I don’t love that term. The answer is complicated and subjective. I believe that “community source” was an overall marketing effort that entailed some open washing, but I also believe that (a) Brad has been pretty clear about what he really meant if you listened closely enough, and (b) not every project that called itself community source followed Brad’s tenets to the same degree or in the same way. I believe that KualiCo’s change in license and withholding of code are a violation of the particular flavor of openness that community source promised, but I’m not sure how much of a practical difference that makes to the degree that one cares about the “commercial friendliness” of the project. Would I participate in work with the Kuali Foundation today if I were determined to work only on projects that are committed to open source principles and methods? No I would not. But I would have given you the same answer a year ago. So, after making you wade through all of those arcane details, I’m sorry to say that my answer to the question of whether Kuali is guilty of open washing is, “I’m not sure that I know what the question means, and I’m not sure how much the answer matters.”

The post Is Kuali Guilty of “Open Washing”? appeared first on e-Literate.

Pearson, Efficacy, and Research

Mon, 2014-12-01 08:03

A while back, I mentioned that MindWires, the consulting company that Phil and I run, had been hired by Pearson in response to a post I wrote a while back expressing concerns about the possibility of the company trying to define “efficacy” in education for educators (or to them) rather than with them. The heart of the engagement was us facilitating conversations with different groups of educators about how they think about learning outcomes—how they define them, how they know whether students are achieving them, how the institution does or doesn’t support achieving them, and so on. As a rule, we don’t blog about our consulting work here on e-Literate. But since we think these conversations have broader implications for education, we asked for and received permission to blog about what we learn under the following conditions:

  • The blogging is not part of the paid engagement. We are not obliged to blog about anything in particular or, for that matter, to blog at all.
  • Pearson has no editorial input or prior review of anything we write.
  • If we write about specific schools or academics who participated in the discussions, we will seek their permission before blogging about them.

I honestly wasn’t sure what, if anything, would come out of these conversations that would be worth blogging about. But we got some interesting feedback. It seems to me that the aspect I’d like to cover in this post has implications not only for Pearson, and not only for ed tech vendors in general, but for open education and maybe for the future of education in general. It certainly is relevant to my recent post about why the LMS is the way it is and the follow-up post about fostering better campus conversations. It’s about the role of research in educational product design. It’s also about the relationship of faculty to the scholarship of teaching.

It turns out that one of the aspects about Pearson’s efficacy work that really got the attention of the folks we talked with was their research program. Pearson has about 40 PhDs doing educational research of different kinds throughout the company. They’ve completed about 300 studies and have about another 100 currently in progress. Given that educational researchers were heavily represented in the groups of academics we talked to, it wasn’t terribly surprising that the reaction of quite a few of them were variations of “Holy crap!” (That is a direct quote of one the researchers.) And it turns out that the more our participants knew about learning outcomes research, the more they were interested in talking about how little we know about the topic. For example, even though we have had course design frameworks for a long time now, we don’t know a whole lot about which course design features will increase the likelihood of achieving particular types of learning outcomes. Also, while we know that helping students develop a sense of community in their first year at school increases the likelihood that they will stay on in school and complete their degrees, we know very little about which sorts of intra-course activities are most likely to help students develop that sense of connectedness in ways that will measurably increase their odds of completion. And to the degree that research on topics like these exist, it’s scattered throughout various disciplinary silos. There is very little in the way of a pool of common knowledge. So the idea of a well-funded organization conducting high volumes of basic research was exciting to a number of the folks that we talked to.

But how to trust that research? Every vendor out there is touting their solutions based on “brain science” and “big data.” How can the number of PhDs a vendor employs or the number of “studies” that it conducts yield more credible value than a bullet point in the marketing copy?

In part, the answer is surprisingly simple: Vendors can demonstrate the credibility and value of their research using the same mechanisms that any other researcher would. The first step is transparency. It turns out that Pearson already publishes a library of their studies on their “research and innovation network” site. Here is a sample of some of their more recent titles that will give you a sense of the range of topics:

Pearson also has a MyLabs- and Mastering-specific site that is more marketing-oriented but still has some research-based reports in it.

How good is this research? I don’t know. My guess is that, like any large body of research conducted by a reasonably large group of people, it probably varies in quality. Some of these studies have been published in academic journals or presented in academic conferences. Many have not. One thing we heard from a number of the folks we spoke to was that they’d like to see Pearson submit as much of their research as possible to blind peer-reviewed journals. Ultimately, how does an academic typically judge the quality of any research? The number of citations it gets is a good place to start. So the folks that we talked to wanted to see Pearson researchers participate as peers in the academic research community, including submitting their work to the same scrutiny that academic research undergoes.

This is approach isn’t perfect, of course. We’ve seen in industries like pharmaceuticals that deep-pocketed industry players can find various ways to warp the research process. But pharmaceuticals are particularly bad because (a) the research studies are incredibly expensive to conduct, and (b) they require access to the proprietary drugs being tested, which can be difficult in general and particularly so before the product is released to the market. Educational research is much less vulnerable to these problems, but it has one of its own. By and large, replicability of experiments (and therefore confirmation or disconfirmation of results) is highly difficult or even impossible in many educational situations for both logistical and ethical reasons. So evaluating vendor-conducted or vendor-sponsored educational research would have its challenges, even with blind peer review. That said, the opinions of many of the folks we talked to, particularly of those who are involved in conducting, reviewing, and publishing academic educational research, was that the challenges are manageable and the potential value generated could be considerable.

Even more interesting to me were the discussions about what to do with that research besides just publishing it. There was a lot of interest in getting faculty engaged in with the scholarship of teaching, even in small ways. Take, for example, the case of an adjunct instructor, running from one school to the next to cobble together a living, spending many, many hours grading papers and exams. That person likely doesn’t have time to do a lot of reading on educational research, never mind conducting some. But she might appreciate some curricular materials that say, “there are at least three different ways to teach this topic, but the way we’re recommending is consistent with research on a sense of building class community that encourages students to feel like they belong at the school and reduces dropout rates.” She might even find some precious time to follow the link and read that research if it’s on a topic that’s important enough to her.

This is pretty much the opposite of how most educational technology and curricular materials products are currently designed. The emphasis has historically been on making things easier for the instructors by having them cede more control to the product and vendor. “Don’t worry. It’s brain science! It’s big data! You don’t have to understand it. Just buy it and the product will do the work for you.” Instead, these products could be educating and empowering faculty to try more sophisticated pedagogical approaches (without forcing them to do so). Even if most faculty pass up these opportunities most of the time, simply providing them with ready contextual access to relevant research could be transformative in the sense that it constantly affords them new opportunities to incorporate the scholarship of teaching into their daily professional lives. It also could encourage a fundamentally different relationship between the teachers and third-party curricular materials, whether they are vendor-provided or OER. Rather than being a solitary choice made behind closed doors, the choice of curricular materials could include, in part, the choice of a community of educational research and practice that the adopting faculty member wants to join. Personally, I think this is a much better selection criterion for curricular materials than the ones that are often employed by faculty today.

These ideas came out of conversations with just a couple of dozen people, but the themes were pretty strong and consistent. I’d be interested to hear what you all think.

The post Pearson, Efficacy, and Research appeared first on e-Literate.

Upcoming EDUCAUSE Webinars on Dec 4th and Dec 8th

Sun, 2014-11-30 20:39

Michael and I will be participating in two upcoming EDUCAUSE webinars.

Massive and Open: A Flipped Webinar about What We Are Learning

On Thursday, December 4th from 1:00–2:00 p.m. ET we will be joined by George Siemens for an EDUCAUSE Live! webinar:

In 2012, MOOCs burst into public consciousness with course rosters large enough to fill a stadium and grand promises that they would disrupt higher education. Two years later, after some disappointments, setbacks, and not a small amount of schadenfreude, MOOCs seem almost passé. And yet, away from the sound and the fury, researchers and teachers have been busy finding answers to some basic questions: What are MOOCs good for, and what can we learn from them? Phil Hill and Michael Feldstein will talk about what we’re learning so far.

This will be a flipped webinar. We strongly encourage you to watch the 14-minute video interview film of MOOC Research Initiative (MRI) grantees before the webinar begins. If you would like more background on MOOCs, feel free to watch the other two videos (parts 2 and 3) in the series as well.

More information on this page. Registration is free.

UPDATE: The recording is now available, both the Adobe Connect and the chat transcript.

Teaching and Learning: 2014 in Retrospect, 2015 in Prospect

On Monday, December 8th from 1:00 – 2:00 p.m. ET we will be joined by Audrey Watters for an EDUCAUSE Learning Initiative (ELI) webinar:

Join Malcolm Brown, EDUCAUSE Learning Initiative director, and Veronica Diaz, ELI associate director, as they moderate this webinar with Phil Hill, Michael Feldstein, and Audrey Watters.

The past year has been an eventful one for teaching and learning in higher education. There have been developments in all areas, including analytics, the LMS, online education, and learning spaces. What were the key developments in 2014, and what do they portend for us in 2015? For this ELI webinar, we will welcome a trio of thought leaders to help us understand the past year and what it portends for the coming year. Come and share your own insights, and join the discussion.

More information on this page. This webinar is available for ELI member institutions, but it will be publicly available 90 days later.

We hope you can join us for these discussions.

The post Upcoming EDUCAUSE Webinars on Dec 4th and Dec 8th appeared first on e-Literate.

WCET14 Student Panel: What do students think of online education?

Fri, 2014-11-21 15:37

Yesterday at the WCET14 conference in Portland I had the opportunity along with Pat James to moderate a student panel.[1] I have been trying to encourage conference organizers to include more opportunities to let students speak for themselves – becoming real people with real stories rather than nameless aggregations of assumptions. WCET stepped up with this session. And my new favorite tweet[2]:

@kuriousmind @lukedowden @wcet_info @PhilOnEdTech Best. Panel. Ever. Massively insightful experience.

— Matthew L Prineas (@mprineas) November 20, 2014

As I called out in my introduction, we talk about students, we characterize students, we listen to others talk about students, but we don’t do a good job in edtech talking with students.  There is no way that a student panel can be representative of all students, even for a single program or campus[3]. We’re not looking for statistical answers, but we can hear stories and gain understanding.

These four students were either working adults (and I’m including a stay-at-home mom in this category) taking undergraduate online programs. They were quite well-spoken and self-aware, which made for a great conversation that included comments that might surprise some on faculty-student interaction potential:

A very surprising (to me) comment on class size:

And specific feedback on what doesn’t work well in online courses:

To help with viewing of the panel, here are the primary questions / topics of discussion:

The whole student panel is available on the Mediasite platform:

Thanks to the help of the Mediasite folks, I have also uploaded a Youtube video of the full panel:

Click here to view the embedded video.

  1. Pat is the executive director of the California Community College Online Education Initiative (OEI) – see her blog here for program updates.
  2. I’m not above #shameless.
  3. As can be seen from this monochromatic panel, which might make sense for Portland demographics but not from a nationwide perspective.

The post WCET14 Student Panel: What do students think of online education? appeared first on e-Literate.

In Which I (Partially) Disagree with Richard Stallman on Kuali’s AGPL Usage

Wed, 2014-11-19 18:32

Since Michael is making this ‘follow-up blog post’ week, I guess I should jump in.

In my latest post on Kuali and the usage of the AGPL license, the key argument is that this license choice is key to understanding the Kuali 2.0 strategy – protecting KualiCo as a new for-profit entity in their future work to develop multi-tenant cloud hosting code.

What I have found interesting is that in most of my conversations with Kuali community people ,even for those who are disillusioned, they seem to think the KualiCo creation makes some sense. The real frustration and pushback has been on how decisions are made, how decisions have been communicated, and how the AGPL license choice will affect the community.

In the comments, Richard Stallman chimed in.

As the author of the GNU General Public License and the GNU Affero General Public License, and the inventor of copyleft, I would like to clear up a possible misunderstanding that could come from the following sentence:

“Any school or Kuali vendor, however, that develops its own multi-tenant cloud-hosting code would have to relicense and share this code publicly as open source.”

First of all, thinking about “open source” will give you the wrong idea about the reasons why the GNU AGPL and the GNU GPL work as they do. To see the logic, you should think of them as free software licenses; more specifically, as free software licenses with copyleft.

The idea of free software is that users of software deserve freedom. A nonfree program takes freedom away from its users, so if you want to be free, you need to avoid it. The aim of our copyleft licenses is to make sure all users of our code get freedom, and encourage release of improvements as free software. (Nonfree improvements may as well be discouraged since we’d need to avoid them anyway.) See

I don’t use the term “open source”, since it rejects these ethical ideas. ( Thus I would say that the AGPL requires servers running modified versions of the code to make the source for the running version available, under the AGPL, to their users.

The license of the modifications themselves is a different question, though related. The author of the modifications could release the modifications under the AGPL itself, or under any AGPL-compatible free software license. This includes free licenses which are pushovers, such as the Apache 2.0 license, the X11 license, and the modified BSD license (but not the original BSD license — see

Once the modifications are released, Kuali will be able to get them and use them under whatever license they carry. If it is a pushover license, Kuali will be able to incorporate those modifications even into proprietary software. (That’s what makes them pushover licenses.)

However, if the modifications carry the AGPL, and Kuali incorporates them into a version of its software, Kuali will be bound by the AGPL. If it distributes that version, it will be required to do so under the AGPL. If it installs that version on a server, it will be required by the AGPL to make the whole of the source code for that version available to the users of that server.

To avoid these requirements, Kuali would have to limit itself to Kuali’s own code, others’ code released under pushover licenses, plus code for which it gets special permission. Thus, Kuali will not have as much of a special position as some might think.

See also

Dr Richard Stallman
President, Free Software Foundation (,
Internet Hall-of-Famer (
MacArthur Fellow

I appreciate this clarification and Stallman’s participation here at e-Literate, and it is useful to understand the rationale and ethics behind AGPL. However, I disagree with the statement “Thus, Kuali will not have as much of a special position as some might think”. I do not think he is wrong, per se, but the combination of both the AGPL license and the Contributor’s License Agreement (CLA) in my view does ensure that KualiCo has a special position. In fact, that is the core of the Kuali 2.0 strategy, and their approach would not be possible without the AGPL usage.

Note: I have had several private conversations that have helped me clarify my thinking on this subject. Besides Michael with his comment to the blog, Patrick Masson and three other people have been very helpful. I also interviewed Chris Coppola from KualiCo to understand and confirm the points below. Any mistakes in this post, however, are my own.

It is important to understand two different methods of licensing at play – distributing code through an APGL license and contributing code to KualiCo through a CLA (Kuali has a separate CLA for partner institutions and a Corporate CLA for companies).

  • Distribution – Anyone can download the Kuali 2.0 code from KualiCo and make modifications as desired. If the code is used privately, there is no requirement for distributing the modified code. If, however, a server runs the modified code, the reciprocal requirements of AGPL kick in and the code must be distributed (made available publicly) with the AGPL license or a pushover license. This situation is governed by the AGPL license.
  • Contribution – Anyone who modifies the Kuali 2.0 code and contributes it to KualiCo for inclusion into future releases of the main code grants a license with special permission to KualiCo to do with the code as they see fit. This situation is governed by the CLA and not AGPL.

I am assuming that the future KualiCo multi-tenant cloud-hosting code is not separable from the Kuali code. In other words, the Kuali code would need modifications to allow multi-tenancy.

For a partner institution, their work is governed by the CLA. For a company, however, the choice on whether to contribute code is mutual between that company and KualiCo, in that both would have to agree to sign a CLA. Another company may choose to do this to ensure that bug fixes or Kuali enhancements get into the main code and do not have to be reimplemented with each new release.

For any contributed code, KualiCo can still keep their multi-tenant code proprietary as their special sauce. For distributed code under AGPL that is not contributed under the CLA, the code would be publicly available and it would be up to KualiCo whether to incorporate any such code. If KualiCo incorporated any of this modified code into the main code base, they would have to share all of the modified code as well as their multi-tenant code. For this reason, KualiCo will likely never accept any code that is not under the CLA – they do not want to share their special sauce. Chris Coppola confirmed this assumption.

This setup strongly discourages any company from directly competing with KualiCo (vendor protection) and is indeed a special situation.

The post In Which I (Partially) Disagree with Richard Stallman on Kuali’s AGPL Usage appeared first on e-Literate.