Halfway through the last school year, Leila Campbell, a young humanities teacher at a charter high school in Oakland, Calif., received the results from a recent survey of her students.
On most measures, Ms. Campbell and her fellow teachers at the Aspire Lionel Wilson Preparatory Academy were scoring at or above the average for Aspire, a charter system that runs more than a dozen schools in California and Tennessee.
But the survey, conducted by a tech start-up calledPanorama Education, also indicated that her students did not believe she was connecting with them. Ninety-six percent of the students at Lionel Wilson are Hispanic, and 92 percent receive school lunch assistance.
“It’s a very different population from where I grew up,” Ms. Campbell, who is white, said in a recent interview in her classroom. “I wasn’t scoring where I wanted to with questions like ‘I feel comfortable asking my teacher for help’ or ‘My teacher really cares about me.’ I was below average, and I don’t want to be below average.”
Aaron Feuer, the co-founder and chief of Panorama Education, center, with Brian Rainville and Jessica Cole. Credit Gretchen Ertl for The New York Times
Panorama is trying to assess how well teachers are doing by conducting scientifically valid student questionnaires that collect data about a variety of factors that might affect a teacher’s performance, from how well she conveys the material and whether she encourages interest in a subject to whether a school fosters a sense of belonging for students.
The company, which is run by two 23-year-old Yale grads with a penchant for computers and data crunching, has run surveys in more than 5,000 schools, and it has been adopted by some of the largest school systems in the nation, including the Los Angeles Unified School District and schools in Connecticut.
Panorama has followed the model of Uber and Airbnb in using the unconventional methods of tech start-ups to reinvent industries that have long been seen as tech backwaters. And its increasing popularity suggests that techniques pioneered by the tech industry — including the collection and analysis of large troves of data — may help address problems in American education.
The firm’s techniques have been widely praised by education experts, and it has won prominent supporters in the tech industry. Mark Zuckerberg, the co-founder of Facebook, and Google Ventures, the search company’s investment arm, are among its largest backers.
Some of its innovations sound small, but they have been instrumental in making its surveys more widely accessible than older educational survey methods. For instance, to reduce the costs of its surveys, Panorama created its own scanning system, which allows it to print and collect students’ answers on regular paper, rather than the expensive bubble-scan sheets more commonly used for collecting responses. The firm says its surveys and analytics services are about half the price of older survey methods.
Panorama also hired a team of software engineers and statistics experts to create a kind of analytics “dashboard” for schools — an interactive panel of graphs and charts that presents practical information teachers need in a comprehensible, rather than overwhelming, user interface.
Teachers can dig into how they performed on a question-by-question basis, and they can monitor their performance by subgroup. The survey reports allow teachers to see if they’re connecting better with boys than with girls, or if students who have trouble with English are having more difficulty in a classroom than those who are native English speakers.
Panorama has also invested heavily in improving survey science. It has sponsored research into the best way to ask questions of students to get the most accurate assessment of what’s going on in the classroom, including one recent study by Hunter Gehlbach, a professor of education at Harvard. Last week, again borrowing from the tech industry, the company announced that it would make the survey open-source, meaning schools can use and amend it free.
Seeing how students think about teachers, and how that perception is affecting what they learn, is an unusual development in public education. Today, schools assess the effectiveness of teachers primarily through standardized test scores and observations by administrators, but both measures have been criticized as too narrow, unable to shed light on the complex interplay between teachers and students on a day-to-day basis.
A trophy wall at Panorama Education’s offices in Boston. Panorama is trying to assess how well teachers are doing by conducting scientifically valid student questionnaires that collect data about a variety of factors that might affect a teacher’s performance.
“Education is just starting to figure out what measurement actually means,” said Aaron Feuer, Panorama’s co-founder and chief executive. “Five years ago we thought test scores were the answer to everything. We’re offering a way to focus on the right metrics.”
But in some ways, what the company is doing isn’t new. Student surveys have long been seen as a potential third metric for education. In 2012, the Measures of Effective Teaching Project, a three-year study sponsored by the Bill and Melinda Gates Foundation, found that when combined with test scores and observations, student surveys made for a more reliable and consistent way to measure how teachers were performing.
“A lot of people are unhappy with an overreliance on test scores, but I don’t think it’s an option to drop test scores and go to nothing,” said Thomas J. Kane, a professor of education at Harvard who directed that study. “Student surveys are the most obvious place to add some other measures that aren’t based solely on test scores.”
Mr. Feuer became fascinated by student surveys when he was attending an urban high school in Los Angeles in which about half of incoming freshmen did not graduate. Mr. Feuer is a computer enthusiast with an appetite for data, so he naturally searched for numbers to explain his school’s low performance, but found few hard statistics. He became active in student government, eventually becoming president of the California Association of Student Councils. In that role, he persuaded California’s Legislature to pass a law that wouldencourage schools to solicit student feedback.
But after Mr. Feuer graduated from high school and began attending Yale, he realized that the law he’d helped push had gone nowhere. As useful as they were, student feedback surveys were too expensive and cumbersome for widespread adoption. Then, in college, he met Xan Tanner, a student who was working as a statistical analyst for the Yale men’s basketball team.
“We were in a military history class talking about his basketball analytics work, and we realized that there’s a pretty strong parallel between coaching athletes and coaching teachers,” Mr. Feuer said. “Athletes, like teachers, are smart, talented professionals, and you can’t reduce what they do to a couple of simple stats.”
Instead, you need close monitoring and a large constellation of data to effectively assess their performance. In 2012, the pair decided to start a tech company devoted to making surveys more widely available for schools. The business proved immediately successful, with dozens of schools signing up to test the program.
It is too soon to tell how widely schools will adopt surveys like Panorama’s, and how deeply surveys might become integrated into the education-reform movement’s effort to find a better way to measure teachers. But some teachers and administrators, including those at Aspire Lionel Wilson in Oakland, say the surveys have been instrumental in how they approach the classroom.
After discovering her difficulties connecting with students, Ms. Campbell began to adjust her classroom manner. “I do a presentation where I open up to them, making myself vulnerable about my college experience, and telling them why I’m working with them,” she said. “They start to get me as a human being. And they’re willing to follow me when I push them harder in history and English.” In surveys since then, Ms. Campbell has noticed an improvement in her results.
“The surveys have been transformational in how I operate,” she said. “I’ve grown tremendously from this data.”