Zoie Saunders, Vermont’s interim secretary of education, has hailed the importance of data in revitalizing the state’s education system since she took office this spring.
Stakeholders say they desperately need data. School districts are building budgets for next school year. The Commission on the Future of Public Education in Vermont is trying to plot a more affordable and successful system. And the Vermont Agency of Education has undertaken new statewide and district-level data reports.
“When I stepped into this role, I immediately heard a call from communities and school board members for more data,” Saunders said in an interview. “If we’re not in the practice of using data to make decisions, and there’s now new information, it takes a little bit of time for people to pause and process that information.”
But the “shift to create a data-driven culture,” as Saunders called it, has come with hiccups. Education leaders say recent state data reports include a variety of significant errors. Some of the future of public education commission’s members worry the agency can’t turn around data requests fast enough. And some lawmakers have expressed concern about several key finance-related vacancies within the education agency.
Saunders said it’s all part of the process, with improvements already underway.
“This is going to be the first time the agency has done this level of comprehensive reporting,” she said of her agency’s data initiatives. “We have to go a little slow to go fast.”
Superintendents find data discrepancies
The first phase of Saunders’s listen and learn tour, her cornerstone public engagement initiative, involved collecting and analyzing key district metrics, culminating in a statewide data profile released in August.
The rollout has garnered grumblings from superintendents across the state, who point to significant flaws in the agency’s data that have not yet been corrected.
Sheila Soule, superintendent of the Addison Northwest School District, said that her team “had to send corrections for enrollment data, student demographics, graduation rates, and expenditure data.”
“It is unclear how they are reporting student achievement data … it is too difficult to interpret but does not align with our data,” Soule wrote in an email. “The most concerning error was around the enrollment data as the AOE showed our enrollment as having increased over the past several years, when in fact it has remained relatively flat (after a significant decline).”
Similar to Soule, Brooke Olsen-Farrell, superintendent of Slate Valley Unified School District, said she was “concerned about the data presented to us” due to “several significant inaccuracies.”
Olsen-Farrell said the AOE’s report indicated the district had a 27% increase in enrollment even though it’s actually experienced declining enrollment.
“The staff retention figures appear to be incorrect, and it’s unclear what sources were used to compile this data,” she added. “While we were given the chance to provide feedback, we have yet to see any revisions to the original report.”
Randi Lowe, superintendent of the Bennington Rutland Supervisory Union, oversees six schools in a district that offers school choice for high school. When the district received its profile, the data didn’t appear to factor in the high school students still under BRSU’s umbrella.
“I realized immediately that something was wrong,” Lowe said.
The profile, she said, included BRSU’s total costs while excluding high school choice students — greatly inflating the supervisory union’s apparent cost per pupil.
“If my district’s data was going (to be used) to talk about spending patterns in the state and it looks like we’re spending significantly more than we actually are,” Lowe said, “that’s concerning.”
The agency’s statewide data profile and localized snapshots for districts were created by an outside firm, APA Consulting. Saunders has acknowledged the errors, some of which she attributed to “inconsistency in coding at the level of the school.” She said that the state plans to release updated profiles later this month.
But it wasn’t Lowe’s first experience with faulty data from the state. Last year, the agency told her that one of the district’s schools qualified for some federal funding generally meant for more economically disadvantaged children.
“I could not for the life of me understand how that determination was made,” Lowe recalled.
Ultimately, she discovered there was an error made, as she suspected, but not before she’d informed the school that it could expect more money.
“I really appreciate that Interim Secretary Saunders is really trying to use data,” Lowe said. But she added a note of caution: “These decisions are important and we don’t want to make them based on faulty information.”
A call for ‘more in depth and comprehensive data’
Amid the frustration from superintendents, the association representing them, as well as those advocating for school business officials and school boards, have raised concerns that the Agency of Education isn’t meeting the data needs of the ongoing commission on the future of public education.
In a letter dated Nov. 1, the organizations — the Vermont Superintendents Association, Vermont School Boards Association and Vermont Association of School Business Officials — noted that the agency had not fulfilled data requests submitted by the commission in August.
More than a dozen specific line items had gone unanswered, according to the associations’ leaders, and the agency presented the commission with the same figures in October as it did in August.
“This Commission requires more in depth and comprehensive data than the types of data being utilized to inform the Agency’s Listen and Learn Tour,” the letter read.
The data compiled by the agency also made comparison difficult, as it uses a metric to count students that doesn’t completely factor in school choice, and it included figures that do not cleanly distinguish expenditures from the state’s Education Fund, the letter noted.
Saunders, for her part, expressed disappointment with the associations’ accusations. And in an interview, she indicated that the delays were due to “data validation,” as the agency ensures it updates all the discrepancies in the local data profiles before reporting out new information.
The agency will soon assess trends on a more localized level, according to Saunders, such as school by school.
“That’s where it’s going to be really important that we’ve done this extra process of data validation,” she said. “It’s all part of the design.”
Shaken confidence
Last month, a group of House lawmakers penned a letter to the Agency of Education expressing concern about vacancies at the department, including the departure of Finance Director Nicole Lee this fall.
The departures, including Lee’s, “have shaken our and the field’s confidence in the AOE’s ability to respond to great need and provide knowledgeable guidance in the coming months,” they wrote. “We are especially concerned about Nicole Lee’s position given its role in the management of the $2.4 billion Education Fund and as a resource to districts during the budget season.”
Lee did not respond to an interview request.
Rep. Emilie Kornheiser, D-Brattleboro, was one of the letter’s signatories and chairs the House Ways and Means Committee, a key body responsible for developing property tax legislation. She also leads the future of public education commission’s finance subcommittee. She said there aren’t enough eyes devoted to tracking education spending in Vermont.
“At this point, the Education Fund is almost equal to the General Fund, and the scale of administrative attention that it gets is just significantly different,” she said, “The accountability systems built into monitoring it, the data systems for collecting and analyzing it.”
Much of the burden falls on the Agency of Education, which in addition to losing its finance director, also currently lacks an assistant finance director and permanent chief financial officer, all key players in the financial team.
As keeper of school districts’ finance data, the Agency of Education works closely with other arms of state government with interests in education costs, including the Tax Department and the Legislative Joint Fiscal Office. This past legislative session, the Agency of Education’s finance director was a regular in state tax committees, furnishing lawmakers with data to help inform property tax decisions.
Due to the complexities of the job, the incoming finance director has in the past apprenticed to their predecessor — something that won’t be possible in the same way after Lee’s departure.
But Saunders downplayed the significance of the vacancies, which she characterized as normal across state government.
“The work of the agency is not dependent on one person,” Saunders said, noting that the agency’s finance department had multiple vacancies upon her arrival, part of the reason she’d hired consulting help. The agency also brought on an assistant data director this fall, and is contracting with a previous education finance director for extra help.
“We recognized we needed some additional support in the short term,” Saunders said. “Because it is such a critical area, and given that so many of our school budgets failed last year, we understood coming into this cycle that we had to be really strategic.”
The agency has one more looming item on its data agenda: the “Dec. 1 letter,” written by the tax commissioner, which contains the state’s annual education property tax forecast.
Last year, the letter sent shockwaves across the state when it predicted an 18.5% tax increase — not quite four percentage points higher than the eventual 13.8% average increase statewide.
This year, some stakeholders fear another round of tax hikes, especially given that no substantial changes to the state’s education finance system came to fruition last legislative session.
Providing data for the Dec. 1 letter is part of the agency’s “routine,” Saunders said, and she wasn’t worried about vacancies impacting its creation.
When the letter arrives, policymakers will have a clearer picture of next year’s property tax bills.
Affordability — and voter discontent — are top of mind, and with the legislative session mere weeks away, lawmakers will once again rely on the agency’s data.
Read the story on VTDigger here: Zoie Saunders wants to make the Agency of Education data driven. That’s easier said than done..