Eric Heilman’s Case for Data Governance Before Operational AI
Presented by:
Eric Heilman, executive director of the Center for Institutional Research in Independent Schools (CIRIS), discusses Clean Data, Clear Vision, a new data strategy handbook for independent schools. Listeners learn why schools overestimate their data quality, how to collect demographic data consistently, and why governed data must come before operational AI tools.
- Clean Data, Clear Vision: An Independent School Data Strategy Handbook, CIRIS’s free data standards handbook written by the 2025–2026 Fellows Lab cohort
- Center for Institutional Research in Independent Schools (CIRIS), Eric Heilman’s organization, an initiative of Maret School
- VC Studio from Veracross, the AI-powered reporting tool discussed as an example of operational AI
- NAIS Data and Analysis for School Leadership (DASL), NAIS’s benchmarking platform, now part of Data Vista, which the handbook’s definitions align with
- Data-Informed Decision Making: A Guide to Institutional Research in Independent Schools, the earlier CIRIS Fellows Lab publication
- Google Data Studio, the free dashboarding tool covered in a CIRIS professional learning group (called Looker Studio from 2022 to April 2026)
- Demystifying Data: Institutional Research at Independent Schools, Eric Heilman’s first Talking Technology with ATLIS episode
Transcript
Peter Frank:
Welcome to Talk Technology with Atlas, the show
Peter Frank:
that plugs you into the important topics and trends for
Peter Frank:
technology leaders, all through a unique independent school
Peter Frank:
lens. We'll hear stories from technology directors and other
Peter Frank:
special guests from the independent school community,
Peter Frank:
and provide you with focused learning and deep dive topics.
Peter Frank:
Bill Stites, TLIS: Along and welcome to Talking Technology
Peter Frank:
with Atlas. I am Bill Stites, the director of technology at
Peter Frank:
Montclair Kimberly Academy in Montclair, New Jersey,
Hiram Cuevas:
and I'm Hiram Cuevas, the director of
Hiram Cuevas:
Information Systems and Academic Technology at St. Christopher
Hiram Cuevas:
School in Richmond, Virginia.
Hiram Cuevas:
Bill Stites, TLIS: So, Hiram, I said it just as we were coming
Hiram Cuevas:
into this. I am shocked and in awe that they're leaving this
Hiram Cuevas:
episode up to just you and I. We have no one looking out for us.
Hiram Cuevas:
Well, Peter's in the background, but we're just going to ignore
Hiram Cuevas:
him for most of the episode.
Hiram Cuevas:
Indeed, I am amazed as well. We shall see if
Hiram Cuevas:
Eric can keep us under control.
Hiram Cuevas:
Bill Stites, TLIS: That's true. Unbeknownst to our guest, as you
Hiram Cuevas:
alluded to, Eric, when we bring him on, what he doesn't realize
Hiram Cuevas:
is he thought we were coming on to talk about the latest work
Hiram Cuevas:
that he's doing at Cirus. But unbeknownst to him, this is just
Hiram Cuevas:
going to be an all zombie podcast. We're just going to
Hiram Cuevas:
talk zombies the entire time and see where we go with our
Hiram Cuevas:
conversations. We're going to discuss the genre on a whole,
Hiram Cuevas:
its impact on modern day society, how it might translate.
Hiram Cuevas:
I'm just making that all up, but that would be our perfect
Hiram Cuevas:
episode. I know that
Hiram Cuevas:
it would indeed. Now the question is, how many
Hiram Cuevas:
Diet Cokes have you had, mr. Stites?
Hiram Cuevas:
Bill Stites, TLIS: Well, you know what? I'm off the diet
Hiram Cuevas:
cokes. I'm full fledged into like black coffee, but I am off
Hiram Cuevas:
the diet cokes, much to the again the shock of our guest.
Hiram Cuevas:
We'll get to all that in just a moment as well. But Hiram, I
Hiram Cuevas:
want to know first and foremost how is your September going?
Hiram Cuevas:
Because for me, it is going by in the blink of an eye, like it
Hiram Cuevas:
always does at the beginning of the school year.
Hiram Cuevas:
Yeah, it has definitely gone by very quickly,
Hiram Cuevas:
and I'm also hoping it goes by even faster because I feel like
Hiram Cuevas:
an ER doctor right now, where it's just nothing but triage and
Hiram Cuevas:
triage and triage and trying to get to the next thing that is on
Hiram Cuevas:
the list. The list keeps on growing; doesn't actually
Hiram Cuevas:
shorten at any point. People are understanding when you say,
Hiram Cuevas:
"Hey, the queue's full, but at the same time, they're like,
Hiram Cuevas:
"Oh, this will only take five minutes. Yeah, nothing ever
Hiram Cuevas:
takes five minutes.
Hiram Cuevas:
Bill Stites, TLIS: It does not. But bring on Halloween season!
Hiram Cuevas:
Oh, indeed. My son sent a group text message just announcing
Hiram Cuevas:
that today is the first day of fall. He's very excited by that.
Hiram Cuevas:
But to kick off the first day of fall, I want to welcome back
Hiram Cuevas:
two-time guest Eric Heilman from Cira. So, Eric, please take a
Hiram Cuevas:
moment and introduce yourself to those that may have met your
Hiram Cuevas:
debut episode. But please introduce yourself, give them a
Hiram Cuevas:
little bit of a backstory on who you are and what you do, and
Hiram Cuevas:
then we can dive into the fun. Thank you guys for having me. My
Hiram Cuevas:
name is Eric Heilman. I am the executive director for the
Hiram Cuevas:
Center for Institutional Research and Independent
Hiram Cuevas:
Schools, also known as CIRIS by the its catchy acronym. We focus
Hiram Cuevas:
on helping schools figure out how to better leverage the data
Hiram Cuevas:
they're collecting to make data-informed leadership
Hiram Cuevas:
decisions and to improve operations to deliver their
Hiram Cuevas:
mission, so we provide professional development
Hiram Cuevas:
webinars.
Eric Heilman:
We'll be talking a lot about what Cirus offers as
Eric Heilman:
the episode goes on. We have had a great relationship with Atlas
Eric Heilman:
over the years. We host a track of sessions at the Atlas annual
Eric Heilman:
conference so that folks who work in tech offices can come
Eric Heilman:
and hear about all the amazing resources Atlas offers, and then
Eric Heilman:
hear about data analysis and institutional research from us.
Eric Heilman:
Always love seeing you guys and hanging out with you. I'm also,
Eric Heilman:
as you've alluded to, a Diet Coke fan. I'm thrilled to hear
Eric Heilman:
that you're off it. It means there's more for me. Also, we
Eric Heilman:
can just toss everything we were thinking about talking about if
Eric Heilman:
we want to talk about zombies. If you stay ready, you don't
Eric Heilman:
have to get ready, and I am ready on that topic. Let's go.
Eric Heilman:
That's awesome.
Hiram Cuevas:
Now, the quick question is: How many Diet Cokes
Hiram Cuevas:
have you already had today?
Eric Heilman:
Okay, well, so I guess in most people's homes
Eric Heilman:
they probably count Diet Coke by the can. I've gotten to the
Eric Heilman:
point where I've abandoned cans, and I'm strictly just going two
Eric Heilman:
liters or the giant jugs from the convenience store. So I'm
Eric Heilman:
not even sure what unit of measurement to answer that
Eric Heilman:
question, I will say a couple gallons so far today. But it is
Eric Heilman:
just past noon, so you know I've got a strong afternoon to go
Eric Heilman:
into. So yeah,
Eric Heilman:
Bill Stites, TLIS: I will send you. When I was doom scrolling
Eric Heilman:
last night, I came across a guy who talks a lot about health and
Eric Heilman:
wellness, and he was debunking all the rumors that. Drinking
Eric Heilman:
too much diet coke is actually bad for you compared to like
Eric Heilman:
drinking too much water, and it was hilarious. And I was
Eric Heilman:
thinking of you, so I'll send that along.
Eric Heilman:
I have a theory that I rarely get sick because
Eric Heilman:
there are so many artificial ingredients in my bloodstream
Eric Heilman:
that when a bacteria virus gets in there, they just can't
Eric Heilman:
reproduce. They're like, "What is fire? I'm basically
Eric Heilman:
pre-embalmed. It's fine. That's a good
Eric Heilman:
point.
Eric Heilman:
You know, would be interesting. I wonder how zombies would react
Eric Heilman:
to it.
Eric Heilman:
Bill Stites, TLIS: Ooh, it's a good point. If you've ever cut a
Eric Heilman:
zombie, it kind of is like the color of what comes out.
Eric Heilman:
Exactly. Maybe zombies just reach the ultimate evolution of
Eric Heilman:
the Diet Coke drinker. It's probably not a marketing angle
Eric Heilman:
they want to take, but you know, we'll just leave it there. But
Eric Heilman:
Eric, the three of us at the beginning of the summer, we got
Eric Heilman:
to spend some time together. It was very nice. We did, and it
Eric Heilman:
wasn't even under the supervision of a state
Eric Heilman:
government. Exactly, exactly. But give people a little bit of
Eric Heilman:
background on what brought
Eric Heilman:
the three of us together and got us so much more
Eric Heilman:
quality time. Yes, you know, about two years ago, I was
Eric Heilman:
trying to think of outlandish schemes to spend more time with
Eric Heilman:
Bill and Hiram, and what I landed on was this. So CIRIS has
Eric Heilman:
been working on basically a data style guide or a data handbook
Eric Heilman:
for independent schools. Every few years, CIRIS puts out a
Eric Heilman:
resource that we think would be useful for the entire
Eric Heilman:
independent school community, and one of the things we noticed
Eric Heilman:
over the years was that while schools are very generally able
Eric Heilman:
to use their data to complete tasks, you know, operational
Eric Heilman:
tasks over the course of the year, like producing report
Eric Heilman:
cards, making admissions decisions, etc. when it comes to
Eric Heilman:
actually doing data analysis, the condition and quality of the
Eric Heilman:
data can be really problematic and difficult to work with. We
Eric Heilman:
all know a lot of school data is siloed across different systems,
Eric Heilman:
and a lot of the interesting and valuable strategic questions
Eric Heilman:
schools have rely on pooling data from different systems and
Eric Heilman:
over time to do some analysis, and that was really difficult
Eric Heilman:
for a lot of schools, primarily because of the condition of the
Eric Heilman:
data itself. So we felt there was a need to help schools
Eric Heilman:
figure out what data should we be collecting, what format
Eric Heilman:
should we be collecting it in, in order to not only to do the
Eric Heilman:
tasks that we need to do day to day, but to really build up a
Eric Heilman:
stock of high value, high quality data that we can use
Eric Heilman:
either for data analysis, and then I'm sure we'll get to this.
Eric Heilman:
But also, if schools are interested in adopting
Eric Heilman:
operational AI tools, what does your underlying data need to
Eric Heilman:
look like? So, CIRIS convened a cohort of thought leaders and
Eric Heilman:
folks that are really experienced across the
Eric Heilman:
independent school community. There was actually 13 of us, me
Eric Heilman:
plus 12 folks who worked together from schools all over
Eric Heilman:
the country to develop what ultimately became Clean Data,
Eric Heilman:
Clear Vision, an independent school data strategy handbook.
Eric Heilman:
And so the cohort broke up into working groups for the first 15
Eric Heilman:
months, and then we all got together on campus in DC at
Eric Heilman:
Murray back in June to sort of polish a draft and finalize some
Eric Heilman:
other resources that are available alongside of the
Eric Heilman:
handbook. And so we got to get together and drink Diet Coke
Eric Heilman:
together, have some snacks. You know, really drill down into
Eric Heilman:
what schools need and what, in our view, is sort of the goal in
Eric Heilman:
terms of data governance and talking through implementation
Eric Heilman:
and that sort of stuff for schools.
Eric Heilman:
Bill Stites, TLIS: So bringing us all together, like I said, it
Eric Heilman:
was a great couple of days. But you mentioned it took 15 months,
Eric Heilman:
having been involved with the first publication that we put
Eric Heilman:
out, it was a much longer time frame than we had originally,
Eric Heilman:
and again over that time we had still culminated with us all
Eric Heilman:
coming together and doing similar work. But who were the
Eric Heilman:
people in the room, and why those people you chose to bring
Eric Heilman:
together?
Eric Heilman:
Yeah, so I knew that school data lives in
Eric Heilman:
different systems because schools have different
Eric Heilman:
operational areas. We've got admissions, we've got student
Eric Heilman:
life and academics, we've got the employee data, we've got
Eric Heilman:
advancement data, and to really be able to make recommendations
Eric Heilman:
for how data should be governed in each of those areas, we
Eric Heilman:
needed folks who were experts in each of those areas. And so,
Eric Heilman:
when we were putting together the cohort, we were really
Eric Heilman:
looking for folks that were subject matter experts, either
Eric Heilman:
in an operational office or in the technical side, because we
Eric Heilman:
wanted the recommendations in the handbook to be implementable
Eric Heilman:
and work in a lot of the large platforms. So, our two hosts
Eric Heilman:
here were part of the cohort. We had some folks from the
Eric Heilman:
Episcopal Academy in Pennsylvania, Riverdale. So we
Eric Heilman:
had folks who were advancement background. We had folks that
Eric Heilman:
were admissions and enrollment management background. We had
Eric Heilman:
folks whose background is in student life data and academics,
Eric Heilman:
and we had some folks who had experience. In the business
Eric Heilman:
office and the HR side of things, and so each of those
Eric Heilman:
groups worked on a chapter of the handbook, and so there are
Eric Heilman:
areas where they had deep knowledge about how offices
Eric Heilman:
actually work with this data and what it needs to be able to do,
Eric Heilman:
and had really good insights about what are the current pain
Eric Heilman:
points and what are some suggestions we could make to
Eric Heilman:
help folks solve those.
Hiram Cuevas:
What was most fascinating for me about that
Hiram Cuevas:
experience was your collection of individuals was so
Hiram Cuevas:
intentional because of the various domains within a school,
Hiram Cuevas:
and it was fascinating to hear. While there were some
Hiram Cuevas:
differences in the types of information we were after, the
Hiram Cuevas:
ultimate goal was actually synergy, so that we actually
Hiram Cuevas:
have everything talking to each other, and that we were able to
Hiram Cuevas:
make use of that information. So we weren't dealing with
Hiram Cuevas:
task-based data. We were actually looking at a further
Hiram Cuevas:
analysis well beyond what I think you articulated so well
Hiram Cuevas:
during the webinar. The task data is what ends up being
Hiram Cuevas:
siloed, and I think that's a really key feature that a lot of
Hiram Cuevas:
schools miss out on. They think they're doing a good job with
Hiram Cuevas:
data, but they're really just scratching the surface.
Eric Heilman:
Yeah, we spent a lot of time talking about that.
Eric Heilman:
You know, through my work with CIRIS and back when I was
Eric Heilman:
working with Mission edat, and we were hired as consultants to
Eric Heilman:
work with schools, I've seen a lot of different school data,
Eric Heilman:
and the one common thing that I've noticed across schools is
Eric Heilman:
that they almost always overestimate the quality of the
Eric Heilman:
data in their system, and because they are able to
Eric Heilman:
complete tasks, and so because they're able to complete tasks,
Eric Heilman:
they assume that their data is in good shape, and that's often
Eric Heilman:
not the case. You know, when you look at even some basic data
Eric Heilman:
around how many employees do you have, you know, this is
Eric Heilman:
something we talked about in the cohort. If you pull a list of
Eric Heilman:
employees from your HR system and you pull a list of your
Eric Heilman:
employees from your student information system, it is not a
Eric Heilman:
guarantee that you're going to have the same number of names on
Eric Heilman:
both lists, or that that they're going to be the same names, you
Eric Heilman:
know. Then you're getting into even basic demographic
Eric Heilman:
identifiers. A lot of schools are missing basic race and
Eric Heilman:
ethnicity identities for a huge chunk of their student body,
Eric Heilman:
which makes any kind of analysis really difficult. And so they're
Eric Heilman:
always surprised when they learn that that's the case. But it is
Eric Heilman:
actually true in many, many, many schools,
Hiram Cuevas:
and that's what makes the that data handbook
Hiram Cuevas:
that you created in the prior iteration that much more
Hiram Cuevas:
important because it really creates that landscape for
Hiram Cuevas:
schools to learn what exactly it is that you need to do in order
Hiram Cuevas:
to create that standardization that is needed in order for
Hiram Cuevas:
proper data governance, and I think independent schools have
Hiram Cuevas:
this nasty habit of being so independent that they think they
Hiram Cuevas:
know how to do everything better.
Eric Heilman:
I think that can be true. I also think that we
Eric Heilman:
just have a lot on our plates, and so you're going to do what
Eric Heilman:
works for you, and that you know how to do. And you know, people
Eric Heilman:
in their individual schools are all coming to different
Eric Heilman:
conclusions about like what is most expedient. A lot of times,
Eric Heilman:
folks are learning how to manage their database system from their
Eric Heilman:
predecessor, and so they're learning what their predecessor
Eric Heilman:
has done, which may or may not line up with what the platform
Eric Heilman:
is actually capable of doing. So, some of the difficulties in
Eric Heilman:
data governance are just awareness of what's even
Eric Heilman:
possible in your database platform, much less
Eric Heilman:
understanding. Okay, there is actually a purpose and a value
Eric Heilman:
to the data we're collecting beyond the task that we are
Eric Heilman:
completing right now, and I think that's one of the big
Eric Heilman:
takeaways we emphasize in the handbook is that the data that a
Eric Heilman:
school is compiling in the process of doing tasks is a
Eric Heilman:
super valuable institutional asset. You know, same as your
Eric Heilman:
physical plant or your tech stack, and there needs to be the
Eric Heilman:
same level of attention to the quality and maintenance of that
Eric Heilman:
institutional asset and your databases as there is to your
Eric Heilman:
buildings, your grounds, your furniture. You know, all the
Eric Heilman:
other assets that a school has, there needs to be an eye on the
Eric Heilman:
actual maintenance and quality of the data that the school is
Eric Heilman:
accumulating. For sure,
Hiram Cuevas:
I love the fact that you brought it up as an
Hiram Cuevas:
institutional asset that it can be quantified and it has
Hiram Cuevas:
inherent value. So often, I think schools miss out on that
Hiram Cuevas:
because they're so concerned with facilities, what it looks
Hiram Cuevas:
like. Whereas data, unless you're a data person such as
Hiram Cuevas:
yourself, to most people it's like, "Oh, I'm buying tires. No,
Hiram Cuevas:
you're buying more than tires. You need tires to run a safe,
Hiram Cuevas:
have a safe journey. You need tires to get to your next
Hiram Cuevas:
destination, and really, that's what your data is doing. It's
Hiram Cuevas:
moving the needle so that your schools can fulfill their
Hiram Cuevas:
missions.
Eric Heilman:
Yeah, and I think that this is a great place to
Eric Heilman:
talk about sort of how the impetus and the goal of the
Eric Heilman:
handbook evolved during the time that we were working together.
Eric Heilman:
When we initially convened the cohort, we were really trying to
Eric Heilman:
address. The issues around doing manual data analysis. So that's
Eric Heilman:
something that in the IR community we've run into over
Eric Heilman:
and over again is that you know a school will be really excited
Eric Heilman:
about what data analysis can help them understand about the
Eric Heilman:
school. So they might get excited about a project like
Eric Heilman:
okay, we want to see if the data we collect in our admissions
Eric Heilman:
process is actually correlated with how students do when
Eric Heilman:
they're actually at the school. That is a very reasonable and
Eric Heilman:
understandable project to do, and it was actually a pretty hot
Eric Heilman:
topic back when schools were debating whether or not to be
Eric Heilman:
test optional. One of the questions they had was, you
Eric Heilman:
know, are the scores we see on these admissions tests actually
Eric Heilman:
correlated with how students end up doing our school? Sadly, that
Eric Heilman:
can be a very thorny data set to put together because the
Eric Heilman:
admissions data is over in this admission system. The transcript
Eric Heilman:
info for grads is over in the SIS. You need a longitudinal
Eric Heilman:
data set, so you have to have many years of data. If you're in
Eric Heilman:
a K through 12, if you admit a student in lower school, by the
Eric Heilman:
time they have graduated, 1011, years later, you may or may not
Eric Heilman:
even have their admissions data anymore, and so it was a big
Eric Heilman:
obstacle to schools to get to these answers. The big obstacle
Eric Heilman:
was getting the data set together, and so we were trying
Eric Heilman:
to address that issue. Then, over the 18 months that we were
Eric Heilman:
working on this thing, there were this explosion of AI tools
Eric Heilman:
coming out, and a lot of the conversation was around
Eric Heilman:
student-facing AI, which absolutely 100% very important
Eric Heilman:
topic. But what was also starting to come out were these
Eric Heilman:
operational AI tools from some of the vendors that we see in
Eric Heilman:
the school space. Veracross has just released this year
Eric Heilman:
Veracross Studio, which essentially will build
Eric Heilman:
dashboards automatically. It's an AI agent that's inside of
Eric Heilman:
your platform already, and you can interact with it the same
Eric Heilman:
way you would with Chat GPT or something like that. And it
Eric Heilman:
produces analysis reports, and it's a cool product. I've seen a
Eric Heilman:
few demos. It's awesome, but the question I had for them was: the
Eric Heilman:
demo data said that you're running this product on. Talk to
Eric Heilman:
me about what that data actually looks like. And of course, you
Eric Heilman:
know that data is pristine. They've gone to great lengths to
Eric Heilman:
make sure that it's complete, that it's accurate and
Eric Heilman:
beautiful. And from what I've seen not very many schools have
Eric Heilman:
underlying data that is in the same condition as what they're
Eric Heilman:
saying. So, I think there's a lot of promise in terms of what
Eric Heilman:
operational AI tools will be able to help schools accomplish.
Eric Heilman:
Whether it's helping to support more differentiated instruction
Eric Heilman:
for students or helping support professional development and
Eric Heilman:
training for employees. Whether it's using AI to help vet
Eric Heilman:
student applicants coming in, there's just a lot of places
Eric Heilman:
where schools are going to have a strong incentive to explore
Eric Heilman:
these operational tools. And at
Eric Heilman:
Bill Stites, TLIS: CIRIS, we want to make sure that when
Eric Heilman:
schools see these demos, that if they do invest in those
Eric Heilman:
products, they're actually going to get the bang for the buck. So
Eric Heilman:
we're trying to give folks a scaffolding for what their data
Eric Heilman:
needs to look like and what they actually need to have in their
Eric Heilman:
systems in order to be able to leverage those emerging tools.
Eric Heilman:
Last week, I was actually just up at Veracross. They ran an
Eric Heilman:
event there for their advancement offices, and it was
Eric Heilman:
great because they brought together admissions, they
Eric Heilman:
brought together development, and they brought together
Eric Heilman:
communications, and they were showing off the VC studio piece.
Eric Heilman:
And here at MKA, we were involved early on in the beta
Eric Heilman:
program with it, saw it in its infancy how it developed. Was
Eric Heilman:
very impressed with the way in which it was actually
Eric Heilman:
developing, but one of the questions I asked during the
Eric Heilman:
demo was, "What if your data is bad? As you're saying, I think
Eric Heilman:
we all overestimate the quality of the data, and I pushed on
Eric Heilman:
them a little bit, and I said, "Listen, I don't want this thing
Eric Heilman:
hallucinating, and because you're dealing with a closed
Eric Heilman:
data set, I would rather have the system tell me that it
Eric Heilman:
doesn't have what it needs in order to generate the response
Eric Heilman:
that we're asking for, and I think that was something that
Eric Heilman:
they acknowledged that they owned right up front. Jeremy
Eric Heilman:
Anderson was doing the demo, and the one thing I love about
Eric Heilman:
Jeremy pulls no punches. He will let you know if it's working or
Eric Heilman:
if it's not. And he was great with that whole thing, and it
Eric Heilman:
was like, yeah, you know, he's run some of those things. You
Eric Heilman:
know, if you've ever sat through a Veracross demo, we're all
Eric Heilman:
familiar with the Sarah Green and the Green family. We were
Eric Heilman:
talking about all of those pieces, and really what that
Eric Heilman:
meant to me. You alluded to it earlier, and it was one of the
Eric Heilman:
things after we broke, after we did all the work this summer, is
Eric Heilman:
I came back and I decided, you know what, I'm going to start
Eric Heilman:
small. I'm going to look at this, and I'm going to start
Eric Heilman:
small, and I just want to get a hold of like who works here, and
Eric Heilman:
I wasn't even talking necessarily about what I'll call
Eric Heilman:
our full-time faculty and staff because we've got a pretty good
Eric Heilman:
handle on that. What I was focused on is who. Our coaches,
Eric Heilman:
who are our substitutes, who are those people that we have listed
Eric Heilman:
in our extended plan enrichment programs, like our after school
Eric Heilman:
things? Who's on our maintenance and security staff? Because you
Eric Heilman:
know we've got security guys coming in and out all the time.
Eric Heilman:
Because it tends to be those numbers, not to mention like you
Eric Heilman:
know Hiram's got his whole DC universe, I think, in his
Eric Heilman:
Blackbaud suite, as far as like parents and students that exist
Eric Heilman:
that are there. But what are those like service level
Eric Heilman:
accounts that we have that inflate slightly, but that
Eric Heilman:
inflate our different pieces? And let's get all of that
Eric Heilman:
together. Let's figure out all of those things so that if we're
Eric Heilman:
reporting out the dazzle, if we're reporting out the index,
Eric Heilman:
if we're reporting to anyone, if we're reporting to a vendor who
Eric Heilman:
needs to know how many people work at the school so they can
Eric Heilman:
price us correctly, how do we get that all in hand? And I
Eric Heilman:
think this also speaks to the governance piece because I was
Eric Heilman:
like, you know what, I get to go through and do this, but it's
Eric Heilman:
not my job. Not it. So what I did, and I did this for people
Eric Heilman:
who don't have deep database access in the Veracross. I ran a
Eric Heilman:
report. I exported. I threw in a spreadsheet, and I gave it to
Eric Heilman:
them. And I said, "Go through this. Fix titles. Fix people who
Eric Heilman:
are here. Let me know who's not here. Substitutes was like the
Eric Heilman:
biggest blow. It was absolutely ridiculous. But going through
Eric Heilman:
all of that and fixing all of that, and then I'll work on it
Eric Heilman:
on the back end to clean up some of like the actual work. But I
Eric Heilman:
need you to identify the people, and then I'm going to import it
Eric Heilman:
in. And I think to me, having participated in both writings of
Eric Heilman:
this, we're not perfect.
Eric Heilman:
Just because we participated in this does not mean we're doing
Eric Heilman:
it perfectly or doing it right. We have a lot of ideas, but it's
Eric Heilman:
building that internal team. It's building that governance
Eric Heilman:
structure. It's building that buy-in. That I think is that
Eric Heilman:
ongoing work, particularly when you have turnover, particularly
Eric Heilman:
when you have people coming in and out of different offices.
Eric Heilman:
And how do you build that? So the question is: Is the work
Eric Heilman:
that was done this past summer in defining those scenarios? How
Eric Heilman:
can schools use the handbook now to help build strong governance
Eric Heilman:
culture and buy-in around the ownership of this work? Yeah,
Eric Heilman:
that's a great question. I think in your ramble there, you
Eric Heilman:
touched on a few things that are important.
Eric Heilman:
Wait, Shio, somebody mark the day and time.
Eric Heilman:
Bill touched on several important things in a ramble. So
Eric Heilman:
you know, maybe you should be leaning into black coffee. One
Eric Heilman:
of the things that caught my ear in what you're just saying when
Eric Heilman:
you said, "Well, it's not my job, and I think that that is a
Eric Heilman:
really important thing to recognize is that in most
Eric Heilman:
schools there is nobody whose explicit job it is to monitor
Eric Heilman:
the quality of the data that's in there, how up to date it is,
Eric Heilman:
how accurate it is. In most tech offices, the concern is: is the
Eric Heilman:
database working? Can people produce report cards? Can we
Eric Heilman:
take attendance? Is the LMS up and running? That doesn't
Eric Heilman:
necessarily extend to have we deleted subs that are out of the
Eric Heilman:
pool, because I don't know. It's not that I don't want to do it.
Eric Heilman:
I have no idea what subs are current. I have no idea what
Eric Heilman:
coaches are current. Right. There's no way I can own it
Eric Heilman:
because I don't know the data. Exactly. So I think really
Eric Heilman:
nailing down whose job is it is really important in schools when
Eric Heilman:
it comes to figuring out who really owns the quality of
Eric Heilman:
what's in the system, so the second thing you touch on is
Eric Heilman:
like how do schools actually use this handbook and how do we
Eric Heilman:
implement or operationalize what's in there? And one of the
Eric Heilman:
things you mentioned that really resonated with me was the
Eric Heilman:
starting small. Schools are all over the map in terms of how
Eric Heilman:
closely they are developing the quality of their data. Some
Eric Heilman:
schools have been paying attention to this, and they're
Eric Heilman:
pretty far along. Some schools that may be hearing this
Eric Heilman:
podcast, this may be the first time they thought about. Oh, I
Eric Heilman:
wonder if our list of employees in there is accurate. And
Eric Heilman:
there's something in the handbook for everybody, no
Eric Heilman:
matter what stage you're at. So in the handbook, each chapter
Eric Heilman:
starts with a list of strategic questions that a lot of schools
Eric Heilman:
ask in a certain operational area, and then it lists what are
Eric Heilman:
the data building blocks that you need to be collecting in
Eric Heilman:
order to answer those questions. And in each chapter, it starts
Eric Heilman:
with the very basic building blocks. And so some schools are
Eric Heilman:
going to open that chapter, and be like, we are nailing the
Eric Heilman:
building blocks one and two. We know we are capturing this data;
Eric Heilman:
we're capturing it well. For them, they may be focused on
Eric Heilman:
fine tuning the building blocks four and five and six in that
Eric Heilman:
chapter. There may be some schools that open it up, and
Eric Heilman:
they're like, "I'm actually not sure that we're doing a great
Eric Heilman:
job of building block one. So there are a couple options for
Eric Heilman:
how schools can implement and operationalize the handbook. The
Eric Heilman:
first one is you can just take the handbook and run reports and
Eric Heilman:
try to self-diagnose and do assessments on your own about
Eric Heilman:
does our data where does it line up with and meet the standards
Eric Heilman:
that are in this handbook. Where are the areas where we're
Eric Heilman:
furthest away and try to start tackling or. In those areas, and
Eric Heilman:
you may find out that there are some very basic building blocks
Eric Heilman:
that we need to start with. Other schools will get through
Eric Heilman:
those and find that oh, we just need to do some tweaks around
Eric Heilman:
some finer tuning. CIRIS also for schools that are CIRIS
Eric Heilman:
members, we are also offering a lot of support programming this
Eric Heilman:
year for folks. So we have also developed a data quality
Eric Heilman:
assessment workbook. So, in addition to the handbook, CIRIS
Eric Heilman:
member schools have access to basically a bunch of exercises
Eric Heilman:
that are structured that you can do to get a sense of where your
Eric Heilman:
data quality is compared to the standard laid out in the
Eric Heilman:
handbook. We're also going to be doing webinars and programming
Eric Heilman:
with schools to support them in doing those data assessments.
Eric Heilman:
And then after that, we are also running support for schools to
Eric Heilman:
do data remediation. And some of those we'll be working on with
Eric Heilman:
vendor partners, so that we can help schools in a
Eric Heilman:
platform-specific way to figure out. Okay, now that we know that
Eric Heilman:
we have a critical need to improve data in area X, here's
Eric Heilman:
how we actually can do that in our system.
Eric Heilman:
Because again, you know, as I mentioned earlier, sometimes one
Eric Heilman:
of the biggest hurdles is just knowing what your platform can
Eric Heilman:
do, and like, okay, how do we actually batch update the data
Eric Heilman:
we have into a better format? Is there a better field that we
Eric Heilman:
could be storing it in, et cetera, et cetera, et cetera? So
Eric Heilman:
there's a lot of initial baby steps you can do to assess the
Eric Heilman:
quality of data you have, and we're also supporting folks in
Eric Heilman:
remediating critical areas of need that they may discover.
Hiram Cuevas:
Eric, I've been so grateful for this because as I
Hiram Cuevas:
started to peel back this onion for our own community here at
Hiram Cuevas:
St. Christopher's, that's when you go. Oh, okay. Now I found
Hiram Cuevas:
this. Now I found this, and it does get a little overwhelming.
Hiram Cuevas:
And to Bill's point, you do have to start small. Just a quick
anecdote here:
we were looking at our employees' education, and
anecdote here:
we had B A, we had b.a.we. had Bachelor of Arts, and it was
anecdote here:
just when you started to create the visualization, like what's
anecdote here:
going on here, and then you're like, okay, you've got to take a
anecdote here:
step back, and it leads you down the road of how am I collecting
anecdote here:
this information? It totally changes how you view data
anecdote here:
collection as part of your data governance, and it's been an
anecdote here:
absolute gem of an exercise for me to go through after going
anecdote here:
through this experience with CIRIS.
Eric Heilman:
Yeah, and that's such a great point, and this is
Eric Heilman:
a great example of task-oriented data versus insight-oriented
Eric Heilman:
data. It's such a good example. So everybody who is hiring and
Eric Heilman:
who is collecting that data understands all those variations
Eric Heilman:
of BA that you just mentioned, all mean bachelor's degree.
Eric Heilman:
However, if you wanted to ask a strategic question like are
Eric Heilman:
employee levels of education actually correlated with their
Eric Heilman:
performance when they get here, in order to answer that
Eric Heilman:
question, which is a very good question, you're going to need
Eric Heilman:
to be able to compile a data set where everybody who's at the
Eric Heilman:
same level of education is labeled the exact same way. Data
Eric Heilman:
analysis packages they only understand strings of text,
Eric Heilman:
right? So, b a b dot a dot are two different things to a data
Eric Heilman:
analysis package. So while the schools are able to execute the
Eric Heilman:
task of hiring people and getting that data, if they
Eric Heilman:
actually want to go back and use that data set for analysis, it's
Eric Heilman:
just really painful to deal with. And so, to your point, one
Eric Heilman:
of the things we talk about in the handbook is the importance
Eric Heilman:
of how you collect it initially. The old saying "garbage in,
Eric Heilman:
garbage out. If you make some small adjustments in terms of
Eric Heilman:
how you're collecting, you're going to get much higher quality
Eric Heilman:
in the funnel at the top, and not have to go back and fix it
Eric Heilman:
when you actually want to answer some strategic questions. One of
Eric Heilman:
the things I am happiest about with the handbook is that when
Eric Heilman:
we were writing it, we not only talked about this stuff, but we
Eric Heilman:
also created templates, and they're linked in there. So you
Eric Heilman:
know we have, and if you're not sure how to collect this, here's
Eric Heilman:
a template that you can link to and make a copy of. It already
Eric Heilman:
has the categories you're going to need in there. They're all
Eric Heilman:
named in a standard way, and if you have people input data
Eric Heilman:
through this template, your data will be clean moving forward. So
Eric Heilman:
you know we were really focused on making not only some
Eric Heilman:
theoretical standard suggestions for data standards, but also
Eric Heilman:
like we want to help people actually do this on a real
Eric Heilman:
practical level. And so, I think that's such a great point that
Eric Heilman:
you made about the easiest way to make sure your data is high
Eric Heilman:
quality is to collect it in that way. Wasn't garbage in, garbage
Eric Heilman:
out one of the titles we bounced around for this. It was either
Eric Heilman:
that, or maybe it was a nickname we were talking about for you,
Eric Heilman:
Bill.
Eric Heilman:
Bill Stites, TLIS: But you know, thinking about that, one of the
Eric Heilman:
things is
Eric Heilman:
like we can come up with what are our internal standards going
Eric Heilman:
to be for putting those things in. Years ago, I wrote a guide.
Eric Heilman:
I've got to update it because our systems have changed, and
Eric Heilman:
we've added different systems. And I need to update what we've
Eric Heilman:
got so that it's in alignment with one of the systems that we
Eric Heilman:
have. We had people from NAIS pop in. We were talking about
Eric Heilman:
Dazzle when we were going on with this. We also, for those of
Eric Heilman:
you that know the podcast, you know Hiram was talking about
Eric Heilman:
things from a Blackbaud perspective, I was talking about
Eric Heilman:
things from a Veracross perspective. We were bringing up
Eric Heilman:
all these different perspectives outside of what was going on in
Eric Heilman:
the school itself, and I think that's the question that I have
Eric Heilman:
for you. What do you hope vendors take from the work that
Eric Heilman:
we've done? Because that, to me, is where the biggest struggle is
Eric Heilman:
right now. Is that we may get it set over like the nightly
Eric Heilman:
scripts may run in Veracross and do a bunch of data cleanup. But
Eric Heilman:
if I've got to get that data into Razors Edge, if I got to
Eric Heilman:
get that data over into Magnus, if I got to get that data into
Eric Heilman:
you name the tool, well, if it's not set up to take it, then what
Eric Heilman:
good is the cleanup? So, what do you hope to see vendors pull out
Eric Heilman:
of this, yeah, that is such a great point. As you mentioned,
Eric Heilman:
we did collaborate with NAIS on this project, and you know,
Eric Heilman:
there's a lot of data that schools collect that is
Eric Heilman:
reportable to Dazzle or now Data Vista, and so we did make sure
Eric Heilman:
that when we were talking about pieces of data that are
Eric Heilman:
reportable for benchmarking and to external organizations that
Eric Heilman:
we checked with them to make sure that what we were
Eric Heilman:
suggesting was in alignment with their definitions, and so part
Eric Heilman:
of what you'll
Eric Heilman:
see in the handbook is you know as we're
Eric Heilman:
going through the data building blocks and recommending formats
Eric Heilman:
and that sort of stuff, we definitely had in mind sort of
Eric Heilman:
the normal external reporting that we want to do. I think
Eric Heilman:
everybody has at times wondered when we look at benchmarks for
Eric Heilman:
peer schools that are being produced by NIS or any other
Eric Heilman:
external organization. Are we sure that everybody's using the
Eric Heilman:
same definition when they're reporting their data? And what
Eric Heilman:
does that mean for the quality of the benchmarks we're seeing?
Eric Heilman:
But in the handbook, we were super clear about this is what
Eric Heilman:
the definition is for Dazzler NAIS or Data Vista. This is what
Eric Heilman:
we're recommending. These match. These are in alignment. In
Eric Heilman:
places where that wasn't possible, we did make note of
Eric Heilman:
that. But there's also just a ton of data that schools collect
Eric Heilman:
that's not reportable to external agencies, and that I
Eric Heilman:
think has been at the root of the difficulty vendors have
Eric Heilman:
addressing school needs. Right, so there's all this data that
Eric Heilman:
we're collecting that does not really have an external standard
Eric Heilman:
definition. It's just however schools are working with it
Eric Heilman:
internally, and so when we try to make the case to a vendor
Eric Heilman:
that oh we want X, well you want X, but a bunch of other schools
Eric Heilman:
want the opposite of X, and there was no sort of unified
Eric Heilman:
standard for vendors to look to to help guide product
Eric Heilman:
development, and so my hope is that with this data standard out
Eric Heilman:
here that does list out. This is the data schools need in order
Eric Heilman:
to glean strategic insight and that sort of stuff. That vendors
Eric Heilman:
now have something centralized to respond to. I think we did
Eric Heilman:
the best we could as we were developing the handbook to check
Eric Heilman:
to see which of our recommendations were
Eric Heilman:
implementable. But there are certainly things that are super
Eric Heilman:
important to schools that currently can't be housed in any
Eric Heilman:
platform. You know, I'm thinking right now specifically of
Eric Heilman:
constituent feedback via surveys. That's incredibly
Eric Heilman:
important information, but it's very difficult to store survey
Eric Heilman:
data in any of the platforms. So I think that now that there is
Eric Heilman:
sort of a unified explanation of like this is what schools need
Eric Heilman:
in order to assess operational efficiency and mission delivery,
Eric Heilman:
and now that there's a unified voice that NAIS has helped
Eric Heilman:
collaborate on, and that so many schools were part of developing
Eric Heilman:
that they actually have a coherent standard to respond to
Eric Heilman:
as they're moving forward in developing new features. Yeah,
Hiram Cuevas:
I'm so glad you brought up that constituent
Hiram Cuevas:
information and some of the challenges associated with that.
Hiram Cuevas:
Because I know I worked in the HR section, and the one that
Hiram Cuevas:
came up almost every time we had a conversation was the
Hiram Cuevas:
punctuated tenure.
Eric Heilman:
Oh God, yeah.
Hiram Cuevas:
You have somebody who comes to the school, then
Hiram Cuevas:
they leave, then they come back, and there are no fields in
Hiram Cuevas:
either platform that actually support that. And that's such an
Hiram Cuevas:
important field because you have overall tenure,
Eric Heilman:
right?
Hiram Cuevas:
You have overall teaching experience or employee
Hiram Cuevas:
experience, and there is just no way to actually cure that or
Hiram Cuevas:
deal with that particular impasse within either system.
Hiram Cuevas:
It's really a struggle,
Eric Heilman:
right? And again, that goes back to there's a very
Eric Heilman:
interesting question for schools to dig into around how is years
Eric Heilman:
of experience of. Employee correlated with all sorts of
Eric Heilman:
desirable outcomes, retention, actual job performance is
Eric Heilman:
somebody who's been working for 30 years more likely to be a
Eric Heilman:
high performer than somebody who has been at it for 15 years. In
Eric Heilman:
order to answer that question, you need to be able to count.
Eric Heilman:
Are you only counting the time they'd been at your school, are
Eric Heilman:
you counting their entire time in independent schools? Are you
Eric Heilman:
counting professional experience before they came to schools? So
Eric Heilman:
the handbook does talk about that sort of stuff, but you're
Eric Heilman:
exactly right. Half the battle is figuring out what these
Eric Heilman:
definitions should be and figuring out how we can store
Eric Heilman:
it. Another one that came up a bunch in the HR chapter was
Eric Heilman:
differentiating between headcount and total FTEs, and
Eric Heilman:
how important it is to have concrete job descriptions that
Eric Heilman:
lay out responsibilities and FTE percentage. Independent schools
Eric Heilman:
have a strong tradition of people wearing many hats, and
Eric Heilman:
sometimes those hats come with extra responsibilities, but
Eric Heilman:
aren't necessarily accounted for as an FTE percentage. And so,
Eric Heilman:
you know, you've got a bunch of people in independent schools
Eric Heilman:
that, on paper, are 100% FTE, but they're actually doing 150%
Eric Heilman:
tasks. Which, setting aside whether that's a good idea or a
Eric Heilman:
bad idea, if you want to ask something like, what is the
Eric Heilman:
optimal dollar spent per FTE in a school, like you really have
Eric Heilman:
to account for how many FTEs you have, not just your head count.
Eric Heilman:
And so that's a really important financial
Eric Heilman:
Bill Stites, TLIS: metric to keep an eye on. But you got to
Eric Heilman:
be recording it, so there's a lot of that in there as well. I
Eric Heilman:
was on the bio and demo side of things, like trying to think
Eric Heilman:
about the data in just the very basic forms, like not even
Eric Heilman:
getting into what each operational area, like how do we
Eric Heilman:
track addresses? You mentioned the BA versus the bachelor's
Eric Heilman:
versus the B.A. dot. All of those pieces. The biggest thing
Eric Heilman:
there, also, I think, was what standard are we using? You know,
Eric Heilman:
we talked a lot about like what does the census use? Like, what
Eric Heilman:
are we aligning to U.S. postal standards? Are we aligning to
Eric Heilman:
census standards? Are we relying on standards set by those for
Eric Heilman:
which we need to report to? And I think it's interesting when
Eric Heilman:
you understand what each system can support. What are your needs
Eric Heilman:
that you have as an institution? We specifically talked about
Eric Heilman:
areas where you may have a larger demographic population,
Eric Heilman:
where you want to get more granular with regard to where
Eric Heilman:
their origins and their nationalities and ethnicities
Eric Heilman:
are from, and how do you get down onto that? But if one of
Eric Heilman:
your reporting agencies is only asking for in a much broader
Eric Heilman:
sense, how do you filter those things up? And we talked just a
Eric Heilman:
few minutes ago about where are you collecting all the data
Eric Heilman:
from, and how do you make people feel as if they're being seen
Eric Heilman:
and not marginalized? Yet, how do we use that data from
Eric Heilman:
reporting? So, I think there's a lot to be said. You know, when
Eric Heilman:
you think about data management. You think about like the method
Eric Heilman:
of extracting, transforming, and loading data between. It's that
Eric Heilman:
transformational sense. You know, Hiram and I, when we look
Eric Heilman:
at database audits, we like, all right, what's the middleware
Eric Heilman:
piece that's pulling it all together? Like, what are those
Eric Heilman:
things that, all right, if our systems aren't going to align
Eric Heilman:
properly, what are the guidelines that we need to have
Eric Heilman:
so that we have an institutional understanding of what it means
Eric Heilman:
when we say someone is X, but they may be reporting like X Y.
Eric Heilman:
We're still acknowledging X Y, but we have to report based on
Eric Heilman:
X. And what does that look like? And I think that sent me in
Eric Heilman:
spirals when I was working on this, and when we brought it to
Eric Heilman:
you and brought it to the group, finally it was like, "What are
Eric Heilman:
we doing? I don't feel like I'm the person to make the call. All
Eric Heilman:
I can say is, this is what is out there at a much higher
Eric Heilman:
level. Whether it's any of those standards that I mentioned,
Eric Heilman:
yeah. So race and ethnicity is a great example because that's
Eric Heilman:
data that schools use pretty heavily, and you go to a lot of
Eric Heilman:
websites and schools will be saying X percent of our faculty
Eric Heilman:
are faculty of color, or X percent of our student body are
Eric Heilman:
students of color, and when you actually dig into that, you get
Eric Heilman:
some really interesting answers about how those numbers are
Eric Heilman:
produced, right? And that particular piece of data is a
Eric Heilman:
challenge to work with because schools can be collecting it for
Eric Heilman:
multiple purposes.
Eric Heilman:
So when we report out that information to Dassel
Eric Heilman:
slash Data Vista, they are aligned with the census because
Eric Heilman:
their primary use case for that data is to allow schools to
Eric Heilman:
compare their population to the surrounding market and their
Eric Heilman:
local geographic area. Local geographic information is
Eric Heilman:
categorized by the census. They are pulling that information
Eric Heilman:
from the census, and so schools need to report their categories
Eric Heilman:
to match the. So that they can make an apples to apples
Eric Heilman:
comparison to their local market. A totally separate use
Eric Heilman:
case for that same data is asking our students of different
Eric Heilman:
racial groups having a similar academic experience in our
Eric Heilman:
school, and that's a very internally focused question. And
Eric Heilman:
Bill, just as you touched on, there are some schools that may,
Eric Heilman:
for example, have a really rich and diverse Asian population,
Eric Heilman:
and so it's not sufficient to classify students as just Asian.
Eric Heilman:
They may want to break down to Chinese students, Japanese
Eric Heilman:
students, Thai students, et cetera, et cetera. And so for
Eric Heilman:
them, they're going to need a much more granular
Eric Heilman:
classification set of categories than a school that may have a
Eric Heilman:
smaller Asian population that may just have a category for
Eric Heilman:
Asian students, and so what we were really trying to do with
Eric Heilman:
the handbook was to come up with a scheme that allowed that kind
Eric Heilman:
of difference between independent schools to choose a
Eric Heilman:
level of granularity that is appropriate for their community
Eric Heilman:
internally, but that can be rolled up to external reporting
Eric Heilman:
categories in a consistent way across schools, so that we can
Eric Heilman:
have more confidence in our benchmarking and in how
Eric Heilman:
different schools are accounting for that sort of thing. We also
Eric Heilman:
made some recommendations about how to collect that information.
Eric Heilman:
That information is really incomplete in a lot of schools
Eric Heilman:
because it's inappropriate and oftentimes illegal to try to
Eric Heilman:
compel people to give that information during the
Eric Heilman:
application process, whether that's students or employees.
Eric Heilman:
And then there's a missed opportunity to actually collect
Eric Heilman:
it once they've entered the community. And so a lot of times
Eric Heilman:
you run into a data set for race and ethnicity that is really
Eric Heilman:
incomplete. There's a lot of unknown or unclassified or just
Eric Heilman:
blanks for community members, or you're frankly in some schools.
Eric Heilman:
Somebody on the administrative team is just deciding what race
Eric Heilman:
and ethnicity people are, and that's something that at Cirus
Eric Heilman:
we find to be pretty problematic. And so that again
Eric Heilman:
is an area where we supply a template, and we're like, "Hey,
Eric Heilman:
maybe when you're gathering this data, you should ask people how
Eric Heilman:
they self-identify. And if they're super young kids,
Eric Heilman:
obviously, you know, you may be getting that information from
Eric Heilman:
their parents and guardians. But you know, there are ways to get
Eric Heilman:
much higher quality data in a consistent way that works not
Eric Heilman:
only for your internal purposes, but is also can be rolled up and
Eric Heilman:
reported out consistently as well. Yeah, the example of
Eric Heilman:
pulling out the yearbook and going through and saying, "Oh,
Eric Heilman:
this person, you know, it's you hear stories
Eric Heilman:
Bill Stites, TLIS: like that and you think people are kidding,
Eric Heilman:
but they're not. No, this is how they would do it. I do want to
be clear:
NIS explicitly says not to do that, so it's not like
be clear:
this happening because they think that's okay. They also
be clear:
agree with us that people should be self-identified. But it was
be clear:
interesting because I mean, you brought that point up of do you
be clear:
identify right? Not what is your race and ethnicity, but do you
be clear:
identify as a person of color? It was interesting in that my
be clear:
wife is a Spanish teacher, and I would often hear her tell
be clear:
stories about people from different Spanish cultures,
be clear:
either identifying or not identifying as a person of color
be clear:
based on where they hail from, particularly the European
be clear:
Spaniards, in terms of where all of that kind of played out. So,
be clear:
the idea of saying, "Oh, you look like, or "I'm going to put
be clear:
you in this bucket, is fraught with peril. Absolutely correct.
be clear:
In census categories,
Eric Heilman:
the way that's reported out to Data Vista and
Eric Heilman:
NIS is that anybody who marks Hispanic or Latinx is counted as
Eric Heilman:
a person of color, and for internal analysis, if somebody
Eric Heilman:
marks as their identity that they are Hispanic, Latinx, and
Eric Heilman:
white, we have no idea if that's the only question you ask. We
Eric Heilman:
have no idea if that means that you are a Caucasian presenting
Eric Heilman:
person from Spanish descent, so you marked both of those, or if
Eric Heilman:
it means that you identify as a person of color, and you are
Eric Heilman:
multiracial, and you've clicked those, and so yeah, that's kind
Eric Heilman:
of granularity we were getting into as we were developing this
Eric Heilman:
and trying to figure out how we knew that. Certainly, if you're
Eric Heilman:
trying to do analysis around the equity of experience for people
Eric Heilman:
of color in your community, you want to count the people who
Eric Heilman:
actually identify as people of color, and you should ask them.
Eric Heilman:
That's all they identify, so that you can do that kind of
Eric Heilman:
analysis. We were pretty attentive to all of that. We've
Eric Heilman:
seen that where you know you send out emails
Eric Heilman:
Bill Stites, TLIS: to affinity groups on assumptions, right?
Eric Heilman:
And then, oh my God, that's never gone poorly for anyone.
Eric Heilman:
No, where could there be any pitfalls or thorny bushes ready
Eric Heilman:
to prick you with the different issues that could come up from
Eric Heilman:
that? It's unbelievable, right?
Hiram Cuevas:
I would say another area for your
Hiram Cuevas:
denominational schools-they're always interested in what
Hiram Cuevas:
religious backgrounds.
Eric Heilman:
Yeah,
Hiram Cuevas:
that's also a very sensitive subject. If you happen
Hiram Cuevas:
to be attending a Catholic school, you mentioned. What
Hiram Cuevas:
other religion you are participating in as a family?
Hiram Cuevas:
We're an Episcopalian school, and we're very interested in
Hiram Cuevas:
that. And we struggle with the gap of the data because so much
Hiram Cuevas:
of it goes unanswered.
Eric Heilman:
Yes, and that's an example of an area where, in
Eric Heilman:
certain jurisdictions and certain states, and depending on
Eric Heilman:
whether or not you accept funding from various levels of
Eric Heilman:
government, it may be illegal to ask that question. And so,
Eric Heilman:
definitely in the handbook, we have flagged areas where we're
Eric Heilman:
like, you might want to chat with your lawyer before you do
Eric Heilman:
this. But if it's all legit, here are some categories. And
Eric Heilman:
again, that's an area where we suggested sort of a tiered
Eric Heilman:
approach, where depending on your community, you may have
Eric Heilman:
very broad religious options for folks to choose from, or you may
Eric Heilman:
be able to drill down into more granular denominational
Eric Heilman:
questions and categories. And so, we took a similar approach
Eric Heilman:
to that as well. But that's an example of information that is
Eric Heilman:
not collected by NAS or these external benchmarking
Eric Heilman:
Bill Stites, TLIS: groups, but they can be super critical to a
Eric Heilman:
school's mission, particularly if they are denominational or
Eric Heilman:
religious. Eric, you know, Hiram and I are just thinking a lot
Eric Heilman:
about AI. Yeah, we mentioned some of these tools that are
Eric Heilman:
getting built into the products that we're using. When you start
Eric Heilman:
thinking about the work that we're doing around institutional
Eric Heilman:
research, around data, what would you say are your rules to
Eric Heilman:
live by? Your best practice for schools when they are
Eric Heilman:
considering using an AI platform that is independent of maybe the
Eric Heilman:
tools that they're using to help them in the work that they're
Eric Heilman:
doing. Like, what do they need to be mindful of? What do they
Eric Heilman:
need to look out for? What should they completely stay away
Eric Heilman:
from? Yeah, so we're just at the beginning of AI tools for school
Eric Heilman:
operations, right? And right now, there's probably a lot of
Eric Heilman:
schools that are thinking, "Why would I ever use AI for
Eric Heilman:
operational stuff? You know, for example, I could easily see a
Eric Heilman:
tool
Eric Heilman:
coming out to read and score admissions
Eric Heilman:
applications, that is something that's probably pretty easily
Eric Heilman:
trainable. It's a very time-consuming task in many
Eric Heilman:
ways, and AI might be more consistent. You know, one of the
Eric Heilman:
things we talk about in the handbook in the admissions and
Eric Heilman:
enrollment management chapter is the importance of rubrics when
Eric Heilman:
you are reading files, and how difficult it can be to calibrate
Eric Heilman:
a group of readers to score consistently. An AI tool could
Eric Heilman:
absolutely be a tool where that kind of task could be automated.
Eric Heilman:
Now, should a school make its ultimate decision about who to
Eric Heilman:
admit or decline based on just an AI? No, I don't think that
Eric Heilman:
should ever happen. Call me a humanist, but if you free up
Eric Heilman:
people's time from reading all these applications, that does
Eric Heilman:
give them more time to actually have face-to-face interactions
Eric Heilman:
with prospective families, which many people would argue is a
Eric Heilman:
higher value use of their time. And I think with the pressures
Eric Heilman:
that are on the independent school financial model, schools
Eric Heilman:
are going to be looking for ways to leverage AI to save money,
Eric Heilman:
just like we see in the corporate world. So I think
Eric Heilman:
there's going to be an increasing amount of pressures
Eric Heilman:
for schools to adopt these tools. That is why we wanted to
Eric Heilman:
make the handbook now, so that schools, when they get to that
Eric Heilman:
point and these tools are available, they have the data
Eric Heilman:
they need to do it. But I think the second thing that's equally,
Eric Heilman:
if not more, important than those points is the topic of
Eric Heilman:
data culture in schools. There are plenty of schools that have
Eric Heilman:
good data right now and are doing great data analysis. They
Eric Heilman:
can produce dashboards. They've done these really cool research
Eric Heilman:
projects around a lot of these strategic questions, and they're
Eric Heilman:
getting really interesting answers. Those answers will go
Eric Heilman:
nowhere if the leadership culture at the school does not
Eric Heilman:
know how to incorporate quantitative feedback into their
Eric Heilman:
decision-making processes, who in the room at the SLT meeting
Eric Heilman:
is asking? Well, do we have data about that? What does the data
Eric Heilman:
say? You're never going to hear me advocate that data should be
Eric Heilman:
the end-all, be-all arbitrator of how schools should run. I
Eric Heilman:
firmly believe that the human lived experience is incredibly
Eric Heilman:
important, comma. We can also incorporate data into that
Eric Heilman:
picture. I don't think either one is more valuable than the
Eric Heilman:
other, but I do think as we are moving forward, you're going to
Eric Heilman:
see more and more data analysis available in some of these tools
Eric Heilman:
that are going to be able to do things like, hey, this AI has
Eric Heilman:
united data about which students have late assignments, which
Eric Heilman:
ones have grades that have dropped this week in their
Eric Heilman:
classes, which ones had a few absences this week, and will be
Eric Heilman:
automatically putting together, for example, a list of students
Eric Heilman:
that may need extra support or checking in on and. Schools have
Eric Heilman:
to have the culture around data to acknowledge that those alerts
Eric Heilman:
may be real, and that even if no human has noticed it, that it
Eric Heilman:
may be an actual viable, an important clue to follow up on.
Eric Heilman:
And so, I think right now the most important preparation
Eric Heilman:
schools can be doing is a paying attention to the data that they
Eric Heilman:
are collecting and the condition of it, and then b really
Eric Heilman:
starting to think about what is the culture in our school around
Eric Heilman:
what data tells us. Do we have the data literacy to read
Eric Heilman:
quantitative analysis results? Do we trust it? Is our natural
Eric Heilman:
instinct to devalue or critique analysis projects in a way that
Eric Heilman:
we don't necessarily do when somebody is sharing an anecdote
Eric Heilman:
about what they believe about what's going on in the school,
Eric Heilman:
and so those two are the things I think are most important right
Eric Heilman:
now for schools to be thinking about.
Eric Heilman:
Bill Stites, TLIS: So, Eric, I don't know about you. I know
Eric Heilman:
Hiram and I could pepper you with questions all day, but
Eric Heilman:
don't
Eric Heilman:
threat me with a good time, Bill.
Eric Heilman:
Bill Stites, TLIS: We've got to land this plane and bring this
Eric Heilman:
ship home, so to speak. But
Eric Heilman:
the
Eric Heilman:
one thing I've always valued about CIRIS is just the
Eric Heilman:
commitment to constant programming. You know, to making
Eric Heilman:
sure that there's information out there, that there's
Eric Heilman:
opportunities to connect, talk to people to learn more. I
Eric Heilman:
always say, you know, you're never the smartest person in the
Eric Heilman:
room. You just need to be in the room with the right
Eric Heilman:
people,
Eric Heilman:
Bill Stites, TLIS: and CIRIS really does bring the right
Eric Heilman:
people together just to have the conversations, whether you solve
Eric Heilman:
a problem or not. It's like just having this conversation. So,
Eric Heilman:
what's coming up for Cirus? What do you got going on from a
Eric Heilman:
programming perspective? And then we'll say goodbye and keep
Eric Heilman:
you on for another half hour as we talk offline. I mean, we
Eric Heilman:
haven't even gone to the zombies yet. Exactly. Yeah. Okay. Cirus
Eric Heilman:
has got a lot of great programming going on this year.
Eric Heilman:
We are running four professional learning groups each semester
Eric Heilman:
this year. In the fall, we've got one geared at the handbook
Eric Heilman:
itself. So Louis Tulo at Ravenscroft, who was part of
Eric Heilman:
the cohort, is running a PLG on establishing a
Eric Heilman:
school-wide data governance framework. So, if you are
Eric Heilman:
interested in getting your school's data together and
Eric Heilman:
making sure that it is high quality, but you are looking to
Eric Heilman:
participate with a cohort of people that are on the same
Eric Heilman:
journey and you want some guidance, we've got a PLG for
Eric Heilman:
that. We're running a PLG around using Google Data Studio, which
Eric Heilman:
is a free dashboarding platform. So, if you are interested in
Eric Heilman:
visualizing and analyzing your data, we've got something for
Eric Heilman:
that. We're running two different PLGs this fall around
Eric Heilman:
statistics skills. So, we've got Statistics 101 to help build
Eric Heilman:
data literacy for schools in any office on campus that you have,
Eric Heilman:
we have a little bit more advanced statistics PLG around
Eric Heilman:
linear regression and regression techniques to establish
Eric Heilman:
relationships between variables, whether that's correlation
Eric Heilman:
between tenure and job performance, or admissions info
Eric Heilman:
and student performance. And then in the spring, we are
Eric Heilman:
diving into some topics that we just mentioned around what does
Eric Heilman:
actual data informed decision making look like in schools? So
Eric Heilman:
actually operationalizing data, we've got one around automating
Eric Heilman:
data quality audit. So once you've got your data governance
Eric Heilman:
framework up and running, how do you, in a sustainable way, make
Eric Heilman:
sure that those standards are being adhered to in the school
Eric Heilman:
community. The other one we're running in the spring is how do
Eric Heilman:
you design statistical and quantitative measures of these
Eric Heilman:
kind of squishy concepts like student engagement or belonging
Eric Heilman:
or inclusion? So we'll be doing a little bit of a deeper dive
Eric Heilman:
into some more complicated statistical concepts for folks
Eric Heilman:
that are interested in digging into that. Then along the way,
Eric Heilman:
we're also doing, as I mentioned earlier, sort of one-off
Eric Heilman:
webinars around data quality assessment. So come join us.
Eric Heilman:
We'll do an activity with you where we give you a query or a
Eric Heilman:
list to pull out of your system, and you can check and see, like,
Eric Heilman:
how do the contents of our report square up to what it
Eric Heilman:
should look like ideally? You know, we're really trying to
Eric Heilman:
help schools identify where the areas where their data looks
Eric Heilman:
great right now, where the areas where they need some help, and
Eric Heilman:
to lay out that action plan of like, okay, where is the
Eric Heilman:
smallest yet most impactful bite that we can start with on this
Eric Heilman:
journey to clean things up? And you know, I'm sure along the way
Eric Heilman:
we'll also have some fun little webinars, some games, and that
Eric Heilman:
sort of stuff. You know, it's always a good time to hang out
Eric Heilman:
with all the fun data folks. Yeah, I've got a big year. We've
Eric Heilman:
got a lot going on. So if you are a member school, you have
Eric Heilman:
access to all that, and if folks are interested in joining
Eric Heilman:
Cirius, all the info is on our website, which I'm sure will be
Eric Heilman:
linked with the pot. It
Hiram Cuevas:
will indeed.
Hiram Cuevas:
Bill Stites, TLIS: Awesome. I can't recommend it enough. You
Hiram Cuevas:
know, I've been hanging on and bothering you for quite some
Hiram Cuevas:
time, and will continue to do so in the years to come. And just
Hiram Cuevas:
really appreciate everything that you bring to the community.
Hiram Cuevas:
The fact that you always organize and participate in the
Hiram Cuevas:
Atlas Conference with the tracks you've got offered there always
Hiram Cuevas:
is a selling point for bringing people in and opening and having
Hiram Cuevas:
those conversations. So, thank you one. Again, Eric, we
Hiram Cuevas:
appreciate your time, and I can speak on both Hiram and my part.
Hiram Cuevas:
You know, thanks for involving us in the project this go
Hiram Cuevas:
around, and look forward to future chances to participate.
Hiram Cuevas:
So, thank you very much. I appreciate having you here
Hiram Cuevas:
today.
Eric Heilman:
Thank you, Eric. All right, thank you, guys.
Peter Frank:
This has been Talk Technology with Atlas, produced
Peter Frank:
by the Association of Technology Leaders in Independent Schools.
Peter Frank:
For more information about Atlas and Atlas membership, please
Peter Frank:
visit theatlas.org. If you enjoyed this discussion, please
Peter Frank:
subscribe, leave a review, and share this podcast with your
Peter Frank:
colleagues in the independent school community. Thank you for
Peter Frank:
listening.