A Brief History of DATAG
New York Schools Data Analysis Technical Assistance Group
In the late 90’s the accountability movement was gaining speed. New York began to give state assessments in grades 4 and 8 ELA and Math, with results of performance by schools publicly announced. Prior to the 4 and 8 tests, there had been Program Evaluation Tests (PET) in multiple grades and in multiple subjects including social studies and science, designed for school improvement, but without public reporting of the results. The ELA and Math tests gave birth to new data reporting responsibilities. Around the state, some of the 12 Regional Information Centers (RICs) had begun to play with the notion of compiling regional data bases to collect the data for reporting to Albany, and to store and report data for local district use.
In New York City and in Herkimer BOCES, a school improvement process had been taking shape for several years called Comprehensive District Educational Planning (CDEP). Fundamental to this process was the analysis of state tests, using a benchmarking process which compared local results with regional results and with state-wide measures. The CDEP process was promoted by the State Education Department (SED) as a mandate for low performing districts, but key district educators and the Staff and Curriculum Development Network (S/CDN--comprised of the curriculum and instructional development administrators from each of the 38 BOCES and the Big 5 Cities) pushed for the CDEP process to be rolled out as a school improvement program to be voluntary for every district. S/CDN was working hard to reduce the paperwork and the reporting mandated by SED, which in the late 90’s involved approximately 90 different reporting mandates and plans to be filed annually. CDEP supporters from BOCES and from within the ranks of the SED were seeking to reduce reporting requirements, and positioned the CDEP process as a comprehensive program that could replace several of the existing reporting/planning requirements.
With Goals 2000 funding, S/CDN and the Regional Information Centers, 6 of which were starting to implement regional data warehouses, and the Big Cities, 2 of which had substantial data processing capacity, organized 2 CDEP summer conferences at the Otesaga Hotel. These conferences were intended to roll out a systematic process that used student test data on state tests supplemented by whatever local districts might have available (multiple measures methodology) as the basis to plan for instructional improvement. Key players came from a handful of BOCES around the state, coordinated by two SED staff members. There were about 20 people identified to do the training sessions, and teams from districts were invited to attend. About 300 districts received the training in two summers and began using the CDEP process for school improvement planning.
At the same time, a small group of data users were attempting to make sense of the available data being collected regionally. They organized into an informal group calling themselves the Cognos Club, named after the software platform being adopted among the RICS to house the assessment data. This group included a few very technical database engineers, some network technicians who were setting up database servers, some statisticians who were working hard to develop meaningful reports, and data end-users who wanted to ensure that the reports coming out of these new warehouses were actually useful for instructional improvement. Several members of this small group overlapped with the CDEP training group, and at the second CDEP summer conference in 2000, a dozen folks came together to discuss the challenges of making instructionally meaningful use of the data warehouses. After dinner, they decided to make the bold move of organizing a special interest group, came up with the original name: New York Schools Data Analysis Technical Assistance Group (DATAG), wrote a mission statement, and identified some interim responsibilities for a few of those present. In attendance were 2 SED folks who had more Goals 2000 money available, and the group was promised $5000 to support a few organizational meetings and seed money to continue to expand their efforts.
Organizing through email and publicizing through word of mouth and person to person contacts throughout the state, the group started to meet at Questar BOCES, where two of the 12 were employed. One or two meetings of the group, with invitations to others followed in the 2000-2001 school year, and the final CDEP Summer Conference included presentations by DATAG members on statistical modeling and multiple measures for instructional planning.
The group expanded slowly, holding meetings in the Albany area, adding members, and pulling in more RIC staff and a few more instructional staff as data warehouses were being implemented in more and more RICs. Meetings were attended by 40 - 50 people and small paid memberships began, with 30 paid members in 2001. In 2002 the group identified a Program Chair to pay more attention to organization of effective programs, and a core group recruited session presenters by seeking out effective models of data use throughout the state. In 2003 a constitution and bylaws were written and DATAG began the process of being recognized as a tax-exempt nonprofit. Four meetings were planned annually, and in 2004 the summer meeting became a multiday summer conference with multiple sessions running concurrently. 65 attended.
In 2005, 102 people attended the summer conference on Long Island, and with the growth of the group and the organizational demands for quality sessions, the May meeting was dropped and the October, December and March meetings with a summer conference became the norm. National presenters were recruited to expand the summer conference, which became a half-day preconference, a full day with a keynote and 4 time blocks with concurrent sessions, and a Friday morning with 2 or 3 large group presentations generally using SED representatives. Keynotes and preconference presenters have included Greg Cizek (psychometrician),Yeo Meng Thum, (psychometrician), Jim Wright, Paul Preuss, Giselle Martin-Kniepp, Nancy Love, Karen Hess, Robert Lee (Chief psychometrician for MA), and Kim Marshall.
Sometime around 2004 or ‘05 DATAG organized a listserv that was open to all, housed at a RIC, and free. It remains free today and has about 1500 readers throughout New York. It is intended for the exchange of ideas about the use of data, and is most frequently used as a Q and A forum where folks post questions about where to find resources, how to handle various accountability issues, and reporting issues.
As the Summer Conference grew and the costs rose, the DATAG Board permitted vendors to support the Conference expenses through the purchase of display tables and paid presentation sessions. Vendors are not permitted to present at the one day meetings during the school year--these remain the purview of district presenters and representatives from the SED. Currently, the Thursday Summer Conference session structure is designed to have one keynote, and about 7 rooms which have 4 or 5 time blocks, allowing for up to 35 conference sessions. Several of these are available for vendors who pay to present. The remaining sessions originate from districts or higher ed contributors, and are identified, recruited, and vetted by the DATAG Summer Conference Committee. Attendance is generally around 225.
As of 2017, the DATAG paid membership is about 300, and attendance at the three school year meetings is around 200. Accountability and APPR mandates appear to be of great value in generating state-wide interest. Because we provide real-world examples of how districts are managing data and using data wisely, and because we have expanded our work considerable to include formative assessments as a tool for instructional improvement, we have been able to grow our participation and membership despite the downturn in resources at the district level.