Chapter 4

RESULTS

Summary

            The researcher analyzed the results of the 2006 eighth grade Illinois Standards Achievement Test (ISAT) math passing percentages available on a school-by-school basis at http://www.isbe.net.  The schools used in this study were public schools that offered eighth grade in Cook (excluding Chicago Public Schools), DuPage, Kane, Lake, McHenry and Will counties.  The appendix contains one table for each county that lists the schools in that county used in this preliminary study.  The total sample size of eighth graders in this study exceeds 55,000.  This number is an approximation based on the total number of students in each school divided by grade levels.

            The data gathering procedure required the researcher to record numerous characteristics of controllable and non-controllable factors specific to each of the 258 schools in this study.  Controllable factors are defined as factors that the school administration exhibits control over.  Non-controllable factors are defined as factors that the school administration has no control over.  Figure 4 shows the controllable and non-controllable factors that were gathered for all of the schools combined.

The controllable and non-controllable factors, together with the eighth grade math ISAT passing percentage, the county, the city, and the school, constituted more than 5,500 individual pieces of information that were entered into a Microsoft Excel spreadsheet in preparation for this analyses.

            This preliminary study served to explore the connections between the eighth grade math ISAT passing percentage and the controllable and non-controllable factors.  Multiple linear regression analysis was used to determine if relationships exist between the eighth grade math ISAT passing percentage and the controllable and non-controllable factors.

 

Results from the Analysis of Controllable Education Factors

            The purpose of the first analysis performed by the researcher was to identify the controllable factors that influence eighth grade math ISAT passing percentages.  The researcher employed the multiple linear regression method described in Chapter III.  The data used in the multiple linear regression analysis included 258 schools with an eighth grade population of more than 55,000 students.

The controllable factors that have an influence on the eighth grade math ISAT passing percentage (y^) are:

1.      pupil-to-teacher ratio, PtTR

2.      average class size, ACS

3.      minutes of math instruction per day, MMI

4.      percentage of teachers with M.A.s or higher, PerMA

5.      average administrator salary (based on a per $1,000 amount), AdminSal.

The results of the multiple regression analysis for the controllable factors are show in Figure 5:

Controllable Factor	P-value from Multiple Regression Analysis
Pupil-to-teacher ratio	0.014
Average class size	0.008
Minutes of math per day	Less than 0.001
Percentage of teachers with M.A.s or higher	Less than 0.001
Average administrator salary	0.003
Figure 5: Controllable Factor P-values

The r-square value is 0.368.

            If persons were interested in predicting the eighth grade math ISAT passing percentage based only on the controllable factors, they could do so by employing the fixed value and the coefficients of the influential independent variables that resulted from the multiple linear regression analysis.  From this analysis, the fixed portion of the results, referred to as the y-intercept, is equal to 63.8 percent.   That is, the percentage of eighth graders who meet or exceed the minimum ISAT math score is 63.8% before the controllable factors are applied.  With the application of the controllable factors, the predictive model equation is:

y^ = 63.8 - 0.82 PtTR + 0.48 ACS - 0.28 MMI + 0.32 PerMA + 0.18 AdminSal

As an example of how the controllable factors predictive model equation works, one needs to calculate the expected percentage of eighth grade students attending Westchester Middle School (Cook County) who meet or exceed the minimum ISAT math score.  The following are Westchester Middle School’s controllable factor values:

a)      pupil-to-teacher ratio is 16.6 students

b)      average class is 20.2 students

c)      minutes of math per day is 43

d)     percent of teachers with M.A.s or higher is 50.2

e)      average administrator salary is $95,128 and

The percent of students who meet or exceed the minimum ISAT math score =

63.8 - (0.82 x 16.6) + (0.48 x 20.2) - (0.28 x 43) + (0.32 x 50.2) + (0.18 x 95.128) = 81.2%

According to this calculation, the expected percentage of eighth grade students who meet or exceed the minimum ISAT math score for this data would be about 81.2%.  The actual percentage of eighth grade students who met or exceeded the minimum ISAT math score was 85.0% or a difference of 3.8%.

            The predictive model equation is not considered a reliable predictive tool unless there is a way of checking how accurately it predicts.  One of the most straightforward ways of checking the predictive equation’s accuracy is to see how well it can predict the passing percentages of the schools used in this analysis.  The process entails using the predictive model equation on each school in the study to calculate the school’s predictive passing percentage.  Then the predicted passing percentage is compared to the school’s actual passing percentage.  The difference between the predicted and actual passing percentage is called a residual.  The smaller the residuals, the better the predictive model equation is for predicting a school’s expected passing percentage.

The first graphical analysis of the residuals from the controllable factors predictive model equation is shown in Figure 6.

Figure 6: Residuals Versus Observations for Controllable Factors Model Equation

 

            Figure 6 shows that the residuals for the controllable factors predictive model equation are well distributed.  The graph in Figure 6 shows a handful of schools that have residuals above 20 and below –20.

            Figure 7 shows another graphical analysis of the residuals from the controllable factors predictive model equation.

 

Figure 7: Residuals Versus Passing Percentage for Controllable Factors Model Equation

 

            Figure 7 shows a plot of the residuals compared to the passing percentage for the controllable factors predictive model equation.  The graph in Figure 7 shows a handful of schools that have residuals above 20 and below –20.

           

Results from the Analysis of Non-Controllable Education Factors

The purpose of the second analysis performed by the researcher was to identify the non-controllable factors that influence eighth grade math ISAT passing percentages.  Again, the researcher employed the multiple linear regression method described in Chapter III.  The data used in this second multiple linear regression analysis included all 258 schools with an eighth grade population of more than 55,000 students.

The non-controllable factors that have an influence on the eighth grade math ISAT passing percentage (y) are:

1.      percentage of black students, PBS

2.      percentage of Hispanic students, PHS

3.      percentage of low-income students, PLI

4.      student mobility rate, SMR

5.      student attendance rate, SAR.

 

 The results of the multiple regression analysis for the non-controllable factors are show in Figure 8:

Non-controllable Factor	P-value from Multiple Regression Analysis
Percentage of black students	Less than 0.001
Percentage of Hispanic students	0.001
Percentage of low-income students	0.004
Student mobility rate	Less than 0.001
Student attendance rate	Less than 0.001
Figure 8: Non-controllable Factor P-values

The r-square value is 0.786.

            If persons were interested in predicting the eighth grade math ISAT passing percentage based only on the non-controllable factors, they could do so by employing the fixed value and the coefficients of the influential independent variables that resulted from the multiple linear regression analysis.  From this preliminary analysis, the fixed portion of the results, referred to as the y-intercept, is equal to -121.3 percent.  With the application of the non-controllable factors, the predictive model equation is:

y^ = 2.25 SAR - 0.21 PBS - 0.12 PHS - 0.11 PLI - 0.18 SMR - 121.3

As an example of how the non-controllable factors predictive model equation works, let us calculate the expected percentage of eighth grade students attending Westchester Middle School (Cook County) who meet or exceed the minimum ISAT math score.  The following are Westchester Middle School’s non-controllable factor values:

a)      the black student percentage is 24.9

b)      the Hispanic student percentage is 17.9

c)      the low-income percentage is 6.7

d)     the mobility rate is 7.6 percent and

e)      the attendance rate is 96.5 percent

The percent of students who meet or exceed the minimum ISAT math score =

(2.25 x 96.5) - (0.21 x 24.9) - (0.12 x 17.9) - (0.11 x 6.7) - (0.18 x 7.6) - 121.3 = 86.7%

According to this calculation, the expected percentage of eighth grade students who meet or exceed the minimum ISAT math score for these factors would be about 86.7%.  The actual percentage of eighth grade students who met or exceeded the minimum ISAT math score was 85.0% or a difference of 1.7%.

            As described in the previous section, the reliability of the predictive equation needs to be tested to determine if it is an accurate means of predicting math ISAT passing percentages based on non-controllable factors.  The same methodology will be used to check the reliability of the non-controllable factors predictive equation as was employed with the controllable factors predictive equation.  The process involves calculating the residuals for each school used in the study and then plotting the residuals to determine whether a pattern exists.

Figure 9 shows the first graphical analysis of the residuals from the non-controllable factors predictive model equation.

Figure 9: Residuals Versus Observations for Non-controllable Factors Model Equation

 

            The residuals for the non-controllable factors predictive model equation are well distributed.  The graph in Figure 9 shows a few schools that have residuals above 20 and below –20.

            The second graphical analysis of the residuals from the non-controllable factors predictive model equation is shown in Figure 10.

 

Figure 10: Residuals Versus Passing Percentage for Non-controllable Factors Model Equation

 

Figure 10 shows a plot of the residuals compared to the passing percentage for the non-controllable factors predictive model equation.  The graph in Figure 10 shows a few schools that have residuals above 20 and below –20.

 

Results from the Analysis of the Combination of Controllable and Non-Controllable Factors

The purpose of the third and last analysis was to identify the combined controllable and non-controllable factors that influence eighth grade math ISAT passing percentages.  Again, the researcher employed the multiple linear regression method described in Chapter III.  The data used in this third multiple linear regression analysis included all 258 schools with an eighth grade population of more than 55,000 students.

The combined controllable and non-controllable factors that have an influence on the eighth grade math ISAT passing percentage are:

1.      percentage of black students, PBS

2.      percentage of Hispanic students, PHS

3.      percentage of low-income students, PLI

4.      student mobility rate, SMR

5.      student attendance rate, SAR

6.      pupil-to-teacher ratio, PtTR.

The results of the multiple regression analysis for the combined controllable and non-controllable factors are show in Figure 11:

Controllable and Non-controllable Factors	P-value from Multiple Regression Analysis
Percentage of black students	Less than 0.001
Percentage of Hispanic students	0.001
Percentage of low-income students	0.006
Student mobility rate	0.001
Student attendance rate	Less than 0.001
Pupil-to-teacher ratio	0.001
Figure 11: Controllable and Non-controllable Factor P-values

The r-square value is 0.795.

            If persons were interested in predicting the eighth grade math ISAT passing percentage based on the controllable and non-controllable factors, they could do so by employing the fixed value and the coefficients of the influential independent variables that resulted from the multiple linear regression analysis.  From this preliminary analysis, the fixed portion of the results, referred to as the y-intercept, is equal to -65.4 percent.  With the application of the controllable and non-controllable factors, the predictive model equation would be:

y^ = 1.78 SAR - 0.23 PBS - 0.12 PHS - 0.11 PLI - 0.15 SMR - 0.57 PtTR - 65.4

As an example of how the controllable and non-controllable factors predictive model equation works, one needs to calculate the expected percentage of eighth grade students attending Westchester Middle School (Cook County) who meet or exceed the minimum ISAT math score.  The following are Westchester Middle School’s controllable and non-controllable factor values:

a)      the black student percentage is 24.9

b)      the Hispanic student percentage is 17.9

c)      the low-income percentage is 6.7

d)     the mobility rate is 7.6 percent

e)      the attendance rate is 96.5 percent and

f)       the pupil-to-teacher ratio is 16.6.

The percent of students who meet or exceed the minimum ISAT math score =

(1.78x96.5) - (0.23x24.9) - (0.12x17.9) - (0.11x6.7) - (0.15x7.6) - (0.57x16.6) - 65.4 = 86.7%

According to this calculation, the expected percentage of eighth grade students who meet or exceed the minimum ISAT math score for these factors would be about 86.7%.  The actual percentage of eighth grade students who met or exceeded the minimum ISAT math score was 85.0% or a difference of 1.7%.

            Once, again the reliability of the predictive equation needs to be tested to determine if it is an accurate means of predicting math ISAT passing percentages based on controllable and non-controllable factors.  The same methodology will be used to check the reliability of the combined controllable and non-controllable factors predictive equation as was employed in the previous two analyses.  The process involves calculating the residuals for each school used in the study and then plotting the residuals to determine whether a pattern exists.

Figure 12 shows the first graphical analysis of the residuals from the controllable and non-controllable factors predictive model equation.

Figure 12: Residuals Versus Observations for Controllable and Non-controllable Factors Model Equation

 

            The residuals for the combined controllable and non-controllable factors predictive model equation are well distributed.  The graph in Figure 12 shows a few schools that have residuals above 20 and below –20.

            The second graphical analysis of the residuals from the combined controllable and non-controllable factors predictive equation is shown in Figure 13.

Figure 13: Residuals Versus Passing Percentage for Controllable and Non-controllable Factors Model Equation

 

            Figure 13 shows a plot of the residuals compared to the passing percentage for the combined controllable and non-controllable factors predictive model equation.  The graph in Figure 13 shows a few schools that have residuals above 20 and below –20.