Ballotpedia:Who Runs the States, SQLI, Methodology

From Ballotpedia
Jump to: navigation, search

WhoRunsTheStates Badge.png

Who Runs the States

Main Report Pages
Main PagePart 1Partisanship InfographicPart 2Part 3

Partisanship Results Report (Part 1)
Executive SummaryState Partisanship AnalysisPartisan Control of GovernorshipsPartisan Control of State LegislaturesPartisan Control of State SenatesPartisan Control of State HousesState Government TrifectasOverall Partisan Control: Bright, Medium and Soft StatesChanges of Partisan Domination over 22 yearsYear-to-Year Changes in State Partisan ControlTrifectas and Presidential Election PatternsConclusionMethodologyAppendix AAppendix B

State Quality of Life Index (SQLI) Report (Part 2)
Executive SummaryState Quality of Life Index (SQLI)About the IndexOverall RankingsDramatic Changes from 1st Half to 2nd HalfIndividual IndicatorsMethodologyAppendices

Partisanship and (SQLI) Overlay Report (Part 3)
IntroductionComparing Partisanship and the State Quality of Life Index (SQLI) RankingsDescription of the dataTrends and correlationsMethodologyKey Values for Fifty-State RegressionsAppendices
Praise or blame is extended to political parties for the economic, educational, health and other quality of life outcomes that result from the policies those parties enact into law. To better understand which political party enjoys power in each of the states, Ballotpedia has analyzed state government control from 1992-2013 using the concept of a "partisan trifecta." A partisan trifecta is defined as when a state's governorship and legislative chambers are controlled by the same political party.

The two major political parties claim that their policies will lead to better outcomes. What does the data show?

At Ballotpedia, we explored these issues in a three-part study, Who Runs the States.

This page contains the section of Part Two pertaining to the Methodology.


The following 19 indicators and indices were included in our study:

  • Best and Worst Governed States (24/7 Wall St.)
  • America's Health Rankings® (United Health Foundation)
  • Comprehensive Annual Financial Report (CAFR) debt-to-GDP ratio (Institute for Truth in Accounting)
  • Best and Worst States for Business (Chief Executive magazine)
  • Top States for Business (CNBC)
  • Best States for Business (Forbes)
  • Government employment's share of the population (Bureau of Labor Statistics via State Data Lab)
  • High school graduation rate (National Center for Education Statistics via United Health Foundation)
  • Personal income per capita (Bureau of Labor Statistics)
  • Poverty rate (Census Bureau via State Data Lab)
  • Real GDP per capita (Bureau of Economic Analysis)
  • State general obligation bond credit rating (Standard and Poor's via Pew Center on the States)
  • State government spending-to-GDP ratio (Census Bureau via State Data Lab)
  • State and local tax burden per capita (Tax Foundation)
  • Tax Freedom Day® (Tax Foundation)
  • Unemployment rate (Bureau of Labor Statistics)
  • Unfunded pension liabilities due per capita (Institute for Truth in Accounting/Census Bureau)
  • Voter turnout (United States Elections Project)
  • The Well-Being Index® (Gallup-Healthways).

More details on each are available below.

We selected these 19 annual indicators and indices and used them to build the State Quality of Life Index (SQLI) for the 50 states covering the years from 1992 to 2012. We did not include the District of Columbia, other U.S. dependencies, or national averages in these data sets. For each indicator or index, we determined whether the high or the low end of the recorded data represented the more socially desirable outcome and then accordingly ranked the states from 1st to 50th for each year for which we had data. A state ranked first was considered to have achieved the most desirable outcome among the states such as the lowest unemployment rate or the highest graduation rate. Although it was easy to select the most socially desirable outcome for most of our indicators, a few of the indicators required us to make judgment calls. For the purposes of the SQLI, we considered less government spending, a lower percentage of the population in government employment, lower taxes, and higher voter turnout to be more socially desirable than their opposites.[1]

Because these rankings measure the states' positions relative to one another, rather than the actual quantities being measured, our index over time only shows how each state's quality-of-life compares to that of other states rather than tracking measurable increases or decreases in the levels of the underlying indicators.[2] For example, a state's graduation rate could show strong improvement over the previous year, but the state's graduation rate ranking could remain the same or even worsen if the other states generally also had similar or even great improvement. The unemployment rate in a state could improve from one year to the next -- but if it improves at a slower rate relative to the rest of the country, then the state ranking will decline. For each indicator or index, we also calculated the states' overall positions by totaling all of their annual rankings.

We then assembled all 19 data sets into a single index measuring quality of life in the 50 states, the State Quality of Life Index (SQLI). We totaled each state's rankings from all data sets available for each year and then gave the states overall annual rankings from 1st to 50th for each year. For each year, all indicators were equally weighted. Because we only had data for six of our data sets for every year, the “basket” of indicators was not constant but rather changed for each year based on the data available for that year. We totaled the states' annual quality-of-life rankings and then calculated the states' overall rankings across the 21 years on our quality-of-life index. All years, regardless of how many data sets were included in that year's rankings, were weighted equally in calculating the overall rankings.

In general, we included more indicators in the SQLI for more recent years. Because our more recent rankings take more indicators into account, a given indicator has less weight in calculating our more recent annual rankings than any given indicator had in the earlier rankings. For example, the six indicators for which we had data for all 21 years each accounted for 8.53 percent of the data used in building our index for a cumulative weight of 51.17 percent. However, those are the overall weights reflecting the various weights these indicators were given in different years. When our quality-of-life rankings from 1992 to 2000 used a constant basket of 10 indicators, the six permanent indicators accounted for 60 percent of the data used in the index. Their collective weight was reduced in rankings for subsequent years which included more rankings until the six permanent indicators accounted only for a third of the data used to calculate the 2010 rankings.

In addition to the six permanent indicators, we used an additional four indicators with data available for all years except the most recent one or two because the necessary data had not yet been released. Our data sets for the poverty rate, real GDP per capita, and state government spending per capita as a percentage of GDP include all years except 2012, while the state and local tax burden data set lacks only 2011 and 2012. These 10 indicators together account for 83.55 percent of the data used in calculating our quality-of-life rankings over the entire period and 100 percent from 1992 to 2000. With the inclusion of additional indicators beginning with the 2001 rankings and the lack of the most recent data for four of the ten long-term indicators, the collective weight of these 10 declined, reaching 55.55 percent in 2010 (the last year with data for all 10), and 46.15 percent in 2012.

Four of the remaining nine data sets cover significantly fewer years because the data was not readily available, at least in a state-by-state, year-by-year format, for some years. The Standard and Poor's credit ratings for state-issued general obligation bonds were only readily obtainable for the years from 2001 on. State Data Lab, whose information about state finances was very helpful to us, had only three and seven years of data available for states' unfunded pension liabilities and debt as a percentage of GDP, respectively. Although the Tax Foundation has calculated state Tax Freedom Day®s since 1990, it only publishes the most recent year's state results because it does not backcast using current government data to accurately recalculate past state Tax Freedom Days. We included the 2012 state Tax Freedom Days. Although included for only one year's rankings, the Tax Freedom Days help provide data similar to that provided by the Tax Foundation's state and local tax burden calculations, which is not yet available for 2011 or 2012. These four, shorter-term data sets contributed 7.58 percent to the overall rankings over the entire period and were a significant factor in the last eight years. They were 17.65 percent in 2009 and 2011, the years in which they had the most weight.

In recent years, media organizations have launched numerous indices, including our last five data sets, to rank the 50 states. Chief Executive magazine launched its Best/Worst States rankings in 2005. Forbes began its Best States for Business index in 2006, followed by CNBC with America's Top States for Business in 2007 and 24/7 Wall St. with the Best and Worst Run States index in 2010. Beyond these economic and business-oriented indices, Gallup, Inc., and Healthways launched the Gallup-Healthways Well-Being Index® in 2008. The four economic indices accounted for only 7.38 percent in the overall index across the study period but 30.8 percent in 2012. The Gallup-Healthways Well-Being Index accounted for just 1.49 percent across the entire study period but 7.69 percent in 2012. For an example of our weighting methodology, please see Appendix D.

Although we used a broad range of indicators and indices to make our index, our index places the most weight on economic and state financial data. We encourage everyone interested in doing so to use our partisanship data and create new indices to compare with it. Our results do not analyze the factors or variables contributing to fluctuations in state rankings. We simply provide the data and aggregate results, without passing judgment as to causes of variations. Each indicator or index is briefly explained below. The years in parentheses reflect the years for which a given indicator or index has been used in our study.

A Note on State Data Lab

We obtained much of our data from the State Data Lab, a website operated by the Institute for Truth in Accounting (IFTA). IFTA is a non-profit organization whose stated mission is “in a nonpartisan manner, to compel governments to produce financial reports that are understandable, reliable, transparent and correct.”[3] The State Data Lab includes information about each state's demographics, economic environment, truth in accounting, and official financial data. Aside from those data sets of State Data Lab that we directly used in assembling the rankings used in this study, the website was also helpful in introducing us to other data sets which we then accessed from the original source. We are grateful to them for compiling datasets in such an easy-to-digest format.

24-7 Wall St.'s Best and Worst Governed States (2010-2012): 0.91 percent weight

24/7 Wall St. describes itself as “a Delaware corporation set up to run a financial news and opinion operation with content delivered over the Internet.”[4] Since 2010, 24/7 Wall St. has annually compiled its “Best and Worst Governed States” list, using a methodology similar to that used by Financial World magazine in 1990s before its 1998 bankruptcy:

“To determine how well the states are run, 24/7 Wall St. reviewed hundreds of data sets from dozens of sources. We looked at each state's debt, revenue, expenditure and deficit to determine how well it is managed fiscally. We reviewed taxes, exports, and GDP growth, including a breakdown by sector, to identify how each state is managing its resources. We looked at poverty, income, unemployment, high school graduation, violent crime and foreclosure rates to measure if residents are prospering.”[5]

In our rankings, we coded 24/7 Wall St.'s best run state as first place.

America's Health Rankings® (1992-2012): 8.53 percent weight

America's Health Rankings® is a report published annually by the United Health Foundation, a nonprofit solely funded by the private UnitedHealth Group; the American Public Health Association, an organization of public health professionals; and Partnership for Prevention, a nonprofit organization of business, nonprofit, and government leaders. The rankings, calculated since 1990, track 24 indicators, including 16 determinants of health in 4 categories and 8 health outcomes:

  • Behaviors: smoking, binge drinking, obesity, sedentary lifestyle, and high school graduation[6]
  • Community and Environment: violent crime, occupational fatalities, children in poverty, infectious disease, and air pollution
  • Policy: Lack of health insurance, public health funding, and immunization coverage
  • Clinical Care: Low birthweight, primary care physicians, and preventable hospitalizations
  • Outcomes: diabetes, poor mental health days, poor physical health days, geographic diversity, infant mortality, cardiovascular deaths, cancer deaths, and premature death

Each indicator is assigned a weight within the index, with a 75-25 split between the determinants and the outcomes.[7] In our rankings, we coded America's Health Rankings®'s healthiest state as first place.

Comprehensive Annual Financial Report Debt-to-GDP (2005-2011): 2.15 percent weight

State Data Lab has calculated each state's government debt as a percentage of the state's gross domestic product. In our rankings, we coded the state with the lowest debt-to-GDP ratio as first place.

Chief Executive's Best and Worst States for Business (2005-2012): 2.51 percent weight

State Data Lab has compiled the annual rankings of Chief Executive magazine's Best and Worst States survey. This survey asks CEOs and other business leaders to “grade states in which they do business among a variety of areas, including tax and regulation, quality of workforce and living environment.”[8] Although State Data Lab only compiled these rankings beginning in 2006, we have also included the rankings from the inaugural survey in 2005.[9] In our rankings, we coded Chief Executive's best state for business as first place.

CNBC's Top States for Business (2007-2012): 1.81 percent weight

Television network CNBC has annually ranked the 50 states since 2007. These rankings take into account 51 measures of competitiveness in the following 10 categories: cost of doing business, workforce, quality of life, infrastructure and transportation, economy, education, technology and innovation, business friendliness, access to capital, and cost of living.[10] In our rankings, we coded CNBC's top state for business as first place.

Forbes' Best States for Business (2006-2012): 2.15 percent weight

Forbes has ranked the 50 states annually since 2006. The magazine uses 35 indicators in the following six areas: business costs, labor supply, regulatory environment, economic environment, growth prospects, and quality of life.[11] In our rankings, we coded Forbes' best state for business as first place.

Government employment as percentage of population (1992-2012): 8.53 percent weight

State Data Lab has compiled data from the Bureau of Labor Statistics and Census Bureau to calculate the number of each state's residents in government employment as a percentage of its total population. The Bureau of Labor Statistics annually calculates the number of people employed by all levels of government within each state using information collected in the Current Employment Statistics (CES) survey. As defined by BLS, “Government employment covers only civilian employees; military personnel are excluded. Employees of the Central Intelligence Agency, the National Security Agency, the National Imagery and Mapping Agency, and the Defense Intelligence Agency also are excluded.”[12] We divided the BLS's government employment figures by the Census Bureau's mid-year population estimates to calculate what percentage of the population was employed by the government for each year. We calculated our government employment rankings for 2012 using the direct BLS data but were unable to modify our data to reflect the BLS's March 29 correction of Delaware's 2011 and 2012 data.[13] The correction meant that Delaware actually had the 17th lowest percentage of government employees, instead of the 16th as initially calculated, and that Delaware switched positions with New Hampshire. However, this minor change would not have affected our overall 2012 rankings. In our rankings, we coded the state with the lowest percentage of its population in government employment as first place.

High school graduation rate (1992-2012): 8.53 percent weight

Using data from the U.S. Department of Education's National Center for Education Statistics, the United Health Foundation has compiled state high school graduation rates back to 1990 for its America's Health Rankings®. The graduation rate is defined as the “percentage of incoming ninth graders who graduate in four years from a high school with a regular degree.”[14] For 2004, 2005, and 2006, when two sets of graduation rates were calculated by the NCES, we used the graduation rates calculated in line with the modified criteria set forth by the No Child Left Behind Act, the criteria used for all years since 2006 as well. In our rankings, we coded the state with the highest graduation rate as first place.

Personal income per capita (1992-2012): 8.53 percent weight

The Bureau of Economic Analysis annually calculates the personal income per capita for each state's residents. According to the BEA, “Per capita personal income is calculated as the total personal income of the residents of a state divided by the population of the state. In computing per capita personal income, BEA uses the Census Bureau's annual midyear population estimates.”[15] In our rankings, we coded the state with the highest personal income per capita as first place.

Poverty rate (1992-2011): 8.16 percent weight

State Data Lab has compiled data from the Annual Social and Economic Supplement to the Current Population Survey (CPS), which was started by the U.S. Census Bureau in 1959 and first included state-level data in 1980. This data set tracks “the percent of people whose income falls below the poverty level according to the CPS.” The Census Bureau recommends using the alternative American Community Survey (ACS) for state-level data, but we selected the CPS data because the ACS only started in 2000.[16] In our rankings, we coded the state with the lowest poverty rate as first place.

Real GDP per capita (1992-2011): 8.16 percent weight

The Bureau of Economic Analysis calculates annual, state-by-state real GDP per capita figures. The 1992 to 1996 statistics followed the Standard Industrial Classification (SIC) system and were calculated in chained 1997 dollars. All real GDP figures from 1997 on have been calculated in accordance with the North American Industry Classification System (NAICS) using chained 2005 dollars. According to the BEA, “real GDP by state is an inflation-adjusted measure of each state's gross product that is based on national prices for the goods and services produced within the state,” and “per capita real GDP by state is calculated by dividing the real GDP for a state by the resident population of the state.”[17] In our rankings, we coded the state with the highest GDP per capita as first place.

State credit rating (2001-2012): 4.24 percent weight

The Stateline news service of the Pew Center on the States has compiled a list of the credit ratings for general obligation bonds issued by state governments. Stateline used the ratings of Standard and Poor's, a private credit rating agency.[18] In our rankings, we coded the states with highest credit rating (AAA) as first place. Alaska, Arizona, and South Dakota did not have credit ratings for 2001-02, 2001, and 2001-05, respectively. We did not rank these three states for the years they did not have credit ratings, meaning that 47 states were ranked for 2001, 48 for 2002, and 49 for 2003, 2004, and 2005. Although these three states were not ranked for the years they did not have credit ratings, they received zeroes, meaning that, for the purposes of our index, we treated them as if they had better credit ratings than the other states.

State government spending-to-GDP (1992-2011): 8.16 percent weight

State Data Lab has compiled this data set from U.S. Census Bureau figures, using the total state government spending reported in the Annual Study of State Government Finances and the nominal Gross Domestic Product (GDP). The Bureau of Economic Analysis defines state GDP as the share of national GDP “originating in all the industries in a state.”[19] State Data Lab's data shows each state government's spending as a percentage of its GDP.[20] In our rankings, we coded the state with the lowest percentage as first place.

State and local tax burden (1992-2010): 7.88 percent weight

The Tax Foundation website describes the organization as “a nonpartisan tax research group in Washington, D.C.,” which works “for simple, sensible tax policy at the federal, state, and local levels.”[21] For each state, the Tax Foundation calcu­lated the total amount paid by the residents in taxes, then they divided those taxes by the state's total income to compute a "tax burden."[22] In our rankings, we coded the states with highest credit rating (AAA) as first place.

Tax Freedom Day® (2012): 0.37 percent weight

The Tax Foundation also annually calculates each state's Tax Freedom Day®, the day that a state's residents have theoretically worked long enough to pay off all federal, state, and local tax obligations for the year.[23] For calculation purposes, the Tax Foundation assumes that state residents “[start] working on January 1, earning the same amount each day and spending nothing” until after paying off all taxes.[24] In our rankings, we coded the state with the earliest Tax Freedom Day as first place.

Unemployment rate (1992-2012): 8.53 percent weight

State Data Lab has compiled the annual state unemployment rates until 2010 from the Bureau of Labor Statistics' Local Area Unemployment Statistics program, a state-federal collaboration. The unemployment rate is defined as “persons who actively looked for work in a four-week period ending with the reference week, but who were not employed during the reference week, divided by the total of employed and unemployed persons in the reference week.”[25] We used statistics directly from the Bureau of Labor Statistics for 2011 and 2012.[26] In our rankings, we coded the state with the lowest unemployment rate as first place.

Unfunded pension liabilities due per capita (2009-2011): 0.82 percent weight

State Data Lab has compiled all the states' unfunded pension liabilities due. We then divided each state's unfunded pension liabilities by its population, as calculated for the Census Bureau's annual estimates. In our rankings, we coded the state with the lowest per capita quantity of unfunded pension liabilities due as first place.

Voter turnout (1992-2012): 8.53 percent weight

Dr. Michael McDonald of George Mason University runs the United States Elections Project. He has calculated even-year state-level voter turnout by dividing the votes cast for the highest office by the voting-eligible population (VEP). McDonald explains that “the voting-eligible population is constructed by adjusting the voting-age population for non-citizens and ineligible felons, depending on state law.” The “Vote for Highest Office” is the total number of votes cast for president in a presidential election year or the highest total cast for a statewide office (i.e., governor) in a non-presidential election year. In states without a statewide office up for election in a given year, McDonald sums up the vote totals from the state's U.S. House elections, something he has also done since 2006 in cases when the congressional vote total exceeds that of statewide elections.[27] Reasoning that people's experiences in the year before an election often contribute to the voter turnout in an election year, we used each even year's set of voter turnout figures twice, for that year itself and for the preceding odd year. For example, we used the 2006 voter turnout data for both 2006 and 2005. In our rankings, we coded the state with the highest voter turnout as first place.

Gallup-Healthways Well-Being Index® (2008-2012): 1.49 percent weight

Research and polling firm Gallup, Inc., and health-improvement company Healthways have worked together since 2008 to produce the Gallup-Healthways Well-Being Index® using data gathered from at least 500 Americans per day for 350 days each year. The Index scores are calculated by using respondents' evaluations of themselves with regards to 42 items. These items are organized into 6 “domains of well-being”: life evaluation, emotional health, physical health, health behavior, work environment, and basic access (to medicine, clean water, etc.). In our rankings, we coded the state with highest level of well-being as first place.[28]

See also

External links


  1. As mentioned earlier, choice of indexes and ranking order of indexes can vary. Readers who disagree with these ways of assessing quality of life can re-calculate a different aggregate index using our dataset, available here. Contact report lead author Geoff Pallay with any comments or questions.
  2. America's Health Rankings®, high school graduation rate, government employment as a share of the total population, personal income per capita, unemployment rate, and voter turnout.
  3. State Data Lab, “About,” accessed March 20, 2013, available at State Data Lab
  4. 24/7 Wall St., “About Us,” accessed March 19, 2013, available at 24/7 Wall Street
  5. 24/7 Wall St., “The Best and Worst Run States in America: A Survey of All 50,” November 27, 2012, available at 24/7 Wall Street Report
  6. Note: High School Graduation Rate is weighted more heavily than other metrics because there is an individual metric for it but it is also included within other indices. However, we view it as a vital statistic that merits additional weighting.
  7. America's Health Rankings®, “About the Report,” accessed March 19, 2013, available at ]
  8. JP Donlon, Chief, “Another Triumph for Texas: Best/Worst States for Business 2012,” May 2, 2012, available at Chief Executive Magazine
  9. Economic Development Authority of Western Nevada, “Best and Worst States States for Business, Chief Executive Magazine,” available at Chief Executive Magazine, accessed via the Internet Archive on March 20, 2013.
  10. Scott Cohn, CNBC, “Categories & Criteria for CNBC's Top States for Business 2012,” July 11, 2012, available at CNBC
  11. Kurt Badenhausen, Forbes, “Best States for Business: Behind the Number,” December 12, 2012 Forbes "Best States for Business"
  12. Bureau of Labor Statistics, “Employment, Hours, and Earnings from the Establishment Survey,” p. 1, accessed on April 5, 2013, available at BLS "Employment, Hours, and Earnings from the Establishment Survey"
  13. Bureau of Labor Statistics, “Corrections to State and Metropolitan Area Employment, Hours & Earnings Data,” accessed on April 5, 2013, available at BLS "Hours & Earnings Data"
  14. America’s Health Rankings®, United Health Foundation, “High School Graduation Rate (1990-2012),” accessed March 20, 2013, available at America's Health Rankings
  15. Bureau of Economic Analysis, “Local Area Personal Income and Employment Methodology,” November 2012, p. I-9, available at BEA
  16. State Data Lab, “Glossary: Poverty Rate (CPS),” cited from the U.S. Census Bureau, accessed March 21, 2013, available at State Data Lab
  17. Bureau of Economic Analysis, “Regional Definitions: Per capita real GDP by state,” May 3, 2011, available at BEA
  18. Stephen C. Fehr, Pew Charitable Trust, “Infographic: S&P State Credit Ratings, 2001–2012,” July 13, 2012, available at Pew
  19. State Data Lab, “Glossary: Nominal GDP,” accessed March 21, 2013, available at State Data Lab
  20. State Data Lab, “Glossary: State Government Spending / GDP,” accessed March 21, 2013, available at State Data Lab
  21. Tax Foundation, “About Us,” accessed March 21, 2013, available at Tax Foundation
  22. Tax Foundation, “State and Local Tax Burdens: All States, One Year, 1977 - 2010,” October 23, 2012, available at Tax Foundation
  23. Tax Foundation, “Tax Freedom Day® by State: 2012,” April 2, 2012, available at Tax Foundation
  24. Tax Foundation, “Tax Freedom Day®,” accessed March 21, 2013, available at Tax Foundation
  25. State Data Lab, “Glossary: Unemployment Rate,” accessed March 21, 2013, available at State Data Lab
  26. Bureau of Labor Statistics, “Employment status of the civilian noninstitutional population 16 years of age and over by region, division, and state, 2011-12 annual averages,” March 1, 2013, available at BLS
  27. Michael P. McDonald, United States Election Project, “Voter Turnout,” accessed March 21, 2013, available at; see also McDonald, “Voter Turnout Frequently Asked Questions,” accessed March 21, 2013, available at George Mason University
  28. Gallup-Healthways Well-Being Index®, “Methodology,” accessed March 21, 2013, available at Gallup