\documentclass[12pt,reqno,twoside,usenames,dvipsnames]{amsart}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{verbatim}
\hypersetup{breaklinks=true,
pagecolor=white,
colorlinks=true,
linkcolor= blue,
hyperfootnotes= true,
urlcolor=blue
}
\urlstyle{rm}
\newcommand{\margincomment}[1]
{\mbox{}\marginpar{\tiny\hspace{0pt}#1}}
\newcommand{\comments}[1]{}
\renewcommand{\baselinestretch}{1.2}
\parindent18pt
\parskip 10pt
\begin{document}
\titlepage
\vspace*{12pt}
\begin{center}
{\large {\bf Notes for : ``How Should We Measure Polarization in Congress?''
}}
January 20, 2020
\bigskip
Eric Rasmusen
{\it Abstract}
\end{center}
\begin{small}
Suppose we have an index of left-right ideology for all members of Congress, and they are in two parties, Democrat and Republican. What do we mean by saying Congress is more polarized than ten years ago?
\noindent
Eric Rasmusen:
Professor, Department of Business Economics and Public Policy, Kelley
School
of Business, Indiana University, Bloomington Indiana.
\href{mailto:erasmuse@indiana.edu}{ Erasmuse@indiana.edu}
Phone: (812) 345-8573. This paper:
\url{http://www.rasmusen.org/papers/sdfsdf.pdf}.
I would like to thanks Aaron Kolb, Michael Rauh, Ken Shepsle, Arthur Silve, and the Frieden Political Economy Lunch for helpful comments.
\end{small}
\newpage
\noindent
{\bf 1. Introduction}
Lord Kelvin famously said that if somthing can’t be measured numerically, it doesn’t exist. What he meant was that if we can’t figure out a way to put a number on some concept, that’s a good sign we don’t know what we mean when we use that concept. “Polarization” is a good example. When we say that Congress is highly polarized, what do we mean?
The vague meaning is that the members of Congress are aligned into two groups that vote together a lot. Of course, that is what having parties does, so one might say that Congress is always polarized. The Founders had hoped to avoid parties—or “polarization” or “factionalism”, to use more colored language—but we’ve long had them. On the other hand, it’s quite possible to have political parties without polarization. That’s what happens if the parties have loose discipline and no ideological content. Local government is often like that. Machine politics can be like that too: the parties are business organizations intended for the material profit of their members, not for high policy preferences. The Japanese factions within the Liberal Democratic Party that ruled Japan from the 1950’s to the 1990’s were like that: well-defined factions around political leaders who differed little or not at all on policy.
And polarization is not just a binary concept. The idea does require that there be a discrete number of poles, if not necessarily (except for language purists) just two poles. But we can speak of Congress becoming more polarized without it moving from unpolarized to polarized: the measure should be continuous.
When we think of parties and polarization, there are two distinct ways Congress might become more polarized:
1. The average opinions diverge more: The average Democratic ideological position becomes further from the average Republican one. Call this ``Party Polarization''? This is easily measured by the difference in means, $|\mu_r- \mu_d|$.
2. The clusters become more dispersed and more overlapping: more Democrats are close to more Republicans and vice versa. Call this ``Cluster Polarization''? This is measurable, sort of, by the sum of the standard deviations, $\sigma_r + \sigma_d$, but that is not satisfactory, because that would be positive even if each party has identical members.
\includegraphics[width=\linewidth]{polarization4.png}%fig 1
\includegraphics[width=\linewidth]{polarization1.png}%fig 2
The essential problem comes up even in one dimension. Suppose ideology is from left to right, with 0 in the center. Let the Democratic congressmen be located at circles $d_1, \ldots d_D$ and the Republicans at dots $r_1, \ldots r_R$, with mean values $\mu_d$ and $\mu_r$ as shown in Figure 2.
Figure 1 shows a Congress in which the parties diverge in their averages. Figure 3 shows a Congress in which the party averages are close but tightly clustered. Which Congress is more polarized?
\includegraphics[width=\linewidth]{polarization3.png}%fiure 3
Shelly Connell suggested that we could come up with one measure for Party Polarization and another for Cluster Polarization and then take a weighted average depending on which we thought was more important to the concept. In fact, we could multiply them, perhaps with exponents for weights, and then if $\mu_r=\mu_d$ the result would be a measure of zero. But I think we can do better, by finding some kind of truly scalar, one-number, measure.
Consider the measure $P^1$:
\begin{equation} \label{e0}
P^1 \equiv \frac{1}{2} \cdot \frac{ \Sigma_{i=1}^D(d_i - \mu_r)^2/D} { \Sigma_{i=1}^D (d_i - \mu_d)^2/D} +\frac{1}{2} \cdot \frac{ \Sigma_{i=1}^R(r_i - \mu_d)^2/R} { \Sigma_{i=1}^R (r_i - \mu_r)^2/R}
\end{equation}
$P^1$ has the idea, but we should do some normalization. Subtract 1, so a random Congress has polarization $P=0$. Put in square roots, for the same reason we use standard deviation instead of variance (it restores original units that got squared). How about this?
\begin{equation} \label{e0}
P^2 \equiv \frac{1}{2}\cdot \frac{ \sqrt{ \Sigma_{i=1}^D(d_i - \mu_r)^2/D}} { \sqrt{ \Sigma_{i=1}^D (d_i - \mu_d)^2/D}} +\frac{1}{2} \cdot \frac{ \sqrt{ \Sigma_{i=1}^R(r_i - \mu_d)^2/R}} { \sqrt{ \Sigma_{i=1}^R (r_i - \mu_r)^2/R}} - 1
\end{equation}
We ought to also do something to squish the top end, so the measure goes from 0 to 1 instead of 0 to infinity. So maybe the following is better:
\begin{equation} \label{e0}
P^3 \equiv 1 - \frac{1}{ \frac{1}{2}\cdot \frac{ \sqrt{ \Sigma_{i=1}^D(d_i - \mu_r)^2/D}} { \sqrt{ \Sigma_{i=1}^D (d_i - \mu_d)^2/D}} +\frac{1}{2} \cdot \frac{ \sqrt{ \Sigma_{i=1}^R(r_i - \mu_d)^2/R}} { \sqrt{ \Sigma_{i=1}^R (r_i - \mu_r)^2/R}} }
\end{equation}
This might be easier to read if we define
\begin{equation} \label{e0}
\sigma_{d} \equiv \sqrt{\frac{ \Sigma_{i=1}^D(d_i - \mu_d)^2}{D}}, \;\;\; \sigma_{d-r} \equiv \sqrt{\frac{ \Sigma_{i=1}^D(d_i - \mu_r)^2}{D}},
\end{equation}
with the equivalents for $\sigma_{d}$ and $\sigma_{r-d} $ (which does NOT equal $\sigma_{d-r} $). Then we have
\begin{equation} \label{e0}
P^3 = 1 - \frac{.5} { \frac{ \sigma_{d-r}} { \sigma_d } + \frac{ \sigma_{r-d} } { \sigma_r } }
\end{equation}
$P^2$ and $P^3$ have some desirable properties. \\
1. Two parties with identical means have zero polarization. \\
2. Polarization rises with the distance between party means.\\
3. Polarization goes to the maximum level ($\infty$ or 1) as the distance between party means goes to $\infty$. \\
4. Moving a party member closer to the party mean increases polarization FALSE? Suppose we move a very leftwing democrat closer to the party mean--that *could* reduce polarization. Maybe. Should it? Hard to say. \\
Consider where there are three clusters. There is a cluster of leftwing Democrats around -4, a cluster of moderate Democrats around -2, and a cluster of Republicans around +5. We merge all the Democrats around -2. I think this would *increase* polarization.
I should look at Silicon Valley articles on agglomeration. How do they measure it?
Compare with the Herfindahl Index.
One big question is how to measure polarization of a single group of individuals. How do we split them into two groups and measure how polarized our result it? I start with the two groups as given. The problems are closely related. I could take Congressional data nad then show to to split into two gropus ex ante.
``Polarization'' comes from ``poles''. There are only two poles ,whether to a magnet or a planet. For a group to be polarized is not the same as saying a group has divergent views. Use examples to illustrate that.
Is ``entropy'' connected with this ?
Can I get my measure to be homogeneous of degree zero?
A two step way to do this is to look at total variance, and then at within-party and between-party variance. ANOVA.
Francesco thought there was a theorem that a two-cluster splitting that minimizes the sum of squared errors from each mean is the same as two-clsuter splitting that maximizes the sum of squared errors from each Republican to each Democrat. Look at
$$
\Sigma_{i=1}^D\Sigma_{j=1}^R (x_i-x_j)^2
$$
I think we can get these properties from many different functions--- indeed, maybe from any strictly monotonic transformation of $P^2$, including $P^1$. We could, for example, even keep the same nice normalizations if we define $\sigma_{d-r}$ using absolute deviations instead of squared deviations, as
\begin{equation} \label{e0}
\sigma_{d}' \equiv \frac{ \Sigma_{i=1}^D |d_i - \mu_r|}{D}
\end{equation}
Is there any reason to prefer $P^3$ except for the fact that it ``feels'' better to someone trained in statistics who is used to the nice maximum likelihood properties of squared deviations?
One problem with these measures is that they are undefined if $\sigma_d = 0 $.
I need to figure out how to use python to plot the distributions.
Prof. Chris Connell of Indiana Math suggests that something might be done with correlations instead of with variances, but he didn't have anything particular in mind. Here, however, we don't match up observations as when correlations are ordinarily used--- we aren't comparing the Republican candidate in a particular election with the Democrat, for example. Note, too, that we are not talking about samples, but populations--- we are just interested in a particular Congress, and we observe every single member, not just a sample. That's why I have $\mu$ and $\sigma^2$, not $\bar{x}$ and $s^2$, and use $D$ instead of $D-1$ (though the $D$'s cancel out anyway).
I was going to go a step further and adapt this to 3 or more parties, but I won't, because that introduces a new problem: a Progressive might be closer to the Republican mean or the Democratic one, and which shall we use? So I will defer that. Let's see if $P$ makes sense. Let's take some examples.
It would be good to look at properties of the measure, whatever it is, to match them up with what we think is the idea of polarization. We could, for example, show that whenever a Republican move closer to the Republican mean, our polarization measure increases. Another approach is to look at some Congress distributions and see if the difference in the measures matches our intuition for which is more polarized. I do a little of that in the Table, using the Python program in the appendix that I wrote with Ben Rasmusen for the computations.
\hspace*{-72pt}
\begin{small}
\begin{tabular} {|l|llll |l | lll | l|lll l |l l|}
\hline
& & & & & & & & && && && & \\
Congressmen: & $d_1$ & $d_2$ & $d_3$ & $d_4$ & & $r_1$ & $r_2$ & $r_3$& & $\mu_d$ &$\mu_r$ & $\sigma_d$ & $\sigma_r$ & $P^2$ & $P^3$ \\
& & & & & & & & && && &&& \\
\hline
& & & & & & & & && && &&& \\
Congress 1: Unpolarized & -2 & -1& \;1 & \;2 & & -3 & \;0 & 3 & &\;0 & 0 &\;2.5 &6& 0.0& .00 \\
Congress 2: Polarized both ways & -4 & -3 & -3 & -2 & & \;2 & 3 & 4 & & -3 & 3 &0.7 & 0.7& 7.0 & .97\\
Congress 3: Parties differ & -6 & -6 & \;0 & \;4 & & \;0 & 2 & 7 & & -2 & 3 & 4.2 & 2.9 & 0.8 &.86\\
Congress 4: Congressmen concentrated & -1 & -1 &\;0 & \;0 & & \;1 & 1 & 2& & -0.5 & .5 &\;.5 &0.7& 2.9 & .94\\
& & & & & & & & && && && &\\
\hline
\end{tabular}
\end{small}
\begin{quotation}
1. The difference between the location of the median Democrat and the median
Republican. This measure gets at one aspect of inter-party heterogeneity.\\
2. The ratio of the standard deviation of ideal points in the Democratic party to that of
the full House, which indicates variation in intra-party homogeneity.\\
3. The proportion of overlap between the two parties’ distribution of ideal points
subtracted from one. Overlap is measured by the minimum number of ideal points
that would have to be changed to yield a complete separation of the two parties, with
all Democrats' ideal points being to the left of all Republicans’ ideal points.\\
4. The R2
resulting from regressing the Member’s ideal point location on party
affiliation.
\end{quotation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\newpage
\newpage
Appendix: The Python program for computing P
\begin{verbatim}
import numpy
##############################
d2 = [-4,-3,-3,-2]
r2= [2,3,4]
d3=[-6,-6,0,4]
r3= [0,2,7]
d4=[-1,-1,0,0]
r4= [1,1,2]
##########################
d=[-1,-1,0,0]
r= [1,1,2]
congressnumber = 4
#########################
def own_variance(party):
k3 = 0
for item in party:
k = item - numpy.mean(party)
k2 = k**2
k3 = k3 + k2
k3 = (k3/len(party))**(1/2)
return k3
DD_sd = own_variance(d)
print(f"The Democrats' standard deviation is {DD_sd}.")
RR_sd = own_variance(r)
print(f"The Republicans' standard deviation is {RR_sd}.")
###############################
def other_variance(party, rivals):
k3 = 0
for item in party:
k = item - numpy.mean(rivals)
k2 = k**2
k3 = k3 + k2
k3 = (k3/len(party))**(1/2)
return k3
DR_sd = other_variance(d ,r )
print(f"The Democrats' standard deviation from the Republican mean in Congress {congressnumber} is {DR_sd}.")
RD_sd = other_variance(r ,d )
print(f"The Republicans' standard deviation from the Democrat mean in Congress {congressnumber} is {RD_sd}.")
#################################
def PP2 (DR_sd, DD_sd, RD_sd, RR_sd):
local1 = (1/2) * (DR_sd/DD_sd) + (1/2)*(RD_sd/RR_sd) -1
return local1
P2 = PP2(DR_sd, DD_sd, RD_sd, RR_sd)
print(f"The P2 polarization of Congress {congressnumber} is {P2}.")
def PP3 (DR_sd, DD_sd, RD_sd, RR_sd):
local1 = 1 - .5 / ((DR_sd/DD_sd) + (RD_sd/RR_sd))
return local1
P3 = PP3(DR_sd, DD_sd, RD_sd, RR_sd)
print(f"The P3 polarization of Congress {congressnumber} is {P3}.")
\end{verbatim}
\bigskip
\noindent
{\bf References}
Aldrich, John H. \&
David W. Rohde (19xx) The Logic of Conditional Party Government:
Revisiting the Electoral Connection,''
Aldrich, John H. \&
David W. Rohde (1998) ``Measuring Conditional Party Government,'' January 1998. Paper prepared for delivery at the Annual Meeting of the Midwest Political Science Association,
Palmer House Hotel, Chicago, Ill., April 23-25, 1998.
CANEN, Nathan, CHAD KENDALL \& FRANCESCO TREBBI,
UNBUNDLING POLARIZATION,'' Econometrica, Vol. 88, No. 3 (May 2020), 1197–1233.
Cox, Gary W. \& Keith T. Poole (2002)
``On Measuring Partisanship in Roll-Call Voting: The U.S. House of Representatives, 1877-
1999,''
American Journal of Political Science , Jul., 2002, Vol. 46, No. 3 (Jul., 2002), pp.
477-489,
https://www.jstor.org/stable/3088393.
Desmet, Klaus,
Ignacio Ortuño-Ortín \&
Romain Wacziarg,
``Culture, Ethnicity, and Diversity,''
AMERICAN ECONOMIC REVIEW
VOL. 107, NO. 9, SEPTEMBER 2017
(pp. 2479-2513,
https://www.aeaweb.org/articles?id=10.1257/aer.20150243.
Polarization: Concepts, Measurement, Estimation,'' Econometrica, Vol. 72, No. 6 (November, 2004), 1737–1772.
Jean‐Yves Duclos Joan Esteban Debraj Ray
2004 https://doi-org.proxyiub.uits.iu.edu/10.1111/j.1468-0262.2004.00552.x
Chapter
Comparing Polarization Measures1
Joan Esteban
and
Debraj Ray
January 2010
http://www.socsci.uci.edu/~mrgarfin/OUP/papers/Esteban.pdf
A Disconnect in Representation? Comparison of Trends
in Congressional and Public Polarization
Seth J. Hill, University of California, San Diego
Chris Tausanovitch,
The Journal of Politics, volume 77, number 4. Published online August 13, 2015. http://dx.doi.org/10.1086/682398
Krehbiel, Keith (2000) ``Party Discipline and Measures of Partisanship,''
{\it American Journal of Political Science }, Apr., 2000, Vol. 44, No. 2 (Apr., 2000),
pp. 212-227,
https://www.jstor.org/stable/2669306.
\end{document}