Social science and discussion lists

[Volokh, Eugene]

        I appreciate Mark's points, but I'm not sure they're quite apt.

        (1)  Of course, it's better to read the entire study and then
respond to its substance rather than draw inferences about it based on
someone's post.  If we had world enough and time, we'd always do that!  And
surely if one wanted to write a scholarly piece about it, one would of
course read the study quite closely.

        But I think it may be a bit too much to demand the same
assiduousness from people who casually respond to a casual discussion list
post.  People on this list are pretty busy.  When I cite a study, I
generally assume that people will respond based on my post and their common
sense -- and in rare circumstances based on some relatively short piece of
text to which I link via a URL -- rather than doing any outside reading.  I
think that this is a fair assumption given the necessarily modest effort
that any subscriber can invest into list participation.

        (2)  Likewise, I don't think that my posts to which Mark refers
purported to be "social science," or could reasonably be characterized as
that -- but I am quite sure that I don't expect courts to rely on those
posts in making their decisions!  Thankfully, discussion list posts are not
judged by as demanding a standard as social science is.  Equally thankfully,
social science is not judged by as forgiving a standard as discussion list
posts are.  So I don't quite see what the off-handedness of some of the
assertions in my posts (and in others' posts) say about the question under
discussion, which is whether courts should rely on the study that Mark cited
and others like it.

        Eugene



Mark Tushnet writes:

>       Two points:  (1)  I would have thought that the way to criticize the
>
>       methodology of the study would be to, well, criticize the
> methodology
>       *of the study,* not of flaws inferred from my summary description of
> it.
>        That's why (among other reasons) I included the citation.
>
>       (2)  One has to put a lot of weight on the distinction between using
>
>       social science "in a wide-ranging way" and using it in some other
> way.
>       For example, in a recent thread Eugene Volokh relied on "social
> science"
>       -- based on his impressions of how universities operate, not on
> anything
>       remotely approaching a systematic study -- to support the
> counterfactual
>       claim that universities with policies against abusive uses of the
>       institution's e-mail system would have reacted differently to
> certain
>       hypothetical statements than the institution actually did to the
> ones
>       actually made.  (Although, I would note, all the institution did,
>       apparently, was receive a complaint through its system, open a file,
> and
>       perhaps ask for a response.)  Now, it seems to me that the *only*
>       sensible way to assess the counterfactual is to investigate how
>       institutions actually operate in a reasonably systematic way.
> Whether
>       that counts as "wide ranging," I don't know -- but (a) I'm
> reasonably
>       sure that it's better than relying on impressions gained from
> reading
>       the newspapers, going on the internet, and reading press releases
> from
>       (biased) interest groups, and (b) I'm also reasonably sure that
> relying
>       on social science in one way or another is inevitable, so the only
> thing
>       to do is make sure that the social science is good.
>
>       ----- Original Message -----
>       From: Ilya Somin <isomin at FAS.HARVARD.EDU>
>       Date: Tuesday, January 8, 2002 4:59 pm
>       Subject: Re: Constitutional social welfare rights?  [long]
>
>       > This is not quite on-topic (con law wise), but I will link it back
>
>       > to the
>       > topic  at the end:
>       >
>       > I think that this study, at least as described in Professor
>       > Tushnet's post,
>       > is not very good.  Since some of my Ph.D. dissertation work
>       > involves survey
>       > research, I though that I would comment on it (though I hasten to
>       > add that
>       > I'm not an expert on surveys on this particular subject).
>       >
>       > The study seeks to isolate the impact of social democracy vs.
>       > laissez faire
>       > by isolating it from individual-level variables such as education,
>
>       > marriage,and (presumably, though Prof. Tushnet doesn't list it),
>       > income. However,
>       > much of the impact of socioeconomic systems on individual
>       > happiness surely
>       > occurs THROUGH the effect of the system on individual-level
>       > variables. For
>       > example, one of the main traditional arguments for social
>       > democracy is that
>       > it increase the happiness of the poor by giving them a higher
>       > income, better
>       > health care, etc. Similarly, one of the main conservative
>       > arguments against
>       > the welfare state is that it deters marriage and encourages
> out-of-
>       > wedlockbirths. To put it in technical statisticial terms, the
>       > validity of this
>       > study is undermined by by serious problems of multicollinearity
>       > and by an
>       > apparent failure to recognize that some of the independent
>       > variables are
>       > also intermediate variables (to those who haven't taken statistics
>
>       > classes,I apologize for this little digression into statspeak; the
>
>       > point should be
>       > intuitively clear even without reference to these technical
> terms).
>       >
>       > A similar problem arises with the individualism-communitarianism
>       > issue. It
>       > seems likely, or at least possible, that social democratic
>       > policies, other
>       > things equal, increase the degree of communitarian values in a
>       > society while
>       > more free market policies promote individualism. European social
>       > democratsthemselves have historically argued that their policies
>       > are in part intended
>       > to promote "social solidarity." If these kinds of effects exist,
> the
>       > increased happiness observed in the US because of its
>       > individualism may be
>       > in part a result of its relative lack of social democracy.
>       > However, I don't
>       > put much stock in this argument - any more than in the argument
>       > that the
>       > study provides in favor of social democracy - because I think the
>       > overallresearch design is  so seriously flawed.
>       >
>       > Another flaw in the study is a common one in crosscultural survey
>       > research:self-reported "satisfaction" may mean different things in
>
>       > differentcultures.  Self-reported satisfaction is relative to
>       > social expectations:
>       > both expectations of how well off one expects to be and
>       > expectations about
>       > the extent to which people are expected to appear happy and
>       > contented when
>       > talking to strangers. To take an extreme example, an Afghan who
>       > lives in
>       > what in the West would be considered dire poverty might well
>       > truthfullydescribe himself as "satisfied" if asked in a survey.
>       > Some cultures have
>       > norms against revealing  dissatisfaction to outsiders; others have
>
>       > normsagainst saying that you're extraordinarily happy.
>       >
>       > In sum, I doubt that this study provides much in the way of good
>       > evidenceeither in favor of social welfare rights or against them.
>       >
>       > Now for the link to con law:
>       >
>       > This example illustrates two pit-falls of allowing courts to use
>       > socialscience evidence in a wide-ranging way. First, they may not
>       > be able to tell
>       > a good study from a bad one (studies arguably as poor as this one
>       > have in
>       > fact sometimes influenced court decisions). Second, not being
>       > trained social
>       > scientists, they are susceptible to overvaluing the results of a
>       > study even
>       > when the author of the study does not. Perhaps, for example, the
>       > author of
>       > the APSR article was cautious enough to note that this study by
>       > itself does
>       > not really provide a strong indepedent argument for more social
>       > welfarerights, but merely some relevant evidence. Are we sure that
>
>       > judges(including judges whose politics we disagree with) will be
>       > equally cautious?
>       > I don't believe that courts should never use social science
>       > research, but I
>       > think that such use should be tightly constrained by a healthy
>       > skepticism of
>       > both the quality of the social science involved (there are many
>       > more bad
>       > studies than good ones), and the ability of the judiciary to use
>       > it well.
>       >
>       >
>       > -----Original Message-----
>       > From: Mark Tushnet <tushnet at LAW.GEORGETOWN.EDU>
>       > To: CONLAWPROF at listserv.ucla.edu <CONLAWPROF at listserv.ucla.edu>
>       > Date: Tuesday, January 08, 2002 3:07 PM
>       > Subject: Re: Constitutional social welfare rights? [long]
>       >
>       >
>       > >Two points in response to Michael McConnell's observation about
> the
>       > >connection between interpretive openness and the democratic
> process:
>       > >(1)  A case like M.L.B. shows that -- descriptively -- the
>       > Supreme Court
>       > >is not committed to the view he offers -- as, indeed, would any
>       > case in
>       > >which the Court found a constitutional violation in circumstances
>
>       > where>he (or, if not he, at least a decent selection of
> knowledgeable
>       > >observers) would acknowledge the availability of alternative
> outcomes
>       > >reasonably supported by the legal materials (other than the sort
> of
>       > >thing I've introduced into the discussion).
>       > >
>       > >(2)  In any event, to the extent that one believes that there
> *are*
>       > >underenforced constitutional norms, or that the Constitution has
> some
>       > >bearing on decision-making by political actors other than judges,
> the
>       > >distinction between the democratic process and constitutional
>       > >interpretation collapses (with respect to such constitutional
>       > norms as
>       > >are applied by non-judicial actors).
>       > >
>       >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ucla.edu/cgi-bin/mailman/private/conlawprof/attachments/20020108/610836ed/attachment.htm