The 22 A* Journals from the ERA list vs. the 20 highest impact journals according to google scholar. A brief look and superficial analysis.

When worrying about the best place to send work one thing we can do is consult the various ranking lists of journals. Those of us seeking permanent jobs are often worried about how these lists are used in evaluating job applications- especially in philosophy, how could an ethicist judge the quality of a logicians work? But are the lists consistent with one another? If they are not then whether one gets an interview or not might be decided based on arbitrary tools.

 

Well here are two lists, the now defunct ERA list produced by government bodies in Australia and the rankings provided by google. From the ERA you see the 22 philosophy journals ranked A*, the highest possible ranking. From google the 20 journals with the highest h5-index scored. H-index being “h5-index is the h-index for articles published in the last 5 complete years. It is the largest number h such that h articles published in 2007-2011 have at least h citations each.”

 

So, first, the lists:

 

ERA:

American Philosophical Quarterly

Australasian Journal of Philosophy

Canadian Journal of Philosophy

Continental Philosophy Review

European Journal of Philosophy

Faith and Philosophy

Hypatia: a journal of feminist philosophy

Journal of Philosophical Logic

Journal of Philosophy

Midwest Studies in Philosophy

Mind: a quarterly review of philosophy

Monist: an international quarterly of general philosophical inquiry

Nous

Philosophers’ Imprint

Philosophical Review

Philosophical Studies: an international journal for philosophy in the analytic tradition

Philosophy and Phenomenological Research

Research in Phenomenology

Synthese

The Journal of Medicine and Philosophy: a forum for bioethics and philosophy of medicine

The New Yearbook for Phenomenology and Phenomenological Philosophy

The Philosophical Quarterly

 

Google:

Synthese 30

Philosophical Studies 25

Mind and Language 24

Nous 21

The Journal of Philosophy 21

Mind 20

Journal of Consciousness Studies 20

Philosophy Compass 19

Philosophy and Phenomenological Research 19

Phenomenology and the Cognitive Sciences 17

The Philosophical Quarterly 17

Political Theory 17

Philosophical Psychology 17

Ethics 16

Minds and Machines 16

Erkenntnis 16

Philosophical Review 15

Analysis 15

Philosophical Perspectives 15

Journal of Philosophical Logic 14

 

Less than half of the journals appear on both lists. The 9 which do being the journal of philosophical logic, the journal of philosophy, mind, nous, philosophical review, philosophical studies, philosophy and phenomenological research, synthese and the philosophical quarterly. But how meaningful is this difference?

 

Consider first Journals on the ERA list but not on google. We might think that some of the Journals which appear on the ERA list appear not to be on the google list as google doesn’t class them as philosophy journals. But so far as I can see this turns out not to be the case. Whilst hypatia appears on the feminism and women’s studies list (but not the gender studies list) it does so in virtue of an h5-index score of 14, meaning it ties with the journal of philosophical logic. Now ties on h5-index scores are broken by the h5-median or “h5-median for a publication is the median number of citations for the articles that make up its h5-index.” Journal of philosophical logic wins this by 22 to 16. In fact we see this for a number of other journals which barely miss out on the top 20:

 

Journal H5-index H5-median
Australasian Journal of Philosophy 14 20
Hypatia 14 16

 

 

 

Midwest Studies in Philosophy, Philosophers’ Imprint, The Journal of Medicine and Philosophy, The New Yearbook for Phenomenology and Phenomenological Philosophy do not have h5-index scores on google. Which makes it impossible to know if they belong on both lists. Given that the AJP and Hypatia tie with the Journal of Philosophical logic on h5 their omission doesn’t indicate a great difference in impact (actually including these makes both lists 22, but I can’t find a way to find all philosophy journals where h5=14, so there may be others that belong here too). Similarly for those journals that aren’t ranked on h5. That these journals don’t appear on google’s top 20 doesn’t indicate that they are substantially lower impact than those which do.

So let us expand the google list to 22, we can then reasonably include the AJP and Hypatia, giving us an overlap of 11 of 22, or exactly half. Also those journals not ranked by google gives us a large error, so we can only estimate the overlap as between 11 and 15.

 

This leaves 6 journals for us to examine. Are they, in fact, lower impact journals

 

Journal H5-index H5-median
American Philosophical Quarterly 8 11
Canadian Journal of Philosophy 9 14
Continental Philosophy Review 6 9
European Journal of Philosophy 9 15
Faith and Philosophy 4 5
Monist: an international quarterly of general philosophical inquiry 9 13

 

With scores ranging from 4-9, average 7.5 (a bit more than half the final journals which make the top 20- but the average is clearly influenced by the 4 from Faith and Philosophy) we might think that these journals are in fact lower impact, at least given google’s ability to find citations (which isn’t perfect). Based on this first glance we might think that the omission of these journals is meaningful and so that sometimes the method of generating the list does produce different results. So one might ask: does reputation or impact matter more? I suppose it depends on the employer.

 

Let’s then consider this from the opposite direction- what did the ERA think of the higher impact journals that they didn’t rank A*

 

 

Journal Rank
Mind and Language A
Journal of Consciousness Studies A
Philosophy Compass A
Phenomenology and the Cognitive Sciences A (but considered primarily a cognitive science journal)
Political Theory A* (but considered a political science journal)
Philosophical Psychology C (but considered primarily a psychology journal)
Ethics A* (but considered primarily a law journal)
Minds and Machines A (but considered primarily an AI journal)
Erkenntnis A
Analysis A
Philosophical Perspectives A

 

So we see first that the exclusion from the ERA list of Political Theory is an artefact of how the journals are classified, as such this is not an important difference. The same point is to be made for Ethics.

 

Philosophical Psychology is ranked C, the lowest possible ranking, along with graduate journals and those that never publish in English. This caused some controversy when the list was released and is likely an artefact of it being mistakenly classed as primarily a psychology journal. Indeed the journal was dropped from A to C between the last draft that was sent for consultation and the official “final” list. Even though this difference appears important it is in fact highly questionable.

 

Other than that we see the remaining journals that make the google top 20 that aren’t A* on the ERA list are given the second highest rating of A, meaning they are journals with a good reputation.

 

Overall then the differences between the lists which seem quite pronounced on first glance turn out to be artefacts of classification as not philosophy journals or near misses and so likely an artefact of using short lists of just 20-22 journals. The exceptions to this are philosophical psychology and faith and philosophy, as well as those four journals not ranked by google. For the latter four perhaps we must just wait for google to perform the calculations. Although I have had no contact with faith and philosophy I suspect the low ranking of philosophical psychology by the ERA was a simple mistake.

 

Given that these two lists do on inspection (all be it a superficial not statistics based inspection) appear quite similar. What does this mean? Just that my paranoia about potential employers using such lists to evaluate my work has dissipated slightly.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s