Academia Striving for Relevance and Useful Knowledge –from Anomalous Citations , Transparency,Open Access (OA) and Open Peer Reviewing to New Article Formats, Microarticles and Data in Brief.Professor Luiz Moutinho

Foundation Chair of Marketing

Adam Smith Business School

University of Glasgow,

Scotland

How does science progress or stagnate)?

“We know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong”. Richard Horton, editor of the prestigious medical journal The Lancet.

Some sociologists of science argue that peer review makes the ability to publish susceptible to control by elites and to personal jealousy and personal interests. The peer review process may suppress dissent discovery and findings against “mainstream” theories. Reviewers tend to be especially critical of conclusions that contradict their own views and theory, and lenient towards those that accord with them

At the same time, established scientists are more likely than less established ones to be sought out as referees, particularly by high-presitge journals or publishers. As a result, it has been argued, ideas that harmonise with the established experts are more likely to see print and to appear in prestigious journals than are iconoclastic or revolutionary ones.

Scientific journals ask also for money from Universities, Research Institutions and anyone wishing to access to the full content of their journals, the online version! Subscriptions to Scientific Journals are amazingly high and totally unjustified! Scientific researchers done by Academia are mainly funded by public grants, therefore the results of research are a Public Good! And the access to these results should not be blocked by “Journals” which are private companies!

Only 8% members of the Scientific Research Society agreed that ‘peer review works well as it is.’ (Chubin and Hackett, 1990; p.192)

An exponentially increasing number of studies and experience-based editors’ opinions are clear and explicit about peer review weaknesses and failures. “A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research. Far from filtering out junk science, peer review may be blocking the flow of innovation and corrupting public support of science.” (Horrobin, 2001)

Horrobin concludes that peer review “is a non-validated charade whose processes generate results little better than does chance.”

“If peer review was a drug it would never be allowed onto the market” affirmed Drummond Rennie (Smith, 2010, p.1), deputy editor of the Journal of the American Medical Association.

It “would not get onto the market because we have no convincing evidence of its benefits but a lot of evidence of its flaws.”

The University of Pennsylvania marketing professor J.Scott Armstrong conduced a meta-analysis of studies of peer review conducted to determine whether there was, in fact, a systematic problem. He concluded that the peer-review system was highly unfair and discouraged innovation. Part of the reason stemmed from known bias: papers from more famous institutions, for instance, were judged more favourably than those from unknown ones, and those authored by men were viewed more favourably than those by women if the reviewers were males, and vice versa.

But others had to do with the less visible bias of belief. “Findings that conflict with current beliefs are often judged to have deficits”,Armstrong wrote.

One early study had psychologists review abstracts that were identical except for the results, and found that participants “rated those in which the results were in accord with their own beliefs as better.” Another found that reviewers rejected papers with controversial findings because of “poor methodology“ while accepting papers with identical methods if they supported more conventional beliefs in the field. Yet a third study involving both graduate students and practicing scientists, showed that research was rated as significantly higher in quality if it agreed with the rater’s period beliefs.

How Peer Review is Evolving to Meet New Article Formats

A new kind of article is becoming increasingly popular on the publishing scene; microarticles are short papers containing useful data, descriptions or other valuable research output that may otherwise go unpublished.

Elsevier has invested in a number of journals to accommodate this new format. February, 2014 saw the launch of the open access journal, MethodsX, which publishes the small, but important, customisations researchers make to methodological protocols.

MethodsX

Because this journal is a true community initative - a journal ‘by researchers,for researchers’ – it was quickly decided that the reviewers needed to be involved and acknowledged in a way that differs from most journals. They also require minimal time investment from the reviewers while providing meaningful feedback to authors.

The MethodsX credo: Acknowledgement and Transparency

They also appreciate the reviewers who provide valuable input to each submission. The journal therefore publishes a reviewer “thank you” note in each article, and reviewers are given the option to be individually named in the acknowledgment.

Of course, if reviewers wish to remain anonymous that is not a problem, but they are happy to see that some reviewers have already chosen to have their names published. This not only makes the process more transparent but allow them, as a journal, to formally thank them for their feedback. They also invite reviewers to submit an article to MethodsX with the publication fee waived.

Genomics Data –

Scientists will often publish an article about only a small piece or particular aspect of their data, leaving the rest ripe for interpretation by others.

Genomics Data’s signature ‘Data in Brief’ articles aim to solve these problems. They provide an avenue for researchers to bring their data -along with the details necessary to understand and reuse it – to the forefront.

Data in Brief – How it works

The research article:an interpretation of the dataThe Data in Brief articles support these elements by providing a thorough description of the data, including quality control checks and base- level analysis.

Data in BriefData in Public

Repository

Research Article(s)

New Insights & Interpretations

Link data and description

Describe the data Interpret the data

Reanalyze

ReuseHow are Data in Brief articles

reviewed?The EB IS RESPOSIBLE FOR THE

REVIEW.Keeps AVERAGE DECISION TIME -1 week.

Reinterpret

Reproduce

Peer review for Results in Physics’ microarticles

All microarticle manuscripts still need to undergo peer review and that process is organised by the journal Editor-in-Chief, Professor Jurgen Buschow. The peer-review process is similar to the one followed for a regular research paper, but the instructions to reviewers differ slightly. Each reviewer receives a short list of questions that require them to judge whether the information in the article is:

- new

- well-described

- scientifically sound

The result is that reviewers are not asked to judge the relevance or importance of a microarticle – that is left up to the readers to decide.

Most articles claim contribution to ‘knowledge’, but it is knowledge nobody is interested in! Far too few journal articles have original ideas, because these have been shot down by reviewers and/or editors who are worried about not being scientific enough, and the most innovative ideas cannot be ‘scientifically proven’.

It is a sad fact, that we are writing for a smaller and smaller ‘mutual admiration club’ of colleagues within a very specialised tribe. Unless we address this issue, some day very soon, somebody cries ‘the emperors new clothes’. Will the taxpayer, the students or industry pay us to do research, which has no impact on economic growth, higher efficiency of our companies and government, more innovation, more sustainability etc.?

And one can only see that it is getting worse in the sense that the (mainly in person hours) are increasing (must be way over $100.000 per article) and less and less relevant.

While brilliant and progressive research continues apace here and here, the amount of redundant, inconsequential, and outright limited research has swelled in recent decades. As a result, instead of contributing to knowledge, the increasing number of low-cited publications only adds to the bulk of words and numbers to be reviewed. Reviewer time and energy requirements multiply by the year. The impact strikes at the heart of academia.

Aspiring researchers, are turned into publish-or-perish entrepreneurs, often becoming more or less cynical about the higher ideals of pursuit of knowledge. They fashion pathways to speedier publication, cutting corners on methodology and turning to politicking and fawning strategies for acceptance.

Academia should revert to its paper focus on quality research and rededicate itself to the sober pursuit of knowledge.

In an era of questioned scientism and methodolatry, with an over-reliance of the same limited methods of research and the use perennial instruments, a new dawn of polymeaures is emerging. Many of them are in the realm of biometrics,physiology, psychophysics and neuroscience.

Journal Ratings are Over-Rated

Journal rankings are mostly a relic of the 20th century. The major legitimate function that ratings of journals used to serve was predicting exposure of an article.

The more dubious use of journal rankings that continues today is that administrators use journal ranking as a surrogate measure for quality of scholarship. Admittedly this practice is limited to administrators who lack the skills to judge an article based on its own merits, but it is sadly more common than one might hope.

Decisions about tenure, promotion, and pay raises use rankings of journals as the measure of quality, in spite of the obvious flaws. Since many “top” journals still publish about the same number of articles in spite of the increase in the number of articles generated and submitted, one can assume that some articles published outside the “top” journals now have quality comparable to the ones that previously appeared in “top” journals.

One crisis across social science is that little agreement exists regarding what constitutes quality. Many “top” journals reject 90% or more of the articles submitted.

Standards of quality are far from universal. The academic community needs to recognise the silliness associated with many of the uses of journal ranking.

Academic journals should change practices so that the materials they publish would make meaningful contributions and have the needed, detailed backup available online.

Citation numbers, for their part, often have less to do with quality of the work than the amount being done in that area. Impact factors and citation stats are self-fulfilling prophecies that feed into the insecurities of the academic who wants to be lauded with acceptance and labels of rigour.

The reality for journal editors today is to chase rankings and impact factors. A journals impact factor is highly subjective and arbitrary. And who are those people making decisions, how do they get their data and how do they analyse and interpret them? The whole bibliometrics systems is a bureaucratic monster that is taking away more and more time for serious and innovative research.

It is a standardisation machine hitting an area, science, which is meant to be diverse and risk-taking to find new knowledge. But when we live in a forcing system we are stuck with it – or leave academic research. Editors devote most of their reports to Editorial Boards to these rankings instead of discussing content and future decisions.

What do“anomalous citations” mean?

The more important is that articles based on obsolete theory, formalistic and shallow surveys and with more space devoted to long statistical tables with fancy indexes (sometimes with 3 decimals) that nobody reads, ending with a section telling how useful this is for practitioners and researchers, at the same time saying that the research has limitations that the authors expect others to handle. Many journals currently have their share of this and it should e reduced; it has no scientific value. (Evert Gummesson 2014).

Does an academic paper contain useful knowledge? (Scott Armstrong 2004).

Sick of Impact Factors ...... and so is science.

The impact factor might have started out as a good idea, but it is time has come and gone. Conceived by Eugene Garfield in the 1970s as a useful tool for research libraries to judge the relative merits of journals when allocating their subscriptions budgets, the impact factor is calculated annually as the mean number of citations to articles published in any given journal in the two preceding years.

By the early 1990s it was clear that the use of the arithmetic mean in this calculation is problematic because the pattern of citation distribution is so skewed. Analysis by Per Seglen in 1992 showed that typically only 15% of the papers in a journal account for half the total citations. Therefore only this minority of articles has more than the average number of citations denoted by the journal impact factor. Take a moment to think about what that means: the vast majority of the journal’s papers – fully 85% - have fewer citations than the average. The impact factor is a statistically indefensible indicator of journal performance; it flatters to deceive, distributing credit that has been earned by only a small fraction of its published papers.

But the real problem started when impact factors began to be applied to papers and to people, a development that Garfield never anticipated.

We submit to time-wasting and demoralising rounds of manuscript rejection, retarding the progress of science in the chase for a false measure of prestige.

A possible solution to the impact factor conundrum is the rise of mega-journals, which publish exclusively online and judge papers only on their novelty and technical competence, and in the potential of article-level metrics to assess the scientific worth of papers and their authors.

And now we have the rising tides of open access and social media. The quality control provided by peer-review expert assessment of manuscripts before publication should simply be a technical check on the work, not an arbiter of its value.

The rate of accrual of citations remains rather sluggish, even in today’s weird world, so attempts are being made to capture the internet buzz that greets each new publication; there are interesting innovations in this regards from the likes of PLOS, Mendeley and altmetrics.org.

These developments go hand in hand with the rise of open access (OA) publishing.

If you use impact factors you are statistically illiterate... …..If you include journal impact factors in the list of publications in your cv, you are statistically illiterate……Strong but true.

(Professor Stephen Curry,Chair of Structural Biology, Imperial College).

Open Peer Review is Finally Available for Scientists!

Online Social Network Seeks to Overhaul Peer Review in Scientific Publishing.

Three Finnish researchers have created an online service that could eventually replace or supplement the current way journals get scientists to peer review submitted manuscripts. Peerage of Science is an innovative social network of scientists to which researchers submit their manuscripts; other members with relevant expertise, altered by keywords in the papers, will then provide reviews that scientific journals can use to decide whether to publish the work.

The current peer review system in which journal editors send potentially publishable manuscripts to experts for review is hotly debated. Many scientists complain that the system is slow, inefficient, of variable quality, and prone to favouritism. Moreover, there is growing resentment in some quarters about being asked to take valuable time to provide free reviews to journals that are operated by for-profit publishers or that do not make their papers open-access. Several suggestions have been made to improve the peer review system, such as introducing credits for reviewers, using social media, and making the process more transparent. Scientists receive one credit for every review they finish, These credits are required to upload a manuscript, which costs two credits divided by the number of co-authors.

The relationships between theory and practice, research and action, and basic and applied knowledge are fundamental to the field of Management.

Engaged scholarship is a participative form of research for obtaining the views of key stakeholders to understand a complex problem or research question. By exploiting differences between these viewpoints, engaged scholarship produces knowledge that is more penetrating and insightful than when researchers work alone.

Engaged scholarship can be practiced in many different forms, including: (1) doing basic science with stakeholder advice, (2) co-producing knowledge with collaborators, (3) evaluation research for professional practice, and (4) action/intervention research for a client showing there are many ways to become involved as a researcher.

Academia Strives for Relevance

Are business Schools relevant? Given the expansion of management education in recent years, the question may seem moot. But, with critics continuing to query the real-world value of research and teaching, relevance has remained an issue for School administrators.

Business Schools have been criticised for focusing on peer-reviewed research at the expense of applied studies. We have inherited a structure of rewarding research excellence in particular that can have a very damaging practical effect on the work of a business school.

• We have created a system in which research has much greater incentives and rewards than teaching which can have repercussions for our universities.

Dan LeClair, senior vice president at the Association to Advance Collegiate Schools of Business (AACSB), which accredits more than 500 institutions worldwide, says deans are under more pressure than ever to justify what they do. “The deans have been telling us that major donors are asking tough questions like ‘you have all these faculty members who you are very proud of, but can you tell me how this research has made a difference?”.

“It is also the alumni and even the provosts and presidents of the institutions. They are all asking schools to not only describe what they are trying to achieve, but also to demonstrate it”.

Business Schools are frequently criticised for over-emphasing academic rigour over relevance to practice. And many believe the structures of the business school world feed the tendency: that promotion is based on articles few managers read; and that accreditation bodies and rankings providers count journal entries, and citations, to assess worthiness.

ESADE Business School, in Barcelona, has developed the Creapolis “innovation park” where companies and faculty work collaboratively on so-called “action research” projects. ESADE dean Alfons Sauquet says he is trying to encourage “team-playing in research”, mixing up faculty and companies. “It is pushing people forward in a transversal area of knowledge, and helping them a lot to become more interactive and balanced.”

• Market shifts mean a bleak future for Business Schools • Waning student appetite for academic programmes. Demand for

more practical studies.• Prestigious MBA courses and even entire Business Schools will

close as universities struggle to carve out a future for business education.

• Higher education is about expanding the mind rather than being directly over-specialised.

• Relating something about business to people who do not have any business experience.... is more than a challenge.

• Central elements of academia, such as the need to publish research, means that scholars might be ill -placed to provide the business education that students want.

“Publish or Perish” becomes the motto that business schools work under and will continue to work under in any university setting, so there is a mismatch between the faculty and what the students and participants are looking for.

Peters (2010) asked MBA Directors how much of their student syllabus was influenced by their own research, and found that the answer was less than 10 percent. More than 90 percent of teaching covers material common to all business students.

The structure of business education could change dramatically over the next few years, with the less well-known university departments in the line of fire.

Is a business school a professional school or is it an academic school? That is part of the whole debate.

The evolution of the business school would reset “hybrid institutions” that do more to offer both academic study and workplaced -centred expertise.

Business Schools operate at a different pace from the communities they serve in business and government, which often frustrates business leaders who need answers to today’s problems today. Most stakeholders also overlook future scenarios.

Management education appears slow to respond to new business challenges.

We need to strengthen advocacy efforts aimed at helping both communities to understand the value of an enhanced partnership between management educators and leaders of business.

Visions for UW-Stout’s Future

• Create a “Convenience University”

• Create an “Alumni Skills Guarantee”

• Create a “Stout Associates Program”

• Create a “Centre for Global Marketplace Development”

How can business leaders participate?

• Governance Process• Task Forces• Accreditation Process Teams and Committees• Conference Presenters and Participants• Curriculum development• Research Areas and Contexts• Assurance of Learning

New Teaching Formats/Content/Delivery

Format

• “Pick, Mix and Match”

• Learning Zones

• Focus on Industry – Relevant “Blockings” (outwith management) (double dose, small dose, Master, Executive Education Levels)

Examples

Pharmaceuticals Airlines Aircraft

Chemistry /IBLS Tourism/Geography Aeronautical/ Mechanical

Engineering

• Cafe-Seminars / Knowledge Cafes• International Masters Consortium (IMC) (JV)

• Discovery Campus Masterschool

• Foresight Component

LWI – Strategic Thinking• Business Retreats• “Think Tanks” – OOB (out of he box) “brain monitoring”• Do Tanks• Management Roundtables (Short “TTS”)• Integrated Workshop Series• Management Factory / Living Lab• Dinners for Reflection• Thought Leader Panels• The New Management Experience

LWI

Connection Academia-Enterprise

• Sabbaticals for Managers• Micro partnerships (reverse sabbaticals)• Management Kiosks• “Great Management Alliance” (GMA) (Government-sponsored,

SME clusters, executives from LCs, PGS-Junior consultants)• Management Club• Corporate Partnerships (course and venture-based)• Series of Local Workshops by Management theme / Industry

sector with Partner Organisations (OCA)

Connection Academia-Enterprise

Industry Dominoes–C-Fertilisation of Ideas/Technologies–C-Industry Expertise Transfer–C-Management Knowledge–CF Experiences

Entrepreneur-in-residence

Cross-Functional ExperiencesIntegrated Management

Connection Academia-Enterprise collector,analyst,facilitator,

disseminator,catalyst.

Enterprise Forum (Corporate DNA Project) RU/C(Government/Consortium Sponsored)

(

Dissemination of research and information

Cross-Management Knowledge(Pools of Knowledge) (Corpknow)

Best Virtual Companies

Mock Co. LabsExperimental Research Best Practice in

Management

Management Files

Cross-Fertilisation of Technologies

Cross-Industry Expertise Transfer

Raw Data

One can measure the importance of a scientific piece of work by the number of earlier publications rendered superfluous by it.

(H. Eves)