The Pain of Complexity

Dmitry ZinovievSuffolk University, Boston

Complexity is one of those fuzzy concepts that everyone seems to understand differently. Other examples: sustainability, resilience, success...

What Is Complexity?

Let's Ask the Mechanical Turk!

Question: Enter three nouns (please, nouns only!) that, to the best of your understanding, are associated with complexity. These words could be synonyms, descriptors, attributes of complexity, actions or any other related words.

Ask 100 mTurk workers, pay a nominal fee for an answer

Reject non-nouns; stem* and aggregate nouns

*We tried both Porter and Lancaster stemmers; Porter is less aggressive, it produced more stems (motifs); some manual tweaking was needed

Direct Survey Results

The stems that were mentioned at least twice (in the order of decreasing frequency):Most frequently mentioned (23+ times): COMPLIC*, DIFFICULTI*, INTRICACI*

Less frequently mentioned (29 times): PROBLEM*, ELABOR*, ENTANGL*, PUZZL*, MULTIPL*, COMPLEX*, COMPUT*, CONVOLUT*, HARD*, RAMIF*, SOPHIST*, CHALLENG*, HUMAN*, INVOLV*, LIFE*, POLIT*, RIDDL*, SCIENC*, COMPOSIT*, CONFUS*, DEPTH*, DESIGN*, DIVERS*, LABYRINTH*, LANGUAG*, MATH*, MAZE*, OBSCUR*, PHYSIC*, QUALITI*, RELATIONSHIP*, SPACE*, STRUCTURE*, TEST*, TRICKI*, TROUBL*, VARIABL*, VARIETI*

The highlighted stems bear strong emotional loadthough, no pain yet!

Use Indirect Data

Don't tell me who you are. Tell me who your friends are, and I will tell you who you are.

Instead of asking people what they think complexity is, let's collect abundant implicit markers off the Internet.

Indirect Data: Possible Sources

EBSCO keywords or subject tags (subscription and harvesting software required; we haz it, but the results may be too academic!)

Blogging sites like LiveJournal (LJ)LJ has individual blogs and community blogs (like forums)

Both types of blogs can and do have user- or moderator-declared interests

The interests are usually carefully chosen to reflect the blogger's/community's online identity

Free access, easy-to-write harvester

Raw Data

All LiveJournal communities that list at least one of the following interests:complexity

complex systems

complexity theory

complexity theories

Found 59 communities, such as mixedtype, abstractthought, ivygreenfanclub, pdmi_logic, investors, gifted_teens, etc.

Many more individual bloggerswe ignored them, their interest lists are way too broad!

Selected 374 most frequently declared interests (the corpus)

Example: Ecologists community

Community ecologists.livejournal.com

150 self-declared (by the moderator(s)) interests

Includes complexity, but also many other keywords; are they relevant? Let's assume they are.

Term Vector Model

Treat communities as documents and interests as words

Calculate generalized similarity between words (and between communitiesbut why?) using Kovacs (2010) algorithm:Two words are similar if they belong to similar documents

Two documents are similar if the consist of similar words

Remark: generalized similarity, as defined by Kovacs, is essentially Pearson correlation in an oblique Cartesian coordinate space

For any two words W and V from the corpus, -1 d(W,V) 1 is the similarity between the words; for any two communities C and D, -1 d(C,D) 1 is the similarity between the communities (a by-product, not needed)

Network Construction (1)

Treat words as nodes in a network

Treat similarities as edges if they are above T0=0.65 (slicing); the choice of T0 is tricky and arbitrary:If T0 is too high, the network is complete and not interesting

If T0 is too low, the network is sparse and disconnected

Gephi it!

Complexity Dragon (a.k.a. the Mindscape of Complexity)

Modular Structure

Modularity=0.56 (not great, but still visible)

psychology, chaos,science, mathematics,complexity theory, ...

philosophy, life,imagination, self-expression,knowledge, ...complexity, honesty, music, simplicity,love, poetry, creativity,empathy, humor, hate, ...writing, art, books,romance, drawing, ...

What Is the Meaning of Modules?

We could just stare at the modules and try to come up with a suitable name

or we could use crowdsourcing through Amazon Mechanical Turk again!

Fragment of the fireball

Let's Ask the Mechanical Turk!

Question: Describe the following group of 50* words with a single most suitable word or a two-word or three-word phrase: ...*There are only 36 terms in the lower left module.

Ask 100 mTurk workers, pay a nominal fee for an answer

Reject non-nouns; stem and aggregate nouns

The nouns are module descriptors (motifs)

Network Construction (2)

Treat original modules as documents and motifs as words

Calculate generalized similarity

Do slicing

Build a network of stems

Gephi it!

The Three Shades of Complexity

The new network has a simple, but not trivial, structure, and is highly modular

The Three Shades of Complexity:Science/Technology

Society/Mind

Creativity/Emotions

Connections:via education

via life/humanities

There are lines here, theyare just too thin...

Semantic Spectrography

mTurk for motifextraction and clustering

Term source formindscape construction

Analysis termselection

Spectrography vs Direct Survey

Direct Survey: 37

Spectrography: 382

4

9

4

104

111

113

37

24

Comparison Summary

Positive:Spectrography is much more detailed: it catches 41% of direct survey terms; direct survey catches only 4.5% of spectrography terms

Spectrography reveals structure

Negative:Spectrography requires a source (or sources) of terms

Successfully Used Elsewhere

D. Zinoviev, D. Stefanescu, L. Swenson, and G. Fireman, Semantic Networks of Interests in Online NSSI* Communities, in Proc. Workshop Words and Networks, Evanston, IL, June 2012, published online (also submitted to Social Networks in 2013)

D.Zinoviev and Z.Zhu, Conceptual Structure of Sustainability: Social and Scholarly Perspectives, Sunbelt XXXIV Social Networks Conference, St.-Pete Beach, FL, February 2014 (also submitted to Social Networks in 2014)

*Non-Suicidal Self Injurya common and epidemically spreading activity among adolescents and young adults

But Where Is the Pain?

SE corner of the emotional module full of NSSI-specific terminology

Communities: artificial_joy, humans_being (sic), the_addicted

Complexity and creativity as attributes of human nature

Thank-You!

Muokkaa otsikon tekstimuotoa napsauttamalla

Muokkaa jsennyksen tekstimuotoa napsauttamallaToinen jsennystasoKolmas jsennystasoNeljs jsennystasoViides jsennystasoKuudes jsennystasoSeitsems jsennystasoKahdeksas jsennystasoYhdekss jsennystaso