the elements of fashion style

Download The Elements of Fashion Style

Post on 02-Jan-2017




3 download

Embed Size (px)


  • The Elements of Fashion Style

    Kristen Vaccaro, Sunaya Shivakumar, Ziqiao Ding, Karrie Karahalios, and Ranjitha KumarDepartment of Computer Science

    University of Illinois at Urbana-Champaign{kvaccaro,sshivak2,zding5,kkarahal,ranjitha}


    I need an outfit for a beach wedding that I'm going to early this summer. I'm so excited -- it's going to be warm and exotic and tropical... I want my outfit to look effortless, breezy, flowy, like Im floating over the sand! Oh, and obviously no white! For a tropical spot, I think my outfit should be bright and colorful.


    Figure 1: This paper presents a data-driven fashion model that learns correspondences between high-level styles (like beach, flowy, and wedding)and low-level design elements such as color, material, and silhouette. The model powers a number of fashion applications, such as an automated personalstylist that recommends fashion outfits (right) based on natural language specifications (left).

    ABSTRACTThe outfits people wear contain latent fashion concepts cap-turing styles, seasons, events, and environments. Fashion the-orists have proposed that these concepts are shaped by de-sign elements such as color, material, and silhouette. Whilea dress may be bohemian because of its pattern, material,trim, or some combination thereof, it is not always clear howlow-level elements translate to high-level styles. In this pa-per, we use polylingual topic modeling to learn latent fashionconcepts jointly in two languages capturing these elementsand styles. This latent topic formation enables translationbetween languages through topic space, exposing the ele-ments of fashion style. The model is trained on a set of morethan half a million outfits collected from Polyvore, a popularfashion-based social network. We present novel, data-drivenfashion applications that allow users to express their desires innatural language just as they would to a real stylist, and pro-duce tailored item recommendations for their fashion needs.

    Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from

    UIST 2016, October 16 - 19, 2016, Tokyo, Japan 2016 Copyright held by the owner/author(s). Publication rights licensed to ACM.ISBN 978-1-4503-4189-9/16/10. . . $15.00


    Author KeywordsFashion, elements, styles, polylingual topic modeling

    ACM Classification KeywordsH.5.2 User Interfaces; H.2.8 Database Applications

    INTRODUCTIONOutfits contain latent fashion concepts, capturing styles, sea-sons, events, and environments. Fashion theorists have pro-posed that important design elements color, material, sil-houette, and trim shape these concepts [24]. A long-standing question in fashion theory is how low-level fashionelements map to high-level styles. One of the first theorists tostudy fashion as a language, Roland Barthes, highlighted thedifficulty of translating between elements and styles [2]:

    If I read that a square-necked, white silk sweater is verysmart, it is impossible for me to say without again hav-ing to revert to intuition which of these four features(sweater, silk, white, square neck) act as signifiers forthe concept smart: is it only one feature which carriesthe meaning, or conversely do non-signifying elementscome together and suddenly create meaning as soon asthey are combined?

    To address this fundamental question, we present a model thatlearns the relation between fashion design elements and fash-ion styles. This model adapts a natural language processingtechnique polylingual topic modeling to learn latent

    {kvaccaro, sshivak2, zding5, kkarahal, ranjitha}@illinois.edu

  • fashion concepts jointly in two languages: a style languageused to describe outfits, and an element language used to la-bel clothing items. This model answers Barthes question:identifying the elements that determine styles. It also powersautomated personal stylist systems that can identify peoplesstyles from an outfit they assemble, or recommend items fora desired style (Figure 1).

    We train a polylingual topic model (PLTM) on a dataset ofover half a million outfits collected from a popular fashion-based social network, Polyvore1. Polyvore outfits have bothfree-text outfit descriptions written by their creator and itemlabels (e.g., color, material, pattern, designer) extracted byPolyvore. These two streams of data form a pair of paralleldocuments (style and element, respectively) for each outfit,which comprise the training inputs for the model.

    Each topic in the trained PLTM corresponds to a pair of dis-tributions over the two vocabularies, capturing the correspon-dence between style and element words (Figure 2). For exam-ple, the model learns that fashion elements such as black,leather, and jacket are often signifiers for styles such asbiker, and motorcycle.

    We validate the model using a set of crowdsourced, percep-tual tasks: for example, asking users to select the set of wordsin the element language that is the best match for a set in thestyle language. These tasks demonstrate that the learned top-ics mirror human perception: the topics are semantically co-herent and translation between elements and styles is mean-ingful to users.

    This paper motivates the choice of model, describes both thePolyvore outfit dataset as well as the training and evaluationof the PLTM, and illustrates several resultant fashion applica-tions. Using the PLTM, we can explain why a clothing itemfits a certain style: we know whether it is the collar, color,material, or item itself that makes a sweater smart. We canbuild data-driven style quizzes, predicting style preferencesfrom a users outfit. We even describe an automated personalstylist which can provide outfit recommendations for a de-sired style expressed in natural language. Polylingual topicmodeling can help us better our understanding of fashion the-ory, and support a rich new set of interactive fashion tools.

    MOTIVATIONAs more fashion data has become available online, re-searchers have built data-driven fashion systems that processlarge-scale data to characterize styles [6, 11, 12, 20, 23],recommend outfits [21, 22, 27, 32], and capture changingtrends [8, 10, 29]. Several projects have used deep learn-ing to automatically infer structure from low-level (typically)vision-based features [15, 28]. While these models can pre-dict whether items match or whether an outfit is hipster,they cannot explain why. For many applications, modelspredicated on human-interpretable features are more usefulthan models that merely predict outcomes [9]. For example,when a user looks for a party outfit, an explanation likethis item was recommended because it is a black miniskirthelps her understand the suggestion and provide feedback tothe system.

    topic distribution

    1 21 2

    STYLEsummer, vintage, beach, american, relaxed, retro, unisex

    ELEMENTshort, denim, highwaisted, shirt, top, cutoff, form, distressed

    STYLEbiker, motorcycle, vintage, summer, college, varsity, military

    ELEMENTjacket, black, leather, shirt, zip, denim, sleeve, faux

    STYLEprom, occasion, special, party, holiday, bridesmaid

    ELEMENTdress, shoe, cocktail, evening, mini, heel, costume

    STYLEparty, summer, night, sexy, vintage, fitting, botanical

    ELEMENTdress, mini, sleeveless, cocktail, skater, flare, out, lace, floral





    topic distribution

    Figure 2: Via polylingual topic modeling, we infer distributions over latentfashion topics in outfits that capture the elements of fashion style. Fashionelements like jacket, black, leather signify the biker, motorcycle style.Conversely, fashion styles like prom, special occasion label groups ofelements such as cocktail, mini, dress.

    This paper presents a fashion model that maps low-level el-ements to high-level styles, adapting polylingual topic mod-eling to learn correspondences between them [18]. Both setsof features (elements and styles) are human interpretable, andthe translational capability of PLTMs can power applicationsthat indicate how design features are tied to user outcomes,identifying peoples styles from the elements in their outfitsand recommending clothing items from high-level style de-scriptions.

    In addition to their translational capabilities, PLTMs offer anumber of other advantages. Unlike systems built on discrim-inative models [21, 27, 32], PLTMs support a dynamic set ofstyles that grows with the dataset and need not be specifieda priori. Moreover, topic modeling represents documents asdistributions over concepts, allowing styles to coexist withinoutfits rather than labeling them with individual styles [6, 11,12, 20]. Finally, the model smooths distributions so that sys-tems can support low frequency queries. Even though thereare no wedding outfits explicitly labeled punk rock inour dataset, we can still suggest appropriate attire for such anevent by identifying high probability fashion elements associ-ated with wedding (e.g., white, lace) and punk rock(e.g., leather, studded), and searching for clothing itemswhich contain them.

    To build a PLTM for fashion, we require data that containsboth style