lda2vec text by the bay 2016

lda2vec (word2vec, and lda)

Christopher Moody @ Stitch Fix

@chrisemoody Caltech Physics PhD. in astrostats supercomputing sklearn t-SNE contributor Data Labs at Stitch Fix github.com/cemoody

Gaussian Processes t-SNE

chainer deep learning

Tensor Decomposition

word2vec

1. king - man + woman = queen 2. Huge splash in NLP world 3. Learns from raw text 4. Pretty simple algorithm 5. Comes pretrained

word2vec

1. Set up an objective function 2. Randomly initialize vectors 3. Do gradient descent

word2vec

word2vec: learn word vector w from it’s surrounding context

“The fox jumped over the lazy dog”Maximize the likelihood of seeing the words given the word over.

P(the|over) P(fox|over)

P(jumped|over) P(the|over) P(lazy|over) P(dog|over)

…instead of maximizing the likelihood of co-occurrence counts.

P(fox|over)

What should this be?

P(vfox|vover)

Should depend on the word vectors.

P(fox|over)

“The fox jumped over the lazy dog”

P(w|c)

Extract pairs from context window around every input word.

P(w|c)

objectiv

Measure loss between w and c?

How should we define P(w|c)?

objectiv

How should we define P(w|c)?

Measure loss between w and c?

w . c ~ 1

objectiv

vcanada . vsnow ~ 1

w . c ~ 0

objectiv

cvcanada . vdesert ~0

w . c ~ -1

objectiv

w . c ∈ [-1,1]

objectiv

But we’d like to measure a probability.

w . c ∈ [-1,1]

objectiv

∈ [0,1]σ(c·w)

objectiv

∈ [0,1]σ(c·w)

SimilarDissimilar

Loss function:

objectiv

L=σ(c·w)

Logistic (binary) choice. Is the (context, word) combination from our dataset?

The skip-gram negative-sampling model

objectiv

Trivial solution is that context = word for all vectors

L=σ(c·w)w

L = σ(c·w) + σ(-c·wneg)

objectiv

Draw random words in vocabulary.

objectiv

Discriminate positive from negative samples

Multiple Negative

L = σ(c·w) + σ(-c·wneg) +…+ σ(-c·wneg)

The SGNS ModelPMI

ci·wj = PMI(Mij) - log k

…is extremely similar to matrix factorization!

Levy & Goldberg 2014

The SGNS ModelPMI

‘traditional’ NLP

ci·wj = PMI(Mij) - log k

…is extremely similar to matrix factorization!

The SGNS Model

L = σ(c·w) + Σσ(-c·w)

ci·wj = log

#(ci,wj)/n

k #(wj)/n #(ci)/n

The SGNS Model

L = σ(c·w) + Σσ(-c·w)

ci·wj = log

popularity of c,wk (popularity of c) (popularity of w)

99% of word2vec is counting.

And you can count words in SQL

Count how many times you saw c·w

Count how many times you saw c

Count how many times you saw w

…and this takes ~5 minutes to compute on a single core. Computing SVD is a completely standard math library.

word2vec

ITEM_3469 + ‘Pregnant’

+ ‘Pregnant’

= ITEM_701333 = ITEM_901004 = ITEM_800456

what about?LDA?

LDA on Client Item Descriptions

LDA on Item

Descriptions (with Jay)

LDA on Item

lda vs word2vec

Bayesian Graphical ModelML Neural Model

word2vec is local: one word predicts a nearby word

“I love finding new designer brands for jeans”

But text is usually organized.

In LDA, documents globally predict words.

doc 7681

typical word2vec vector

[ 0%, 9%, 78%, 11%]

typical LDA document vector

[ -0.75, -1.25, -0.55, -0.12, +2.2]

All sum to 100%All real values

5D word2vec vector

[ 0%, 9%, 78%, 11%]

5D LDA document vector

[ -0.75, -1.25, -0.55, -0.12, +2.2]

Sparse All sum to 100%

Dimensions are absolute

Dense All real values

Dimensions relative

100D word2vec vector

[ 0%0%0%0%0% … 0%, 9%, 78%, 11%]

[ -0.75, -1.25, -0.55, -0.27, -0.94, 0.44, 0.05, 0.31 … -0.12, +2.2]

Sparse All sum to 100%

Dimensions are absolute

Dense All real values

Dimensions relative

dense sparse

100D word2vec vector

[ 0%0%0%0%0% … 0%, 9%, 78%, 11%]

[ -0.75, -1.25, -0.55, -0.27, -0.94, 0.44, 0.05, 0.31 … -0.12, +2.2]

Similar in fewer ways (more interpretable)

Similar in 100D ways (very flexible)

+mixture +sparse

can we do both? lda2vec

-1.9 0.85 -0.6 -0.3 -0.5

Lufthansa is a German airline and when

#hidden units

Skip grams from sentences

Word vector

Negative sampling loss

German

word2vec predicts locally: one word predicts a nearby word

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units#hidden units

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

German

Document vector predicts a word from

a global context

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

We’re missing mixtures & sparsity!

German

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

We’re missing mixtures & sparsity!

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

Now it’s a mixture.

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

Trinitarian baptismal

Pentecostals Bede

schismatics excommunication

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

#topicsDocument weight

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

topic 1 = “religion” Trinitarian baptismal

Pentecostals Bede

schismatics excommunication

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

Milosevic absentee

Indonesia Lebanese Isrealis

Karadzic

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

topic 2 = “politics” Milosevic absentee

Indonesia Lebanese Isrealis

Karadzic

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

Sparsity!

0.34 -0.1 0.17

41% 26% 34%

-1.4 -0.5 -1.4

-1.9-1.7 0.75

0.96-0.7 -1.9

-0.2-1.1 0.6

-0.7 -0.4 -0.7 -0.3 -0.3-1.9 0.85 -0.6 -0.3 -0.5

-2.6 0.45 -1.3 -0.6 -0.8

#topics

#topicsfox

#hidden units

#topics

#hidden units

Word vector

Topic matrix

Document proportion

Document weight

Document vector

Context vector

34% 32% 34%

41% 26% 34%

99% 1% 0%

@chrisemoody

lda2vec.com

+ API docs + Examples + GPU + Tests

@chrisemoody

lda2vec.com

@chrisemoody Example Hacker News comments

Topics: http://nbviewer.jupyter.org/github/cemoody/lda2vec/blob/master/examples/

hacker_news/lda2vec/lda2vec.ipynb

Word vectors: https://github.com/cemoody/

lda2vec/blob/master/examples/hacker_news/lda2vec/

word_vectors.ipynb

@chrisemoody

lda2vec.com

human-interpretable doc topics, use LDA.

machine-useable word-level features, use word2vec.

if you like to experiment a lot, and have topics over user / doc / region / etc. features, use lda2vec. (and you have a GPU)

If you want…

?@chrisemoody

Multithreaded Stitch Fix

@chrisemoody

lda2vec.com

CreditLarge swathes of this talk are from

previous presentations by:

• Tomas Mikolov • David Blei • Christopher Olah • Radim Rehurek • Omer Levy & Yoav Goldberg • Richard Socher • Xin Rong • Tim Hopper

“PS! Thank you for such an awesome idea”

@chrisemoody

doc_id=1846

Can we model topics to sentences? lda2lstm

“PS! Thank you for such an awesome idea”doc_id=1846

@chrisemoody

Can we model topics to images? lda2ae

TJ Torres

and now for something completely crazy4Fun Stuff

translation

(using just a rotation matrix)

English

Spanish

Matrix Rotation

deepwalk

Perozz

et al 2

learn word vectors from sentences

vOUT vOUT vOUT vOUTvOUTvOUT

‘words’ are graph vertices ‘sentences’ are random walks on the graph

word2vec

Playlists at Spotify

context

sequence

‘words’ are song indices ‘sentences’ are playlists

Playlists at Spotify

contextErik

Bernhar

Great performance on ‘related artists’

Fixes at Stitch Fix

sequence

Let’s try: ‘words’ are items ‘sentences’ are fixes

Fixes at Stitch Fix

context

Learn similarity between styles because they co-occur

Learn ‘coherent’ styles

sequence

Fixes at Stitch Fix?

context

sequence

ningGot lots of structure!

context

sequence

context

sequence

Nearby regions are consistent ‘closets’

?@chrisemoody

context dependent

oldberg

Australian scientist discovers star with telescopecontext +/- 2 words

context dependent

context

Australian scientist discovers star with telescope

oldberg

context dependent

context

Australian scientist discovers star with telescopecontext

oldberg

context dependent

context

BoW DEPS

topically-similar vs ‘functionally’ similar

oldberg

?@chrisemoody

Crazy Approaches

Paragraph Vectors (Just extend the context window)

Content dependency (Change the window grammatically)

Social word2vec (deepwalk) (Sentence is a walk on the graph)

Spotify (Sentence is a playlist of song_ids)

Stitch Fix (Sentence is a shipment of five items)

Guess the word given the context

~20x faster. (this is the alternative.)

vIN vINvIN vINvIN vIN

SkipGram

vOUT vOUT

vOUT vOUT vOUTvOUT

Guess the context given the word

Better at syntax. (this is the one we went over)

vDOC = a vtopic1 + b vtopic2 +…

Let’s make vDOC sparse

This works! 😀 But vDOC isn’t as interpretable as the topic vectors. 😔

vDOC = topic0 + topic1

Let’s say that vDOC ads

softmax(vOUT * (vIN+ vDOC))

theory of lda2vec

pyLDAvis of lda2vec

LDA Results

context

History

I loved every choice in this fix!! Great job!

Great Stylist Perfect

LDA Results

context

History

Body Fit

My measurements are 36-28-32. If that helps. I like wearing some clothing that is fitted.

Very hard for me to find pants that fit right.

LDA Results

context

History

Sizing

Really enjoyed the experience and the pieces, sizing for tops was too big.

Looking forward to my next box!

Excited for next

LDA Results

context

History

Almost Bought

It was a great fix. Loved the two items I kept and the three I sent back were close!

Perfect

All of the following ideas will change what ‘words’ and ‘context’ represent.

What about summarizing documents?

On the day he took office, President Obama reached out to America’s enemies, offering in his first inaugural address to extend a hand if you are willing to unclench your fist. More than six years later, he has arrived at a moment of truth in testing that

The framework nuclear agreement he reached with Iran on Thursday did not provide the definitive answer to whether Mr. Obama’s audacious gamble will pay off. The fist Iran has shaken at the so-called Great Satan since 1979 has not completely relaxed.

Normal skipgram extends C words before, and C words after.

OUT OUT

On the day he took office, President Obama reached out to America’s enemies, offering in his first inaugural address to extend a hand if you are willing to unclench your fist. More than six years later, he has arrived at a moment of truth in testing that

The framework nuclear agreement he reached with Iran on Thursday did not provide the definitive answer to whether Mr. Obama’s audacious gamble will pay off. The fist Iran has shaken at the so-called Great Satan since 1979 has not completely relaxed.

A document vector simply extends the context to the whole document.

OUT OUT

OUT OUTdoc_1347

fromgensim.modelsimportDoc2Vecfn=“item_document_vectors”model=Doc2Vec.load(fn)model.most_similar('pregnant')matches=list(filter(lambdax:'SENT_'inx[0],matches))

#['...Iamcurrently23weekspregnant...',#'...I'mnow10weekspregnant...',#'...notshowingtoomuchyet...',#'...15weeksnow.Babybump...',#'...6weekspostpartum!...',#'...12weekspostpartumandamnursing...',#'...Ihavemybabyshowerthat...',#'...amstillbreastfeeding...',#'...Iwouldloveanoutfitforababyshower...']

lda2vec text by the bay 2016

Technology

home :: bay-lake region planning commission - text marinette...

goodnews bay district,...

james bay and northern quebec agreement - … · james bay...

san juan county shoreline master program update...doe bay...

2015 bay star sport & bay star -...

christchurch city council footer text flood taskforce flood...

state legislative district reference map...cv redfish lk...

mbrc standard drawing - mbrc-4004 open space signage...

moreton bay bay

ivanof bay american bay fox bay boulder bay alaska...

a4 landscape template 2 - royal holloway, university · web...

te ara hura map january 2019 - waiheke island€¦ · man...

lda2vec text by the bay 2016 with notes

2-bay / 4-bay turbo nas ts-251+ / 451+ use your turbo nas...

emu bay structure plan - kangaroo island council bay... ·...

mgunby.wikispaces.commgunby.wikispaces.com/file/view/goin'+gangsta,+choosin'+cholita.pdfbernstein's...

pavilion park - spicerhaart · 2019-12-11 · marina...

seagate business storage 1-bay, 2-bay, and 4-bay nas … ·...

bfc brochure unbolded text - bay county, michigan ·...

tasmania catchments...