connecting talent to opportunity.. at scale @ linkedin
DESCRIPTION
Text Analytics Summit 2012, San Francisco.TRANSCRIPT
Connecting Talent Opportunity.. at scale
Anmol BhasinDirector of Engineering Recommendations & Personalization
LinkedIn Confidential ©2013 All Rights Reserved
Text Analytics SummitSan Francisco November 14, 2012
Anmol
World’s Largest Professional Network
LinkedIn Confidential ©2013 All Rights Reserved 3
Members Worldwide2 new
Members Per Second100M+
Monthly Unique Visitors180 M+ 2M+
Company Pages
…..
Our MissionConnect the world’s professionals to make them
more productive and successful.
Our VisionCreate economic opportunity for every
professional in the world.
Members First!
Linkedin fills in big shoes..- Members perceive LinkedIn profiles to be their
professional identity of record.
- Companies turn to LinkedIn for finding, engaging and hiring top talent.
Connecting Talent to Opportunity is crucial for our business!
LinkedIn Confidential ©2013 All Rights Reserved 6
Members Worldwide2 new Members Per SecondUse Linkedin to Hire..Monthly Unique Visitors
175M+ Company Pages
…..
*
85%
Recommendations
50%
10
Real Time Talent Match
11
12
Automated Resume to Profile Link
Job Recommendations
14
Address job seeker* need: find dream job– Huge cost of consumption
Lag between view and application is in hours/days– Extremely high level of expectation
– No forgiveness for less than perfect recommendations
Accuracy is key!
(*) 20% active, 60% receptive -- 10/12 Job Seeker Survey, 20K in 7 countries
Problem Definition
17
Corpus StatsJob
User Base
Filtered
titlegeocompany
industrydescriptionfunctional area
…
Candidate
Generalexpertisespecialtieseducationheadlinegeoexperience
Current Positiontitlesummarytenure lengthindustryfunctional area…
Similarity (candidate expertise, job description)
0.56Similarity
(candidate specialties, job description)
0.2Transition probability
(candidate industry, job industry)
0.43
Title Similarity
0.8
Similarity (headline, title)
0.7
...derived
Matching
Binary Exact matches: geo, industry, …
Soft transition probabilities, similarity, …
Text
Transition probabilitiesConnectivityyrs of experience to reach title education needed for this title…
EnsembleScorings
How LinkedIn matches people to jobs
~250B Member Job Pairs a day!
the magic is in the models
features
19
Feature Engineering – Entity Resolution
Companies
Huge impact on the business and UE Ad targeting TalentMatch Referrals
‘IBM’ has 8000+ variations- ibm – ireland- ibm research- T J Watson Labs- International Bus. Machines- Deep Blue
K-Ambiguous
Asonam’11, KDD’11
Open to relocation ? Region similarity based on profiles or network Region transition probability
predict individuals propensity to migrate and most likely migration target
Impact on job recommendations 20% lift in
views/viewers/applications/applicants
Feature Engineering – Would you move
21
What should you transition to .. and when ?
Months since graduation
Prob
abili
ty o
f sw
itch
Insights
23
Where are you likely to stay ?
24
Power of aggregation..
Beforeemployees worked at
Yahoo! (247)Google (139)Microsoft (105)Oracle (93)IBM (68)
Beforeemployees worked at
Microsoft (1379)
IBM (939)Yahoo! (608)Oracle (558)
Different Strokes for
different folks
Demographic Segmentation Students (or recent grads) US vs International members Industry Specific models
e.g. Finance vs Technology
Behavioral Segmentation Job Seekers (Active) Daily Users vs Monthly Users
Segmented Models
27
Types Active Passive receptive Not a job seeker
Modeling Ordered logistic reg.
Impact ~10x application rate between Active and Passive receptive
Job Seeking
28
[Zhang, 2012]
Job Seeking Socially Contagious?
Capturing User Interests
29
ContentSocial Graphs
http://inmaps.linkedinlabs.com/
…
Behavior
PVsActions (clicks)
Queries
Big Data A/B is the
new
A/B TestingIs A better than B.. Let’s test
323232
A/B TestingIs A better than B.. Let’s test
Beware of - novelty effect- cannibalization- potential biases (time, targeted population)
- random sampling destroying the network effect
Don’t forget to A/A test first
(“Seven Pitfalls to Avoid when Running Controlled Experiments on the Web”, KDD’09“Framework and Algorithms for Network Bucket Testing” WWW’12 submission)
Enjoy testing furiously!
33
Some final remarks Most people aren't actively looking for jobs.
– Many people are but most aren’t– Complicates evaluation and training
Important not to offend– JYMBII: I am more senior than that!– What is the price of a bad recommendation? (PYMK vs. JYMBII)
You can’t always get what you want– Every employer wants the hottest candidate.– The perfect candidate already works for you.
It takes a village
Credits
Engineering : Abhishek Gupta, Adam Smyczek, Adil Aijaz, Alan Li, Baoshi Yan, Bee-Chung Chen, Deepak Agarwal, Ethan Zhang, Haishan Liu, Igor Perisic, Jonathan Traupman, Liang Zhang, Lokesh Bajaj, Mario Rodriguez, Mitul Tiwari, Mohammad Amin, Monica Rogati, Parul Jain, Paul Ogilvie, Sam Shah, Sanjay Dubey, Tarun Kumar, Trevor Walker, Utku Irmak
Product : Andrew Hill, Christian posse, Gyanda Sachdeva, Mike Grishaver, Parker Barrile, Sachit Kamat
Alphabetically sorted
