revving up the force.com formula engine
DESCRIPTION
Do your formulas run more like a wagon than a sports car? Join us as we take you "under the hood" of the Force.com formula engine and show you how to tune it for maximum performance. We'll explore some common designs that make formulas slow to a crawl, then discuss how to use formula fields to improve the scalability of applications handling very large data volumes.TRANSCRIPT
Revving Up the Force.com Formula Engine
Bud Vieira, salesforce.com, @aavraDaisuke Kawamoto, salesforce.com, @daisukeSfdcNathan Shilling, Appirio, @nds9619
Safe HarborSafe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
You are a Force.com developer, architect, or administrator ...
… you want to know why your queries & reports that use
formula fields are so slow ...
… so you can design a solution that
scales & performs better.
You need to understand how formula evaluation works under
the hood!
Index selective formula fields!
Reduce run-time overhead!
Takeaways …
Why use formula fields?
Data Logic Display
How does a formula field work?
Filtering on formula fields[SELECT … FROM Invoice__c WHERE Portfolio_Formula__c = ‘US Portfolio’LIMIT 150]
List View
Report
To be indexed, a formula
MUSTbe deterministic
How does indexing work?
Edit Record
Click SAVE Save to Object
Write to Index TableEvaluate
Formula
A field is deterministic …
… when the value of the fieldDOES NOT CHANGE
between save events on its record
What prevents formula fields from being indexed?Formula Element Can be Indexed? Why?
Contact__r.LastName No Values change between formula record updates
< (Today() + 3) No Date functions change in real time
Opportunity.Amount No Special functionality
Country__c Depends Formula field could be non-deterministic
Fields that Force.com cannot index
Exhaustive list of fields: http://bit.ly/XL8ybp
Solution #1
Store lookup values on the same object
Solution #1
1. Create custom field on object2. Define workflow rules3. Add field update actions
Or use Apex triggers
Solution #2
Split the formula
Solution #2IF(AND((Invoice_Total__c > 1000),(Recurring__c),(Collective__c)),IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c >= 10000)), "US Over 10K",IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c >= 5000)), "US Over 5K",IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c < 5000)), "US Under 5K",IF((Invoice_Total__c >= 8000),IF(Contact__r.Country__c="Canada", "CAN Over 8K",IF(Contact__r.Country__c="Italy", "IT Over 8K",IF(Contact__r.Country__c="Australia","AU Over 8K","Other"))),IF((Invoice_Total__c >= 4000),IF(Contact__r.Country__c="Canada","CAN Over 4K",IF(Contact__r.Country__c="Italy", "IT Over 4K",IF(Contact__r.Country__c="Australia", "AU Over 4K","Other"))),IF(Contact__r.Country__c="Italy", "IT Under 4K",IF(Contact__r.Country__c="Canada","CAN Under 4K",IF(Contact__r.Country__c="Australia","AU Under 4K","Other”))))))) ,NULL)
SELECT … FROM Invoice__c WHERE Formula__c = ‘US Over 8K’SELECT … FROM Invoice__c WHERE Formula1__c = ‘US’AND Formula2__c = trueAND Invoice_Total__c >= 8000
IF(Contact__r.Country__c="United States", "US"IF(Contact__r.Country__c="Canada", "CAN",IF(Contact__r.Country__c="Italy", "IT",IF(Contact__r.Country__c="Australia","AU","Other"))),NULL)
IF( AND(Recurring__c, Collective__c ),TRUE,FALSE)
Country (lookup)
Invoice is BOTHRecurring AND Collective
Country (lookup)
Invoice is BOTHRecurring AND Collective
Invoice Total
Solution #3Spell it out in your QUERY
Age__c = TODAY() - CloseDate
SELECT Id, Name FROM Opportunity WHERE Age__c > 14 AND Age__c <= 21
SELECT Id, Name FROM Opportunity WHERE (CloseDate > LAST_N_DAYS:14) AND (CloseDate = LAST_N_DAYS:21)
Demo
Index selective formula fields
Call Customer SupportGet your selective formula fields indexed
save thousands of ms (runTime)
When to index formula fields
INDEX
Deterministic?
Frequently used?
Selective?
Formula complexity canEXPLODE
run-time SQL
The formula field in this simple SOQL query …
SELECT Formula__c FROM Invoice__c
… has moderately complex logic …IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))))))),NULL)
… which generates run-time SQL like this!
Where is all the SQL coming from?
• Spanning objects• Nesting logic• Nesting other formula fields
All at RUN TIME
Solution #1 – Store static values
• Workflow actions• Triggers• Integrations
Now the runtime SQL statement looks like this!
Save 100s of ms
Solution #2 – Split up the formula
Save about 100 ms
IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))))))),NULL)
IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", "Other" )))),NULL)
IF( (Invoice_Total__c > 1000), IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))),NULL)
North America
APAC and EMEA
Making formulas scream!
Customer case studies
Nathan ShillingRegional Practice LeadCertified Technical Architect@nds9619
All About Appirio
Appirio is a global services provider that helps enterprises reimagine their business and become more agile using crowdsourcing and cloud, social and mobile technology.
▪ More than 600 enterprise customers successfully moved to the cloud
▪ Strategic partner of salesforce.com since 2006
▪ On-demand access to 600,000 of the world’s top developers, designers and data analysts through Appirio’s community
Business Challenge• Users have multiple managers
in a hierarchy • User Z sometimes has data
rolling up to Manager A
Details – Business Challenge• The object we will report on is assigned to only one leaf
member of the hierarchy• The user running the query can be a manager to multiple
members of the hierarchy• The query should show all records at or below the Managers
top-level node• Standard Role Hierarchy clearly doesn’t solve this!
Technical Solution – Query “My Team” records• Formula is needed to determine “My Team” query• “My Team” cannot be indexed• “My Team” references Normalized data on Hierarchy record
• If(Lead.Hierarchy__r.Parent__r.Parent__r.Level__c == User.Level__c…• If(Lead.Hierarchy__r.Parent__r.Level__c == User.Level__c…
• First try: Formula complexity taxed the query optimizer• Lead object contains more than 10 million rows at any given
time
Denormalization & static values solve our problem
All the pieces put together…• Store de-normalized hierarchy level values on Lead• Store User’s top-level hierarchy value on User• Simplify “My Team” formula to compare just those two sets
of fields
And the results querying from 10 million rows…• First Iteration: 30 second query returning 1000 rows
• Using formulas to compare normalized reference fields
• Second Try: 6 second query returning 1000 rows• Reduced formula to use de-normalized static values• Still not fully performant because formula is not indexed
• Final Result: subsecond query returning 150 rows• Negotiated business requirement change to only show
“Last 30 Days” of created Lead data• A selective query using an index always wins!
Takeaways …
Index selective formula fields!
Reduce run-time overhead!
Related DevZone hands-on, mini-workshop
Wednesday
9:15AM or
3:45PM
Bud Vieira
Architect Evangelist@aavra
Daisuke Kawamoto
Architect Evangelist@daisukeSfdc
Nathan Shilling
Regional Practice Lead Certified Technical Architect
Appirio@nds9619
We want to hear from YOU!
Please take a moment to complete our session survey
Surveys can be found in the “My Agenda” portion of the Dreamforce app
Hands-on Work Shop @ Developer Zone
Wednesday 9am & 1 pm
Appendix
Formula Limits