revving up the force.com formula engine

Revving Up the Force.com Formula Engine

Bud Vieira, salesforce.com, @aavraDaisuke Kawamoto, salesforce.com, @daisukeSfdcNathan Shilling, Appirio, @nds9619

Safe HarborSafe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

You are a Force.com developer, architect, or administrator ...

… you want to know why your queries & reports that use

formula fields are so slow ...

… so you can design a solution that

scales & performs better.

You need to understand how formula evaluation works under

the hood!

Index selective formula fields!

Reduce run-time overhead!

Takeaways …

Why use formula fields?

Data Logic Display

How does a formula field work?

Filtering on formula fields[SELECT … FROM Invoice__c WHERE Portfolio_Formula__c = ‘US Portfolio’LIMIT 150]

List View

Report

To be indexed, a formula

MUSTbe deterministic

How does indexing work?

Edit Record

Click SAVE Save to Object

Write to Index TableEvaluate

Formula

A field is deterministic …

… when the value of the fieldDOES NOT CHANGE

between save events on its record

What prevents formula fields from being indexed?Formula Element Can be Indexed? Why?

Contact__r.LastName No Values change between formula record updates

< (Today() + 3) No Date functions change in real time

Opportunity.Amount No Special functionality

Country__c Depends Formula field could be non-deterministic

Fields that Force.com cannot index

Exhaustive list of fields: http://bit.ly/XL8ybp

Solution #1

Store lookup values on the same object

Solution #1

1. Create custom field on object2. Define workflow rules3. Add field update actions

Or use Apex triggers

Solution #2

Split the formula

Solution #2IF(AND((Invoice_Total__c > 1000),(Recurring__c),(Collective__c)),IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c >= 10000)), "US Over 10K",IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c >= 5000)), "US Over 5K",IF(AND(Contact__r.Country__c="United States",(Invoice_Total__c < 5000)), "US Under 5K",IF((Invoice_Total__c >= 8000),IF(Contact__r.Country__c="Canada", "CAN Over 8K",IF(Contact__r.Country__c="Italy", "IT Over 8K",IF(Contact__r.Country__c="Australia","AU Over 8K","Other"))),IF((Invoice_Total__c >= 4000),IF(Contact__r.Country__c="Canada","CAN Over 4K",IF(Contact__r.Country__c="Italy", "IT Over 4K",IF(Contact__r.Country__c="Australia", "AU Over 4K","Other"))),IF(Contact__r.Country__c="Italy", "IT Under 4K",IF(Contact__r.Country__c="Canada","CAN Under 4K",IF(Contact__r.Country__c="Australia","AU Under 4K","Other”))))))) ,NULL)

SELECT … FROM Invoice__c WHERE Formula__c = ‘US Over 8K’SELECT … FROM Invoice__c WHERE Formula1__c = ‘US’AND Formula2__c = trueAND Invoice_Total__c >= 8000

IF(Contact__r.Country__c="United States", "US"IF(Contact__r.Country__c="Canada", "CAN",IF(Contact__r.Country__c="Italy", "IT",IF(Contact__r.Country__c="Australia","AU","Other"))),NULL)

IF( AND(Recurring__c, Collective__c ),TRUE,FALSE)

Country (lookup)

Invoice is BOTHRecurring AND Collective

Country (lookup)

Invoice is BOTHRecurring AND Collective

Invoice Total

Solution #3Spell it out in your QUERY

Age__c = TODAY() - CloseDate

SELECT Id, Name FROM Opportunity WHERE Age__c > 14 AND Age__c <= 21

SELECT Id, Name FROM Opportunity WHERE (CloseDate > LAST_N_DAYS:14) AND (CloseDate = LAST_N_DAYS:21)

Demo

Index selective formula fields

Call Customer SupportGet your selective formula fields indexed

save thousands of ms (runTime)

When to index formula fields

INDEX

Deterministic?

Frequently used?

Selective?

Formula complexity canEXPLODE

run-time SQL

The formula field in this simple SOQL query …

SELECT Formula__c FROM Invoice__c

… has moderately complex logic …IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))))))),NULL)

… which generates run-time SQL like this!

Where is all the SQL coming from?

• Spanning objects• Nesting logic• Nesting other formula fields

All at RUN TIME

Solution #1 – Store static values

• Workflow actions• Triggers• Integrations

Now the runtime SQL statement looks like this!

Save 100s of ms

Solution #2 – Split up the formula

Save about 100 ms

IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))))))),NULL)

IF( (Invoice_Total__c > 1000), IF(CONTAINS(Country__c , "United States") &&(RPE__c > 500000), "US Portfolio", IF(CONTAINS(Country__c , "United States") &&(RPE__c <= 500000), "US Non-Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c > 400000), "CAN Portfolio", IF(CONTAINS(Country__c , "Canada") &&(RPE__c <= 400000), "CAN Non-Portfolio", "Other" )))),NULL)

IF( (Invoice_Total__c > 1000), IF((RPE__c > 300000), IF(CONTAINS(Country__c , "Italy"),"IT Portfolio", IF(CONTAINS(Country__c , "France"),"FR Portfolio", IF(CONTAINS(Country__c , "Australia"),"AU Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Portfolio","Other")))), IF(CONTAINS(Country__c , "Italy"), "IT Non-Portfolio", IF(CONTAINS(Country__c , "France"), "FR Non-Portfolio", IF(CONTAINS(Country__c , "Australia"), "AU Non-Portfolio", IF(CONTAINS(Country__c , "Japan"), "JP Non-Portfolio","Other" ))))),NULL)

North America

APAC and EMEA

Making formulas scream!

Customer case studies

Nathan ShillingRegional Practice LeadCertified Technical Architect@nds9619

All About Appirio

Appirio is a global services provider that helps enterprises reimagine their business and become more agile using crowdsourcing and cloud, social and mobile technology.

▪ More than 600 enterprise customers successfully moved to the cloud

▪ Strategic partner of salesforce.com since 2006

▪ On-demand access to 600,000 of the world’s top developers, designers and data analysts through Appirio’s community

Business Challenge• Users have multiple managers

in a hierarchy • User Z sometimes has data

rolling up to Manager A

Details – Business Challenge• The object we will report on is assigned to only one leaf

member of the hierarchy• The user running the query can be a manager to multiple

members of the hierarchy• The query should show all records at or below the Managers

top-level node• Standard Role Hierarchy clearly doesn’t solve this!

Technical Solution – Query “My Team” records• Formula is needed to determine “My Team” query• “My Team” cannot be indexed• “My Team” references Normalized data on Hierarchy record

• If(Lead.Hierarchy__r.Parent__r.Parent__r.Level__c == User.Level__c…• If(Lead.Hierarchy__r.Parent__r.Level__c == User.Level__c…

• First try: Formula complexity taxed the query optimizer• Lead object contains more than 10 million rows at any given

time

Denormalization & static values solve our problem

All the pieces put together…• Store de-normalized hierarchy level values on Lead• Store User’s top-level hierarchy value on User• Simplify “My Team” formula to compare just those two sets

of fields

And the results querying from 10 million rows…• First Iteration: 30 second query returning 1000 rows

• Using formulas to compare normalized reference fields

• Second Try: 6 second query returning 1000 rows• Reduced formula to use de-normalized static values• Still not fully performant because formula is not indexed

• Final Result: subsecond query returning 150 rows• Negotiated business requirement change to only show

“Last 30 Days” of created Lead data• A selective query using an index always wins!

Takeaways …

Index selective formula fields!

Reduce run-time overhead!

Related DevZone hands-on, mini-workshop

Wednesday

9:15AM or

3:45PM

Bud Vieira

Architect Evangelist@aavra

Daisuke Kawamoto

Architect Evangelist@daisukeSfdc

Nathan Shilling

Regional Practice Lead Certified Technical Architect

Appirio@nds9619

We want to hear from YOU!

Please take a moment to complete our session survey

Surveys can be found in the “My Agenda” portion of the Dreamforce app

Hands-on Work Shop @ Developer Zone

Wednesday 9am & 1 pm

Appendix

Formula Limits