Viz: A Visual Analysis Suitefor Explaining Local Search

Behavior

Steven Halim (School of Computing, NUS)Roland Yap Hock Chuan (School of Computing, NUS)Lau Hoong Chuin (School of Information Systems, SMU)

User Interface Software and Technology 2006Montreux, Switzerland

http://www.comp.nus.edu.sg/~stevenha/viz

OutlineShort Background:

Local Search (LS) for attacking NP-hard Combinatorial Optimization Problems (COPs),

The problem: Understanding and Tuning LS…Our Approach:

Explaining Local Search using Visualization (Viz)

Viz DEMOQ & A

Local Searche.g. Tabu Search, Iterated Local Search,

Simulated Annealing…is commonly used for attacking NP-hard

(intractable) Combinatorial Optimization Problem (COP):e.g. Traveling Salesman Problem (TSP)

Simplified explanation of Local Search:

Background – Local Search

Local Search algorithms have tunable configurations:Parameter values,Components, and/or Search strategies.

These configurations affects its behavior performance…

To make a LS perform well in context is not straightforward.Its heuristic/stochastic behavior is not well

understood.Empirical analysis is needed.

Background – Tuning Problem

Bottleneck !!!

Common Practice: Ad Hoc Tuning

Implement the local search

Evaluate its performanceTextually/Statistically…

Good or Give up

Not good

Stop

Modify the local search using something??

(“ad hoc”)

We propose: Visual Diagnosis

Implement the local search

Evaluate its performanceUsing Visualization Tool…

Good or Give up

Not good

Stop

Modify the local search after understanding what

is happening.

The tool helps us to think more systematically.

Why Use Visualization?Visualization is a cognitive tool.Human brain is a powerful pattern finding

machine.As long as the data is visualized in a way that

conform with the rules of human visual perception.

Information Visualization: “The use of interactive visual representations of abstract data to amplify cognition.”Facilitates hypothesis formation in empirical analysis.Allows users to answer questions they didn't know

they had.

Explaining Local Search Behavior

via Visualizationhttp://

www.comp.nus.edu.sg/~stevenha/viz

Explaining Local Search BehaviorIt is more useful to understand how and why

something works,Rather than just simply know that something works…

By understanding the underlying principles,We can adjust properly if presented with different scenarios!

To understand LS, we need to answer these questions:

1. Does it behave like as what we intended?2. How good is the local search in intensification?3. How good is the local search in diversification?4. Is there any sign of cycling behavior?5. How does the local search algorithm make progress?

Explaining Local Search Behavior6. Where in the search space does the search spend

most of its time?7. How far is the starting (initial) solution to the global

optima/best known solution?8. Does the search quickly find the global optima/best

known solution region or does it wander around in other regions?

9. How wide is the local search coverage?10. What is the effect of modifying a certain search

parameter/component/strategy w.r.t the search behavior?

11. How do two different algorithms compare?

The behavior of heuristic and stochastic local search

is hard to explain without appropriate tools!

How to Understand LS Behavior?Various local search analysis tool:

Data Analysis (text-based):Solution QualityRobustnessSpeed/Running TimeRun Time/Length Distribution [Hoos 98; Hoos &

Stuetzle, 05]Fitness Landscape (Statistical) Analysis [Jones, 95;

Watson et al., 05]Information Visualization:

Visualization of Fitness Landscape/properties/search behavior

Visualizations for Local SearchExisting Visualizations:

Objective Value over TimeFitness Distance Correlation [Jones, 95; Watson et al.

05]Problem Specific, e.g. TSP [Klau et al., 02]2-D Animation of 2 Dimensional Problem [Syrjakow &

Szczerbicka, 99]N-to-2-Space Mapping [Kadluczka, 04]Search Trajectory Analysis [Lau et al., 05; Halim et al.,

06a,b]

Analogy of Search Trajectory Visualization:Finding the Highest Mountain

1. Without anchor points, the behavior of the pink trajectory is hard to be explained.

2. Do several local search runs with different configurations, record diverse local optima/anchor points (circled).

3. With anchor points, the behavior of the pink trajectory is as follows: trapped in region that contains red/blue anchor points, thus failed to visit good solutions, the green/orange anchor points.

4. The behavior of the pale blue trajectory is as follows: after reaching a local optima, it diversifies to another place. It manages to reach green and orange anchor points, and thus its performance is better than the pink trajectory in Figure 3.

Search Trajectory VisualizationIn Practice

Layout the points in Abstract 2-D Space•Points that are close in N-dimensional space in terms of distance metric (hamming, permutation distance, etc)are laid out close to each other in the abstract 2-D space and vice versa.•This utilizes human strength in discerning 2-D spatial information.

Layout First Phase: •The anchor points are measured with each other using distance metric.•The anchor points are installed greedily in abstract 2-D space •Re-optimize using the Spring Model layout algorithm.

Layout Second Phase: •Again, using Spring Model algorithm, the points along search trajectory are aid out in abstract 2-D space using these anchor points as reference.

Presentation Aspects:•Color coding is used to enhance our understanding: blue: good, green: medium, brown: poor anchor points.•The search trajectory is animated over time.

Anchor Points are quite close to each other.

Trapped in cycling near Anchor Point C

Only covers regions near Anchor Point D and E

Anchor Points are quite close to each other, but their quality are different.

Trapped in cycling near Anchor Point C because it is attractive (green)

Only covers regions near Anchor Point D and E, which are good regions (blue and green)

Explaining Local Search Behavior

using Viz: Local Search Visual Analysis

Suitehttp://

www.comp.nus.edu.sg/~stevenha/vizNote: we use the updated version of Viz

for the demo.

Design GoalsEnable algorithm designer (human) to understand LS

behavior.Must obey Information Visualization Rules:

Overview first, Zoom and Filter, then Details on Demand.Utilize human visual perception strengths.Interactive.Responsive.

Aim for widespread use:Generic:

Support various (/all) Local Searches.Support various (/all) Combinatorial Optimization Problems.

Viz as Analysis Suite.Facilitates information sharing with other algorithm

designers.

Using Viz

Viz Features (Overview)Multi-Source VisualizationsVisual ComparisonAnimated Search PlaybackInformation Visualization + Data AnalysisColor and HighlightingMultiple Level of DetailsViz GUI is:

User friendly, use familiar widgetsInteractiveResponsiveCustomize-able

Multi-Source VisualizationsThe local search playback are viewed

from several angles:Generic

Search TrajectoryObjective ValueFitness Distance CorrelationEvent Bar

Algorithm SpecificProblem Specific

Coordinated Visualizations

Human is creature of comparison.Actually, human is not a perfect judge.

But, good at relative comparison when given a baseline.Two comparison modes:

JuxtaposeSuperimpose

Useful for understanding:Behavior: Same LS & configuration, different parts of

searchRobustness: Same LS & configuration, two runsEffect of Modifications: Same LS, different configurationsOverall behavior: Two totally different LS

Visual Comparison

AnimationsLocal Search info is too big: cannot be presented at

one time.But rather: animated over time.

(the search trail is drawn using alpha blending).

Local Search typically runs in thousand iterations…Analyzing from start to end may be overwhelming

We use Event Bar for quick reference to interesting regions.Viz computes highlights/index points for generic events

(e.g. new best found solution)

Animated Search Playback

Info Visualization + Data AnalysisWe combine strengths from both worlds.

Information VisualizationData Analysis (Textual/Statistics)

Viz DEMO

http://www.comp.nus.edu.sg/~stevenha/viz

Viz is Useful for:Local Search Algorithm Analysis tool:

IndividualAddressing Tuning ProblemDebuggingImproving algorithm

CollaborativeResearchers can annotate and publish *.viz file for

information exchange.Benchmarking

Pedagogical tool:For Studying/Teaching Local Search Algorithms.

Presentation tool: To illustrate behavior of a new Local Search Algorithm.

SummaryWe have seen that proper visualization can be used to help

explaining/understanding local search behavior.Some of these are hard to derive without proper visualizations.A necessary step for tuning/improving local search.

To be useful, a visualization tool should utilizehuman visual perception strengths.

Viz’s capability in explaining local search is so far promising.Lessons for UIST:

Regardless whether your research is on Local Search or not. Hopefully you can get something from what I share here with my visualization tool Viz

Careful application and extrapolation of existing techniques in a new domain can result in fascinating extensions. (multidisciplinary).

Many other fields can be enhanced using proper Information Visualization tools, developing specific visualization for these domains may be fruitful.

End of TalkTime for Q & A

Thank you for your attention

For further details and to download Viz, please visit:

http://www.comp.nus.edu.sg/~stevenha/viz