Evolution-enabled application-programming interfaces

Ralf LämmelData Programmability Team

Microsoft, Redmond

API asbestos and its remedy

Source of the term “software asbestos”:Klusener, Lämmel, Verhoef:“Architectural modifications to deployed software”,2005, Science of Computer Programming

C++ code for window creation in pre-.NET

HWND hwndMain = CreateWindowEx( 0,"MainWinClass", "Main Window",WS_OVERLAPPEDWINDOW | WS_HSCROLL | WS_VSCROLL,CW_USEDDEFAULT, CW_USEDEFAULT,

CW_USEDDEFAULT, CW_USEDEFAULT,(HWND)NULL,(HMENU)NULL, hInstance, NULL);

ShowWindow(hwndMain, SW_SHOWDEFAULT);UpdateWindow(hwndMain);

.NET version

Form form = new Form();form.Text = "Main Window";form.Show();

Table of contents

• Call to arms

• More examples

• Categorization

• Techniques

• Show stoppers

Evolutionary API transformationsWhy is that a worthy research

challenge?

• Improve quality of APIs.• Support effective obsoletion.• Decrease platform complexity.• Orthogonal to model-driven xyz.• Improve productivity of developers.

API evolution samples

• CodeDom: program generation

• Xml.Schema: XML schema processing

• DOM vs. XML binding: API-fication

API asbestos:My first .NET program

Sigh!Main.cs(…,…): warning CS0618:

'System.CodeDom.Compiler.CodeDomProvider.CreateGenerator()‘ is obsolete.

Use CodeDom API for program generation CodeCompileUnit compileUnit = new CodeCompileUnit(); // ... build some cool code objects CSharpCodeProvider provider = new CSharpCodeProvider(); ICodeGenerator gen = provider.CreateGenerator(); IndentedTextWriter tw = new … // create a text writer gen.GenerateCodeFromCompileUnit(compileUnit, tw,

new CodeGeneratorOptions());

Typical developer support

CodeCompileUnit compileUnit = new CodeCompileUnit(); // ... build some cool code objects CSharpCodeProvider provider = new CSharpCodeProvider(); // ICodeGenerator gen = provider.CreateGenerator(); IndentedTextWriter tw = new … // create a text writer provider.GenerateCodeFromCompileUnit(compileUnit, tw, new CodeGeneratorOptions());

Elimination of the obsolete pattern

That is, we get rid of the extra generator object, anddirectly use the Generate… method of the provider object.

CodeCompileUnit compileUnit = new CodeCompileUnit(); // ... build some cool code objects CSharpCodeProvider provider = new CSharpCodeProvider(); // ICodeGenerator gen = provider.CreateGenerator(); IndentedTextWriter tw = new … // create a text writer provider.GenerateCodeFromCompileUnit(compileUnit, tw, new CodeGeneratorOptions());

How to …?

• Find offending “Create” calls.• Get a handle on factory (provider).• Find all uses of created generator.• Setup provider itself as generator.

More API asbestos:My second .NET program

Sigh!

Main.cs(…,…): warning CS0618: 'System.Xml.Schema.XmlSchema.Compile(System.Xml. Schema.ValidationEventHandler)' is obsolete: 'Use System.Xml.Schema.XmlSchemaSet for schema compilation and validation.'

Use Schema object model (SOM) for schema processing

XmlSchema schema = new XmlSchema(); // … do something cool with the schema schema.Compile(

new ValidationEventHandler(ValidationCallbackOne));

An attempted modernization

XmlSchema schema = new XmlSchema(); // … do something cool with the schema XmlSchemaSet set = new XmlSchemaSet(); set.Add(schema); set.ValidationEventHandler +=

new ValidationEventHandler(ValidationCallbackOne);

// schema.Compile( // new ValidationEventHandler( // ValidationCallbackOne)); set.Compile();

How to get this right? Is the above guess correct anyway?

Deprecated classes, methods, fieldsin Java world

Source: Henkel & Diwan: “CatchUp! Capturing and Replaying Refactorings to Support API Evolution”International Conference on Software Engineering, 2005

Burdens implied by API evolution

• API producer• Upward compatibility• Extending maintenance• Investments

• Tools• Processes• Documentation

• API consumer• Re-engineering for eradication• Complexity due to multiple versions

An example for API-fication or translation

• Given: a C# program that accesses XML business data through DOM.

• Wanted: a converted C# program that uses XML binding instead.

• Needed: (This is the naïve version.)• The relevant XML schema(s)• An XML binding technology• A dedicated C# program transformation• Potentially: IDE component

Untyped (DOM-based) XML functionality

Typeful XML functionality

Kinds of API evolution

• Refactoring• Service introduction• Service extension• Service elimination• Service restriction• API-fication• API translation

Example: Replacement of DOM by XML binding

• Refactoring: Prepare the introduction of XML binding technology• Translation: Replace DOM patterns by schema-derived object model• Extension: Inject additional validation functionality into the application

Compare to kinds of grammar adaptation:Lämmel: “Grammar Adaptation”,Formal Methods Europe, 2001

• Weak and strong obsoletion• Provision of facades• Provision of parallel versions• Migration guides, strategies, wizards• Byte-code transformations• Source-code transformations

Techniques for API evolution

What about refactoring?

Record refactorings on the API developer’s site

Replay refactorings on the API user’s site

Source: Henkel & Diwan: “CatchUp! Capturing and Replaying Refactorings to Support API Evolution”International Conference on Software Engineering, 2005

We cannot expect in general:• a refactoring suite to be sufficiently expressive,• refactoring to be practical for API developer,• a change scenario to be semantics-preserving.

So what?

Musts• Source-code transformation• Source-code model with high fidelity• Type analysis, simple DFA/CFA• Test harness for system under transformation• Simple feedback mechanisms

Shoulds• More advanced user interaction• More advanced DFA/CFA/effect analyses• API protocols based on temporal logic• Security models and other constraints• Coverage of multiple (“all”) languages• Story for 80/20 conversions

The envisaged transformation framework

Potential show stoppers

• Language issues

• Incompleteness issues

• Legal issues

Language issues

• Reflection

• Threads

• Side effects

• Third party frameworks

• Multitude of languages

Incompleteness issues

• API producer can’t convert

• API consumer won’t convert

Legal issues

• How to sell 80/20 solutions?• Can we promise correctness?• How to prove correctness to client?• Development may rely on client code?• Support may rely on client code?