down the rabbit hole with typed data -...
TRANSCRIPT
Down the Rabbit Hole with Typed Data
http://2015.midcamp.org/session-proposal/down-rabbit-hole-typed-data
Matthew Radcliffe mradcliffe
@mattkineme
Use Case: Xero Integration• Xero accounting platform with a nice Restful API
• Post invoices from Open Atrium cases. Invoice done. Click send. Back to coding.
• Post bank transactions from Commerce / Ubercart payment transactions. Auto-matching Bank Statements to Bank Transactions is a huge cost savings for a small company.
• PHP-Xero
• It’s abandoned - upstream fixes no longer an option.
• A fork on Github will never be canonical.
• xero declares OAuth :(
Escaping Procedural Reality
if ( (count($arguments) == 0) || ( is_string($arguments[0]) ) || ( is_numeric($arguments[0]) ) || ( $arguments[0] === false ) ) { $where = false; //it's a GET request if ( !in_array($name, $valid_get_methods) ) { return false; } $filterid = ( count($arguments) > 0 ) ? strip_tags(strval($arguments[0])) : false; if(isset($arguments[1])) $modified_after = ( count($arguments) > 1 ) ? str_replace( 'X','T', date( 'Y-m-dXH:i:s', strtotime($arguments[1])) ) : false; if(isset($arguments[2])) $where = ( count($arguments) > 2 ) ? $arguments[2] : false; if ( is_array($where) && (count($where) > 0) ) { $temp_where = ''; foreach ( $where as $wf => $wv ) { if ( is_bool($wv) ) { $wv = ( $wv ) ? "%3d%3dtrue" : "%3d%3dfalse"; } else if ( is_array($wv) ) { if ( is_bool($wv[1]) ) { $wv = ($wv[1]) ? rawurlencode($wv[0]) . "true" : rawurlencode($wv[0]) . "false" ; } else { $wv = rawurlencode($wv[0]) . "%22{$wv[1]}%22" ; } } else { $wv = "%3d%3d%22$wv%22"; } $temp_where .= "%26%26$wf$wv"; } $where = strip_tags(substr($temp_where, 6));
• XeroOAuth-PHP
• Xero went off in their own direction.
• Tried it, contributed to it, no thanks.
• Suffers from similar code standard/quality issues as PHP-Xero.
Escaping Procedural Reality
• “xero make”: model data as associative arrays.
• “xero query”: pass-through for PHP-Xero library.
• “form helper”: builds form array snippets.
• “xero reference”: field type/widget/formatter.
Escaping Procedural Reality
• Essentially:
• Make HTTP OAuth--signed Requests
• of Xero API Types
• structured as XML
Escaping Procedural Reality
Through the Drupal 8 Looking Glass
• Guzzle 3*
• Extendable, Catchable exceptions, OAuth, XML/JSON encoding, etc…
• XeroBundle provided XeroClient:
• Library code is small and maintainable.
• Inherits what it does from Guzzle Client.
* At the time I stared through the Looking Glass, Guzzle 3 was the version in core. Guzzle 5 is included in Drupal 8 core at the time of writing this session.
• Symfony 2
• Serializer* transform data from/to formats.
• Serialization module
• Extends Serializer to transform Drupal typed data from/to formats.
Through the Drupal 8 Looking Glass
* The Serializer Component. http://symfony.com/doc/current/components/serializer.html
• Typed Data
• A way of describing and managing complex data types.
Through the Drupal 8 Looking Glass
Figuring out Xero Types
• Why Typed Data?
• Need to describe Xero types into something Drupal and Symfony understand.
• Data types: primitive and complex
• Are these entities or not?
Xero Types• Accounts
• Invoices
• Payments
• Credit Notes
• Contacts
• Users
• Bank Transactions
• Amount
• Line Items
• and more…
Xero Types• Accounts
• Invoices
• Payments
• Credit Notes
• Contacts
• Users
• Bank Transactions
• Amount
• Line Items
• and more…
Xero Types• Accounts
• Invoices
• Payments
• Credit Notes
• Contacts
• Users
• Bank Transactions
• Amount
• Line Items
• and more…
Xero Types• Accounts
• Invoices
• Payments
• Credit Notes
• Contacts
• Users
• Bank Transactions
• Amount
• Line Items
• and more…
Oh, Dear! I’ll be Late for Release!• Typed Data, the white rabbit; a means to let
me use Serializer and Guzzle.
• And so down I went after it*.
• In the past couple of years, Typed Data API has changed and so I chase after it.
• One of the remaining Drupal 8 APIs with little documentation.
* Alice in Wonderland. Lewis Carroll. 1865. Page 4. PDFFreeBooks.org. 2015.
Typed Data API Docs
https://www.drupal.org/node/1794140
My Personal Challenge: Request edit permissions for this documentation page Sunday. Work on PostgreSQL and
SQLite instead.
How to read a Map• Typed Data API provides most of these, yay!
• There’s one complex data type that is not an entity.
• Map: the new associative array.
• Next challenge: how to create this elusive data type class.
<?php namespace Drupal\xero\Plugin\DataType; !/** * Xero Payment type. * * @DataType( * id = "xero_payment", * label = @Translation("Xero Payment"), * definition_class = "\Drupal\xero\TypedData\Definition\PaymentDefinition" * ) */ class Payment extends XeroTypeBase { protected function getPropertyDefinitions() { return $this->definition->getPropertyDefinitions(); } }
plugin typeplugin id plugin label
Data Definitions• Complex data type needs definition classes to
describe its properties (or child data types).
• Symfony 2 Constraints
• Label
• Property data type
• When I began, definitions were associative arrays.
class PaymentDefinition extends ComplexDataDefinitionBase { public function getPropertyDefinitions() { if (!isset($this->propertyDefinitions)) { $info = &$this->propertyDefinitions; ! $info['Invoice'] = DataDefinition::create(‘xero_invoice’) ->setRequired(TRUE) ->setLabel('Invoice'); ! $info['Account'] = DataDefinition::create(‘xero_account’) ->setRequired(TRUE) ->setLabel('Account'); ! $info['Date'] = DataDefinition::create(‘string’) ->setRequired(TRUE) ->setLabel('Date'); ! $info['Amount'] = DataDefinition::create(‘float’) ->setRequired(TRUE) ->setLabel('Amount'); } return $this->propertyDefinitions; } }
List Data Definition
• Data definitions can be lists of data types i.e. multiple values or an indexed array equivalent.
• This is important for Xero because the API returns a list of items in addition to other odd types like Tracking Categories, Line Items, and Addresses.
MAD Tip
• New in 2015 that could save time: ClassiPHPy
• https://github.com/EclipseGc/Classiphpy
• Generate your definition classes via associative arrays with Pharborist.
• New in 2015 that could save time: ClassiPHPy
• https://github.com/EclipseGc/Classiphpy
• Generate your definition classes via associative arrays with Pharborist.
drupal module upgrader uses Pharborist to transform Drupal 7 PHP into Drupal 8 PHP (if possible).
Extra Bonus MAD Tip
MAD Tip
Denormalize Me• Normalizer classes transform data into Typed
Data and vice versa.
• Normalization is obscured from use because it runs as part of serialization.
• “$context” was a difficult parameter to understand.
• TypedDataManager use is recursive and confusing.
• Normalization classes must be defined as services in the service container.
• Must have a tag so that Symfony 2 knows to treat it differently than other services.
Denormalize Me
services: xero.normalizer: class: Drupal\xero\Normalizer\XeroNormalizer arguments: ['@typed_data_manager'] tags: - { name: normalizer }
xero.services.yml
class XeroNormalizer extends ComplexDataNormalizer implements DenormalizerInterface { ! protected $supportedInterfaceOrClass = 'Drupal\xero\TypedData\XeroTypeInterface'; ! public function __construct(TypedDataManager $typed_data_manager) { $this->typedDataManager = $typed_data_manager; } ! public function denormalize($data, $class, $format = NULL, array $context = array()) { if (!isset($context['plugin_id']) || empty($context['plugin_id'])) { throw new UnexpectedValueException(‘...’); } ! $name = $class::$xero_name; $plural_name = $class::$plural_name; ! // Wrap the data in an array if there is a singular object returned. if (count(array_filter(array_keys($data[$plural_name][$name]), 'is_string'))) { $data[$plural_name][$name] = array($data[$plural_name][$name]); } ! $list_definition = $this->typedDataManager->createListDataDefinition($context['plugin_id']); $items = $this->typedDataManager->create($list_definition, $data[$plural_name][$name]); ! return $items; }
Building the Mock Turtle
• Unit tests are awesome and fast, but Typed Data Manager is big and scary. :(
• Typed Data calls itself so mocking is not straight-forward. :(
• Need to mock the service container. :(
• What is necessary to have decent coverage and reduce complexity?
PHPUnit
• DataDefinitionInterface::getPropertyDefinitions seems to be easy to generate coverage for because it’s just returning a an array of more data definitions.
• Am I getting anything out of this test, or is it just cruft to make my coverage green?
class PaymentDefinition extends ComplexDataDefinitionBase { public function getPropertyDefinitions() { if (!isset($this->propertyDefinitions)) { $info = &$this->propertyDefinitions; ! $info['Invoice'] = DataDefinition::create(‘xero_invoice’) ->setRequired(TRUE) ->setLabel('Invoice'); ! $info['Account'] = DataDefinition::create(‘xero_account’) ->setRequired(TRUE) ->setLabel('Account'); ! $info['Date'] = DataDefinition::create(‘string’) ->setRequired(TRUE) ->setLabel('Date'); ! $info['Amount'] = DataDefinition::create(‘float’) ->setRequired(TRUE) ->setLabel('Amount'); } return $this->propertyDefinitions; } }
Typed Data Mocking• The create static methods for Typed Data will
call its manager:
• Mocks cannot just return a single known value for complex data types
• Mocks must carefully re-construct the order of how create will be called on all definitions in a complex data definition.
public function createInstance($data_type, array $configuration = array()) { $data_definition = $configuration['data_definition']; $type_definition = $this->getDefinition($data_type); ! if (!isset($type_definition)) { throw new \InvalidArgumentException(format_string('Invalid data type %plugin_id has been given.', array('%plugin_id' => $data_type))); } ! // Allow per-data definition overrides of the used classes, // i.e. take over classes specified in the type definition. $class = $data_definition->getClass(); if (!isset($class)) { throw new PluginException(sprintf('The plugin (%s) did not specify an instance class.', $data_type)); } return $class::createInstance($data_definition, $configuration['name'], $configuration['parent']); }
public function setUp() { // Typed Data Manager setup. $this->typedDataManager = $this->getMockBuilder('\Drupal\Core\TypedData\TypedDataManager') ->disableOriginalConstructor() ->getMock(); ! $this->typedDataManager->expects($this->any()) ->method('getDefinition') ->with(static::XERO_TYPE, TRUE) ->will($this->returnValue(['id' => static::XERO_TYPE, 'definition class' => static::XERO_DEFINITION_CLASS])); $this->typedDataManager->expects($this->any()) ->method('getDefaultConstraints') ->willReturn([]); ! // Validation constraint manager setup. $validation_constraint_manager = $this->getMockBuilder('\Drupal\Core\Validation\ConstraintManager') ->disableOriginalConstructor() ->getMock(); $validation_constraint_manager->expects($this->any()) ->method('create') ->willReturn([]); $this->typedDataManager->expects($this->any()) ->method('getValidationConstraintManager') ->willReturn($validation_constraint_manager); !
public function testGetPhoneNumber() { $string_def = DataDefinition::create('string'); $country = new String($string_def); $area = new String($string_def); $number = new String($string_def); ! $this->typedDataManager->expects($this->any()) ->method('getPropertyInstance') ->with($this->phone, $this->callback(function($subject) { return in_array($subject, array( ‘PhoneCountryCode', ‘PhoneAreaCode', ‘PhoneNumber') ); })) ->will($this->onConsecutiveCalls($country, $area, $number)); ! $this->phone->set('PhoneCountryCode', '01'); $this->phone->set('PhoneAreaCode', '805'); $this->phone->set('PhoneNumber', ‘255-8542'); ! $this->assertEquals('01-805-255-8542', $this->phone->getPhone()); } }
\Core\Validation\ConstraintManager') ->disableOriginalConstructor() ->getMock(); $validation_constraint_manager->expects($this->any()) ->method('create') ->willReturn([]); $this->typedDataManager->expects($this->any()) ->method('getValidationConstraintManager') ->willReturn($validation_constraint_manager); ! // Mock the container. $container = new ContainerBuilder(); $container ->set('typed_data_manager', $this->typedDataManager); \Drupal::setContainer($container); ! // Create data definition $definition_class = static::XERO_DEFINITION_CLASS; $this->dataDefinition = $definition_class::create(static::XERO_TYPE); } }
A poem about mocking the container
What I learn not to do
I resign myself to that fate.
Besides, core does it too.
So please do not hate
this container so small
is not actually in my app at all.
I do not load,
nor invoke the container directly
to get variables in my code.
I use dependency injection, actually.
So forgive the misuse I leverage.
All I want is pretty coverage.
Alice’s Evidence
• Typed Data helps to use Drupal 8 effectively:
• Dependency Injection
• Better code
• Code re-use on and “off the island”.
public function buildForm(array $form, FormStateInterface $form_state) { ! $config = self::config('xero.settings'); ! $account_options = array(); ! try { $context = array('plugin_id' => 'xero_account'); $accounts = $this->query ->setType($context['plugin_id']) ->setMethod('get') ->setFormat('xml') ->execute(); ! foreach ($accounts as $account) { // Bank accounts do not have a code, exclude them. if ($account->get('Code')->getValue()) { $account_options[$account->get('Code')->getValue()] = $account->get('Name')->getValue(); } } } catch (RequestException $e) { } catch (\Exception $e) { } ! $form['defaults']['account'] = array( '#type' => 'select', '#title' => t('Default Account'), '#description' => t('Choose a default account.'), '#options' => $account_options, '#default_value' => $config->get('defaults.account'), );
public function __construct(XeroClient $client, Serializer $serializer, TypedDataManager $typed_data, LoggerChannelFactoryInterface $logger_factory) { $this->client = $client; $this->serializer = $serializer; $this->typed_data = $typed_data; $this->logger = $logger_factory->get('xero'); }
public function execute() { try { $this->validate(); $this->explodeConditions(); ! $data_class = $this->type_definition['class']; $endpoint = $data_class::$plural_name; $context = array('plugin_id' => $this->type); ! if ($this->data !== NULL) { $this->options['body'] = $this->serializer->serialize($this->data, $this->format, $context); } ! $request = $this->client->{$this->method}($endpoint, $this->options['headers']); ! // Add query parameters via getQuery() instead :-( if (!empty($this->options['query'])) { $q = $request->getQuery(); foreach ($this->options['query'] as $key => $value) { $q->set($key, $value); } } ! $response = $request->send(); $data = $this->serializer->deserialize($response->getBody(TRUE), $data_class, $this->format, $context); ! return $data; }
Typed Data Summary
• Strong typing useful for RESTful APIs.
• It is complex, but powerful.
• And difficult to test.
• But what else..?
What else..?• The floor is open to ask, discuss, or share:
• Typed Data uses (e.g. entity adapters)
• Questions
• Other Drupal 8 APIs
• http://2015.midcamp.org/session-proposal/down-rabbit-hole-typed-data