Thursday, 4 May 2017

Security-oriented Architecture

Split your monoliths into myriad services and suddenly your established security boundaries fizzle into non-existence.

I would assert that all services need to think about security, namely:
  • authentication: who is calling me?
  • authorisation: are they allowed to call me?  are they allowed to request this particular data?
  • validation: is the message valid?
  • confidentiality: is my sensitive data secure?  what is sensitive data?
  • integrity: how do I know the request has not been tampered with?
  • non-repudiation: how do I know the request came from a legitimate source
Yes, all services, even the internal ones.  Don't think of hackers as spotty teenage opportunists.  Think of  them as sophisticated criminal organisations that spend millions on stealing precious corporate data, or threatening brands with ransom attacks.  Loyal as I'm sure your staff are, are they immune to greed, revenge or blackmail?

That's not to say you might not answer the questions above differently for internal services, as opposed to edge-facing ones, but consider these issues you must.


Share:

Tuesday, 24 March 2015

Microservices: New spin, old hat?


This year's QCon 2015 conference in London was abuzz with talk of microservices, and if you, like me, have seen protocols and standards like CORBA, RMI, J(2)EE and SOAP come (and go), you would be forgiven for sighing and thinking "what's new"?  However, it seems that Google, Amazon, eBay and others would beg to differ.

In a nutshell, microservices:
  • Are small (they fit in your head)
  • Run in their own process
  • Are independently replaceable, upgradeable and deployable via automated/test deployment mechanisms
  • Are backwardly (and forwardly) compatible
  • Communicate with each other via lightweight mechanisms, typically an HTTP resource API
  • Are organised around business capabilities.  "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure." -- Melvyn Conway, 1967
  • Involve products, not projects
  • Require decentralised governance
  • Require decentralised data management (independent persistence)
  • Automate their infrastructure (e.g. AWS)
  • Are designed for failure of dependent services
  • Prefer asynchronous calls to synchronous ones
  • Involve evolutionary design
  • Are fully developed, owned and supported by single teams
  • Compete with each other in a market-place
Much of this will be familiar, but what is new and very interesting is the degree to which microservices apply the familiar principles of loosely-coupled, encapsulated software components to the organisation developing and managing those components.

Traditionally structured companies will balk at all of this, and perhaps dismiss this as yet another architectural fad, but I'm not so sure.  Check out these two excellent videos on Spotify's Engineering Culture.







Further reading:
Share:

Monday, 16 March 2015

Supporting Your Public Web API


No matter how beautiful and well-designed your new web API is, unless it is well supported, you may find that you become a victim of your own success.  Below are 4 suggested key elements of a supportable web API, based on my experiences at IG.


1. Documentation, documentation, documentation!

Unless it is easier to refer to your API documentation than to contact you, you will be contacted.  Don't assume anything is obvious.  Document your API in detail. Define your business terms.  Explain common interaction patterns.  Provide client code snippets and working samples in a variety of language bindings.   Provide a FAQ.

Consider providing a client SDK (Yes,I know REST is meant to be easy, but the reality is that unless your client is already a developer, they will struggle.  Make it easy for them to use your API.)

Consider providing a moderated forum to allow clients to answer each other's questions.  Provide an interactive API explorer allowing clients to try out APIs and see the raw HTTP requests and responses.


2. First and Second Line Support

Provide a "contact us" form instead of direct phone numbers.  The page should encourage clients to refer to API documentation, FAQ, and forum (if one exists).

Provide first line, "customer sensitive" support to answer common queries, or at least to ensure queries are well formulated before passing onto second line support.  If your organisation already has a customer services help desk, use them! They may balk initially at the perceived technical nature of the support, but should be helped past this by through training and provided access to simple debugging tools, e.g. the API explorer or a facility to search service log files.

Provide dedicated second line support via a rota consisting of the developers who built the API.

Maintain a searchable log of frequently answered questions (and answers).  Update the public FAQ from this log.


3. Effective error handling

Give API clients as much error information as possible in request responses.  If inputs are incorrect - tell them why.  If error codes are used - ensure these are documented.

Log every request in detail, including request authentication headers and body (subject to client privacy constraints).  Use a transaction id pattern to allow the flow of API requests to be tracked across multiple back-end systems.  Provide easy access to system logs, e.g. via Splunk, which provides a searchable index over a number of configured log files.  Splunk additionally allows requests to be charted and management dashboards to be created.  Senior management will want to know just how popular the API is.


4. Provide a barrier to entry

If, despite all of the above, the demand for your API is still overwhelming, consider:

  • throttling your API, with chargeable tiers for increased quotas
  • requiring that a verified account to be opened before an API can be used
  • providing support to valued customers only (or on a subscription basis).




Share:

Wednesday, 14 January 2015

What is SOA?

What is SOA?  SOA or Service-Oriented Architecture is a software architecture pattern in which applications or systems are constructed from underlying (and usually distributed) software services that conform to a specific set of characteristics, namely:

1.Contract based
2. Location transparency
3. Autonomous
4. Abstract
5. Reusable
6. Composable
7. Stateless
8. Discoverable
9. Extensible
10. Loosely coupled

The primary goal of SOA is software development agility, i.e. the ability to respond the change easily, and cheaply, thus allowing businesses to rapidly respond to changing markets.

Services are typically (but by no means exclusively) implemented as web services, i.e. they operate over the ubiquitous web HTTP protocol, and are implemented either using XML-based SOAP or the lightweight (and more popular) REST paradigm.

The highly distributed nature of this architecture has resulted in a need for a runtime platform (the Enterprise Service Bus, ESB) to help manage the operation of these services, but also to handle complex enterprise integration scenarios involving multiple (and often legacy) platforms, protocols and security models - products like Oracle Service Bus or the open source Mule ESB.

In addition to being an integration solution, the service-oriented pattern also offers an opportunity for easier implementation of high-level business processes using Business Process Management (BPM) solutions that are often offered as part of the ESB product suite.

Service-oriented architecture is highly relevant to the rapidly growing cloud computing model where applications are built in the cloud as orchestrations over public cloud-based APIs, and we are seeing ESB vendors like WSO2 and MuleSoft in or moving into this space, offering cloud-based versions of their products, as well as API management products.

Some will argue that SOA is not new, and that distributed programming involving services has existed for a long time, and they are right.  However those early efforts stumbled on the hurdles of platform incompatibility and standards complexity.  Now, with ubiquitous, and easy to use protocols like REST, web service API development is growing rapidly, and the integration possibilities seem endless.

(What is SOA, Service Oriented Architecture)

Share:

REST Web Service Paradigm Shift

Moving your Service-Oriented Architecture (SOA) from RPC style (web) services to SOAP document-oriented services, or REST resource-oriented services requires a difficult mental paradigm shift.  The same kind of shift that was required for monolothic and procedural programmers to move to object-oriented programming styles.  This change is not easy, and not all will pass muster.  Failing to make the change will mean the difference between Just a Bunch of Web Services (JABOWS) or a fully-fledged SOA.

Ensuring that this mind shift does happen will require good organisation and strong, well-articulated guidance and governance.  Until the new pattern becomes habit, people will very quickly slip into old habits. 

Below are some guidelines and implementation suggestions from this blog:

Share:

Saturday, 6 September 2014

Favourite Funny Programming Jokes

funny programming joke picture - c++ c you have no class

“Knock, knock.”
“Who’s there?”
...very long pause....
“Java.”

~

Programming is like sex:
One mistake and you have to support it for the rest of your life.

~

The generation of random numbers is too important to be left to chance.

~

Two strings walk into a bar and sit down. The bartender says, “So what’ll it be?”

The first string says, “I think I’ll have a beer quag fulk boorg jdk^CjfdLk jk3s d#f67howe%^U r89nvy~~owmc63^Dz x.xvcu”

“Please excuse my friend,” the second string says, “He isn’t null-terminated.”

~

A programmer was crossing a road one day when a frog called out to him and said, "If you kiss me, I'll turn into a beautiful princess."

He bent over, picked up the frog and put it in his pocket. The frog spoke up again and said, "If you kiss me and turn me back into a beautiful princess, I will stay with you for one week."

The programmer took the frog out of his pocket, smiled at it and returned it to the pocket. The frog then cried out, "If you kiss me and turn me back into a princess, I'll stay with you and do anything you want." Again the programmer took the frog out, smiled at it and put it back into his pocket.

Finally, the frog asked, "What is the matter? I've told you I'm a beautiful princess that I'll stay with you for a week and do anything you want. Why won't you kiss me?"

The programmer said, "Look, I'm a programmer. I don't have time for a girlfriend -- but a talking frog, now that's cool!"
Share:

Thursday, 10 July 2014

Excel REST Trading API Example

As much as I have a love-hate relationship with Microsoft, I have to admit the Excel has won me over as a power-user tool, particularly in the world of finance, where it may be used to access powerful back-end analytics servers to provide real-time yield curves and such like.

REST APIs such as IG's Web Trading API make this even easier, as this Excel VBA code snippet to get a list of open positions will hopefully demonstrate.  The sample assumes that a login /session request has already been executed and the client and account session tokens have been obtained.  The API key is the unique authorisation token required to access the API.

Public Function positions() As Collection

    ' Set up the HTTP access framework
    Set oXMLHTTP = CreateObject("MSXML2.XMLHTTP.6.0")
    ' Set the API URL
    Call oXMLHTTP.Open("GET", IG_API_HOST + "/positions", False)
    ' Set the HTTP request headers:
    ' Set the account session token
    Call oXMLHTTP.SetRequestHeader("X-SECURITY-TOKEN", m_accountToken)
    ' Set the client session token
    Call oXMLHTTP.SetRequestHeader("CST", m_clientToken)
    ' Set the API key
    Call oXMLHTTP.SetRequestHeader("X-IG-API-KEY", m_apiKey)
    ' Set the content type to JSON
    Call oXMLHTTP.SetRequestHeader("Content-Type", "application/json; chartset=utf-8")
    ' Set the requested response type to JSON
    Call oXMLHTTP.SetRequestHeader("Accept", "application/json; chartset=utf-8")
    ' Execute the request
    Call oXMLHTTP.send
    
    If oXMLHTTP.Status = 200 Then 
        ' Successful. A list of zero or more positions will have been returned
        ' Extract the response as a JSON object
        Dim data As Dictionary
        Set data = JSON.parse(oXMLHTTP.responseText)
        ' Get the list of positions from the response object
        Dim positions as Collection
        Set positions = data.Item("positions")
        ' If positions exist, iterate over the list         If Not positions Is Nothing Then
            Dim aPosition As Object
            ' Do something with the position, e.g. extract the instrument name of the underlying market
            For Each aPosition In positions
                  Dim instrumentName as String
                  Set instrumentName = aPosition.Item("market").Item("instrumentName")
                  ' etc
            Next
    Else 
        ' An error occurred
        MsgBox oXMLHTTP.responseText
        Set positions = Nothing
    End If
        
End Function
Share:

Monday, 30 June 2014

IG REST Trading API

Hot off the press is the news that IG (my esteemed employer) has launched its REST Trading API. 

The API provides REST-based and streaming access to a range of popular asset classes (including indices, forex, commodities, binaries and options), and includes functionality to:
  • Trade using risk management tools such as stops and limits
  • View real-time and historical market prices
  • Analyse market instrument and client sentiment information
  • Review account balances and trade history


IG already offers a well-established FIX API for corporate clients, but now with its new REST Trading API, IG's technology platform becomes much more easily and widely accessible, allowing for example:

  • Corporate partner integration without the start-up costs and complexity of FIX 
  • Trading strategy automation
  • Retail trading app development


The REST Trading API labs site offers getting started guides, API documentation, a downloadable sample trading app, and an interactive API companion app for trying out the REST APIs.  



The API will only be available in the UK initially, but the wider roll-out is imminent, so be sure to follow the labs site on Facebook or Twitter for updates.  

Related:

·         ForexMagnates: IG Opens API to Public Development, Launches IG Labs
·         FOW.com: IG rolls out new tech offering for traders


Share:

Tuesday, 19 February 2013