Brandon Werner – Good friends, good coffee, and good ideas shared around a room.

Google Plus Closing: Your Apps Are (Probably) Impacted.

January 24, 2019January 28, 2019

Chances are you may have read the note about Google+ closing with a disinterested sigh. After all, you never integrated your app with Google+ social buttons or APIs.

You may want to think again.

If you added Google Sign-In to your application over the last eight years, you probably followed the advice given to developers at the time to use the Google+ People API. This was recommended even if you just wanted profile or email address of your users, something usually served by the ID token provided by the Google Identity service. You’ll want to make the change below, or you could start seeing failures in the next few months.

When Google Accounts were Google Plus Accounts

When Google released Google+ in 2011, they made significant changes to which APIs they wanted developers to call for user information. There was no reason to believe Google+ wouldn’t be successful, so these APIs seemed a future proof way to ensure developers would be getting the best information for a user, including the additional social information Google+ provided. Most of us in the cloud software industry know this impulse; we want to move customers and developers to where our resources are strategically focused and not duplicate APIs across different services.

All the ways to get user information

To understand Google’s motivations, it’s helpful to understand all the ways you can get data about a user on modern cloud platforms such as Google and Microsoft. This is due to our mutual commitment to standards, particularly OAuth2 and OpenID Connect.

Using the Identity Service

When a developer writes code to sign a user in to their application, chances are very good they are using OAuth2 and OpenID Connect. You can think of OpenID Connect as a layer above OAuth2 meant specifically to sign in users and prove they have an account with the identity service. All major identity services use OAuth2 and OpenID Connect, including Facebook, Amazon, Google, and Microsoft.

OpenID Connect specifies two ways to get information about the signed in user:

By looking in to the ID token which comes back to the app when authenticating the user.
By calling a UserInfo endpoint with an Access Token. Any additional information outside of what is in the ID token usually requires additional consent.

The original idea of the ID Token was for it to act as a verifiable calling card of sorts on the internet, able to be passed around from service to service so each could easily customize its experience to the user without needing to call a backend service (like the UserInfo endpoint).

In practice this use is much more restricted. Since the ID token can be passed around to other services on the internet, and that can cause disclosure of the Personally Identifable Information (PII), most of the industry has moved towards a very restricted set of PII claims in the ID token (email, Full Name, etc.) requiring a call to either the UserInfo endpoint or a Graph service for all other user data.

Using the Graph Service

Most modern platforms also have a “Graph API” that you can call to get information about a user.

For Microsoft this User graph endpoint is very extensive and well beyond what can be provided through a UserInfo endpoint. Google’s new People API is equally extensive.

All of these options to get user information leads to developer confusion. Although the ID token allows you to not need additional network calls to identify the user, most apps need updated data and end up calling the UserInfo endpoint anyway. Some SDKs built exclusively to the OAuth2 standards like App Auth will rely on UserInfo for user data, whereas SDKs from the services themselves like MSAL from Microsoft and Google SignIn will use the Graph API.

There is no easy answer. For years we at Microsoft resisted implementing the UserInfo endpoint and pointed developers to the Graph API to reduce developer confusion. Alas, to better support developers who want to use their SDK of choice we recently relented and added a UserInfo endpoint of our own.

Google Points Developers To The Graph

Google made much the same decision in recommending developers use their Graph API for Google+. Part of this change was to point developers who needed information about a Google (now Google+) user in their app to use the Google+ People API:

https://www.googleapis.com/plus/v1/people/me

This was the default guidance for the entire eight year span Google+ was available. This was maintained as interest in Google+ was reduced and Google Accounts lost their Plus branding.

Image from Google showing they reverted their Account Button branding back to before Google+ — Google recently reverted their Account Button branding back to before Google+

Why Your App May Be Impacted

Most app developers took a dependency on Google+ People API without knowing it, even if the rest of their app had no ties to Google+ at all. Since most app developers aren’t identity experts, they had little awareness they were taking this dependency or the other options available to them.

This API is now closing down.

This means soon developers will begin seeing failures in their applications. You can easily see if you’re impacted by searching for the /v1/people/me endpoint in your code.

How To Fix Your Google Identity Platform Sign-In

There are two paths to fixing your Google sign-in. Which you choose depends on how you use the Google platform.

Option 1: Use the ID token you get from Google

If you are only using Google accounts to sign-in users to your app and you have a backing user account in your own database, then just sticking with the standards and using the ID token should be fine. This is usually for apps that allow you to associate a Google account after account creation since you’re not interested in the full profile.

The best practice is to use the sub in the ID token as the user identifier. The sub will map to the old Google+ profile ID. You should never use the email address as the primary key to your own user database. Consider this a good time to update to the sub claim if you do so.

If you are confident you are always getting the token from Google using TLS and using it only for your app, then ID Token verification isn’t really necessary. Even Google agrees.

However, if you either receive an ID token from another app or send it along to your backend, you’ll want to validate it. Even though Google offers a token validation endpoint, I can’t recommend you just call an API to validate a token. This introduces a round-trip to a server and can impact your service if the endpoint goes down. Instead, you should validate the token yourself using the keys that Google provides in their configuration endpoint. Google has documentation on how to validate the ID token.

Option 2: Transition to the new Google People API

If you are using Google account to sign-in users to your app and rely on data from Google about that user, then migrate to the new People API.

The best part of using the new People API is that if you previously were using scopes of the Google+ People API, your users won’t be prompted for re-consent when using the new API!

This is available for the following old Google+ People API scopes:

email
profile
https://www.googleapis.com/auth/plus.login
https://www.googleapis.com/auth/plus.me
https://www.googleapis.com/auth/plus.profile.agerange.read
https://www.googleapis.com/auth/plus.profile.emails.read
https://www.googleapis.com/auth/plus.profile.language.read

Technology

Thanks To Siri and Kinect, Web 3.0 can now…

June 29, 2012January 28, 2019

Savas Parastatidis wrote a blog entry on the semantic web and machine learning, covering the existing history of this area and discussing the future of semantic processing as well as the false starts the area has had in past. For hope of a future, he calls to Apple’s Siri technology and discusses how the knowledge web will require loose processing of the existing web instead of taking in to account the models proposed in OWL, micro formats, etc. which have failed. Although a good goal, and Google and Bing have proven some capability at this, I think this approach – which requires AI and un-aided relationship reasoning – is doomed to be another failure in the goal of the semantic web. Instead, I see something even more amazing occurring which could finally give us Web 3.0 dreamers the world we wanted ten years ago (indeed, the path was laid out for us well by Nicola Guarino and Christopher Welt) – saving the environment and giving us a richer future for our interaction with our devices.

The First Generation of NUI Arrived Last Year

The last year has given us huge advancements in NUI interfaces such as Siri, TellMe, and Kinect. These interfaces allow for us to interface with knowledge either locally or world wide through voice and motion gestures. This has proven to be a lot of fun for customers, and opens up technology to more people in a new and engaging way. However it was not long until customers discovered the limitations of the first generation of these NUI devices. These limitations can be itemized as the following:

The accessible knowledge is scoped to the local knowledge of the device or service
The interface is limited only to the ontologies (read: keywords, models) that the device or service understands
The interface is locked to the platform and the platform provider (for ease of use)
The experience degrades substantially as soon as the query for knowledge extends beyond any of the above restrictions
The requirement of backend processing of queries against a single point of knowledge leads to service slowing down or not being available

These restrictions are understood if one knows the following fact: the internet is dumb.

The academic and engineering world has known this for some time, and have tried various attempts to rectify this problem.

This is the history so far:

First, they tried web services and the ability to “discover” knowledge providers through a self-describing knowledge warehouse (SOAP, UDDI, XML)
Second, they tried to mark up the existing websites to deliver knowledge the same way we share knowledge with people (XHTML, RDF, microformats)
Finally, they tried to just parse the web as it exists, using probability and graph analysis to determine relevancy.

The third is where we are now, and it is ridiculous state of affairs.

Mining the Internet Kills Us and The Planet

This “third way” of mining the internet just doesn’t scale. Certainly it as created new technologies to process large amounts of data, including map reduce and bulk synchronous parallel algorithms becoming mainstream. Hadoop offered through both Amazon and Microsoft on a PaaS level demonstrates the customer demand this has created. However this is more a reflection of the problem presented to the young Google trying to mine knowledge from the web ten years ago than it is any solution, or even a good one. If one considers the developer cost of using any of the other approaches listed above (discoverable services, self-describing web pages) vs. the huge amount of server space, energy use, and mindshare being used to parse the unstructured internet, the allegation that we software engineers are perpetrating something liken to a crime against humanity holds merit.

One would think that agreeing to – and using – any of the methods that would lead to a more rich and discoverable web of knowledge would receive attention and mindshare considering the consequences of mining the internet as it is today. What we have given the world instead are competing and diverse standards such as schema.org, RDFa, microformats, xfn, and the like accompanied by breathless blog posts about why a certain standard will change the world. In the face of these standards, many of which are backed by different companies to different ends (IBM for RDF, Google for schema.org) the engineers have not picked a winner but instead sat out the battle. This results not only in a dumb web staying dumb, but in the increase in the use of computing and natural resources to mine knowledge that could so easily be made to be discovered by the very people who seek to find ways of discovering it.

Facebook shows us the way forward: The Semantic Markup Rush

Something interesting happened while this was going on: Facebook introduced “likes” and the Open Graph. While many lamented the proliferation of these buttons across the internet mucking up the web, some went about doing something that for ten years standards could not get them to do: they started marking up their websites with semantic knowledge.

For the first time IMDB wasn’t just a webpage, but a rich and machine readable database of movies. Amazon.com became a rich resource of book data, and every blog finally had “articles” which had an “author” that had a “first name” and a “last name”. It happened for one reason: the profit motive. Companies who participated in the Open Graph got the benefit of the “link back” when users shared content on their Facebook Walls and profiles. This meant that anyone who didn’t participate in the Open Graph were left out in the competition for eyeballs and money. Even those who use advertising as their chief revenue stream could use the rich and seamless sharing with Facebook as a way of increasing the network effect of their sites.

You could argue that Twitter could also provide this benefit and without the requirement of semantic markup, but this is where vision is important: Facebook anticipates the future need for machine learning to make the user experience more compelling, allowing users to discover products and services that relate to each other in richer ways. The need is not for “dumb” links that surround the user, but knowledge of the objects, and their type, that surround the user. Here we see Facebook part ways with the ten year old philosophy of Google, which relies on graph analysis to feel out links a user puts on their profile. Facebook goes towards the true knowledge web of links to objects. It’s telling that Google doesn’t pitch the semantic tags of their schema.org proposal in their tutorial on how to add the Google Plus button to a website, instead indicating only the description and name properties. This highlights the gap in understanding in competitor’s minds of what makes Facebook so useful to customers now and in the future.

However, for NUI interfaces to become truly useful and the knowledge web to take off, this gathering of objects is just the first step.

Siri, Kinect: The Next Great Markup Rush

In order to solve the local knowledge problems of the NUI interfaces, we have to agree on a shared ontological taxonomy. Again, I don’t believe that those who propose the crowd sourced model previously put forward by the likes of MIT or Stanford will get us where we need to go. Going forward there is an opportunity for those who are developing these next generation NUI interfaces to lead on a new standard. Anyone that wishes to get in front of Kinect or Siri and have their weather content, flight information, or recipes hit the maximum number of eyeballs with participate.

The only way this works is if, like Facebook with the Open Graph, taxonomy is forced to be developed and registered during the NUI app creation process. This ensures that developers get the immediate benefit of their labor – the ability to get in front of customers through a NUI interface – while at the same time there are no semantic breaks in the taxonomy that is being developed. This would not only change the web, but also allow for the benefits I listed above:

NUI interfaces would break out of their limited scope
Reasoning engines could actually work with a structured taxonomy allowing better functionality over time
Our computer resources used to parse the dumb web would be eliminated

Although those in the open source world will be disappointed in this proposal of a commercial entity building the semantic structure as the price for access to their NUI experience, history has proven that most developers are coin operated and require ROI. The App Store boom has proven this. Now that we have NUI interfaces that are requiring this level of reasoning and interaction, the time is ripe to push forward.

No matter what the source, we cannot accomplish this new goal without a killer NUI app as well as the means to write semantic content for it.

Let’s hope it happens. The Semantic Web has waited long enough.

Cloud Computing

Facebook, Data Durability, and hacking HBase

December 20, 2010January 28, 2019

I found myself catching up on what’s been happening on the other side of the fence in the HPC Distributed computing world and in particular the Hardoop stack. Boots on the ground implementations of distributed computing are where theory meets the harsh reality of customer demands, network latency and commodity hardware.

That’s why I found this article recently written by Ars Technica about Facebook’s choice of using HBase over MySQL to be fascinating.

The debate around using Hardoop, and in particular the distributed file system HDFS and HBase, vs. existing file system and SQL databases has been heating up in the past year. This is an interesting fight. Microsoft says SQL Server Datacenter is just fine, and Dryad uses it along with its distributed file system Cosmos. Oracle says they’ve been doing parallel dataset analysis and storage since 2001 using SQL with no problem. Meanwhile NoSQL is gaining traction and people are wondering if you need object relational mapping solutions at all.

SQL based HPC solutions give you reliability but fall down when it comes to high throughput and distributed processing, especially in long running queries. Non SQL database solutions are fast and very distributed but the cost of that distributed nature is data loss. Very smart people have struggled with how to best reduce that data loss in massively distributed systems, but most proposals still has potential data loss at levels most cloud companies would find too dangerous. This was a big topic of conversation at the IEEE International Symposium on High Performance Distributed Computing, and Abdullah Gharaibeh & Matei Ripeanu’s paper on this, Exploring data reliability tradeoffs in replicated storage systems, covers a lot of ground.

Facebook’s own distributed platform, Cassandra, has this problem as well according to the article:

For many, it’s surprising that Facebook didn’t put the messaging system on Cassandra, which was originally developed at the company and has since become hugely popular across the interwebs. But Ranganathan and others felt that it was too difficult to reconcile Cassandra’s “eventual consistency model” with the messaging system setup.

“If you’re going to front the data with some sort of a cache…then it becomes very difficult for use to program a cache where the database heals from underneath,” Ranganathan says.

This is an odd statement, since all massively distributed database solutions have this problem, including HBase, which Facebook picked over Cassandra.

Then I read the following statement:

But HBase did require modification. Facebook engineer Nicolas Spiegelberg and another company developer spent a year adding commits to the platform, mainly in an effort to minimize data loss. “The goal was zero data loss,” says Spiegelberg. Committers updated HDFS Sync support, added some ACID properties, and even redesigned the HBase master.

What exactly is Facebook doing here? Usually the best way to scale a distributed database in a massive way is to minimize data loss, not prevent it. It’ll be interesting to see how this new platform works out for them.

Language Theory

Haskell: Generative Type Abstraction and Type-level Computation

November 10, 2010January 28, 2019

I haven’t had lunch with Simon for awhile (it’s hard since he’s in Cambridge and only visits main campus occasionally) but he publishes papers at an amazing pace, and every time his language, Haskell, and the world gets a lot smarter. Here he introduces the concept of “roles” to aid in the problems that occur with type-level computation.

Generative Type Abstraction and Type-level Computation (Extended Version), by Simon Peyton Jones, Dimitrios Vytiniotis, Stephanie Weirich, Steve Zdancewic:

Modular languages support generative type abstraction, ensuring that an abstract type is distinct from its representation, except inside the implementation where the two are synonymous. We show that this well-established feature is in tension with the non-parametric features of newer type systems, such as indexed type families and GADTs. In this paper we solve the problem by using kinds to distinguish between parametric and non-parametric contexts. The result is directly applicable to Haskell, which is rapidly developing support for type-level computation, but the same issues should arise whenever generativity and non-parametric features are combined.

Lambda The Ultimate describes the environment in which the paper finds itself and provides larger context:

In fact, this isn’t the first time coercion lifting has caused trouble. In capability security terminology, coercion lifting is a “rights amplification” operation, and there are previously known examples of seemingly innocuous coercion lifting across an abstraction/implementation boundary resulting in Confused Deputies. There’s no discussion of this connection in the paper, and the paper cannot solve the problem discussed at that link, which exposes an much deeper issue than confusing parametric/non-parametric contexts.

Cloud Computing

What I’ve Been Working On: Office365 in Beta!

November 10, 2010January 28, 2019

Use to be at Microsoft products had predictable release cycles. Released in “waves”, you could anticipate a period of creativity and envisioning at the beginning, a period of hard work coding and hacking in the middle (this is when you sleep in your office), and a period of release and downtime often accompanied with a ship party. Around this time most PMs and architects were already busy doing the envisioning for the next wave, but mostly you kicked back and had a life for a little while. This is how Microsoft had learned to ship software that had a “big bang” release and a version number.

In the cloud, that cycle doesn’t exist.

Our team has been working very hard on Office 365, the successor to BPOS, almost before BPOS was released two years ago. Combined with constant improvements to the existing service, such as the addition of PowerShell, speed and reliability improvements, and an introduction of a more refined service dashboard, it has left our team panting from the exciting but exhausting pace.

It’s an awesome time to be at Microsoft and an awesome time to be in the Microsoft Online team. I can’t wait to get the feedback from our customers on their experience with Office 365.

If you haven’t yet, sign up for the Beta today.

Cloud Computing

What I’ve Been Working On: Microsoft Online Now Supports…

July 27, 2010January 28, 2019

For the last few months, we have been working hard on something great for our customers and partners.

For the first time businesses and consumers who prefer BlackBerry® devices will be able to purchase both Exchange Online services and Hosted BlackBerry® services together in the cloud without needing any additional servers or software. This feature provides the full range of BlackBerry features such as push email, calendar and address book through their device.

Previously, this required a full BlackBerry® Enterprise Server (BES) installed on-premises. Before now, for small businesses and businesses using cloud services their usual route was the Blackberry® Internet Service (BIS) which provided email functionality but lacked calendar and address book support. Some popular cloud providers required a separate connector download and the customer to have a Blackberry Enterprise Server (and license) on site to get true BES functionality.

Great User Experience

Now, all you need is a BlackBerry® device with service from your telecom provider, a subscription to Exchange Online and a subscription to Hosted BlackBerry Service.

Further, businesses will be able to administer their BlackBerry® licenses and devices for their users in a portal experience that provides great ease of use. We can do this through leveraging our knowledge of both our customer’s Exchange mailboxes and their BlackBerry® devices, which enable us to give them a Better Together unified licensing/provisioning experience. With just 4 clicks through a wizard – a company administrator can move all of his Exchange users over to BlackBerry® with licenses automatically allocated in the process.

This is the initial launch of the portal, and we have much more to do as learn from our customers and partners on what is important to them in a BlackBerry Administration experience. As this progresses, feel free give us feedback or ask any questions on Microsoft Online Technet forums.

Technology

Un-PC Reality

September 27, 2008January 28, 2019

One of the things in all the better Leadership training seminars I’ve been to in my career has been the insistence and dedication to reality. Usually the best strike hard by announcing a string of facts about what is changing that is causing organizational problems and asking everyone to confront them.

There are some that have taken this advertising campaign as a validation that their world has not changed. Some that continue to believe we can maintain the way things have existed – the way our revenue and ecosystem has worked in the past – and confuse that desire with the goals of the PC Campaign. Certainly, Windows as a platform is not going anywhere – the marketshare numbers still show remarkable power. The problem is Google and others have proven that not only is that not a hindrance to their efforts – they can use it against us by layering a platform on top of it. That is the reality we find ourselves in. We risk becoming a pretty TCP/IP stack for the larger world.

That is why it is dangerous to assume “I’m a PC” means an old client PC with a tower/flat panel screen in the corner of the Den running Windows 95. It’s tempting to want that old model back, but the reality is Windows has moved away from the PC. What Windows is the air we breathe? Are you talking just of the client? What about Windows in the cloud? If I spend 90% of my time on my MacBook Pro inside mesh.com using Silverlight – am I then – in your opinion – still running Windows?

To borrow from Emerson: there is no wall where I, the device, ends and you, the operating system, begins. Windows now comes to see us without bell. The walls are taken away.

I’m still trying to figure out if we have seriously – each of us – looked that reality dead in the face. There is a sense of urgency that needs to spread through everyone – an urgency that I’ve seen so I’m optimistic we can achieve a lot more in the future than we have in the past. What we have to guard against, however, is assuming that this future looks like the past – a PC on every desktop running Microsoft software.

Perhaps we should second the new mission statement with a vision “the network, through every device, running Microsoft software”

I think in some there is real fear of this new reality- fear that needs to be addressed – a way forward clearly communicated for them. That I believe would help Microsoft regain its spirit – which never relied on the current products – but always the future. We should aspire to make our customers fans – fans of the brand and the innovation, not of just the PC.

Language Theory

The Rise Of Functional Programming: F#/Scala/Haskell and the failing…

September 16, 2008January 28, 2019

Over at Lambda The Ultimate, the best academic programming blog on earth, there is a large debate going on regarding what the future of languages will be for 2008. The most important thing to emerge from the discussion is the larger role functional programming will play. It seems like a safe bet. This year has seen the explosion of interest and creation of functional languages such as Apple OS X’s Nu, Java’s JVM using Scala and Microsoft Research’s .Net language F#.

I am ecstatic at this change.

The Failure Of Lisp

It’s hard to understand where it came from. Certainly one can argue the broader academic community had nothing to do with it, the old guard Common Lisp hackers are still as fickle and as judgmental to new comers as ever. Also, the old standards in Lisp languages, Franz and LispWorks have not lowered their prices to anything approachable to the casual developer. There are open source ANSI Lisp implementations without all the supporting engines and functionality, such as SBCL. In fact, my most linked thing I’ve ever written in my career is the installation walk-through I did for installing SBCL and Allegro which includes adding your repository and packages for CLOS and automatically compiling the FASL files, especially dealing with the asdf differences between the implementations. The complexity of this in itself points to problems with portability and configuration in Lisp. However, even that project that targeted Lisp’s Bread and Butter, the parsing of semantic ontologies for the Semantic Web, was met in the message boards with worries on if there would be enough developer participation using such an odd language, and recommendations on moving it to Java.

In reality, Common Lisp showed its failure as a community by sitting out this enthusiasm that has been generated around functional programming languages. It didn’t have to be that way. I recall my first awareness of functional programming’s growth was the awesome work of Lemonodor‘s blog and Sriram Krishnan posting “Lisp Is Sin“. I was happy at the time that Lisp was getting such attention, as well as functional language architectures in general. I imagined that as OO languages had grown so verbose and feature dense that even the IDEs to develop your applications run in to the tens of gigabytes, a new evolution “Back To The Future” was inevitable. Even more, I believe long suffering Lisp deserves to be back in favor again, it’s certainly spent its time in purgatory. Yet, it didn’t happen. You can blame the old 50 year old men sitting on IRC channels for that. It was the most thorny and un-inspiring community I’ve ever participated in, despite my extreme interest in the language. It’s jaw dropping that a language with such promise has sat out the resurgence, and speaks to what an un-friendly and un-inviting community can do a technology platform. I would be the first to march it off to the grave.

The Rise Of Functional Languages

The interest in functional programming actually grew up around more academic but pure languages like Scheme and Haskell. Although these languages sit within their own island and lack many of the “dirty” aspects of Lisp’s CLOS environment that make it easy to access OS and hardware resources, they are still strikingly useful in learning things that are the staple of functional languages, such as Closures and Lambdas. Indeed, one could argue that the movement to move Closures in to OO languages (first C#, now Java) was in part due to the rise of awareness of functional languages.

Further, it seems to me that functional programming languages answered two prayers of those more ambitious engineers who don’t seem to want to stick with the script and Java worlds they were taught in college. Those two large wins, far more important than the semantic features of functional languages that have gotten all the attention, are architecture foundations of functional languages:

Referential Transparency / Side Effects
Concurrency

Referential Transparency

To those coming from a pure OO world, Referential Transparency and the restriction of side-effects can be something hard to get their heads around. The best way I describe this concept is by hitting at the root of their assumptions: Everything they deal with are dead. The objects are dead, the variables are dead, the entire atmosphere is dead, as if something had come along and killed everything in your stack and you have to assemble your program by only what’s been given to you, nothing more. There are no instances, objects do not “come alive” and have state; a state that you have to poke in to and a state that can change at any time. A function will always do what you expect, and nothing can come along and change that behavior.

One of the things that seems to appeal to developers most about the promise of SOA architectures happening in enterprise environments, if you’re smart enough to pry it out of them, is that they get the same referential transparency in services. No one can override a service (besides versioning, which is explicit to the developer) and a service will only return what it did earlier in your code and earlier in the year. This forces developers to design services that have the same relationship to the world as functional programmers write their functions for. This is perhaps the trickiest part of migrating enterprise teams to a services based model, their expectations of the mutableness of the services they are accessing and their inability to anticipate what working in that world will be like. Especially for those who use tools or libraries to convert service interaction in to an object, the interaction can be jarring.

However, the soon find the predictability and the safety of such an environment liberating. In much the same way OO programmers were use to making their objects or variables immutable to maintain their contracts and relationships with other objects, often sacrificing many of the benefits that OO programming promised their stack, now they have immutability and transparency in an environment where functional paradigms are key, they do not expect to be able to “embrace and extend” services. They are what they are. This tends to cascade out to the living instantiated code a developer writes as well, as there is no point in entering the world of the living if what you have to return to is a dead function.

This was hinted at in an article in the ACM Queue magazine by Terry Coatta, entitled “From Here to There, The SOA Way“. He states,

Objects are still a very good way to model systems and they function reasonably efficiently in the local context. But they don’t distribute well, particularly if one tries to use them in a naive way. A service-oriented architecture solves this problem by dealing with the latency issues up front. It does this by looking at the patterns of data access in a system and designing the service-layer interfaces to aggregate data in such a way as to optimize bandwidth, usage, and latency.

Not that SOA limitations are the only thing that is affecting the consciousness of a software engineer, the other issue is the large rise in the complexity of managing a large enterprise library written in an OO language. One of the largest pain points of any application of large size is the management of graphs and graphs of live objects and the living data within them. When software engineers experience the lack of side-effects in functional languages, it’s a breath of fresh air.

Concurrency

A funny thing happened on the way to those multi-core processors. People loaded their applications on them and noticed nothing got much faster, particularly when it came to transaction intensive tasks. Turns out Intel and AMD left out an important fact about their Moore’s Law cheating multi-core environment: you can’t ring as much performance out of it without changing the way you manage concurrency and threads. Sequential programming could always rely on going faster as the single processor speed got faster, but as multicores come in to play that isn’t always the case. You want to farm off transactions to occur on separate processors, and in the living world of mutable objects and variables, breaking out two transactions to work concurrently that operate on the same living data is a bad idea. Add structural programming’s solution to this problem, optimistic and pessimistic locking, and you have dead-locks in short order.

Functional programming has been a natural place to explore parallel processing and new ways of doing atomic transactions because of the reasons above. More important, these atomic structures can be composable which is lost when doing locks in structural programming. A lot of the buzz has been generated around the idea of software transactional memory, where execution blocks can be flagged and managed and built upon. The best introduction to this topic is the paper by Tim Harris entitled Concurrent Programming Without Locks. Although this use to be expressed only in the confines of Concurrent Haskell, others have shown how the same techniques can be used in other functional languages, such as F# using nothing more than PowerList.

This experimentation is one of the large reasons why functional languages have become more important as software engineers wrestle with the problems and promise of multi-core processors in transaction processing. Although not every engineer will be interested in the deeper details of STM or other strategies in concurrent programming, the fact that these libraries will emerge and only be available in the functional realm will force software engineers to learn the core concepts and bring even more visibility to the functional programming space.

Functional Hybrids: Functional Programming Is Now Approachable

The other driver for adoption of functional programming languages, besides the architectural benefits it has to solve current problems, is the fact that languages such as F# and Scala have adopted a more hybrid model in their language design, where a developer isn’t forced completely outside her comfort zone. Scala is a combination of functional and deeper OO methodologies (as in SmallTalk) and has access to the entire Java library, significantly reducing the learning curve. The same can be said for F# and .Net and Nu and Objective-C. This does have draw-backs however, as both F# and Scala have not been able to use more of the STM strategies that Concurrent Haskell allows because the underlying thread architecture of the VMs they run against are built for structural programming languages. It is easy to see how this can be fixed, however, and allow those using hybrid functional languages the same power as those who express their ideas in Haskell or even Lisp.

As I said, I am excited about this new resurgence in functional programming languages, and I am enthusiastic 2008 will have even more to offer those who are just getting their toes wet. I personally know some college freshman who started out using Nu as their first language, and are already contributing to the community. The future of software engineering is bright.