The Geek Will Inherit the Earth
DDD Scotland has been a great weekend of lots of geek-ery, fun and fire. It was also my first experience of speaking at a large conference and gave me the opportunity to talk about some of the things I had learned whilst developing on the Windows Azure platform over the past year, working with the European Environmental Agency and RiskMetrics.
The Infrastructure Access Layer, or IAL for short is a simple concept, nothing new, in-fact you have all been using something similar for years when programming against your databases. I have just expanded the concept to the cloud (and the ability to create “cloud ready” applications).
The Infrastructure Access Layer is simply a way to abstract away your underlying infrastructure from your business logic or processing. This means that I can easily change how my applications interact with the infrastructure just by changing a method within the IAL and the business logic would be none the wiser.
Lets take a simple example, I may be developing an applications that takes work items and processes them. I create a method in the IAL called GetWorkItem which will return to my business logic the work item to be processed. This method can then call out to Azure Queues to get a work item, or maybe in the future you will want to switch out where we are getting the work item from, maybe we want to use a web service to call another location, use blob storage to store large work items or listen for a work item to be passed onto a service bus. This can all be dealt with in a single place.
All the business logic knows and cares about is that it is going to call the GetWorkItem method and get a work item back, not where it came from or how it received it it.
Even with more and more people are moving toward the cloud, there are still companies that aren't quite ready to make the jump, but realise that in the near future may have to and want to know how to create applications that will be able to transition easily to the cloud when they are ready.
By programming against the Infrastructure Access Layer companies will be able to make this move a lot smoother, by just replacing the needed classes so that they interact with the cloud rather than the old infrastructure.
A great example of this would be using a messaging system such as MSMQ as part of your on premise application, and then being able to easily switch this out for Windows Azure Queues or the App Fabric Service Bus when moving into the cloud. Having to make minimal if any changes to your business logic.
As I talked about in my last post the cloud can end up getting everywhere and you can soon end up breaking the principle of DRY (Don’t Repeat Yourself).
With the IAL you can easily keep the cloud contained, letting a minimal number of project have access to the cloud, making SDK upgrades and maintenance easier.
It is indeed. As I said at the start, the IAL is a simple concept that allows you to abstract away your underlying infrastructure from your business logic and also help create cloud ready applications for when people are ready to make that push to the cloud.
I'm sure everyone is aware of the recent volcanic eruption in Iceland and the cloud of ash that has been spewed out and more importantly the knock on effects this ash cloud has had on not just Europe but the whole world.
The ash cloud started out in Iceland where it eventually moved and spread out to cover the UK and Europe, even some reports say that it got as far as Canada. Grounding planes and crippling transport for a number of weeks.
This got me thinking, not all clouds are nice white clouds, some are evil volcanic clouds that actually do more harm than good especially when not contained to where we want them to be. (the ash cloud would have been fine if it had not strayed into European airspace).
The same is true in your applications, when developing for the cloud you can find that the cloud can end up spreading throughout your applications and if your not careful touching every section of code you write.
This can then be a real problem when the underlying SDK to interact with the cloud changes and you have to go through all of your application looking where to refractor and make changes.
To combat this I contain any code that will access the cloud within few specialised “cloud” projects within my application.
This means that throughout the application if I want to access the Azure message queue, I call a single method from one of my cloud project that will return me a message, meaning that no other part of you application needs to know how to deal with the cloud.
Containing your cloud not only allows you to keep to the DRY principles and code re-use it, makes maintenance of your application easier and can reduce code smells. One of the biggest benefits is that it also allows you to easier test your applications without having to rely on the cloud or developer fabric, as you can create your cloud methods in a mock-able way.
This afternoon I noticed that I was able to select Europe and Asia regions to deploy my Azure application.
I have yet to see official word of these locations being available, but this must be for the move from CTP to production.
Along with my talk, Ben Nunney will be giving a presentation on Powershell for developers. Showing how developers can make PowerShell work for them, whilst proving that the command line is still cool.
Here is a breakdown of the event with overviews of each talk:
Cloud Coffee is the latest in a wide range of coffee shops on the high-street. What makes Cloud Coffee different is that they use latest “cloud” principles, keeping costs low by using utility based staffing, maintaining a decoupled working environment and ensuring good scalability to meet customer demand. Cloud Coffee has very quickly risen to high street fame.
This analogy will help you understand how things work in the cloud, specifically Windows Azure. During this talk Dominic will elaborate and expand upon on the concept of ‘Cloud Coffee’, sharing some of the lessons he learned whilst developing one of the first production applications for the Windows Azure Platform.
It's a long time since DOS ruled the world, and the classic command shell was in serious need of an upgrade. Enter PowerShell - focusing on accuracy, extensibility, and congruent syntax, it is one of the most powerful and misunderstood things to come out of Microsoft in recent years. Who says a command shell can’t be interesting? By the time the session is over you’ll have seen what PowerShell can do for you as a developer, know exactly how to get there, and who knows - you may have even fallen in love with the command line all over again.
You can find out more information and register for the event here.
After months of hard work the Eye On Earth project has been launched and released to the public (go take a look).
The project undertaken for the European Environmental Agency allows users to view and rate up to date environmental information about both the air and water quality around Europe, utilizing many bleeding edge technologies such as Windows Azure, SQL Azure, Silverlight, the new Bing Maps control along with many more.
Here is just a short snippet from the official press release, that can be read in full here.
Eye On Earth gets not just a fresh look and feel for both air and bathing water information, it also becomes one of the newest applications built on Windows Azure and SQL Azure hosting the Geo Observatory Data Store. The site’s user interface provides interactive information from Europe-wide to street level, based on data from air-quality measurement stations and citizens’ input. It translates scientific data into easily understandable terms.
I will be posting more information about the project and what I have learned in helping to developing the solution, especially with regards to Azure in future posts.
Site Link - http://eyeonearth.eu/
I mentioned in my last post that I presented on “What Is Windows Azure?” at the first Reading Geek Night. This presentation was aimed at people who are new to the world of Windows Azure and aimed to be a quick introduction to wet their appetites.
In good presentation style (or was that lack of time) my presentation contained minimal text and relied on my presentation skills and images to convey the message, so here is a quick run down of my presentation:
Cloud Platform – Windows Azure is a cloud based platform for built from commodity machines, that allows you to develop and deploy, scalable, available applications in Microsoft's data centres across the world.
Utility Computing – Much like your gas and electricity you are only charged for what you consume, therefore giving a very small barrier to entry and making it ideal for small to mid-size businesses. When you need more compute power all you have to do is spin up more servers, no more buying servers based on your peak load, just scale and pay for what you use.
Web & Worker Roles - This is possibly my favourite slide, and where I think the whole concept of Windows Azure falls into place for everyone. Azure is build up on virtual machines consisting of worker and web roles I like to think of these much like the Barista and coffee makers in a coffee shop. Your order is taken by the Barista / web role and passed to the back end coffee makers / worker role for your order to be completed. If at any time either of these “fall over” there will be another role to step into its place and continue the order.
Azure Storage – There are three types of Azure storage Blobs, for binary data, Queues for queuing up messages to be processed by the worker roles, and tables which are flat file databases which will easily scale as needed.
Relational Database – although Windows Azure ships with table storage, you can also use a relational database structure using SQL Azure and SQL Management studio.
Developer Experience – Windows Azure offers a familiar developer experience for any .NET developer with plug-in’s for Visual Studio and a local developer fabric to develop on dev machines. Azure also offers SDKs for both Java and php.
Thanks for everyone who attended the talk and I hope you enjoyed it as much as I did. You can find more information about Windows Azure at http://www.microsoft.com/windowsazure/ along with the SDK and developer tools that you will need to get started.
I have recently been working on a big Azure application that we have been wanting to deploy to the cloud and test, to make sure that no unexpected bugs occur whilst the application is in the wild and give the opportunity to see how it performs.
Now the big problem has been deploying our application to the cloud every time we wanted to update the application online, we had a dev sit there and manually publish the service, upload the package and initialize the instances. This obviously wasn't great as one of the developers on the team would have to take time out from his tasks / bugs to upload the application.
However, the Azure team came through with some great releases just over a week ago that would allow the management of our services via an API. As soon as I read the blog post announcing the Windows Azure Service Management API I knew that my prayers had been answered and we would be able to automate deployment of our application out to the cloud. This surprisingly was much easier to understand and implement than I imagined as a sample called csmanage.exe was included with the release giving a hook into all of the API’s (documentation).
The most time consuming part turned out to be wrapping the calls to csmanage.exe into build scripts and watching a number of builds and deployments fail before getting it right (no one likes to wait 15mins only to see a red light).
So here’s what I did to get our application deploying on the daily build:
Seting up the Service
Setting up the service requires you to upload a certificate to your Azure login, this can either be done with the following handy command line argument:
Or use Eric Nelson’s walk through “Using IIS to generate a X509 certificate”.
Now we just need to log into the portal and upload the certificate, taking note of your Subscription ID and certificate thumbprint.
Configure csmanage.exe
With this information we can now configure the csmanage tool to talk with the service management API.
Packaging the Cloud Service
When deploying prior to the release of the management API I would normally right click the Cloud service within Visual Studio and let it handle the packing of the application itself, however during the build we will need to do this automatically. So we can dig out cspack.exe a command line application to package up your solution that was shipped with the Azure SDK.
When packaging the application we need to give the cspack executable a number of arguments these are, the service definition so that it knows what configuration to expect, any web roles and their containing folder, any worker roles with the folder containing the dll and also the name of the dll and an optional out parameter. I normally use the out parameter to ensure that I have the correct name and location of the package for the next step.
Uploading cspkg to Azure Storage
When deploying with the csmanage.exe we need to ensure that the packaged solution is available in blob storage on the same account you are deploying to, to do this I have created a simple command line application that takes the cspkg file and puts it into a folder called “packages” in our storage area.
Using csmanage.exe
Now that we have our package in Azure storage and csmanage.exe configured we can use the management API to call out to our service and deploy to Azure.
I first suspend and delete any service that is currently running:
This leave me safe in the knowledge that the creation of a package will not fail due to a running service.
Now the important part using csmanage.exe to create a deployment in our hosted service, within this command I am selecting the hosting area I wish to create the deployment, the production slot as well as the location of the package (blob storage) and any configuration that will be used in the deployed service.
Once the service is successfully deployed we can set the status of the deployment to a running state and allow it to initialize itself.
Once the deployment is fully initialized users can start using the application.
Creating Build Targets
With the correct commands figured out and correctly deploying my application to the cloud I decided to write some build targets to do this during our daily builds. To do this I used the msbuild exec command (I know this is probably really ugly but it gets the job done, well at least until the Azure boys give some built in tasks).
The only problem I found with this that the csmanage.exe will return exit codes and so the exec command will need to ignore these, or will treat them as errors during the build.
Multiple Deployments
On the current project we are actually deploying the application to a number of different sites for test, performance etc. so have also had to create a number of scripts to ensure that the endpoints are correct for each of the deployments. Added to this we needed to check that the csmangae.exe and the cloud apps configuration (cscfg) are set up correctly using the sdc tasks and their xml setters to get this working.
Now at the end of each day the latest iteration of our application is uploaded and deployed to Azure, leaving us devs to be happy coders and not worry about deployment :) .
