Monday 24 December 2012

Buzzword Bazinga: What is Big Data? featuring MapReduce and NoSQL


The term 'Big Data' describes a vast amount of data, or chat, generated from two main sources: people and computers. People spend a lot of time talking on social networks like Twitter and Facebook and producing on-line content about their companies, affiliated organisations and themselves. Computers and devices, now IP enabled and connected to the internet, now have the ability to send their instrumentation information into internet. The latter phenomenon is known as 'the internet of things': a description of how connected devices and sensors, from shoes to thermostats to cloud server farms, now have a voice and can make information about how they are used and what they experience accessible to data crunching algorithms. These algorithms would then produce insights that can be feed back into the real world and affect everything ranging from peoples behaviour to how corporations are run.

Handling all this data is a massive computational challenge. Compounding the challenge is that data is  growing exponentially, may be stored over the place and even kept in different formats. To address these challenges two key technologies have evolved to help us along: MapReduce and NoSQL.

But before we delve into these let's break down the attributes of Big Data into a bit more detail.


Friday 21 December 2012

Cloud Recipes: Build and Deploy an Azure Web Role with CC.net and GitHub

In this recipe we will create an Azure Web Role (Cloud Service Project), add the solution to a Git Repository, Download CruiseControl.Net server and set-up a build and deployment script to build from the Git Repository and deploy to the Azure Web Role.

To complete this recipe you need the following:
  • The latest release of Cruise Control.NET (1.8.2.0) - Download and install here 
  • A Microsoft Azure Subscription - sign up for a three month free trial here.
  • A Git Account - sign up for a free account here.
  • Putty Gen and Putty Agent can be downloaded here.
  • Git  
  • The .Net framework 4 or above
  • The Azure SDK 1.8 and all the goodies
  • Windows Azure Authoring Tools  
  • Windows Azure Tools for Visual Studio 
  • Visual Studio IDE 2012
I created this recipe on one machine with everything installed in one place. In the real world you would have a dedicated build server.


Wednesday 19 December 2012

Digital Trends in 2013

Every year, around this time, there are a lot predictions on what the digital trends for the next year will be. They are often produced by Marketeers and therefore have pithy names which are a clever amalgamation of two words which never before have co-existed. In a lot of cases, that is for a very good reason. Some examples from 2012 include: "Idlesourcing", "Eco-cycology", "Flawsome", "Recommerce", "More-ism" sourced from here. ...

This year I decided Tech-Rash needed its own list so I tried to come up with my own trends backed by catchy names. I attacked the problem by coming up with a catchy name first and then thinking how/if it could become a trend. I came up a few but then decided to cast the net out and 'crowd-source' the rest of the content for this post. I got back a range of responses ranging from silly (like mine) to silly, but could catch on, to serious. Thanks to @davidknipeTA, @michael_hook, @jgbibby, @cs_andrews and @ev2000  for their contributions.

As an aside, the idea of silly things turning into something serious and useful has already been coined by Seth Godin (Marketeer Extraordinaire) in his article Ridiculous is the new Remarkable.


Friday 14 December 2012

Cloud Recipes: Building a Facebook Canvas App using Azure and BitBucket

Welcome to the first official entry into Tech-Rash's Cloud Recipes. Cloud Recipes, I hope, will became a useful reference for cool things you can do on the Cloud in thirty minutes our under.

This Cloud Recipe has been shamelessly ripped off a talk given by Richard Conway from ElastaCloud who runs the UK Windows Azure User Group. In the talk he demonstrated setting up a Facebook application in 15 minutes based on a new .Net 4 MVC Facebook Template provided in the ASP.Net Fall 2012 Update. He used BitBucket as a Source Repository and rigged up an Azure Website to Auto-publish from the BitBucket Repo. I gave it a go and to see if I could achieve in 15 minutes and I failed but did learn a valuable lesson that churning out Facebook Apps can be a very quick process.

Monday 10 December 2012

UKWAUG Talk: RavenDb on Azure in 15 minutes

On Friday 30th of December I attended the UK Windows Azure User Group and spoke about about running RavenDB - a great NoSQL technology - on an Azure VM. Thanks to the UKWAUG for hosting the event and the other speakers for some really interesting and inspiring content. The pizza and beer went down very well too.

The other talks included:

  • Cost saving Azure Solution Architecture
  • Creating a Facebook Application in 15 Minutes on an Azure Web Site with BitBucket as source control
  • Robot Ball Wars - A tech showdown using Sphero controlled from Smart Devices and a NetDuino with various inputs such as a Wii Remote. The showdown culminated in two Sphero balls dancing in tandem to Gangnam style which was bizzare but lots of fun.

Useful Links:


Wednesday 28 November 2012

A pragmatist’s view: What is Technical Architecture and how do you do it?

As a Technical Architect in a digital agency I am often am asked what the role actually means or does. This question comes from everyone including Technical Architects (TAs) and one that is actually quite difficult to answer. The reason for this is that the definition of Technical Architecture in the real world has a very broad scope and is different in different companies, projects and from person to person. It is often becomes a religious argument what being a TA means or what the best way of doing it is. Forgetting best practices, patterns, methodologies, techniques and opinions just for a minute, here a broad view ‘what’ a Technical Architect does and ‘how’ they do it. I'll even look at why in some cases. I'll leave where and who up to you.

Thursday 15 November 2012

Dealing with a Vampire Launch: What to do when your site bursts into flames

We're gonna need a montage.
I've written about Vampire Launches in the past. For new comers, a 'Vampire Launch' is when a site or application is launched and has a brief and glorious moment in the sun but then suddenly and horrifically erupts into flames. Vampire launches do happen and when they do, an expert firefighting team needs to be deployed to deal with technically complex and mysterious issues in a pressured and emotional environment. Having been involved in a few crises in my time and watching and participating in their resolution, these are some things I've learnt along the way.

Tuesday 13 November 2012

Effective Resource Management in Web Application Development

Mmmm. Chocolate.
My wife and I recently found ourselves in a frustrating predicament. Over a year ago we made a decision that we needed to move house and therefore needed to save money so we could afford a deposit. We started our saving initiative my way. I created an elaborate spreadsheet which listed our outgoings in meticulous detail. Well, the outgoings that I knew about like bills and mortgage payments. I then added in lines on our living expenses and luxuries. I was very generous about how much we needed to live and how much we could splash out on for fun. I crunched some numbers and the end result looked pretty good. We could save a tidy sum every month and be living in a new house before you could say ‘white picket fence’.

Thursday 1 November 2012

Social networks: If you build it, will they come?

Don't do what Costner did or you will fail.
Social is pretty big now, right? If you ask me what main things the next generation have which I didn't have growing up, I'd say Facebook, Twitter and Apps. It's no surprise then that businesses are desperate to get involved and use social networking to boost their profile and their profits. And while they're at it they will create a couple of apps too.

There are many different strategies to leverage social media. Some companies try to use what's already there. They will litter their web site with Social calls to action. Do you 'Like' this page about the latest innovation in washing powder? Of course I do! Which person of sound mind wouldn’t? Would you like to tweet and then re-tweet this great deal on life insurance? Did you have to ask? #Death is trending right now.

Tuesday 9 October 2012

Building a Cloud Platform: Weapons of War


I've just returned home from a big development build. For the last five months we've been in a secret room racing to get version 1.0 of a cumulonimbus size cloud project out. It wasn't easy. Getting to 1.0 was a real battle because... well, you don't know! Because you weren't there... man!

Now that I am back on civilian soil and fighting off night terrors from the horrors of war, I can reflect on what went well and what when badly. This could be a book in its own right; but for this post I'm going to cover The Weapons of War - or the essential tools needed when building any product or platoon... I mean platform.

Friday 25 May 2012

Web application development is transforming, but that doesn't mean everything has to change


The way we build web and mobile device applications is changing fundamentally. What this change is - in a nutshell - is application code moving from the server to the client. The server being big computers sitting in a data centre somewhere and the client being a modern day browser. To explain this further, let’s call the old way of building web sites as web 2.0 and the new way as web 3.0.

In the past, when web 2.0 was all the rage, web sites were built with a lot of server side code written languages such as c#, php or java. This code ran on various web server application platforms such as Microsoft ASP.Net. The code ran on the server and spat out HTML which was rendered on a browser and allowed the user to interact with the application. There was probably was a bit of JavaScript but none of which was essential to the application doing what it needed to do.

Friday 20 April 2012

What defines a successful project?

Single Fist Pump!
A colleague asked me over coffee, “What do you do?” I sipped my full fat cappuccino and reflected on the question. Sipping coffee and reflecting go well together so I may have milked the situation a bit more than was required. 

“Hello!” he interrupted. “Did you hear me?”

It was a good question erring on the side of existential and difficult to sum up in a tweet sized thought bite. 

For a bit of context, I have a technical leadership role which can mean many things to many people - including other people in technical leadership roles. I had to dig deep and try find an answer that came from my heart and wasn’t ripped off from the Steve Jobs Autobiography. 

“To make sure we deliver projects... successfully.” I said with the thousand yard stare of a visionary. 

“To deliver projects?” he smirked. “Truly inspirational.”

Wednesday 4 April 2012

[definition] Pushing Tin

Pushing Tin [pa oosh ing tihn]
Adjective
1. Setting up and managing technical infrastructure in the days before CLOUD architecture
2. A movie with John Cusack and Billy Bob Thornton in which Billy Bob gets really, really lucky.

Tuesday 13 March 2012

Avoiding the pitfalls of joint Web Site and Web Service development

How to turn a Balrog into a puppy dog.
Developing web and device applications these days is more often than not dependant on 3rd party web services. This integration is one of the biggest culprits behind projects being delivered late and costing more than expected. I’m not talking about fully developed and functioning web services, of course; but web services developed in tandem with the web site. There are a number of reasons why developing web sites and dependant web services together can cause severe delays in and over-spends in project.

Friday 10 February 2012

RavenDB Vs SQL - A follow up

The fight they were all waiting for
This is a following on post from RavenDB is it all it’s cracked up to be. A few have commented that to provide any meaningful interpretation of the load test they needed to know more about the SQL implementation which is fair enough. So, here we go.

Tuesday 7 February 2012

Is RavenDB all it's cracked up to be?

It all started with a Vampire Launch.
As a seasoned technologist I’m always happy to bare old battle scars and discuss them candidly with my peers and starry eyed groupies* over a few pints down the local. Technology, after all, always has problems and we have taken it upon ourselves to carry that giant boulder up the hill and start afresh when it rolls back down again.

Take, for example, a recent run in with a search implementation on a property website. We selected a search technology built on Lucene.Net - a high performance, enterprise search platform that is fast becoming an industry standard.  Choosing a Lucene based search provider for a web site heavily reliant on search functionality seemed like a great idea at the time. Fast forward to go live and we soon discovered that the search didn’t hold up very well under significant load. This is what I call a “Vampire Launch” i.e. a web site having a brief moment in the sun but then suddenly erupting into flames. One stressful week later and the search was back online and taking the strain of public exposure. This time however, it was now driven by trusty Microsoft SQL.

Before I cover up this grizzled scar (until the next showing that is), it's worth highlighting three important lessons harvested along the way.

Lesson 1: If you are trying out a new technology make sure your POC includes load testing.

Lesson 2: If you are using a new technology make sure the providers of the technology are able to support it.

Lesson 3:  Make sure you build using interfaces and abstraction in case you ever need to swap out a nasty search implementation.

So how does this all relate to RavenDB?