The Fog of Technology

A few weeks ago there was a flurry of media interest about the recent rapid improvements in the field of robotics and autonomous machines, and the potential threat they may at some point pose to mankind.

The article reminded me of something Jared Diamond wrote in his book Collapse, how societies choose to survive, or fail, about collective psychology in the face of some perceived danger. He describes how people living downstream from a recently constructed Dam in a valley were interviewed about how concerned they were about the possibility of the dam wall breaking, and the valley being flooded. In a settlement quite far downstream from the dam, residents were moderately concerned about the danger, and openly discussed what they might do in an attempt to evacuate the area in the event of an emergency. Strangely, as the interviewers approached people living in settlements closer to the wall, they found that they seemed to be less concerned about it than those living further away. People living right beneath the wall, of course, seemed to be completely unconcerned (if we ignore statistically insignificant exceptions), and were apparently convinced that it was impossible that anything could ever go wrong, and that there was nothing to worry about. The possibility that concerned people living near the dam wall may already have moved away needs to be taken into account, but this strange phenomenon is otherwise clearly obvious.

Interestingly, property prices of such areas reflect what one would generally expect i.e. that they are far lower right next to the location of concern. A good example is that of the housing suburb of Melkbosstrand near the Koeberg nuclear power station near Cape Town, where the area itself is considered to be quite nice (probably because high quality housing needed to be provided for the mostly foreign contractors that originally built the plant), but the average property prices are far lower than those in similar suburban areas outside the immediate danger zone, where there’s a reasonable general expectation of a safe escape in the event of a serious emergency.

This highlights the commonly known, but sometimes very strange looking aspect of human nature – irrational optimism. The case with the dam wall demonstrates how it comes into play when people’s circumstances change for the worse: The closer we find ourselves to some unexpected apparent danger, the less likely we are to believe that the threat is real. This was also disturbingly demonstrated in the heartbreaking film The Pianist, where targeted people were often not able to convince themselves of the horrors that lay ahead, despite overwhelming evidence pointing to the obvious.

The news reports began (I think) with this New York Times article, and were quickly picked up, and rehashed by most of other mainstream media for about a week, and then quickly, seemed to disappear again.

I can’t help but wonder whether it’s this same optimism that collectively takes hold of the popular imagination when it comes to the problem of potentially dangerous new technology. Discussion about this goes right back to the early pioneers of computers and AI, and it’s become so embedded in pop culture over the years with works like 2001: A Space Odyssey, the Terminator films, The Matrix, and countless others, that it seems people have become quite immune to any alarming developments that should probably set the alarm bells ringing.  It seems we pop our heads out every now and again, after hearing something worrying, and then quickly retract them back into the apparent comforting safety of our shells.

We seem to have become accustomed to relegating news of such developments quickly into some entertainment rubbish dump of the mind, the place for things that should not be taken too seriously, possibly joining other suppressed fears as the stuff of possible future apocalyptic movies.

If you’re interested in looking into this in more depth, try reading Bill Joy’s prescient 2000 Wired article. It will soon be a decade since that was written, but the field has literally exploded with ‘progress’ since then; especially the last two years, after a long era of relative stagnation. For some reason, new developments now seems to be happening at a quicker pace than ever before, and I think the need for some serious, high level discussion about this is becoming very pressing, and should probably be taken as seriously as the matter of  dealing with the dangers of nuclear weapons, and their proliferation.

If there is one positive side effect of the coming energy shortages which the world will likely soon have to deal with, (alongside less paper junk mail — see the Hirsch Report), it’s that it may soon be increasingly more difficult to power our complex societies at current levels as it has been in the past, and that funding for these projects may be a lot more difficult to come by under those circumstances. This could slow down developments, and perhaps give people a chance to think about these matters in a more lucid way.

A few links:

Robot builds copies of itself

New military robots

Race to build robot army

Robotic ‘insect’ takes off

Search Breakthrough?

News sites have been abuzz this week with talk about the new Wolfram Alpha search engine, and it feels a little like Google might end up with a serious challenger from what seems like out of the blue. Google doesn’t appear to have been left too far behind though, and seem to be quite far along in the process of developing an answer-engine based search tool of their own.

Search has been one of those areas that most have become accustomed to being disappointed with. Over the last few years things have improved somewhat with sites like Clusty, and the Google Operators, but a search tool that’s able to infer semantic meaning from non-trivial questions hasn’t seemed close until very recently.

Alpha is still very rough around the edges and is easy to confuse with even simple questions, but if the hype it to be believed, its going to continue improving and expanding broadly.

Some interesting links: background, twine article, & some wired examples

Slashdot also has an interesting posting about their terms of service, which are a little surprising.

Silverlight vs WPF

Before version 2.0, Silverlight’s differences with WPF seemed quite clear: it was a browser plugin with a retained mode graphics system (similar to WPF), with the UI declaratively created in a restricted subset of XAML, and programmed using a subset of the .NET Framework. It appeared to have been released as a competitor to Flash, or perhaps introduced as a new take on the old Java Applet idea,  allowing developers to craft modules of their UI’s with powerful new WPF tools (e.g. Expression Blend).

It seemed to have its place, and didn’t appear to be the offspring of some grandiose plan intended to allow Microsoft an opportunity of grabbing dominance in web UI development, although there were some early worries that Microsoft would compete unfairly, with some questioning their commitment to keeping the technology cross platform.

Originally the idea of running WPF in the browser seemed interesting — cross platform WPF, but only in a limited sense, and likely to have restrictions on a lot of the things that made WPF so compelling (e.g. 3D graphics); a teaser for developers if you will. But it felt mostly like the familiar march of Microsoft technological improvements of the past, with the usual complaints that normally accompany new releases.

Silverlight 3 looks set to introduce some improvements and features that are going to change the above balance dramatically. Version 2.0 already introduced some surprising features e.g. cross-machine service calls and local machine access, blurring the distincintion between web, and desktop applications somewhat. Versions 3.0 is apparently even set to have out-of-browser capabilities!?

Was Silverlight just a facade around an MS plan to make WPF platform independent, and try and create a universal UI platform? I think not, but one could be forgiven for arriving at that conclusion. Also, how will these new Version 3 features impact WPF? It looks like it’s going to have a serious competitor in the form of an ever improving, platform independent subset of itself, dressed up in a Silverlight guise.

However it plays out, It seems clear that WPF and Silverlight are going to play very big parts in UI development in future, and I think the distinction between web and desktop applications is going to keep getting more blurred until there isn’t much of a distinction left at all.

More important than all of the above though, is what Microsoft have done for developers with all of this. Whether or not Silverlight is technically superior to Flash is, I think, a separate argument. There’s no question that the Visual Studio\WPF\Expression development tools combination is going to be extremely hard for anyone to compete with, and this is one area that Microsoft have excelled at over the last few years. They seem to have gotten it perfectly right this time.

If you’d like to explore this in little more depth, this article is well worth reading.

Here’s a MIX09 video of Silverlight out-of-browser being demonstrated.

Silverlight evolves

If you’re interested in Silverlight, but haven’t yet tried it out, its now very easy to get started. There’s a single download that includes the Silverlight runtime, the SDK, and the Visual Studio service pack. There are a few other downloads for Expression, DeepZoom and a SilverLight tookit, and it would probably be better to install those too before getting started.

After the Silverlight Tools install runs, you should see the Silverlight project templates in Visual Studio:

Select Silverlight Application, then the top option on the dialog to create an ASP.NET Web Project to be the host:

Visual Studio generates two projects which should look very similar to this:

Here are some excellent tutorials for getting started that are more thorough:

Silverlight.net getting started video.

ScottGu’s excellent tutorials on Silverlight 2, and Expression Blend.

I didn’t get the option when right clicking the xaml files in VS to open them up with Expression (as demonstrated in the getting started video) because I had an older version (1.x) of Expression installed. Expression 2, and its Service Pack 1 need to be installed before that option becomes available in Vistual Studio. It wasn’t necessary to re-install the tools as well; the order of the installs doesn’t seem to matter.

The ASP.NET project that gets generated has two files that we’re interested in, in my case they’re called SilverlightTestPage.aspx, and SilverlightTestPage.html. There’s also a Default.aspx page that isn’t in use, and can be ignored for now. Visual Studio has generated code for two different ways of hosting silverlight, one in plain html, the other using an ASP server control to do the hosting. The one we’re interested in is the aspx file, but let’s briefly take a look at how it’s done in html. If you open up the body section of the html host file you’ll see something like this:

<body>
    <!-- Runtime errors from Silverlight will be displayed here.
	This will contain debugging information and should be removed or hidden when debugging is completed -->
<div id='errorLocation' style="font-size: small;color: Gray;"></div>
<div id="silverlightControlHost">
		<object data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="100%" height="100%"><param name="source" value="ClientBin/SilverlightTest.xap"/><param name="onerror" value="onSilverlightError" /><param name="background" value="white" /><param name="minRuntimeVersion" value="2.0.31005.0" /><param name="autoUpgrade" value="true" />			<a href="http://go.microsoft.com/fwlink/?LinkID=124807" style="text-decoration: none;">     			<img src="http://go.microsoft.com/fwlink/?LinkId=108181"     			alt="Get Microsoft Silverlight" style="border-style: none"/>			</a>		</object>
		</div>
</body>

It’s done using the <object> tag with the type=”application/x-silverlight-2, which most modern browsers will know how to deal with. We’re more interested in the aspx page, where the equivalent thing is done in only a few lines. The entire body block looks like this:

<body style="height:100%;margin:0;">
    <form id="form1" runat="server" style="height:100%;">
        <asp:ScriptManager ID="ScriptManager1" runat="server"></asp:ScriptManager>
<div  style="height:100%;">
            <asp:Silverlight ID="Xaml1" runat="server" Source="~/ClientBin/SilverlightTest.xap"
               MinimumVersion="2.0.31005.0" Width="100%" Height="100%" /></div>
</form>
</body>

It uses the ScriptManager and Silverlight server controls for the embedding.

By just adding some regular ASP.NET markup close to the ScriptManager block, adding some basic xaml to the Page.xaml in the Silverlight project, and hitting F5, they’ll pop up together on the same page in your brower. If you select to view the source in the browser you’ll see that the code that was rendered is similar to that in the html page, but uses Javascript to create the Silverlight object instead.

Some of the early demos of Silverlight 1.0 seemed, at the time to be both interesting and disappointing. Disappointing mainly because of the very limited feature set, but it then also seemed that Microsoft was pushing it as a lightweight ‘sandboxed’ version of WPF that was probably aimed at challenging Flash in the browser. The idea of being able to have WPF running in the browser sounded interesting, but it was hard to imagine it upstaging the WPF platform. That it was implemented to be a small, platform independent subset, that necessarily had many limitations (e.g. no 3D graphics, because it couldn’t leverage the muscle of DirectX) must have resulted in a lot of people dismissing it as something that couldn’t be considered a serious development platform, at least not then. I certainly didn’t think it would amount to much at the time, and was surprised at how much attention it started getting (esp. after Silverlight 2 betas became available) on blogs from people whose opinions I trust (e.g. Tim Sneath and Scott Guthrie ). The main sticking point for me was that it wouldn’t use the Milcore subsystem that sits on top of Direct X, which seemed to be where the WPF magic emanated from. I assumed that in order to be portable it would have to based on a simplified foundation, which on other platforms would probably use their equivalent of GDI+.

I browsed over the chapter of it in one of my WPF books, and figured I’d wait a while before looking into it more seriously.
I saw a demo of a pre-release version of Silverlight 2.0 during the Mix08 conference in Cape Town (which caused a bit of excitement beacause Brad Abrams was going to be there, and personally do a lot of the demonstrations). A couple of people from our company that weren’t involved in Web development and hadn’t registered for the event decided to come along at the last minute after they heard that some Microsoft luminaries were scheduled to be there. The lectures and demos didn’t disappoint. One of the demos (which were started from scratch) walked through the then new pre-release Silverlight tools. The speaker went through the motions of creating a simple Silverlight application, which called through to a WCF Service, that used LINQ to SQL for the data access. I remember being very surprised that things had come along as quickly as they had.

There was a Microsoft Expression event in January 09 given by Michael Koester, and I think everyone that attended came away from that feeling impressed. I remember thinking that the distinction between WPF and Silverlight had blurred quite significantly, and now thinking back, I find it hard to remember wich demos were done with WPF, and which with Silverlight.

Silverlight 3 beta 1 tools are now available, and there’s no question that it’s going to play a big part in front end development going forward. How it finds its place, and comes to co-exist with the desktop development platforms in future (esp. with the Silverlight 3 browserless features) is going to be interesting to see unfolding.

Code-Behind or Single-File

I’m busy wading through Stephen Walther’s book, revisiting ASP.NET after spending the last while on a desktop application full time.

Two things about it have struck me — how much its changed (over the last few years), but also how much it feels the same. I’ve still got quite a distance to walk, but my overall impression so far is that the original ideas behind ASP.NET turned out to be very solid, and that a massive amount of work has been done since then to modernise what was essentially a very elegant platform.

I was suprised by something early on, and it left a bit of a niggle. He mentions that there are a couple of different options when it comes to organising code, and that he prefers using the Single-File method of placing the code in script blocks above the markup.  He explains which options to pick in Visual Studio to get it arranged that way (The default project templates still use code-hehind ) , and then goes ahead and does it for all the samples.

I think I was so surprised because I’ve seen so much code-beside in WinForms and WPF recently that its became almost expected, and also remember using it in older versions of ASP.NET. It seemed like a bit of an affront to the senses when looking at it at first, perhaps because its associated somewhere with the style of writing old interpreted ASP, which is now very outdated.

His reasons for choosing it are quite interesting. He dismisses the argument about a clean separation of code and markup by arguing that the main reason for doing that would be to support code re-use, and because that doesn’t happen much with Aspx pages, the benefits of separation just aren’t there. He thinks its preferable to have to manage fewer pages, rather than more just for the sake of it.

This is a compelling argument, and I find it hard to argue with, but haven’t yet decided if I’m completely swayed. He mentioned that there’s a big debate raging about the issue in a lot of forms, so I decided to take a look.

There are lots of  strong opinions about this, but I found one particular discussion that seems both interesting and fair.  The bottom line: It depends on personal preference and personality.  Please don’t let that dissuade you from reading the article, there are technical considerations (e.g. compilation issues) that need to be taken into account as well!

Problems with var

The (relatively) new C#3 var keyword has worried me since I first saw it. The first thing that came to mind was the old VB variant datatype that used to allow the avoidance of strong type checking. That C# is stronlgy typed is one of the reasons why I think it feels so solid. It turns out that it wasn’t introduced to allow weakly typed code to slip past the compiler, and that it doesn’t even allow weakly typed data declarations (more below),  but was required to make other new language additions work, e.g. anonymous types:

var o = new {Name="Cole", Gender="Male", Breed="Ragdoll"}

The compiler will create an anonymous class for you (and keep it hidden away). This class will have 3 public properties  with their respective getters and setters (Name, Gender and Breed). This looks convenient, and could cut down a lot of code, especially when retrieving or passing data around, but it also looks like it might add to the problem. It’s easy to imagine a lot of code like this adding up to something that’s hard to read and maintain, I noticed not long ago that a lot of the new language features were popping up all over our code, and not all of them seemed easy to read, but take a look at this:

var query = from c in db.Cats join o in db.Owners on c.OwnerId equals o.Id select new   { OwnerName = (o.Name ?? , c.ClubId), CatName = c.Name, Breed = c.Breed };

In this case the necessity of the existence of the var keyword starts to become more clear. What kind of a datatype is it returning? And this is not even a complicated query! var Allows you to leave it up to the compiler. The MSDN documentation gives a concise description:

“Beginning in Visual C# 3.0, variables that are declared at method scope can have an implicit type var. An implicitly typed local variable is strongly typed just as if you had declared the type yourself, but the compiler determines the type.”

So var isn’t about allowing weakly typed data, but sometimes it looks that way. What it can do is allow an undisciplined coder to make his code difficult to read and maintain if used carelessly, but is that really a big problem? Isn’t that why code reviews, code guidelines and colleagues’ opinions are so useful?

This reminds me of  some of the arguments I heard around the time of the first release of WPF. There was a lot of concern that Microsoft was making a big mistake — finally yielding to the UNIX philosophy of allowing the mechanism, but not restricting policy. Microsoft had, when it came to UI frameworks, restricted policy for almost 20 years. WPF would allow UI’s to look however the designers pleased, it would now be very easy to do — and even encouraged!

I’ve come to think of these as good things,  developers are being handed powerful tools, that need to be handled with care, and I think it should be our responsibility to learn how to use them properly. There’s definitely no lack of guidance if an effort is made.

Here are a few links to some interesting discussions about this; from some of these it looks like this may even turn into something of a minor religious war within the C# community.

Discussion at StackOverflow with some very interesting  points of argument.                                                                                 Scott Hanselman’s var != dim.
MSDN documentation (the thread at the bottom of the page is an interesting read).

Are C#3 feature additions just sugar?

While reading my first C# book shortly after version 1 of the famework had been released, I was initially surprised at how similar it looked to Java which had been the main language of instruction in our Computer Science courses, and almost didn’t continue past the first few chapters. The author’s enthusaism for C# was infectious though, and after persevering with it for a while the distinction started to become more clear. Feature by feature, the differences started adding up to something that really did feel like a new language, by the latter stages of the book I remember feeling excited about making a move to the language and the platform.

Early versions of C# were criticised for their similarity to Java around that time, but over the years as new features were added, that argument seemed to fade away. With each new version of the language, lots of new feature additions have been included. Looked at individually they can sometimes seem underwhelming, but comparing large sections of code from different versions of the language can surprise even experienced C# coders how much things have changed since its first release.

I’m hoping to post examples of a lot of the newer language additions from C#2 onwards, but will start with a small new C#3 feature that reminded me of the original property concept back in C#1.

Properties were one of the features that were hyped a lot in C#1. The idea wasn’t new, and was probably borrowed from the COM world where the get_ , and set_ prefixes for accessor methods had been a convention for quite a while. The small change to the language that enabled accessing the property, using the . operator rather than calling the accessor methods e.g object.PropertyName was a very small syntactic change (that initially felt uncomfortable for those with a C++ background), but it resulted in code that (after some mental adjustment) looked neater than the equivalent construct with a field and accessor method in C++ or Java.

The property access syntax was then described as a better balance between a field and a property — depending from where you’re accessing the property, (inside or outside the class) you’ll see the right thing. It abstracted away implementation details of the class while still allowing easy access.  C#3 has now introduced a new change to the property concept that is similarly interesting, Automatic Properties.

Ignoring coding style and guidelines (which I’ll talk about in another post), take a simple pre C#3 class e.g:

public class Cat { string _breed; string _coatPattern; public string Breed { get { return _breed; } set { _breed = value; } } public string CoatPattern { get { return _coatPattern; } set { _coatPattern = value; } } }

The equivalent thing with Automatic Properties now looks like this:

public class Cat { public string Breed { get; set; } public string CoatPattern { get; set; } }

So the idea has been taken further: the backing fields will be created by the compiler, and access from within the class will be through the property accessors. Clearly if the idea is to manipulate the data before storing it in the backing field then this won’t work, and will need to be done the regular way. But most properties are just direct accessors to backing fields, so this small change cut out large amounts of clutter.

If we wanted to make the property read-only, an obvious thing would be to try this:

public class Cat { public string Breed { get; } //.. } but this causes a compile error, the correct way to do it is like this: public class Cat { public string Breed { get; private set; } //.. }

Generic event handlers save time, and look better!

I started this entry with a discussion about keeping up to date with language and framework changes, but it became longer than intended so I decided to move it to a separate post below. This example ties into what was discussed there. I’m hoping to add random posts over the months with lots of examples like this.*

Before the .NET Framework 2.0 was released, if you wanted to pass custom information through to an event handler when an event was fired,  a delegate needed to be declared that specified a class that was derived from System.EventArgs.  Framework 2.0 introduced the generic System.EventHandler delegate, which allows any class derived from EventArgs to be used with the event handler, making it much easier (and cleaner) to pass arbitrary data through to an event handler.

This example uses a MessageMaker class that has the generic event MessageMade. The MessageEventArgs class has a property for the Message itself as well as one for a Priority, the idea being that the MessageMaker could create both prioritised messages as well as plain messages with no notion of importance.

public enum MessagePriority { Low, Moderate, High }

The MessageEventArgs class made slightly cleaner with C#3 Automatic Properties:

public class MessageEventArgs : EventArgs { public string Message { get; set; } public MessagePriority MessagePriority { get; set; } public MessageEventArgs(string message) { Message = message; } public MessageEventArgs(string message, MessagePriority messagePriority) { Message = message; MessagePriority = messagePriority; } }

The MessageMaker, with the generic event:

public class MessageMaker { public event EventHandler<MessageEventArgs> MessageMade; protected virtual void OnMessageMade(MessageEventArgs mea) { if (MessageMade != null) MessageMade(this, mea); } public void SimulateMessageCreation(MessageEventArgs mea) { OnMessageMade(mea); } }

and the MessageMonitor whose event handler watches for the events:

public class MessageMonitor { public MessageMonitor(MessageMaker messageMaker) { messageMaker.MessageMade += new EventHandler(MessageProcessor); } void MessageProcessor(object sender, MessageEventArgs mea) { if (mea.MessagePriority != null) { Console.WriteLine("Prioritised message detected"); Console.WriteLine(string.Format("Message:{0}, priority:{1}", mea.Message, mea.MessagePriority)); } else { Console.WriteLine("Regular message detected.."); Console.WriteLine(mea.Message); } } } class Program { static void Main(string[] args) { MessageMaker messageMaker = new MessageMaker(); MessageMonitor messageMonitor = new MessageMonitor(messageMaker); messageMaker.SimulateMessageCreation(new MessageEventArgs( "Many more margaritas", MessagePriority.Moderate)); System.Console.ReadLine(); } }

The main point here is that no delegate needs to be declared, and we can inspect the properties in the MessageEventArgs class within the event handler to examine this (arbitrary) information, and act accordingly.

* The MSDN samples are often very good, but every now and then a bit lacking.  I like to write snippets and my own examples even when the provided ones are clear, as things seem to hang around longer when done that way.  Examples on this blog shouldn’t be interpreted as an indication of my opinion about the quality of the MSDN samples related to the topic in question.

On choosing the right path, and staying current. part 1

A few years ago Charles Petzold wrote the strongly titled article “Does Visual Studio Rot the mind?”  It did a lot of mileage in email forwards at the company where I then worked, and caused quite a bit of discussion and debate at the time. I’m quite sure our groupthink must have reflected reactions pretty much the world over in MS development communities, as he put across some very strong opinions about some universal programming truths – this from a revered programming ‘Guru’ that most people could hardly have thought of as being extremely opinionated in any way,* but likely rather associated with a clutch of positive traits that so many of us often wish we had more of.  A whole generation of programmers cut their teeth on his original Programming Windows tome, and I still find it hard to believe that a single person was able to write that book.  It could only have been done with a staggering knowledge of (certain parts) of the Win32 API, and an iron will.** It took me the better part of a year to wade through most of it (while I was a student), and was probably the biggest reason that I veered toward the Windows platform at the time.

I’ve had quite a few discussions about how to keep up to date with language and framework changes over the years with friends and colleagues, and have found that the general consensus is an affirmation of Charles Petzold’s suggestions, that we willingly accept the train of events that allow us to arrive at a point where most agree that it’s simply not possible to keep up to date with the rapid pace of change anymore, at least not with all of the significant changes and technology updates on the sprawling Windows / .NET platform.  This is something that a lot of developers have strong opinions about, and is sometimes good fuel for argument. It’s taken me a long time to make up my mind about this, but I’ve come to believe that there are many individual cases that can change ones mind about this if presented in the right way.

The numbers themselves sound impressive; over 13000 classes in Version 3.5 of the Framework, but numbers themselves sometimes aren’t enough to convey the scale of what this means. The way I like to think of it is to compare it to an experience I had several years ago when I was first attempting to find my way around the university library. After arriving at the computer science section I felt quite stunned – thousands, many thousands of books… it would be impossible to read through even a small fraction of them. Surely it would be stupid to even try!?  This led to one of those moments that don’t come very often, that can lead one to question all sorts of things that we take for granted e.g. that there’s a direct correlation between effort and  success, or  that expert knowledge and expertise come as a result of hard work and determination. This seemed to drive home an unsettling idea that there are many other factors that come into play here.***  When you’re trying to find your way as a student its probably natural to question a lot of things that have long been taken for granted, probably because the mental models of our ‘world’ that have served us so well until then, and which are still (mostly) being carried around, seem to break down far more often during this period than we seem to remember.  I think this may contribute to the ease with which some young adults quickly change opinions about some important things — by deferring to authority (or subconsciously picking one of many other options from a rather long list) at a time when an important gap needs to be filled. A common example is that of a student who suddenly comes to believe that Windows is ‘bad’ and UNIX is ‘good’ because someone in a position of academic authority said that things were so. I think that this ‘specialization’ idea might be one of these beliefs that is often unfortunately instilled around this time.

I find it interesting how often IT gets compared to Medicine (with medical students being required to commit masses of encyclopedic type knowledge to memory) when this topic comes up, and have also noticed some interesting correlations between opinions about this, age, and number of years of programming experience.  A large part of me likes to believe we’ve been conditioned to accept that there’s simply too much to digest;  that it’s mandatory to pick a niche area and try and find a comfortable space within it.  There are some interesting counterexamples of this that I’ll discuss later.

In practice (in the Windows world), this “must-specialize” meme normally translates into a graduate developer either being assigned to, or choosing to work on a project that falls into one of the following categories:  web, desktop, middle layer, database or security. There is at least one other category (mainly) in the UNIX world of large scale concurrent applications mostly for the telecommunications industry. This area should probably be avoided if at all possible by developers who are not completely comfortable on UNIX, and has caused untold misery and suffering for those who have wondered in thinking that it might be an easy environment to adjust to.  Survival in these cases can sometimes depend on serendipity; which for some might come in the form of a sympathetic veteran taking the newcomer under their wing until they’re able to tread water. I’ve always loved the expression “anything is easy if you can do it”, and it applies very well to all things UNIX.  I don’t think there’s anything inherently more difficult about UNIX once you’re at ease on the platform, but there’s absolutely no question that it requires a lot more effort to move from ‘beginner’ to ‘competent’ on UNIX than it does on Windows.  I also think its true that its harder to move from ‘competent’ to ‘accomplished’  on Windows than it is on UNIX, with Jeffrey Richter’s Advanced Windows book compared with W. Richard Stevens’ Advanced UNIX book being a good example of this.  These are both classic books, but to try and pick which is better is to completely miss the point;  there’s not a whole lot of difference between the platforms for developers in terms of difficulty after reaching an ‘intermediate’ level, where programmers generally start focusing almost entirely on the problem at hand rather than the platform, with it almost becoming second nature, and disappearing into the background as time goes by.

This touches on two other topics that I hope to post on in future:  the misdirected energy that causes a lot of smart people to fall into the platform ‘culture war’ trap, and another about different ways of approaching learning. Even when just talking about talking about this; I get reminded of something that Andrew Tanenbaum said about his experiences at MIT. ‘Learning at MIT is like trying to drink water from a fully opened fire hydrant”. I’ll leave this here for the moment with one piece of advice: If you you’re not able to, or have never attempted to create or edit a non-trivial multi-file Makefile, there are much easier routes into the UNIX world. Once inside, changing direction is quite easy and natural.

Since the web & desktop areas are so broad, most new and intermediate developers have a realization at some point that it’s a lot less easy than they thought to become adept at all of the different areas under that umbrella, especially after they’ve been assigned to a medium or large sized project which has been split into different groups, where interaction between different modules or layers of code is done mostly through a set of well defined interfaces.  This is especially true with projects that are technically complex, where each developer needs to be very familiar with a lot of complexity, and perhaps tens of thousands (and continually increasing) lines of code just within their area.  A developer in an environment like this can easily spend 6 months mostly in one particular layer of code, without getting a strong understanding and handle on the other layers of the project.

This is something that Fred Brooks talks about in the Mythical Man-Month when he discusses the value of having people on a project with varied technical skills. In practice it seems that an arrangement like this often works well, and the related idea that surprised me the most was the suggestion that it’s not necessary to have a team full of language experts (pedants) , which could possibly even be counterproductive!?  Possible reasons could be the tendency to focus on some areas at the expense of others, more competition instead of co-operation, technical disagreements, an inherent ‘natural’ belief that being an expert in one area qualifies one to make significant decisions in another area , and so forth. Clearly presented arguments like this only seem to muddy the waters further — we’ll have to return to this argument later.

I think there are some problems with the education system where this kind of thinking is fostered. Perhaps a better approach could be the encouragement of initially shining the focus on the ‘big picture’. I do acknowledge that graduate developers often only run into these issues in their first year or two of work, but doesn’t that make the case for extending the technical training time (e.g. by a year) at most universities or institutions, where students could be assigned to projects in a way that reflects successful real-world projects, seem like a sensible idea?

There’s quite a bit more of this to come, and I’ve got some interesting examples of  different cases to discuss, but this will have to wait for another post.

*Is  this an indication that the common perception might not reflect reality closely?

** On second thought, perhaps the common perceptions are wrong, and extreme traits should be expected.

*** This might make for another post.

First Post

I’ve been undecided about doing a blog for a few years, vacillating between the positions of:

• “It’s not me – I’m an introvert”
• “I’d like to try it, and its easy to get started, but much less easy to keep up to date”.  I remember reading an article a few years ago, in which someone famous (can’t remember who) was being interviewed, and asked their opinion about keeping a blog. The answer went something along the lines of  “I think they’re great, and have thought of doing one, but I’m not sure that I have the time to commit to it now etc.  .. a stale blog is worse than no blog etc.’  That seemed like a strong argument at the time, and reminded me of the anecdote about the old chess master Emanuel Lasker (who was atypically tardy among his peers) who, when asked one day why he’d stopped wearing his wristwatch responded, “when I wear it, I feel tyrannized by the damn thing!”
• “Some of my friends do it & I’d like to try it, but don’t have the time” *

Anyway I’ve got some time now, and have a few other incentives e.g. making a few positive lifestyle changes before getting settled down into  a new job (after which, past experience has shown it gets easier to have those urges to do new things be sabotaged by bullet 3 above). I’m also hoping that this might make it somewhat easier in my current quest to find the right job.

I’m planning on keeping the posts mostly technical, and related to things that I’ve been involved with in the past, or hope to do in future. For my (also new) non-technical blog, click here.

*   This pattern of thought is of course either just your mind making a subconscious decision to avoid an initial burst of pain that often goes along with (sometimes even minor) lifestyle changes, or just plain laziness.