Shey’s Rebellion

I’ve always equated the use of DataSets anywhere outside the data layer with code smell, it’s a sure sign that a proper domain model doesn’t exist; it’s also just as likely that the application isn’t layered.  Lets look at some of the reasons you should avoid using DataSets.

DataSets don’t properly abstract the database, the developer has to be intrinsically aware of the database schema, he has to know the name of the columns and the type of the columns you are returning in the DataSet.  Anytime a developer wants to use the DataSet he will have to look at the underlying query which creates the DataSet and pick out the field names used in the query.  Instead of understanding the business process, instead of adding value, your developer will be mucking around with database queries trying to figure out which field name to use. 

Additionally, since DataSets are weakly typed, they don’t tell you the type of the column, the developer must know the type of the field and cast the field to the appropriate type.  Every time you cast your run the risk the cast will fail, the value returned could be null or different type.

Since your application/presentation layer is so tightly coupled to the database your design becomes brittle and difficult to change– every time your database schema changes, it’s effects will ripple throughout your system creating a maintenance nightmare. 

What troubles me more though is that DataSets don’t give you the opportunity to create a business logic layer, because DataSets are just data containers you can’t add functionality to them.  Two DataSets can’t interact with each other, you can’t abstract problems. Instead you’re forced to write procedural code to work with DataSets and you’ve surrendered all the benefits of object oriented programming.

Most of the problems associated with DataSets can be solved by using classes that represent your domain model within a well-defined business layer, essentially your represent your data tables with classes.  ORM is immensely useful here, and I’ll discuss it in detail in my next post.

Sometimes, we need someone to remind us, we’re doing it wrong.

Here’s my quick list– you’re doing it wrong if:

• you’re not using ORM.
• you’re using (ADO.NET) data sets
• you’re not refactoring
• you can’t deploy in one step
• you’re not always learning.

stop doing it wrong, start learning how to do it right:  AgilePakistan.org

I’ve always considered Software Developers to be a special breed.  Good software developers are driven– they set aside time to hone their technique and learn new methods and technologies, they strive to do better.

If this sounds like you, then read on:  I’m trying to form study group of programmers who are interested in studying software development.  The group would meet once or twice a month to present new ideas, discuss technology related articles and read and refactor code.   

The main focus of the group would be Object oriented programming and design; initially I would like to cover topics such as ORM, Domain Driven Design and Inversion of Control.

Obviously this is for Karachi only, if you’re still interested drop a note in the comments.

As a software developer, I have unintentionally surrounded myself with other software developers who share the same ideals as myself. If you asked any them if good design is worth the effort, they will answer with an unequivocal yes!  They may ask “How can you question design?  It is the heart of software development.”

Project managers will stand behind us, nodding in agreement, because no project manager is brave enough to openly say that design isn’t a worthwhile activity, instead they’ll rephrase their disagreement: “We have a deadline approaching and we need to get this functionality out of the door fast!” A dangerous phrase which means they want you to churn out code as fast as you can without regard to future costs.

Software developers use the term “Technical Debt” to describe the obligation an organization incurs when it chooses a design or approach that’s quicker in the short term but increases complexity of the software and is more costly in the long term. 

Imagine, skipping the dishes after dinner, instead of washing them you let them pile up in your sink, you’ll save time now, but eventually, you’ll have a huge mess in your kitchen and you won’t have any dishes to eat on. Even worse, imagine that your in-laws drop in on a surprise visit and you don’t have any dishes to serve them food on—these imaginary in-laws are clients asking you for new functionality.

Technical debt prevents you from meeting your customer’s needs quickly, modifications to the code base take exponentially longer and as your technical debt accumulates you expose yourself to more risk.

Sometimes, just like financial debt, it is beneficial to take on technical debt to exploit an opportunity, but like credit card debt, the interest on technical debt can catch up on you, all too often we don’t manage our debt, it spirals out of control and we spend most of the our money paying of the interest, not the principle.

Software development is a series of compromises, but we can’t fool ourselves into believing that we can trade good design for greater development speed.

The problem with sacrificing design is that the code becomes unstable, adding a feature will break the software in other places; the code becomes to modify, a simple change requires modifying other parts of the system and finally, the code becomes harder to read, developers spend days trying to figure out which files need to change.  Eventually, your developers will become frustrated and they’ll start proposing the big rewrite of the system.

Development in the absence of design yields systems which have a high level of coupling, where functionality is spread across the entire application and often duplicated in several locations across the code base.

As the application evolves, these duplicate functions become unsynchronized, making bug fixes more difficult and time consuming. Maintaining a code base with high coupling is difficult and time consuming, your developers waste time looking for the code to modify searching for the duplicate code in ten different files, instead of developing new functionality; they waste time re-understanding the code. All of these problems compound to make maintenance and further development extremely expensive.

Programmers are lazy and if good design actually increased the effort and time required we’d be against it.  Software design is an effortful activity, but one which leads to a net increase in productivity and efficiency.

For software to remain relevant and profitable, it must change often, add value by growing in features, if your code is brittle and fragile, then adding new features to your application is difficult, eventually you’ll have to turn away customers because you can’t meet their requirements in time. Extrapolate this over five years and imagine the profit you had to forgo because your code is unstable.

Good design ensures functionality is confined to a single location, it encourages code reuse, makes maintenance easier and it makes it easier to add new features, in short good design saves you money.

Everyone knows that source control is fundamental to the practice of modern software development, but most people don’t know how to use it. Today I want briefly comment on comments.

Comments with your commits are important; I assumed this was obvious, why should you comment your commits? Comments quickly tell you: what changed, why it changed, which files you will need to Diff and if change is of interest to you. If your developers use good commit comments, then your repository will not only contain every change you have ever made, it’ll also have an explanation of all those changes. As a project develops and matures, these kinds of comments are invaluable, especially when you consider developer turnover.

If you don’t comment your commits, then the only information you have is that Joe committed a file in March. Now you’ll spend 30 minutes figuring out what changes he made; if you’re lucky Joe’s still around and he still remembers why he made those changes–tracking down Joe, that’s another 30 minutes. In all likelihood, Joe doesn’t remember why he made those changes, and without comments there’s nothing to cue his memory, but you really need to know why he made this change, so you insists that he checks his emails from March, maybe something in his email will jog his memory.

Yes, you’ve just wasted 2 hours of development time because you didn’t comment your commits.

BooLangStudio for Visual Studio 2008!

What about us 2005 guys?

Web Services for NAV 5.0. Finally.

Please, just let it die. For those who still love C++ take a gander at D.

Holy smokes, I didn’t know that .NET 3.5 supported parallel programming.

Bravo Microsoft.

The end of RDMBS is near! This isn’t another ORM rant, but let me be honest: I don’t like relational databases and I can’t wait for the day when I don’t have to read another query. RDBMS don’t fit our model of computing.

Working with the object oriented paradigm you quickly realize that your objects don’t fit in relational databases—they’re square pegs and your database is a round hole. RDBMS came before object oriented programming, before the web; our requirements have changed since the 70’s. So we’ve invented pretty toys like Active Record and NHibernate to fit our square pegs into databases but in the end they’re kludges, elegant kludges because we’re still stuck with RDBMS.

In 2008, how do we use databases? I bet most of us use them to only store data? For many of our applications we need key lookups, easy scalability, ability to store massive amounts of data and none of the other overhead that is associated with administration. Enter Document Oriented Databases– Imagine giant hash tables floating in the sky, no schemas or tables. No fixed data types, just keys and data. CouchDB and Amazon’s SimpleDB are two such database systems, which happen to be built in Erlang. You can read more about CouchDB, but the important thing to know is that it’s a distributed database system which works well for semi-structured data.

Now imagine this Reduce(Map(F(CouchDB)) and Bam! If you think I’m doing a lot of hand waving here, I am, but look at Google, what makes their infrastructure so scalable? How does Google achieve their massive scalability? BigTable and Map/Reduce! Document oriented Databases and Map/Reduce are changing the way we look at scaling and Map/Reduce is quickly becoming the essence of scalable high-performance computing (and the end to RDMBSs).

There’s a lot more to this, where it’s headed I’m not sure, but it’s making my spidey sense tingle. There are other closely related topics that should be explored, column oriented database and Hadoop, but that’s for later, I need to breathe.