Thoughts on Enterprise Architecture and Project Management

Extension for 14th Carnival of Enterprise Architecture

I have extended the deadline for submissions to the 14th Carnival of Enterprise Architecture to Saturday, January 17, 2009, with publication the next day on the 18th. You can get further info for submissions here, and thanks!

14th Carnival of Enterprise Architecture Coming January 15, 2009

The 14th Carnival of Enterprise Architecture that I host through Blog Carnival is coming up on January 15, 2009, with submissions due by January 14, 2009. If you have an EA- or IT-related post that you'd like featured through the Carnival, you can submit the Permalink URL and related information here.

This is a very easy and effective way for EA and IT bloggers to get their messages across to a wider audience, and I hope that EA and IT bloggers will consider submitting their best pieces to the Carnival. Please note that you do not have to be the author of the material to submit it, as long as it: a) meets the guidelines below; and b) honors the author's copyright and fair use guidelines.

As always, here are a few pointers for article submissions to the carnival:

* Off-topic material, sales pitches (direct or disguised) for products & services, and content spam will be not be published. Prior to every carnival episode, I delete a number of pieces of obvious spam, vendor ads, and off-topic material prior to publication - and it's a complete pain in the rear, believe me. Please don't waste your time or mine and thanks in advance.

* Feel free to nominate blog posts of others for the Carnival, but make sure to identify the author!

* Material posted in previous editions of the Carnival will not be re-published, regardless of who submitted it and when it was originally submitted.

Again, the deadline for submissions is January 14, 2009, and the carnival will be published on January 15, 2009. I look forward to all worthy contributions!

The 14th Carnival of Enterprise Architecture that I host through Blog Carnival is coming up on January 15, 2009, with submissions due by January 14, 2009. If you have an EA- or IT-related post that you'd like featured through the Carnival, you can submit the Permalink URL and related information here.

This is a very easy and effective way for EA and IT bloggers to get their messages across to a wider audience, and I hope that EA and IT bloggers will consider submitting their best pieces to the Carnival. Please note that you do not have to be the author of the material to submit it, as long as it: a) meets the guidelines below; and b) honors the author's copyright and fair use guidelines.

As always, here are a few pointers for article submissions to the carnival:

* Off-topic material, sales pitches (direct or disguised) for products & services, and content spam will be not be published. Prior to every carnival episode, I delete a number of pieces of obvious spam, vendor ads, and off-topic material prior to publication - and it's a complete pain in the rear, believe me. Please don't waste your time or mine and thanks in advance.

* Feel free to nominate blog posts of others for the Carnival, but make sure to identify the author!

* Material posted in previous editions of the Carnival will not be re-published, regardless of who submitted it and when it was originally submitted.

Again, the deadline for submissions is January 14, 2009, and the carnival will be published on January 15, 2009. I look forward to all worthy contributions!

One Reason We're In A Recession...

Looks like these idiots thought that my "Measuring Technical Debt" post yesterday merited attention due to the "great money management tips" I offered....

There are myriad ways people lose money in the financial markets, and this is obviously one web-related way to do so.

Disclaimer: I am NOT an investment advisor. Not federally insured either. Some investors can and do lose money. Your mileage may vary. Specific investment performance is not guaranteed. LOL

Measuring Technical Debt

A number of bloggers including yours truly have been revisiting the concept of technical debt over the past month or two, given the parallels between it and the financial kind making headlines every 10 minutes; and also the fact that projects are being reined-in across numerous organizations due to budget, economic, and headcount issues caused by difficult current economic conditions.

Martin Fowler recently posted an interesting piece about how one would go about measuring technical debt, and I'll add a few other items to consider.

One thing about technical debt that often gets overlooked is that it cannot be reasonably measured or estimated until it has been incurred - that is, after the fact. Financial debt scenarios are estimated in a more straightforward way because certain critical factors such as interest rates and payback periods are known up front, reducing the uncertainty, but not eliminating it.

Martin also mentions that measurements of technical debt are subjective because the resulting system is measured against a system in, as he put it, "an imaginary state." While this is true in a substantial number of cases, I believe that some (not all) of the subjectivity in such measurements can be mitigated if an overall design/architecture exists up front. How much design and architecture? That's also quite subjective, but such artifacts would establish a baseline that facilitate objectivity in measuring and controlling technical debt.

Then we come to the debt measurements themselves. The most straightforward metric with technical debt is time - how much time will/did it take to build/refactor assuming that the overall system was "clean?" Those estimates come in units of time as that what the project team utilizes daily and reports in standups or project meetings. However, time as a form of 'debt' usually doesn't resonate with non-technical managers - but money does. For example, I can tell a project sponsor or product owner something along the lines of Martin's example - 'if the system was 'clean,' we incurred 2 days of interest on our technical debt." While that usually gets acknowledged, the ramifications aren't clear.

What does get their attention is when that 2 days of interest is converted into monetary terms. The straightforward way of doing that is: (Cost of involved team members per day) X (Number of days of interest). For this to work, we must know how much 1 day of labor per team member (hours work also) is costing the project and multiply it by the number of team members involved. Then multiply it by the days (or hours) of interest paid. Now, we have an approximation of how much the technical debt cost in financial terms, which will definitely get management attention if its high enough (and hopefully not high enough to get the messenger shot).

Again, keep in mind that these measurements are made after the technical debt is incurred, and are rough approximations, not cold hard facts. Having a design/architecture as a baseline improves these measurements, otherwise the 'imaginary state" (or design/architecture after the fact) as Martin describes must suffice.

If anyone has better methods or optimizations on this, I'm all ears and eyes.

Carnival of Enterprise Architecture #13 - December 2, 2008

Welcome to the December 2, 2008 edition of Carnival of Enterprise Architecture.

Business Process Management

Ian Louw presents Being Human - A Natural Inhibitor to BPM Adoption? posted at Business Process Management (BPM) - InSights.

Enterprise Architecture

Alan Inglis presents "Correct" is a contextual thing... posted at Chief Architect.

Alan Inglis presents Architecture as diagrams is an anti-pattern! posted at Chief Architect.

Jimson Lee presents Customer Data Integration Between Open Source SugarCRM Package and 3rd-party Applications posted at CRM Help Desk Software.com, saying, "An interesting story involving integrating and customizing multiple databases, including their CRM, Call Center for support, and bug/defect tracking for all releases of their software."

Mike Kavis presents Toolbox for IT - Knowledge Sharing Communities posted at Enterprise Architecture & Other Enterprise Topics.

Other

Paul Wallis presents IT (still) exists for one reason posted at Keystones and Rivets, saying, "Wallis continues his argument that IT's role is to manage the flow of data between business assets."

Robert McIlree presents The Bastardization of Agile posted at Technology Architecture & Projects, saying, "Agile is sometimes used not as originally intended, and the results are often predictable."

Robert McIlree presents Proposed Federal Uber-CTO: It Isn't About The Technology posted at Technology Architecture & Projects.

Madeleine Begun Kane presents Life-Saving, Spam-Fighting WordPress Plugin posted at Mad Kane's Humor Blog. 

Madeleine Begun Kane presents Twitter Jitters posted at Mad Kane's Humor Blog.

http://www.panisphere.com/google-apps-growing-popularity-on-corporate-world presents Google Apps Growing Popularity on Corporate World posted at PaniSphere, saying, "Google Apps which is a collection of software is becoming more popular among businesses due to its relatively cheap price compared to other software provided by key players including Microsoft and SAP."

SOA

M presents Generating the WSDL before writing your Web Service posted at SandboxM.

Mike Kavis presents Toolbox for IT - Knowledge Sharing Communities posted at Enterprise Architecture & Other Enterprise Topics.

Mike Kavis presents Enterprise Mashups - The Icing on your SOA posted at Enterprise Architecture & Other Enterprise Topics.

Paul Wallis presents Understanding SOA posted at Keystones and Rivets, saying, "Wallis explains simply how SOA works, how it can be used and, describes why a properly planned and implemented Service Oriented Architecture can create a flexible way of aligning business and IT."

That concludes this edition. Submit your blog article to the next edition of Carnival of Enterprise Architecture using our carnival submission form. Past posts and future hosts can be found on our blog carnival index page.

Reminder: Carnival of Enterprise Architecture #13 Coming December 2, 2008

A friendly reminder that the 13th Carnival of Enterprise Architecture that I host through Blog Carnival is coming up on December 2, 2008, with submissions due by December 1, 2008. If you have an EA- or IT-related post that you'd like featured through the Carnival, you can submit the Permalink URL and related information here.

This is a very easy and effective way for EA and IT bloggers to get their messages across to a wider audience, and I hope that EA and IT bloggers will consider submitting their best pieces to the Carnival.

A few pointers for article submissions to the carnival:

* Spam, which includes pitches for commercial software or services, content spam, and off-topic material will be not be published.

* Feel free to nominate blog posts of others for the Carnival, but make sure to identify the author!

* Material posted in previous editions of the Carnival will not be re-published.

Again, the deadline for submissions is December 1, 2008, and the carnival will be published on December 2, 2008. I look forward to your contributions!

A friendly reminder that the 13th Carnival of Enterprise Architecture that I host through Blog Carnival is coming up on December 2, 2008, with submissions due by December 1, 2008. If you have an EA- or IT-related post that you'd like featured through the Carnival, you can submit the Permalink URL and related information here.

This is a very easy and effective way for EA and IT bloggers to get their messages across to a wider audience, and I hope that EA and IT bloggers will consider submitting their best pieces to the Carnival.

A few pointers for article submissions to the carnival:

* Spam, which includes pitches for commercial software or services, content spam, and off-topic material will be not be published.

* Feel free to nominate blog posts of others for the Carnival, but make sure to identify the author!

* Material posted in previous editions of the Carnival will not be re-published.

Again, the deadline for submissions is December 1, 2008, and the carnival will be published on December 2, 2008. I look forward to your contributions!

Is The Call For A Federal CIO/CTO "Reinventing Government" By Another Name?

I spent some time recently going a bit deeper into President-elect Obama's call for a Federal-level Chief Technology Officer and came to the realization that it, and the various responses by individuals and groups, strongly resemble major parts of the Clinton Administration's (specifically then-Vice President Al Gore) attempt to "Reinvent Government" (and, to be completely fair historically and politically, the 1994 Republican Party "Contract with America" that helped them sweep congressional seats wasn't any better - or worse.) History shows that while the effort led to some marginal improvements here-and-there and saved a few billion dollars that were quickly spent on something else, it really didn't accomplish very much and the results were nowhere near what the political speeches trumpeted to the electorate in that day.

As I googled and found various documents, articles, and other artifacts from those efforts, I came across an article written in February, 1995 by the late management guru Peter Drucker in The Atlantic Monthly. Dr. Drucker posits at length why "reforms" like this don't usually work, and proposed how they could do so. As with almost all of his writings, this one is well worth reading and really resonated with me.

In light of the Obama CTO proposal and Friday's (11/14/2008) news that there is little or no government oversight around the $700 billion of our money being dished out to bail out banks and insurers, Dr. Drucker's commentary, although made years ago, takes on a greater sense of urgency to reform and rightsize the federal government. Some highlights:

"There has been no lack of publicity about the Gore initiative since. Press release after press release has announced the reinvention of yet another agency or program; big conferences, one chaired by the President himself, have been convened, and any number of TV appearances made. Of all the domestic programs of the Clinton Administration, this is one of the few in which there actually have been results and not just speeches. Yet neither the public nor the media have shown much interest.

...There are good reasons for this. In any institution other than the federal government, the changes being trumpeted as reinventions would not even be announced, except perhaps on the bulletin board in the hallway. They are the kinds of things that a hospital expects floor nurses to do on their own; that a bank expects branch managers to do on their own; that even a poorly run manufacturer expects supervisors to do on their own--without getting much praise, let alone any extra rewards."

On proposals to "save money" that have no real effect, particularly to taxpayers and constituents:

'Even if all of these proposals were to be enacted, the results would be trivial. The proposed Agriculture Department saving of $3.6 billion over five years works out to about $720 million a year--or around one percent of the annual department budget of almost $70 billion. A saving of $12.5 billion looks like a lot of money. But over two years the federal government spends $3 trillion. An annual saving of $6 billion -and this is many times more than Congress is likely to accept--would thus be a cut of no more than two tenths of one percent of the budget. Surely the only way to describe the results of Gore's efforts so far is with the old Latin tag "The mountains convulsed in labor only to give birth to a ridiculous, teensy-weensy Mouse." '

The commentary above also reminds us that lobbyists, special-interest groups, and Congress always view the lack of budget growth as a "cut" in funding. To actually lower or terminate spending would be akin to heresy, and the 'savings' Dr. Drucker indicates above (in mid-1990s dollars) are minuscule and don't matter much in the end given the total amount of spending.

Dr. Drucker then goes further as to why the government must restructure:

'Any organization, whether biological or social, needs to change its basic structure if it significantly changes its size. Any organization that doubles or triples in size needs to be restructured. Similarly, any organization, whether a business, a nonprofit, or a government agency, needs to rethink itself once it is more than forty or fifty years old. It has outgrown its policies and its rules of behavior. If it continues in its old ways, it becomes ungovernable, unmanageable, uncontrollable.

The civilian part of the U.S. government has outgrown its size and outlived its policies. It is now far larger than it was during the Eisenhower Administration. Its structure, its policies, and its rules for doing government business and for managing people go back even further than that. They were first developed under William McKinley after 1896, and were pretty much completed under Herbert Hoover from 1929 to 1933.'

So, a new Federal CIO/CTO is going to come in, specify a load of technology to 'correct' all of this ancient and bloated organizational and process infrastructure, and declare victory? How many of us still follow very-explicit business processes from 1933? Or 1988, for that matter? Yes, there are some, but very few. In this particular case, the 'old-way' of government going about its business is the rule, not the exception. How is an infusion of 21st-Century technology supposed to improve or enhance business processes originally made in the 19th and early-20th century if the processes themselves aren't changed?

For those that want to play the 'blame-game' and lay this dilemma all at the feet of specific partisan or ideological interests:

'In fact there is no point in blaming this or that President for the total disarray of our government today. It is the fault neither of the Democrats nor of the Republicans. Government has outgrown the structure, the policies, and the rules designed for it and still in use.'

Dr. Drucker then wisely provides a roadmap for what organizations (not just government) should (and should not) do when confronting these issues:

'The first reaction in a situation of disarray is always to do what Vice President Gore and his associates are now doing--patching. It always fails. The next step is to rush into downsizing. Management picks up a meat-ax and lays about itself indiscriminately. This is what the Republicans and (as of last December) President Clinton now promise. In the past fifteen years one big American company after another has done this--among them IBM, Sears, and GM. Each first announced that laying off 10,000 or 20,000 or even 50,000 people would lead to an immediate turnaround. A year later there had, of course, been no turnaround, and the company laid off another 10,000 or 20,000 or 50,000--again without results. In many if not most cases, downsizing has turned out to be something that surgeons for centuries have warned against: "amputation before diagnosis." The result is always a casualty.

But there have been a few organizations--some large companies (GE, for instance) and a few large hospitals (Beth Israel in Boston, for instance)--that, without fanfare, did turn themselves around, by rethinking themselves. They did not start out by downsizing. In fact, they knew that the way to get control of costs is not to start by reducing expenditures but to identify the activities that are productive, that should be strengthened, promoted, and expanded. Every agency, every policy, every program, every activity, should be confronted with these questions: "What is your mission?" "Is it still the right mission?" "Is it still worth doing?" "If we were not already doing this, would we now go into it?" This questioning has been done often enough in all kinds of organizations--businesses, hospitals, churches, and even local governments--that we know it works.

The overall answer is almost never "This is fine as it stands; let's keep on." But in some--indeed, a good many--areas the answer to the last question is "Yes, we would go into this again, but with some changes. We have learned a few things." '

If the incoming Obama administration is serious about 'real change' in the federal government, they're going to need to confront reality along the lines of Dr. Drucker's advice above. So will Congress, if they can possibly get past the lobbyists and special-interest groups - a tall order but it may be possible. The answer is definitely not hiring someone as a pseudo-overlord CIO/CTO to throw technology at the problems and declare victory - that's akin to "patching over the problems" and always fails in the end.

On the other hand, 'real change' will eventually be forced upon us by huge, swirling budget deficits and immense, unsustainable government debt - and that time will come much sooner than most people think. We have an opportunity now to examine, diagnose, address, and correct these problems in a rational, sensible manner.

Or have it mandated to us by the rest of the world, which would most likely result in 'amputations without diagnosis,' and that game has no winners.

Proposed Federal Uber-CTO: It Isn't About The Technology

Disclaimer: I have consulted with various US federal government and state agencies in the past, and the views expressed herein are mine alone, and not those of the US or any state government agency.

President-elect Obama has proposed the appointment of a Federal Chief Technology Officer to oversee and overhaul federal IT operations - bringing US government IT into the 21st century with updated technologies, transparency, connectivity, etc. etc.

A great number of industry pundits and bloggers have jumped on the bandwagon with many suggestions that not only encompass what to do, but who could be qualified to do it - Bill Gates, Vint Cerf, Jeff Bezos, etc.

Todd Biske has a number of thoughtful suggestions about what a Fed CTO might accomplish, and my commentary is going to be a riff on his views, with a bit of a difference: I'm positing that a Fed CTO will be rendered largely helpless and ineffective unless various federal laws and regulations that drive and directly affect government IT are overhauled - including broad and mandatory standardization. Unless this happens, and fairly quickly, no amount of technology or infrastructure is going to turn this bloated boat around anytime soon.

The federal government operates under myriad rules and regulations that are often specifically tailored to agency goals and operations. In addition to US statutes passed by Congress and signed into law by presidents, we have the Code of Federal Regulations (CFR), Executive Orders, the decisions of various federal judges/panels, the Tax Code (part of statute, but a monster in its own right), agency/commission orders, procurement rules, OMB compliance mandates, rules, regulation, more rules, and on and on and on....

And I haven't mentioned, at least directly, the Clinger-Cohen Act (a.k.a. The Information Technology Reform Act of 1996) yet. That one mandated CIOs for all government agencies in addition to creation and 'mandatory' utilization of the Federal Enterprise Architecture Framework (FEAF) - which is essentially a Zachman framework adapted specifically for the federal government. The positive impact that Clinger-Cohen has had on government IT operations is highly debatable, and needs to be revisited and perhaps overhauled as well.

Confused enough yet? I don't know anyone that wouldn't be. It's a big mess, and it can be untangled, but it will take time, a great deal of effort, and of course, money...but that's what the Federal Reserve's printing presses are for - given that they're currently rolling along smartly for banks, insurers, credit card companies, and perhaps the automobile industry.

One of the fundamental challenges a Fed CTO faces is that most government agencies are very insular when compared against their sister agencies - they operate their piece of the government largely in a vacuum, and that's primarily due to all of the specific laws and regulations that they must honor and account for. A further conundrum is that a great deal of these laws and regulations are specific to an agency, and not the others. It's no surprise then, that agency processes/procedures, and thus, their IT, mirror this specificity and complexity. 

Then there is the issue of procurement. Suffice it to say that complexity and obfuscation rear their respective ugly heads here also, and needs to be overhauled. The focus on procurement issues is not just protecting the taxpayer from getting ripped off (which they often are anyway) but that the government derives value from the purchases - immediately and on-going. This would suggest that procurement needs to be streamlined, incremental, much more nimble to the market and emerging useful technology, and swing away from 'big-bang" projects/deployments that take years, cost much, and often fail.

Lastly, there are the issues of us, the people of this great land. We want transparancy, but do not violate our privacy. We want cost-effectiveness, but responsive and supportive agency functions, when we desire and need it. We value security, not just of the homeland, but of ourselves and our individual histories as represented by the data that the government specifically collects about us. 

What the new federal CTO faces is not just a 'technology' or IT problem, nor can it be solved simply by some name-brand IT whiz throwing interoperable, SOA-infused, Web/Enterprise 2.0 technologies at it. We have fundamental problems with the enormous bureaucracy that we have created over the years, and that has to be changed and optimized first before the technology comes in, because just throwing technology at the problem is not going to work.

Finally, I have a viable candidate for the CTO position, even though he is not as well known as some of the heavyweights bandied about by IT pundits and bloggers...how about this guy? I think he'd fit the position well.

Carnival of Enterprise Architecture #13 Coming December 2, 2008

The (Lucky) 13th Carnival of Enterprise Architecture that I host through Blog Carnival is coming up on December 2, 2008, with submissions due by December 1, 2008. If you have an EA- or IT-related post that you'd like featured through the Carnival, you can submit the Permalink URL and related information here.

This is a very easy and effective way for EA and IT bloggers to get their messages across to a wider audience, and I hope that EA and IT bloggers will consider submitting their best pieces to the Carnival. Please note that you do not have to be the author of the material to submit it, as long as it: a) meets the guidelines below; and b) honors the author's copyright and fair use guidelines.

As always, here are a few pointers for article submissions to the carnival:

* Off-topic material, sales pitches (direct or disguised) for products & services, and content spam will be not be published. Prior to every carnival episode, I delete a number of pieces of obvious spam, vendor ads, and off-topic material prior to publication - and it's a complete pain in the rear, believe me. Please don't waste your time or mine and thanks in advance.

* Feel free to nominate blog posts of others for the Carnival, but make sure to identify the author!

* Material posted in previous editions of the Carnival will not be re-published, regardless of who submitted it and when it was originally submitted.

Again, the deadline for submissions is December 1, 2008, and the carnival will be published on December 2, 2008. I look forward to all worthy contributions!

Modeling the Unthinkable

I've noticed a theme lately in a few books and current business periodicals about models - primarily financial and risk management in nature - not taking into account scenarios that have little or no chance of occurring - which at some point, do happen. Of course, since the models didn't account for such events, the organizations utilizing them promptly lost their shirts in the markets, went bust, got bailed out by the government, laid off thousands, ad nauseum.

This isn't exactly news - in the mid-1990s, the hedge-fund Long Term Capital Management (LTCM) made a fortune - for awhile - using computerized trading models to exploit pricing differences in various securities and derivatives. They also compounded those activities with a lot of financial leverage. Then in 1998, certain events that could 'never happen' conspired to bring them down like a house of cards: Russia defaulting on its debt, and subsequent market turbulence and credit markets seizing-up. The Federal Reserve hastily arranged a bailout by investment banks, and injected capital to stabilize the markets - which responded positively and led directly to...the dot-com boom and subsequent bust in 2000-2001.

Sound familiar, and like we're always doomed to repeat the past?

This is yet another lesson that a number of Black Swan scenarios (see "The Black Swan: The Impact of the Highly Improbable") must be thought out and accounted for in models. This isn't an easy task by any means, but it is necessary. The reason it isn't easy is that we must leave the comfort of the workaday, modestly-predictable existence that we see almost every day and become "Chicken Little" - assuming that "the sky is falling" and developing scenarios and responses to them that protect (to the extent possible) assertions and positions the main model espouses. Another conundrum is developing and selecting the "right" outliers and  scenarios to incorporate - again, this can get maddening, but if we don't do it, we're doomed to repeat the past.

What this means for enterprise, data, and software architects is that risk scenarios are acknowledged and to the extent possible accounted for, as models are developed and validated. It also means that models are critically reviewed by peers and stakeholders top-to-bottom. This should include vetting models against realistic scenarios that probably won't happen, but render cataclysmic outcomes if they do. Case in point that happens enough to mention: data architects developing logical and physical models that while well thought-out and designed, fail to take into account data volumetrics, flows, and the capacity of the production systems/databases to handle the load the models directly or indirectly propose. Then in test or even production, with the systems, network, and databases down or at best, crawling on their knees, it's a really bad time to think "damn, we should have simulated data flows and volumetrics to validate the data models!" Next!

The ultimate question in doing hypotheticals like this is how much risk you and your organization are willing to absorb. Ignoring that minute chance that a 'Black Swan' will arrive will usually work most of the time, but when it does arrive - and it eventually will - the results are never pretty. For anyone.