I've been thinking about standards for service-oriented architecture after a discussion with architects in our CTO team and our IITS division (IITS was formerly known as IDX). If you're working on service-oriented architecture, the sheer number of standards you have to be aware of is astounding.
Of course, you'll need to be aware of the basic XML standards, XSLT, XPath, etc. On top of that, you have to worry about security, where the WS-Security standard plays an important role, as do SAML, X.509, WS-Policy, and possibly WS-SecureExchange, and WS-Federation. Once you have a handle on security, you'll need to start thinking about data exchange standards. Specifically, you should probably pay attention to the WS-MetaDataExchange standard, as well as standards from organizations like OASIS. In the long run, meta data will be the toughest nut to crack. Once you've defined an approach for managing meta data, you'll need to start paying attention to business process specifications. In this realm, the leading standards are the BPDM, BPEL, and BPDM standards. These in turn, are based on the ubiquitous WSDL standard. So you've now read through more than a dozen standards documents, involving thousands of pages of specifications. And you haven't created a service yet.
When you're thinking about implementation, you'll need to worry about messaging and transport. So you'll need to look at various messaging standards, depending on your implementation environment. If you're in a Microsoft shop, you'll probably need to work with BizTalk, Windows Communication Foundation, and .NET. If you're working in a Java shop, EJB might play a role, as might the JAX standards, JMS, and maybe even JBI.
The JBI (Java Business Integration) standard has emerged to provide some level of compatibility between enterprise service bus implementations. The newest standards on the block are the Service Component Architecture (SCA) and the Service Data Objects (SDO) standards.
If you believe the press, everybody is doing SOA. We have all done the neccessary ROI analysis, and are realizing the agility, flexibility, and reuse that SOA provides. Not. We're too busy trying to figure out how to apply all these standards, or how to find a single vendor that supports enough of a subset of the standards so that their own products work together. Or even worse, we're trying to find products from multiple vendors that work together using the same version of some subset of these standards.
There are simply too many standards to worry about. Is SCA the answer?
Thursday, June 14, 2007
Monday, March 19, 2007
Services, Schmervices
Service-oriented architecture (SOA) was flagged as the "most-despised" buzzword in a survey by Network computing. I can understand that. "Service-Oriented" is to a system as "High Definition" is to the latest consumer product. We have high-definition television, high-definition radio, high-definition phones, high-definition linoleum, and even high-definition dishwashers. It seems that in the last year or two, every minor new product from software vendors is the answer to every problem you've ever had developing your own service-oriented architecture.
But developing software with services, or better yet, software as a service, really is different. There isn't a lot of good guidance out there. You'd think after twenty or twenty-five years of figuring out how to develop distributed systems, somebody would have the answer.
About a dozen years ago, distributed objects were the way to go. Client-server was so yesterday. You just had to throw a system together from CORBA or DCOM components, and you were cool. The problem was that it wasn't easy, and it was really hard to get the granularity right so the system would scale and perform. Here we are a dozen years later, and service-oriented is the way to go. The web is so yesterday. You just throw a system together with some web services and an enterprise service bus, and you're cool. The problem is that it isn't easy, and it's really hard to get the granularity right so the system scales and performs. Hmm, does anybody see a pattern here?
But this time it's different....
But developing software with services, or better yet, software as a service, really is different. There isn't a lot of good guidance out there. You'd think after twenty or twenty-five years of figuring out how to develop distributed systems, somebody would have the answer.
About a dozen years ago, distributed objects were the way to go. Client-server was so yesterday. You just had to throw a system together from CORBA or DCOM components, and you were cool. The problem was that it wasn't easy, and it was really hard to get the granularity right so the system would scale and perform. Here we are a dozen years later, and service-oriented is the way to go. The web is so yesterday. You just throw a system together with some web services and an enterprise service bus, and you're cool. The problem is that it isn't easy, and it's really hard to get the granularity right so the system scales and performs. Hmm, does anybody see a pattern here?
But this time it's different....
Sunday, March 18, 2007
Let's restart this
It's been far too long since the last post. Since I last jotted anything down, things have changed a bit. We finally brought someone in to dedicate time to our production support team, so I quickly handed things over to him. And he's done a great job, bringing our backlog down and keeping it down.
I've moved into a new role where I'm now focused on IT strategy. My employer is probably the best around at execution: when we make commitments, we keep them. But we often think so much about the current project, the current quarter, the current problem, that we lose sight of the direction we're supposed to head.
So the posts will head another direction. I'll be looking at service-oriented architecture, business process management, web 2.0 & enterprise 2.0 stuff. And since I get to think about this stuff full time now, maybe I can get myself to post a little more frequently.
I've moved into a new role where I'm now focused on IT strategy. My employer is probably the best around at execution: when we make commitments, we keep them. But we often think so much about the current project, the current quarter, the current problem, that we lose sight of the direction we're supposed to head.
So the posts will head another direction. I'll be looking at service-oriented architecture, business process management, web 2.0 & enterprise 2.0 stuff. And since I get to think about this stuff full time now, maybe I can get myself to post a little more frequently.
Sunday, October 22, 2006
And a month later...
It's been way too long since I wrote the last entry. Things at work have been kind of hectic. In the last four weeks, we reorganized, pushed out our third release of the year, reorganized again. Hopefully, the organization will stabilize enough so that we don't have to reorganize again before Thanksgiving.
In terms of the Scrum, we're moving forward. We see benefits, but our old habits continue to haunt us. We had to rearrange priorities during our release to handle those last-minute show-stoppers. Performance challenges after the release due to a fairly major architecture change required some additional reprioritization.
Overall, scrum works. The daily scrums are great at communicating progress. Every day, each team member communicates the tasks he or she is working on, the tasks planned for the next day, and obstacles that are preventing progress. That works great! We try to keep interruptions at a minimum. This has been somewhat successful. The team has now come together very well, and they're taking ownership of things that need to get done.
But we have encountered a few big challenges. First, while the daily scrums are great for communicating progress, they aren't so good at ensuring that everything is finalized, and tasks are actually completed. We haven't been good about communicating estimates. And we have a challenge with communicating between our on-shore scrum team and our offshore non-scrum team.
I contact Jeff Sutherland, one of the inventors of the methodology. Our model is similar to what he calls a "Type C Scrum", where the priority list is constantly rearranged. His company, PatientKeeper, has been successfully executing Type C Scrums for about two years. He sent me an advance copy of a book he's been working on. We'll be reviewing it both to provide feedback to Jeff and to see if we can apply some of the lessons learned by someone doing this a lot longer than we have.
Stay tuned for more updates.
In terms of the Scrum, we're moving forward. We see benefits, but our old habits continue to haunt us. We had to rearrange priorities during our release to handle those last-minute show-stoppers. Performance challenges after the release due to a fairly major architecture change required some additional reprioritization.
Overall, scrum works. The daily scrums are great at communicating progress. Every day, each team member communicates the tasks he or she is working on, the tasks planned for the next day, and obstacles that are preventing progress. That works great! We try to keep interruptions at a minimum. This has been somewhat successful. The team has now come together very well, and they're taking ownership of things that need to get done.
But we have encountered a few big challenges. First, while the daily scrums are great for communicating progress, they aren't so good at ensuring that everything is finalized, and tasks are actually completed. We haven't been good about communicating estimates. And we have a challenge with communicating between our on-shore scrum team and our offshore non-scrum team.
I contact Jeff Sutherland, one of the inventors of the methodology. Our model is similar to what he calls a "Type C Scrum", where the priority list is constantly rearranged. His company, PatientKeeper, has been successfully executing Type C Scrums for about two years. He sent me an advance copy of a book he's been working on. We'll be reviewing it both to provide feedback to Jeff and to see if we can apply some of the lessons learned by someone doing this a lot longer than we have.
Stay tuned for more updates.
Thursday, September 14, 2006
Scrum and Priorities
So our first daily Scrum started on Monday and it went pretty well. In meeting with the developers, they confirmed what I originally suspected. Our throughput is low, not because the problems we have to solve are too complex, but because the developers are constantly interrupted to handle the next emergency.
So a prioritized product backlog is the right answer--with each item in the backlog being a defect (or more specifically, the tasks necessary to fix a defect). But we can't follow the traditional scrum methodology and create a 30-day-long backlog. We have to adjust our backlog on a daily basis to ensure that we are handling problems in the right order. Since we don't have a prioritized list, I started the team out by just having them pick defects from our defect tracking system and get working on them. I also committed to minimizing interruptions. That might be easier said than done, but we'll try. It takes a long time to break old habits.
Today (Thursday), I distributed a first cut at a prioritized list of defects. I'm also trying to recruit someone to serve as our "product manager", and maintain that prioritized list.
In the next couple of days, I'll regroup with the team to measure our progress in the first Scrum week. Stay tuned for more details.
So a prioritized product backlog is the right answer--with each item in the backlog being a defect (or more specifically, the tasks necessary to fix a defect). But we can't follow the traditional scrum methodology and create a 30-day-long backlog. We have to adjust our backlog on a daily basis to ensure that we are handling problems in the right order. Since we don't have a prioritized list, I started the team out by just having them pick defects from our defect tracking system and get working on them. I also committed to minimizing interruptions. That might be easier said than done, but we'll try. It takes a long time to break old habits.
Today (Thursday), I distributed a first cut at a prioritized list of defects. I'm also trying to recruit someone to serve as our "product manager", and maintain that prioritized list.
In the next couple of days, I'll regroup with the team to measure our progress in the first Scrum week. Stay tuned for more details.
Saturday, September 09, 2006
More Thoughts on Scrum
I've been doing a little more thinking about Scrum, and have had a little success in getting it kick-started at work. Our first sprint, with a team of seven excellent developers, starts on Monday.
Here's some more thinking on the subject. The basic theory behind scrum is based on traditional process control theory. At first glance, it appears to be simply "management by to-do list". But there really is method behind the madness. Most existing software development processes assume that software development is a defined process--similar to that of building a car on an assembly line. If you've been doing software for a while, I'm sure you realize that is absolutely not the case.
Scrum is different. It acknowledges that software development is really an empirical--unpredictable--process. The methods used to control an empirical process are vastly different from those used to control a defined process. For a defined process, you can follow a recipe or series of steps to produce the correct output. For an empirical process, you closely examine the inputs prior to performing a process and the outputs once the process is complete. You put controls in place to react and change course when things go haywire.
So it sounds ideally suited to certain software development activities. But I'm trying to apply it to production support--maintenance of a system that is already in production. As with any support organization, our reacts to constantly changing priorities: we scramble to deal with the latest "emergency." Maintenance releases occur weekly on a regular basis, and sometimes as frequently as once a day. To complicate things even further, 80 percent of our support team resides in India, while the other 20 percent is located here in the U.S.
Scrum is really intended more for longer development periods, prioritized task lists that don't change, and development teams that have the luxury of being co-located and being left alone. There has been some talk about evolving Scrum to deal with concurrent development periods, and even weekly maintenance releases. And that will be the subject of my next blog.
Here's some more thinking on the subject. The basic theory behind scrum is based on traditional process control theory. At first glance, it appears to be simply "management by to-do list". But there really is method behind the madness. Most existing software development processes assume that software development is a defined process--similar to that of building a car on an assembly line. If you've been doing software for a while, I'm sure you realize that is absolutely not the case.
Scrum is different. It acknowledges that software development is really an empirical--unpredictable--process. The methods used to control an empirical process are vastly different from those used to control a defined process. For a defined process, you can follow a recipe or series of steps to produce the correct output. For an empirical process, you closely examine the inputs prior to performing a process and the outputs once the process is complete. You put controls in place to react and change course when things go haywire.
So it sounds ideally suited to certain software development activities. But I'm trying to apply it to production support--maintenance of a system that is already in production. As with any support organization, our reacts to constantly changing priorities: we scramble to deal with the latest "emergency." Maintenance releases occur weekly on a regular basis, and sometimes as frequently as once a day. To complicate things even further, 80 percent of our support team resides in India, while the other 20 percent is located here in the U.S.
Scrum is really intended more for longer development periods, prioritized task lists that don't change, and development teams that have the luxury of being co-located and being left alone. There has been some talk about evolving Scrum to deal with concurrent development periods, and even weekly maintenance releases. And that will be the subject of my next blog.
Friday, September 08, 2006
Processes
I've been thinking about software development processes recently. I've actually been interested in them for quite a while. While at Andersen Consulting, I worked on an applied research process where we applied some of the latest object-oriented techniques on projects for a defense contractor. Those processes later became the object track for Andersen's Method/1.
Since then, I've applied both Extreme Programming and the Rational Unified Process to various projects. I've also worked on a very large-scale project where we followed a "homegrown" process influenced by the company's product development processes. This process is very much a waterfall process. As I said, the project is large, and in my experience, the larger the project, the harder it is to apply newer, less rigid processes.
But like any waterfall processes, it has its weaknesses. Because requirements must be specified up front, they often arrive late in the cycle and are rarely as complete as we'd like them to be. Because requirements are late, we struggle to complete designs in time for a thorough review. Because we rush through design reviews, we struggle with defects after the code is written. Although we test pretty thoroughly, the system is complex and we've introduced a lot of "hidden" defects into production. The burden of bringing those production defects under control is what's currently on my mind.
We have a team of about 30 people working on production support. The team does a great job of responding to problems quickly, but it's a thankless job. We're keeping our heads above water, but just barely. Our fix rate is just slightly larger than our incoming defect rate.
So I've decided to give Scrum a try. Scrum is a process, not a methodology. Based on process control techniques, it is almost more a philosophy than a process. With Scrum, software is developed by relatively small teams in 30-day sprints. The team controls what they develop, how they develop, and when they develop. This isn't the ideal scenario for Scrum. I'm not talking about a product development activity, I'm talking about software maintenance. I'll also have an up-hill battle to get our leadership team to accept this "untried" process.
I'll post updates as we make progress.
Jeff
Since then, I've applied both Extreme Programming and the Rational Unified Process to various projects. I've also worked on a very large-scale project where we followed a "homegrown" process influenced by the company's product development processes. This process is very much a waterfall process. As I said, the project is large, and in my experience, the larger the project, the harder it is to apply newer, less rigid processes.
But like any waterfall processes, it has its weaknesses. Because requirements must be specified up front, they often arrive late in the cycle and are rarely as complete as we'd like them to be. Because requirements are late, we struggle to complete designs in time for a thorough review. Because we rush through design reviews, we struggle with defects after the code is written. Although we test pretty thoroughly, the system is complex and we've introduced a lot of "hidden" defects into production. The burden of bringing those production defects under control is what's currently on my mind.
We have a team of about 30 people working on production support. The team does a great job of responding to problems quickly, but it's a thankless job. We're keeping our heads above water, but just barely. Our fix rate is just slightly larger than our incoming defect rate.
So I've decided to give Scrum a try. Scrum is a process, not a methodology. Based on process control techniques, it is almost more a philosophy than a process. With Scrum, software is developed by relatively small teams in 30-day sprints. The team controls what they develop, how they develop, and when they develop. This isn't the ideal scenario for Scrum. I'm not talking about a product development activity, I'm talking about software maintenance. I'll also have an up-hill battle to get our leadership team to accept this "untried" process.
I'll post updates as we make progress.
Jeff
Subscribe to:
Posts (Atom)