A study conducted by Atos Origin measuring the productivity of OutSystems’ Agile Platform shows that OutSystems is 22% cheaper for Agile software development. With OutSystems, the study delivered equal levels of quality in a shorter time period even with an increased workload.
study measured the productivity of a project executed by Atos Origin for one of its customers using OutSystems’ Agile Platform against the productivity of other Agile projects at Atos Origin; non-agile projects implemented in 4GL languages; as well as projects in a global Quantitative Software Management (QSM) database.
We measured the following three criteria: "Development Scale of effort" + "Software" + "Development time."
In this blog I will not describe OutSystems’Agile Platform, for this, I refer to a previous blog: OutSystems , Agile Application Lifecycle Management. in which I discuss the productivity characteristics of OutSystems and the project which has been successfully implemented.
OutSystems starting points:
This was the team’s first project with OutSystems. The people involved in this project have participated in the online certification and a 2 week bootcamp. They are experienced .NET software engineers but have never implemented a project with OutSystems before. The OutSystems project was a new experience for the project members, except for the engagement manager, committed to the project. The Engagement Manager was the liaison between the business and the IT team and played a crucial role in translating business requirements into development requirements.
Project Characteristics:
- Project size: 444 function points
- Development methodology: Agile, OutSystems Agile methodology (Time Boxed by 2 week Sprint)
- New application project
- Contract type: Fixed Price / Fixed Date
- Team Size: 1 Engagement Manager, Delivery Manager 1, 3 Developers
- First OutSystems project for the developers
- Team Background: Microsoft .NET developers
Comparison criteria:
The study compared the new OutSystems project with:
- An agile project in Microsoft. NET in the Atos Origin Software Factory (the SDMC)
- A Unified Process Microsoft 4GL - all other 4GL languages under Microsoft. NET, in the Atos Origin Software Factory (SDMC)
- A comparison with international QSM traditionally conducted business in 2005 (This is the global benchmark in the QSM database.
For the record, there were no comparison made with 3rd generation languages (3GL’s like Java) since the productivity of these environments is significantly lower. Note: I do not include figures in this Blog from other projects due to the sensitivity of the information, however, that type of information can be shared with customers by special request.
Productivity comparison plan:
Atos Origin set up the known project metrics from the various projects in a database using the SLIM (Software Lifecycle Management) Data Manager, the QSM device for setting up project metrics in the database. Larry Putnam, founder of QSM defined the QSM Software Process Comparison and implemented in the SLIM toolset.
The comparison plan was devised to compare Agile projects implemented in .NET; traditional, non-Agile 4-GL projects; and all projects from the QSM database (non-Atos Origin projects). The last of which contains the comparison of all measured data and projects that provided a type of international benchmark.
Next, project metrics
- Turnaround (Start and end of project)
- Effort by business group (in hours)
- Size (in Function Points and/or Lines of Code)
- Quality during the project (number of errors)
- Global Sourcing Ratio (successfully run in
) - Influence factors on productivity (this includes a number of parameters under which can influence productivity, including: clean order, complexity, development process, project conditions)
- Other specific KPIs
- Specific project conditions
Development with OutSystems has led to a very positive picture regarding productivity. OutSystems themselves claim two-to-four times higher development productivity and ten times higher application management productivity levels. In this study, Atos Origin concluded that an inexperienced team (with only initial training in OutSystems but who were experienced. NET developers) were
22.9% more productive with OutSystems Agile Platform than developing a similar project with a 4GL, and 29% more productive with OutSystems when compared to a comparable Microsoft .NET project. When compared with the QSM global benchmark, OutSystems is 260% more productive.
Team Efficiency
The Team Efficiency (QSM PI - Productivity Index) is 20.8. The higher the better. The Team Efficiency has to do with the processes and tooling. The result of these two parameters leads to productivity and efficiency. Overall, the team efficiency was good
The Time Pressure (QSM MBI - Manpower Buildup Index) is 6.8. This index measures the time of software development on a 14 point scale. If this index is above 8 you have a problem with time and results are no longer good, with the team under "above normal" stress levels. The pressure with OutSystems was high but still good for this project. Generally, it can be said that the (positive) pressures in Agile projects are quite high. You can consider this to be "healthy" time pressure.
The Quality
The Quality of the program is determined by the number of defects in the system test filtering. Of course, the lower the better. The quality for the OutSystems project was 78. That is not distinctive from other 4GL projects. This is just good. The international average is somewhat lower, but of course biased because this OutSystems project has been developed using Agile methodology, and an Agile project by definition is based on changes and refinements during the process.
Productivity and Turnaround
If productivity in function points is compared to an overall conversion factor e.g. SLOC (Service Lines of Code) for the construction phase, you can see OutSystems delivers a great deal of code in a short period of time. Looking at the x-axis that shows the lines of code versus time on the y-axis, you see that you have more code in less time than average. It is even beyond the normal scale of the number of lines of code. OutSystems delivers code rapidly.
Productivity and Effort
The cost per function point unit converted to Service Lines of Code (SLOC) against the effort (effort,) instead of time, we get the following table. Here we can conclude that OutSystems’ Agile Platform is highly productive.
Productivity-influencing Factor
This number is determined by a set of 18 questions which are given a particular rating. The higher the factor number, the more complex it is, resulting in negative impact on productivity. I will not name all the factors but you can think of the programming language, presence of a standard project management as Prince, clean order (to what extent the specification is ready and approved), technical complexity and other project conditions. The productivity impact factor for this project was 51 compared to the standard average of 41. This was therefore a more complex matter than average.
Summary
Developing with OutSystems is very interesting. It is a highly productive development environment. We only studied a new project implemented by a team that was new to OutSystems (although experienced in.NET). Given this, we can assume even higher productivity levels for a project being implemented by people experienced in OutSystems. In addition, OutSystems is a lifecycle management platform and claims to be 10x faster when it comes to managing the application but we have not yet investigated this.
For the development of new software projects, OutSystems’ Agile Platform is 22.9% more productive than a standard Agile .NET web development project. Since it is a lifecycle management environment which allows you to manage the production application with one-click-deployment that can be monitored, maintained and managed, OutSystems will also probably be more productive when it comes to application management and change.
The duration of the project was more than 40% lower than average and the relative speed was more than 25% higher than an average project. The quality is comparable to other Atos Origin projects.
In times of shrinking budgets and rising expectations for web and mobile applications, doing Agile with OutSystems represents a very good alternative. Where time-to-market is also crucial – OutSystems represents a good alternative when compared to other mainstream development paths.
Hans ten Berge Principle consultant Atos Origin Hans ten Berge BLOG: For more information: OutSystems , Agile Application Lifecycle Management http://en.wikipedia.org/wiki/OutSystems_Systems
