Fail Fast and Agile SCRUM for Software Development

Fail Fast and Agile Scrum makes Sense for software Development. I am exited with this methodology.
This is one of the best methodology evolved in recent times for Software development and seems software development has attained a new level of maturation.

I want to divide this blog into 2 parts:
1. Fail Fast
2. Agile Scrum
Both have close relation and little distinction.

Fail Fast:
You might have come across Fail Fast Exception in Java API. Similar to that, Projects/Proof of Concepts/Research can be executed in Fail Fast Methodology.
My view of this methodology is plan the project in such a way that if something is not going in right direction or the hypothesis goal of the project is not meeting, then there should be a event/alert with minimum time frame of execution of project. This help in decision of further course of action for project.

For doing this, FMEA can be used as tool.
To give a brief on FMEA:
FMEA (Failure Mode Effective Analysis) is one of the tool of Six Sigma. This tool help us analyze risk and ways in which project can be failed. Below are the steps:
1. List the ways in which a item can go wrong
2. Once you know the way in which item can go wrong, place an action taken for each to counter potential failure.

Advantage of this methodology is:
1. Cost effective resource utilization
2. Quick Delivery of Product.
3. Lesser risk.

Agile Scrum:
This methodology is also in-line with above methodology. Scrum is a iterative incremental process for Software Development. Group of Developer, Business Analyst/End user and testing team are allocated to the project. Time frame for each release can be 1 to 3 weeks.
In order for this methodology to work, below process needs to be in place:
1. Good tracking tool for requirements and analysis.
2. Efficient build and Deployment process for source code and Database change control process.

I don’t know if this is a good picture to depict Agile Scrum but intention/interpretation is ,
1. Train is moving towards ultimate destination I, e Attaining full application goals and maturity level.
2. Developer is busy in dumping the coal in each bogie, correlating to incremental code release.
3. If they miss putting the coal in one bogie, they can put in other.
This methodology enables below:
1. Align Software Development with changing Business requirements.
2. Quick turn around for the business requirements.
3. Need for lesser Documentation due to closer interaction between Developers and BA/Users.
4. Developer are relieved of Release stress I, e earlier in water fall or traditional iterative model, If they cannot deliver code in one release they have to wait till next release(greater than one month) and also project manager pressure.

Downside of this methodology is:
1. No or less documentation.
2. On boarding of new developer/Team member takes longer as there is no documentation of functionality.
3. Project could be in risk due to Developer dependency and longer on boarding of developer.

Conclusion:
Definitely this methodology has its advantages and disadvantageous. Gains out weight the disadvantages. If this methodology can be used with required documentation, good coordination of Business and Development team, efficient build and deployment process, projects will be executed in time and a quality product will be developed.

What is you delivery methodology? Please post your views.

Introduction to Finger printing verification

Out of my interest on secure systems, first started with finger print verification/matching.

Background:

Currently in order to identify a person, we use identity cards.

Anybody can impersonate with ID cards. But with finger prints, person need not carry any ids with them in order to verify a person’s identity.

Thought Fingerprint would be one of the cheapest identification mechanisms for authentication.

Below are the steps for introduction of JAVA API For Fingerprint Reading/Verification/Matching:

Step 1 :

Bought Microsoft Finger Print Reader which is USB Compatible.

This comes with its own driver for reading the finger print reading.

This can be used for logging in to the system or websites etc.

Step 2 : Researched on Finger Print reader APIs.

Found Griaule as candidate for my testing.

This software has both capability for identification and matching.

More Griaule SDK details can be found in :

http://www.griaulebiometrics.com/page/en-us/fingerprint_sdk

Step 3 : Install

Install the Fingerprint SDK for Java.

Step 4 : Programming

Griaule evaluation software comes with sample java code for :

Capturing of finger prints

Identification- one-to Many

Matching- one-to-one

I have tested this API with simple program using Microsoft Fingerprint Reader in MS Vista, it works great.

Productivity, Efficiency with Quick Deployment and Testing Environments

Below are the question project team needs to ask:
1. Are you giving the release after release for the bugs?
2. Is your release cycle dragging or delay in delivering?
3. Is your bug list growing?

If your answer is yes for one or more questions, then below could be the candidate reasons:
· Deployment sucks, takes tooooo long
· Inaccessible Testing environment for developers due to any reasons.
· No Rigor by QA Team on testing the functionality and bugs.

Believe it or not , most of the productivity is lost due to above reasons.
For example if your team has 10 developers, developer is losing 1 hr per day due to deployment for testing, team will be loosing 10hrs per day.

Loss of productivity = No. of lost hrs * No. of Team members. (Worst Case Scenario)

Some other parameters can be added if you want :)

Reason could be many,but, we want the way out:
1. Testing environment should be easy i.e. :
Checkout the code from CVS/SVN ,run ANT or any build script and application is ready for testing.

Most of the projects have lengthy process of deployment.
1. Checkout from version control takes time, I have seen checkout takes upto 1 hr, specially in cases of Offshore/Onshore model projects.
Solutions: Expedite this process by moving the server to the place where developer checkin the most.
2. Build process, could take long.
This could be due to not well written ant tasks and dependencies
Solution: This could be avoided by dividing the complex ANT tasks to small tasks. i.e. There may be some code in the project that may not change regularly ,this is a candidate for moving to different ant task.
3. Configuration changes
a. This process is complicated by no. of variables that needs to be replaced in different config files
Solution : This could be done in 2 ways :
a. Deployment guide:
i. Create a deployment guide as the project is developed, Keep update for every change that requires the config file change.
ii. This has the drawback of human errors, below process avoids this drawback.
b. Automated Config Replacing scripts :
i. Given the config files and the Variables and its values in files , script should replace all the variables in the config file.

2. Simple Deployment guide :

Contents could be
a. List of files that needs to be changed with complete dir path
For each of the file, highlight the variables in colors which need to be changed
b. Any project notes to the deployer should be mentioned in this document.

Notes :To avoid the human errors while changing,automate the variable replacement in the config files

Deployment guide also requires testing.
i. This can be done by giving the deployment guide to any team member of other project and He should be able to deploy without any instructions from the project team.

Conclusion:
Invest time on Deployment process (version control, Build and Deployment) to reduce the time and easy deployment and access to various test environments. This leads to Productivity, Efficiency which leads to quick and ontime delivery.

What’s your deployment time?

YSlow: New Dimension to Web Application Performance

This blog explains the 5 rules I have implemented out of 14 Rules and improved the web application performance.

By implementing below rules my pages are coming as though they in my local system.
I am thrilled with the response.
Business Analyst asked if this was the prototype html? so that shows how fast web pages are served.
Now I have been assigned to improve other use cases :)

Below are the 14 YSlow rules:
1. Make Fewer HTTP Requests
2. Use a CDN
3. Add an Expires header
4. Gzip components
5. put stylesheet at the top
6. move the scripts to bottom
7. Avoid CSS expression
8. Make JS and CSS external
9. Reduce DNS Lookup
10. Minify JS
11. Avoid Redirects
12. Remove duplicate scripts
13. Configure ETags
14. Make Ajax cacheable

I have implemented 5 rules for the phase I in my module i,e
3. Add an Expires header :
Added the header as below :
int seconds=432000;
long now=System.getCurrentSystemMillis();
response.setDateHeader("Expires",now+seconds);
response.addHeader("Cache-Control","max-age=3600");

5. Put stylesheet at the top
6. Move the scripts to botton

10. Minify JS
Minify JS and HTML. Removed white spaces using the simple editor.

12. Remove duplicate scripts
Remove duplicate and unnecessary scripts,html.
Lot of times unnecessary/duplicate html and scripts may not show in jsps.
Specially with dynamic content in loops of jsp.
But look at when it turns to html you can see how much content it generates and some of the content from the loops can be removed or move out of the loops.

Conclusion : YSlow has definitely a new way of improving the performance of web applications at client side ,extending to the best practices of Server side.

More details in the video by Steve Souders


Which rule did you implement????

Software project jump start

Jump Start Projects
Why Project have to start from scratch? Companies which are small/big/Corporate does not have Common services group. Even if they do, they are part of Architectural group which has the documentation and suggests the best practices but not follow with rigor.

Below are the some of the pain areas in the development cycle :
1. In Analysis/Design phase too much time spent on the Research for Basic services by development team
2. Team trying to gather reusable components from other teams
3. Lack of Code sharing within the organization,which most of the companies faces issues with. :)

This team I am proposing is the extension the Architectural services group.

Project Jump start Team
This could be the part Architectural team.

Responsibility of the team is to jump start the project by ready made components
Below are the some of the common components across the projects
1. Exception Handling
2. Logging
3. E-mail
4. DB Access(JDBC,AKD(EMatrix,Oracle OIM etc...)
5. Struts (Configure the struts framework)
6. Caching etc


Below are the features/services provided by this team :
1. One of the Representative from Common Services engages in the early phase of Development by providing above components ready
2. Train the team with the reusable components
3. Suggest the Design Changes to some extent not overlapping with Architectural services.
4. Best Practices are shared in the team in regards to Performance,Code Reusability and Documentation of code.


Below are the benefits :
1. Projects can benefit by reducing 20% of their development time
2. Since the code is tested and used in different projects , this components are bug free.
3. Since the Development is Equipped with right tools and Training, project is better Managed and Ontime delivery.
4. Project Development team can concentrate on Business logic and not worrying about Basic services.


Pitfalls
1. Process made but not followed constrained only for documenting but not let the team knowing
2. Too much time sent on Training and research


Early engagement of right resources and reusable components can significantly reduces the Development time and Companies prosper in all round.


Post your comments and feedback.

Java Application Performance improvement

This blog provides with the tips for Java Application Performance improvement.
These are some of the lessons/thoughts learned from experiences.

Here we go :

1. Often even the experienced developer makes the mistake of adding the code which is either unnecessary or function calls which are not necessary.
   
2. When database is the issue :
   1. Use the batching to commit,play with batch size and see the result.
   PreparedStatement.addBatch
   2. Create indexes on the table also see that not many indexes are created as index comes with some overhead while insert and delete ,as it has to reorder the index.
   
3. When the process is intensive and involves processing of thousands and millions of records records,
   There are 3 strategies
   1. Delay the sub-process
   2. Divide and Conquer
   3. Hybrid of above both
   

1. Delay the sub-process : One of the delaying sub-process could be to delay the committing sub-process by adding to batch of batchSize(e.g..100) records.
2. Divide and Conquer : This can be archived in 3 ways;
1. Let say you have 100,000 records to process,delegate to 2 or 3 threads to process the records.
2. Divide between machines : Quoting the same example of processing 100,000 records can be divided between the machines. In this case process will use different CPU cycles for processing.

I suggest the combination of threads and RMI could be used for Divide and Conquer and see that there is not much network latency between the machines. RMI Example quoted in RMI Tutorial is good example to use a costly resources(such as Sun Solaris box with high configuration).

This strategy is best implemented with application hosted in distributed environment with Load Balancing.

Conculsion :
You dont need humungus boxes(machines) to process,divide the work between threads and boxes, dont believe refer google architecture.

More in my next blogs....

Post your comments and suggestion,what do you think?