I have been actively involved in working with one of my clients to plan ,design and architect a solution to help them analysis their business critical data. The final solution consisted of a Data Warehouse and Data mart using Oracle tool set, an ETL solution using the Informatica tool set and a User Analysis solution using Business Objects. The lessons learnt during this engagement are discussed below in the following buckets:

1. Impact of end Users – The perception, awareness and domain knowledge of the users, all play an important role in the successfully development of the solution and its acceptance by the end users.

2. Data – The various issues which surround the core of the solutions need to be understood and addressed include transparency, confidence etc

3. Project Management- Successful completion of the project and the quality of the final deliverables are impacted by these issues which include – well defined process and milestones, change management etc.

Impact of the Users
The success of the solution implemented is gauged by its acceptance and use by the final end users. Hence it is very important to understand the difference between the users (business users vs. IT users), skills of the users (Expert vs. Novice) as well as the “Change readiness” of the users. The following sections describe the key issues:

Impact of type of Users
While the IT users and business users are the two main users of the system, it is important to understand and cater for the difference in approaches/perceptions of these two groups. An illustrative example is the reaction of these two groups to demos of the software products during the software selection phase. An informal review at the end of the demo of one of the product (let us call in Product X) showed that the IT users were comfortable with the concept of multi-dimensional cubes put forward and were in favor of the product while the Business users were nervous about this paradigm and found Product X to be complicated and difficult to use. The analysis of the answers to specific questions in the feedback forms further strengthened the point that the business users liked the features of Product X over the other products  and rated it better in answers to specific questions. But the "gut feel" was that it was more complicated and hence it was not finally chosen !

My recommendations ?

1. The views of the final users of the system (Generally the Business users) must be given more weight since the acceptance of the delivered solution is dictated by them.

2. To ensure conformity with the overall policy, care must be taken to make sure that the decision is in line with the overall enterprise wide standards mandated for similar applications (if available)

3. In case of disagreement, it is worthwhile to repeat the demos or going in for further software selection process and strive for a consensus rather than force a decision one way or the other.

Impact of Users skill levels
Every organization has its own mix of super users as well as the novice users. It quickly becomes apparent that the more vocal and knowledgeable users will quickly hijack the process leading to a solution which may not be palatable for the whole group.

My recommendations ?

1. During the requirement and design process, care must be taken to include representatives of both these groups as well as proactively solicit feedback from both the groups. If required, I recommend that separate one on one interviews in a personal non threatening environment be done to obtain the feedback from the non vocal group.

2. Design of user interface – Since both the groups need to use the system, I recommend that a simple wizard based interface be developed to perform common activities for the novice users and allow the super users to use the available functionality of the BI tool. As the novice users get more comfortable, they will adopt the best practices way of using the out of the box functionality.

3. Understanding of data – While the power users “clearly know” that abc1256 is the SKU for widget A, it would be foolhardy to assume that all the users have similar familiarity of the data. I recommend that there is provision for prompts/look up lists for the novice users.

Impact of User Perceptions
As one of my colleagues put it, “In system implementation (as well as in real life), perception becomes reality”. Hence it becomes crucial for the team to pay close attention to the perception of the users.

The main impact includes:

1. Perception about the team – Make sure that the users are clear about the roles and responsibility of the different team members. Incidents where two different groups/roles repeat the same questions raise concerns in the business user’s minds about the competency of the users as well as effective communication between the team members.

2. Perception about the process - Make sure that the users understand the journey to the destination. They must be clearly educated on the whole process to the end goal and why the team is engaged in specific activities. Any ambiguity would lead to the users doubting the process and hence the final solution.

3. Perception about the data- The final solution is not only as good as the data but also as good as the user perception of the data. Any “gotchas” in the data would lead to the users loosing confidence in the data and the final solution.

My recommendation to avoid these issues is:

1. Have a detailed user kick off /education session to ground the user expectations on the activities ahead

2. Have regular (Recommendation is weekly) where the business users are informed on the current state of the project and any of their questions answered.

3. Have regular individual feedback sessions to obtain feedback on specific questions like performance of the team, level of comfort with the progress etc.

Impact of user awareness
Since the objective of the engagement is to move the users away from their mechanical way of doing business to a more automated approach, it is implied that the users have the “frog in the well” syndrome and are not aware of the options available to exploit. These lead to the following:

1. Most of the times the users are not aware of the data analysis/Business Intelligence tools available and the functionality it provides out of the box as well as the common paradigms for analyzing the data. Hence they are not able to participate fully in the design sessions.

2. The flip side of an increasing awareness is that the requirements and expectations of the users also change. As they become more aware of the functionality of the product for example, the requirements quickly start changing and the revised requirements becomes what they initially had in mind but where not able to express. The impact is felt not only on the system functionality but also on the performance of the end solution and easily escalates into scope change issues.

My recommendations are

1. The user must be provided with an awareness/training session BEFORE beginning of the requirements/design sessions. This should include a simple prototype as well as hands on use by all the client team members.

2. Document the requirements sessions and track decisions/discussions in meetings through signed off meeting minutes.

3. Keep close track of the changes in requirements and have periodic “reality checks” to ensure that scope creep is avoided. The meeting minutes can be used to arbitrate in any conflict in perceptions.

Impact of users business domain
Every business has its own quirks and these needs to be factored in to ensure the success of the project. An illustrative example in this engagement was the FDA requirement that the end solution delivered must be a “validated” system. My recommendations are:

1. Ensure that there is strong business lead who can be an effective interface between the client team and the technical team as well as inspire trust and confidence in the users

2. Review and analyze the impact of all process requirements mandated by the clients IT strategy (Emerging standards, recommended tools and technology, recommended process) as well as other government mandated requirements.

3. Conduct training sessions to ensure that all the engagement team members are well versed in these requirements and understand the impact on the activities.

4. Appoint an external third party entity to periodically review the engagement and certify that these requirements are being met. The results can be presented to the top client management and become part of the final deliverables.

Data
The most critical factor for the success of the solution is reliable data which can provide reliable analysis. The most common issue involved in this core component is its perception and understanding by the users.  

One Version of the truth
Many data warehouse initiatives fail because the users do not have confidence in the data and hence tend not to use the solution. During our interviews, one of the end users declared that she would not use system “x” for data analysis since it gave incomplete or incorrect data. On further digging we found that this was caused because System “x” truncated certain data to fit it into the report template and the incorrect data was a one time situation caused by a failed ETL process.

Based on this and other observations I recommend the :

1. All users must be involved and must sign off on the enterprise wide definition of the data (e.g. There is one universal definition of key elements like patient, event, seriousness of events etc)

2. If feasible aim for a universal data Model (Common definitions used by all systems which facilitates exchanges of data/comparison of data across systems)

3. All calculations must be agreed upon by all the users and it must be pre-calculated when ever possible in the Data Marts. This would avoid different formulas be used by users in the presentation tool to calculate predefined key metrics.

4. All coded values must be decoded in the data mart. This not only makes the analysis of the data easier for the users (since they do not have to figure out that Y in the Event Type column stands for Serious and N stands for not serious but also makes the reports generated through this solution more user friendly and readable to the users downstream.

Data Transparency
During our interviews with the Business users, one of the concerns raised by them was the fact that the data capturing, transformation and presentation process was a black box to them and this sometimes makes them doubt the data. They then run validation reports on the source systems to confirm the correctness of the data.
My recommendations:

1. Make the data capture process transparent to the business users. This could be through a dash board which allows them to view the various transformation performed on the data, the frequency of the updates and other relevant meta data,.

2. Also have a an proactive alerting system in place which would alert the users (based on predefined business rules) to any issues with the ETL process.

3. In addition introduce factors like – completeness factor (Which indicates how complete the data is, whether essential fields are missing in that record etc) as well as confidence factors (Statistically generate a value which indicates how much of confidence we can have in that record of data). This will boost the analysis and help the users weed out data which could skew the results.

Project Management issues
The best team can fail if the execution is not done properly. Based on our experience in facing and resolving some of the issues, I have the following recommendations to make:

1. Set up a team of people who can perform periodic reviews of the deliverables and ensure that it is following the best practices approach

2. Aim to fuse the process which the team is comfortable with the process which is recommended by the client’s IT department.

3. Provide for time and effort which this and use of client recommended tools would entail.

4. Document the process and ensure that the deliverables are signed off.

5. Manage scope closely.