Not drinking the Kool Aid

View previous topic View next topic Go down

Not drinking the Kool Aid

Post  Type2 on Tue Feb 03, 2009 3:03 pm

You are an experienced modeler, architect and developer. You've been hired by a Fortune 500 company because you can recite Kimball methodologies in your sleep and their in-place systems are anything but properly dimensional. Your supervisor sends you to all Kimball University classes and distributes Kimball tips to the entire team as soon as they come in.

Yet when your first design sees light of day it is quickly dissected and you are pushed to implement using the company status quo. Your efficient and clean star schema is massaged into text fields on the fact table, "banning" of junk dimensions, parent-child relationship fact tables…stuff like fingernails on a chalkboard.

Your supervisor continues to push Kimball in meetings and talks, but when it comes time to implement, decisions are based on how hard it will be for the in-place ETL team to do their job, not the end user.

Do your employers/supervisors/team members talk the talk but not walk the walk? What do you do when this happens?
avatar
Type2

Posts : 6
Join date : 2009-02-03

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  ODMonk on Tue Feb 03, 2009 5:21 pm

1) Is the DW the end or is there a cube somewhere?
2) Who's pushing the changes and who's relenting and allowing the changes to occur?

Owen

ODMonk

Posts : 1
Join date : 2009-02-03

View user profile

Back to top Go down

If I was in that position.

Post  Chris Rutherford on Tue Feb 03, 2009 5:38 pm

Hi there,

First I'd chat with my supervisor and explain my frustration. See if there is some underlying reason why he wants to make life easier for the ETL people and not the end users. Depending on his answer you'd not have to read any more.

I'd do what was requested of me but at the same time I'd get some of the more prominent business users (we all know the type, the ones that ask for all the tricky stuff from the data warehouse, want a million reports etc) interested in what a proper dimensional data warehouse can do.

It should not be that difficult given that in your example there is a bunch of Kimball tips floating about in that environment. Once the business users find out that they can have a simple star schema to answer 80% of their questions via a cube or some such technology they should be beating a path to your door. After all my job is to make theirs (the end user) easier so they can deliver real meaningful information to the business with which to make decisions.

In my down time (don't get much but what I do get) I'd mock up a quick dimensional model (I use WhereScape RED and no I don't work for those guys I just love the software) from your source data. Sounds like you should already have the original as it was massaged into something akin to dog tucker, so just show that to the business users.

Now if I don't get any buy in from the business (after extensive prototypes examples etc) then it's not worth flogging a dead horse. Ideally the data warehouse should be designed with the end users in mind not what's easiest for the ETL folks. Now that said some times it's just a complete hair pulling exercise depending on your source systems so you have to be realistic.

If all that fails and I really want to make proper dimensional data warehouses then I'd consider my future employment with that company.

Cheers,

Chris

Chris Rutherford

Posts : 2
Join date : 2009-02-03
Location : New Zealand

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  tod mckenna on Wed Feb 04, 2009 7:14 am

1.) Quit
avatar
tod mckenna

Posts : 9
Join date : 2009-02-03

View user profile http://blog.todmeansfox.com

Back to top Go down

Re: Not drinking the Kool Aid

Post  tod mckenna on Wed Feb 04, 2009 7:19 am

Ok, more seriously. I had a similar situation here at the bank. The solution for me was mostly an education one. First I had to tell them that their "star schemas" were not dimensional models (or even star schemas, for that matter). Then after some debate, I had to show them how one looks (a working prototype was developed). After that, I had to prove that the real dimensional model worked better than what was in place. Now things are sailing along.

Oh, and if you really want to impress management, create a robust date dimension and proceed to show him/her how easy it is to get end of month numbers, quarter breakdowns, and holiday sales statistics!
avatar
tod mckenna

Posts : 9
Join date : 2009-02-03

View user profile http://blog.todmeansfox.com

Back to top Go down

Re: Not drinking the Kool Aid

Post  cbusch on Tue Feb 10, 2009 3:41 pm

Getting over the ETL hump can be difficult for people who are thrown into the dimensional world. I'd suggest giving the ETL team a set of templates that handle the standard transformations. Many of the tools on the market provide these. Once you have the templates in place, there should be no excuses.

cbusch

Posts : 4
Join date : 2009-02-03
Age : 56
Location : Albany NY

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  Purushothaman.VS on Wed Feb 11, 2009 2:50 am

This probably is the problem everywhere, i mean in my experience i have hardly executed a project which follows a perfect Dimensional Model, it is mostly a tailored version which is nowhere close to Dimensional model but more towards the ER model, sometimes on the pretext of keeping the ETL simple or sometimes some other excuse, but the end result is the same.
I believe some of the reasons for this, at least in my organisation is that the DBA's are involved in the Dimensional Modelling and second the ignorance of the ETL team to the BI and the End user requirement.
The solution out of it could be to engage the ETL team in some educative discussions related to the BI and the contribution of a proper dimensional model in making the Reporting easier and faster. The team has to understand the final goal of the Data warehouse and that it is to satisfy the Business users. Even Kimbal has, in his various articles, strongly advocated the fact that no Data warehouse is good, no matter how proper it may be, if the business doesn't accept it and the acceptance of the business would depend on their requirements being addressed. Besides, the Dimensional Modelling should be an activity which should be done more with the Requirements and BI tool to be used in mind and not the ETL. The consideration of ETL comes quite later once the Dimensional model is frozen after checking its feasibility to support the end user requirements. Once this understanding is achieved, i feel the path becomes a lot easier.

I hope one day, we are able to achieve this consensus and model data warehouses which are Dimensional model in real sense and the concepts explained by Kimball could become a reality in all the places........... till then, all the best with the tough times guys

Purushothaman.VS

Posts : 3
Join date : 2009-02-10

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  TStahr on Fri Feb 27, 2009 11:25 pm

Type2 wrote:decisions are based on how hard it will be for the in-place ETL team to do their job, not the end user.

1. Try to hide the complexity of the incoming data behind some intermediate views to make it easier for the ETL team to load.
2. Get a new ETL team. Assuming ETL is being done with a packaged tool, ETL developers are pretty easy (and cheap) to find.

TStahr

Posts : 4
Join date : 2009-02-27

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  Richard on Sun Mar 08, 2009 4:51 pm

Nice to see others feel my pain :-)

Go to this link and scroll down and check out the fact table in a "standard data warehousing star schema". How do you aggregate car_colour? Yikes!

Richard

Posts : 2
Join date : 2009-02-03

View user profile

Back to top Go down

Re: Not drinking the Kool Aid

Post  Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum