In-Memory Database - Does star schema still relevant ?

View previous topic View next topic Go down

In-Memory Database - Does star schema still relevant ?

Post  endro.koesno on Fri Oct 11, 2013 2:31 am

Hi all,

Now, many databases start to offer In-Memory Option (SAP HANA, Microsoft SQL Server, Oracle).

If the OLTP and OLAP can be in a single database, does Star Schema Modelling still relevant ?
Do we need a new analytic data modelling in the advent of In-Memory Databases ?

Thanks.

endro.koesno

Posts : 2
Join date : 2013-10-11

View user profile

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  ngalemmo on Fri Oct 11, 2013 11:21 am

Using memory instead of disk is simply another mechanism to store data.  Modeling is a method to organize and arrange data so that you can do useful things with it.  Why do you feel modeling needs to change?

Also, OLTP databases have long been used for analytics. That is why data warehousing came about... because trying to do analytics in an OLTP database was a stupid idea born out of necessity. (Hardware was really expensive.)
avatar
ngalemmo

Posts : 3000
Join date : 2009-05-15
Location : Los Angeles

View user profile http://aginity.com

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  endro.koesno on Fri Oct 11, 2013 10:27 pm

In-memory database make the processing of millions of rows in seconds possible.

SAP has been marketing that with such capability, user can directly query the OLTP data to get information they need.
It makes the Real-Time Analytics possible, so it is unnecessary to transform from 3NF model to star-schema model.

With in-memory-database capability, do we still need datawarehouse ?


endro.koesno

Posts : 2
Join date : 2013-10-11

View user profile

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  ngalemmo on Sat Oct 12, 2013 3:52 pm

The SAP BI 'solution' has been a kludge since it was first introduced. Hana has not changed that, it simply made a poorly designed approach faster.

The reasons to create a data warehouse have nothing to do with hardware performance. The reasons to do so are:

1. Simplify the data model.
2. Resolve data retention and obfuscation issues. You don't want to retain long term data in an operational system. Privacy issues require obfuscation in analytic systems, something you cannot do to the operational system.
3. Stability. Operational systems change at the moment, analytics requires stability in most cases, as well as the ability to recreate a past moment in time.
4. Create derived data. The creation of aggregate data sets, derived attributes, segmentation etc... have no place in an OLTP system.
5. Sandboxes. Statistical analysis of data sets require iterative processes that regularly create and destroy data sets.

I'm sure there are more, but that's what I can think of off the top of my head. Besides, memory does't have the cost/performance benefit of disk. Eventually it will, but even then, organizations will still be implementing data warehouses... only the hardware will change... as it has continued to do so since the concept was first introduced.
avatar
ngalemmo

Posts : 3000
Join date : 2009-05-15
Location : Los Angeles

View user profile http://aginity.com

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  BoxesAndLines on Mon Oct 14, 2013 10:42 am

endro.koesno wrote:In-memory database make the processing of millions of rows in seconds possible.

SAP has been marketing that with such capability, user can directly query the OLTP data to get information they need.
It makes the Real-Time Analytics possible, so it is unnecessary to transform from 3NF model to star-schema model.

With in-memory-database capability, do we still need datawarehouse ?

Teradata espouses the same pattern as well. While a well partitioned data structure can run very fast on Teradata, the resulting overly complex SQL eliminates all but the very well versed SQL experts from extracting data. And unfortunately, imitating a dimensional model in views doesn't work.
avatar
BoxesAndLines

Posts : 1212
Join date : 2009-02-03
Location : USA

View user profile

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  ngalemmo on Mon Oct 14, 2013 12:49 pm

And I know from experience, Netezza can do it without the balancing act needed with Teradata. Realistically, 'millions of rows' ain't what it used to be.
avatar
ngalemmo

Posts : 3000
Join date : 2009-05-15
Location : Los Angeles

View user profile http://aginity.com

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  bluexblue on Fri Dec 26, 2014 5:45 am

My doubt would be .. do we need a star schema with in-memory data warehouse? Can't the ODS meet all reports and dashboards requirements with the speed of in-memory solutions (except the benefit of the readability of the star schema of course)?

And sorry for reviving an old thread

bluexblue

Posts : 1
Join date : 2014-12-26

View user profile

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  ngalemmo on Mon Dec 29, 2014 2:00 pm

Can't the ODS meet all reports and dashboards requirements with the speed of in-memory solutions (except the benefit of the readability of the star schema of course)?

This is a question you need to ask yourself. Typically an ODS does not have the historical longevity of a data warehouse, nor it's stability. Can you do queries off an ODS… of course, that is what it is there for.
avatar
ngalemmo

Posts : 3000
Join date : 2009-05-15
Location : Los Angeles

View user profile http://aginity.com

Back to top Go down

Re: In-Memory Database - Does star schema still relevant ?

Post  Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum