TDWI -- The Data Warehousing Institute Germany -- has become the go-to center for business intelligence, big data, analytics, and data warehousing. Today, the association sees itself as a mastermind of the German scene and a vendor-neutral platform where users, suppliers, consulting firms,and scientists can touch base and share ideas. I spoke with executive committee member Professor Peter Gluchowski about current trends in big data and other areas.
Gluchowski heads the Department of Business Informatics specializing in systems development and application systems at Chemnitz University of Technology, where his research interests include aspects of the practical structure of dispositive and analytical systems for decision support. He is the author of numerous publications documenting practical findings in this field.
What have been the most exciting trends in the past 20 years?
Gluchowski: Over the last two decades, the pendulum has swung between front end and back end. Accordingly, emphasis in tool development has also shifted back and forth. Colorful graphical user interfaces emerged in the early 1990s, but at that time there was no back-end data provision. This situation was remedied later on that decade by, for example, data warehousing. Then development reverted to business applications. Although the various trends only lasted for about four or five years, together they brought about a great deal of progress. Nowadays, both back end and front end have reached a good position. Theyíve been joined by strong focus on organizational matters, which is reflected by intensive consideration of structural organization and process organization in BI.
What are the current trends?
Gluchowski: The current trends were outlined in the papers and discussions at the TDWI Conference in Munich, which was attended by over 1,000 participants and was a resounding success. The effects of the global trend towards digital transformation are especially evident from the high interest in big data analytics. Innovative concepts and tools such as R, Spark, and Hadoop are being examined and discussed in depth. On the other hand, weíre also still working on long-standing themes like data governance, agile business intelligence, self-service business intelligence, and of course all aspects of data modeling, such as data vault modeling.
What trends are set to emerge in the coming years?
Gluchowski: Enterprise-wide BI solutions now have to meet a wide range of different requirements. On the one hand, the aim is to build systems for a consistent supply of data which are distinguished by high quality, stability, and availability. On the other hand, we also see business-oriented end-users who want far more autonomy than in the past. We need to create flexible solutions for them that can be quickly adapted to changing needs. The situation boils down to stability versus flexibility.
How do businesses enter the world of business intelligence?
Gluchowski: Most of them have been dealing with business intelligence for years, sometimes without realizing it! Business reporting for instance is part of business intelligence and thatís something which almost every company engages in. Although there are still some substandard applications in use, improvements are being made by nearly all companies. We recommend setting up a broad, stable, quality-assured database with flexible architecture that can be adapted to changing requirements.
What role does big data play?
Gluchowski:Big data concepts and tools are deployed whenever conventional technologies fail. Whether, where, and when this point has been reached is up to each company to decide for itself. Many businesses find it difficult to define the corresponding business case. The advantages of big data arenít always obvious. Once the business case has been defined, itís time to start exploring technological implementation and new tools.
Big data needs a powerful infrastructure. Is Germanyís infrastructure up to speed?
Gluchowski: Although the situationís fine in densely populated areas, many rural districts are still stuck with very poor broadband. However, federal and regional government are endeavoring to improve online access. Then again, large companies in particular have been slow to embrace cloud services as part of the infrastructure. By contrast, small businesses and startups have fewer reservations, so young businesses are obtaining the speed and flexibility they need much faster.
Are you saying that long-established firms need to get to grips with new technology as soon as possible?
Gluchowski: Of course. They have a cautious attitude to modern technology, but they need to overcome their misgivings.
Are the strict data protection regulations in Germany a snag?
Gluchowski: Data privacy is considerably more restrictive in Germany than elsewhere. In analystsí views, itís a barrier, although it does benefit consumers. Data protection needs to be adapted to new circumstances, not abandoned altogether. But this problem isnít insurmountable. Mail-order firms, for instance, have been studying customersí data and consumption behavior for decades, yet ways and means have nonetheless been found to deal with data protection. Good results can still be achieved with anonymous or pseudonymous data.
Germanyís main railway operator Deutsche Bahn recently opened up its data safe. Is DB setting a good example? Why arenít other companies following suit?
Gluchowski: Although Deutsche Bahn is a trailblazer in the corporate world, the real heroes are public authorities. Federal agencies for instance have already achieved a lot under the National Action Plan for the implementation of the G8 Open Data Charter. Cities like Cologne provide plenty of data, and more and more public agencies are doing likewise. Itís no surprise that commercial companies are a little sluggish in this respect. In the past, businesses were reluctant to share information with other parts of the supply chain. Itís quite conceivable that itís company policy for firms to keep their data close to their chest.
Does TDWI believe it has a part to play in educating businesses?
Gluchowski: One key task of TDWI is to create transparency. We need to explain the opportunities that can be created by disclosing data, but also the risks involved. Only then can a business weigh up whether this route makes sense.
What are the long-term goals of TDWI?
Gluchowski: Subjects like data management and data analysis are becoming increasingly important, especially as a result of digital transformation. One goal is obviously to derive concrete information by interpreting business intelligence. Weíre the foremost community regarding business intelligence and analytics, at least in the German-speaking countries. We organize a string of successful formats which are constantly being expanded and improved. Moreover, we mustnít overlook the new generation of data analysts who achieve rapid findings, in some cases with unusual or unconventional methods, and are very successful in the corporate world.